The example below shows how a standalone program can be instrumented with the libcpc routines to use hardware performance counters to measure events in a process. The program performs 20 iterations of a computation, measuring the counter values for each iteration. By default, the example makes the counters measure external cache references and external cache hits; these options are only appropriate for UltraSPARC processors. By setting the PERFEVENTS environment variable to other strings (a list of which can be gleaned from the -h flag of the cpustat or cputrack utilities), other events can be counted. The error() routine below is assumed to be a user-provided routine analogous to the familiar printf(3C) routine from the C library but which also performs an exit(2) after printing the message.

#include <inttypes.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <libcpc.h>
int
main(int argc, char *argv[])
{
int cpuver, iter;
char *setting = NULL;
cpc_event_t event;

if (cpc_version(CPC_VER_CURRENT) != CPC_VER_CURRENT)
    error("application:library cpc version mismatch!");

if ((cpuver = cpc_getcpuver()) == -1)
    error("no performance counter hardware!");

if ((setting = getenv("PERFEVENTS")) == NULL)
    setting = "pic0=EC_ref,pic1=EC_hit";

if (cpc_strtoevent(cpuver, setting, &event) != 0)
    error("can't measure '%s' on this processor", setting);
setting = cpc_eventtostr(&event);

if (cpc_access() == -1)
    error("can't access perf counters: %s", strerror(errno));

if (cpc_bind_event(&event, 0) == -1)
    error("can't bind lwp%d: %s", _lwp_self(), strerror(errno));

for (iter = 1; iter <= 20; iter++) {
    cpc_event_t before, after;

    if (cpc_take_sample(&before) == -1)
        break;

    /* ==> Computation to be measured goes here <== */

    if (cpc_take_sample(&after) == -1)
        break;
    (void) printf("%3d: %" PRId64 " %" PRId64 "\n", iter,
        after.ce_pic[0] - before.ce_pic[0],
        after.ce_pic[1] - before.ce_pic[1]);
}

if (iter != 20)
    error("can't sample '%s': %s", setting,    strerror(errno)); 

free(setting);
return (0);
}

This example builds on Example 1, but demonstrates how to write the signal handler to catch overflow signals. The counters are preset so that counter zero is 1000 counts short of overflowing, while counter one is set to zero. After 1000 counts on counter zero, the signal handler will be invoked.

First the signal handler:

#definePRESET0 (UINT64_MAX - 999ull) #definePRESET1 0 void emt_handler(int sig, siginfo_t *sip, void *arg) { ucontext_t *uap = arg; cpc_event_t sample; if (sig != SIGEMT || sip->si_code != EMT_CPCOVF) { psignal(sig, "example"); psiginfo(sip, "example"); return; } (void) printf("lwp%d - si_addr %p ucontext: %%pc %p %%sp %p\n", _lwp_self(), (void *)sip->si_addr, (void *)uap->uc_mcontext.gregs[PC], (void *)uap->uc_mcontext.gregs[USP]); if (cpc_take_sample(&sample) == -1) error("can't sample: %s", strerror(errno)); (void) printf("0x%" PRIx64 " 0x%" PRIx64 "\n", sample.ce_pic[0], sample.ce_pic[1]); (void) fflush(stdout); sample.ce_pic[0] = PRESET0; sample.ce_pic[1] = PRESET1; if (cpc_bind_event(&sample, CPC_BIND_EMT_OVF) == -1) error("cannot bind lwp%d: %s", _lwp_self(), strerror(errno)); }

and second the setup code (this can be placed after the code that selects the event to be measured):

struct sigaction act; cpc_event_t event; ... act.sa_sigaction = emt_handler; bzero(&act.sa_mask, sizeof (act.sa_mask)); act.sa_flags = SA_RESTART|SA_SIGINFO; if (sigaction(SIGEMT, &act, NULL) == -1) error("sigaction: %s", strerror(errno)); event.ce_pic[0] = PRESET0; event.ce_pic[1] = PRESET1; if (cpc_bind_event(&event, CPC_BIND_EMT_OVF) == -1) error("cannot bind lwp%d: %s", _lwp_self(), strerror(errno)); for (iter = 1; iter <= 20; iter++) { /* ==> Computation to be measured goes here <== */ } cpc_bind_event(NULL, 0); /* done */

Note that a more general version of the signal handler would use write(2) directly instead of depending on the signal-unsafe semantics of stderr and stdout. Most real signal handlers will probably do more with the samples than just print them out.

rd %pic, %rN ! All UltraSPARC wr %rN, %pic ! (ditto, but see text)

rdpmc ! Pentium II only

If the counter context is not active or has been invalidated, the %pic register (SPARC), and the rdpmc instruction (Pentium) will become unavailable.

Note that the two 32-bit UltraSPARC performance counters are kept in the single 64-bit %pic register so a couple of additional instructions are required to separate the values. Also note that when the %pcr register bit has been set that configures the %pic register as readable by an application, it is also writable. Any values written will be preserved by the context switching mechanism.

Pentium II processors support the non-privileged rdpmc instruction which requires [5] that the counter of interest be specified in %ecx, and returns a 40-bit value in the %edx:%eax register pair. There is no non-privileged access mechanism for Pentium I processors.

As described above, when counting events, some processors allow their counter registers to silently overflow. More recent CPUs such as UltraSPARC III and Pentium II, however, are capable of generating an interrupt when the hardware counter overflows. Some processors offer more control over when interrupts will actually be generated. For example, they might allow the interrupt to be programmed to occur when only one of the counters overflows. See cpc_strtoevent(3CPC) for the syntax.

The most obvious use for this facility is to ensure that the full 64-bit counter values are maintained without repeated sampling. However, current hardware does not record which counter overflowed. A more subtle use for this facility is to preset the counter to a value to a little less than the maximum value, then use the resulting interrupt to catch the counter overflow associated with that event. The overflow can then be used as an indication of the frequency of the occurrence of that event.

Note that the interrupt generated by the processor may not be particularly precise. That is, the particular instruction that caused the counter overflow may be earlier in the instruction stream than is indicated by the program counter value in the ucontext.

When cpc_bind_event() is called with the CPC_BIND_EMT_OVF flag set, then as before, the control registers and counters are preset from the 64-bit values contained in event. However, when the flag is set, the kernel arranges to send the calling process a SIGEMT signal when the overflow occurs, with the si_code field of the corresponding siginfo structure set to EMT_CPCOVF, and the si_addr field is the program counter value at the time the overflow interrupt was delivered. Counting, and thus the subsequent delivery of the signal on overflow is disabled until the next call to cpc_bind_event(). Even in a multithreaded process, during execution of the signal handler, the thread behaves as if it is temporarily bound to the running LWP.

Different processors have different counter ranges available, though all processors supported by Solaris allow at least 31 bits to be specified as a counter preset value; thus portable preset values lie in the range UINT64_MAX to UINT64_MAX-INT32_MAX.

The appropriate preset value will often need to be determined experimentally. Typically, it will depend on the event being measured, as well as the desire to minimize the impact of the act of measurement on the event being measured; less frequent interrupts and samples lead to less perturbation of the system.

If the processor cannot detect counter overflow, this call will fail (ENOTSUP). Specifying a null event unbinds the context from the underlying LWP and disables signal delivery. Currently, only user events can be measured using this technique. See Example 2, above.

By default, the library binds the performance counter context to the current LWP only. If the CPC_BIND_LWP_INHERIT flag is set, then any subsequent LWPs created by that LWP will automatically inherit the same performance counter context. The counters will be initialized to 0 as if a cpc_bind_event() had just been issued. This automatic inheritance behavior can be useful when dealing with multithreaded programs to determine aggregate statistics for the program as a whole.

If the CPC_BIND_EMT_OVF flag is also set, the process will immediately dispatch a SIGEMT signal to the freshly created LWP so that it can preset its counters appropriately on the new LWP. This initialization condition can be detected using cpc_take_sample() to check that both ce_pic[] values are set to UINT64_MAX.