There has been talk on the Perfctr mailing list (which gets hopelessly spammed these days) for including support for multi-threaded processes, but thus far I’ve seen nothing that does what I want. So, without further ado, here’s how to patch your kernel to support multi-threaded applications.

I assume you know how to install the basic Perfctr driver, and compile your kernel to add support for it. If possible, compile it as a module, as this is easiest if you need to change things (unless you also change stuff in the kernel header files, in which case you probably want to recompile the complete kernel). Let’s further assume your kernel source lives in /usr/src/linux, further referred to as toplevel. I’ll also assume we’re building the Perfctr module here.

The first thing that needs to be done is make sure that child processes set up their kernel data structures such that performance counter data can be stored at context switch (you want to use virtual counters, i.e., per-process counters). Therefore you need to add a function that does exactly this. We’ll call it __vperfctr_set_child_perfctr(struct task_struct*, struct task_struct*). You also want to be able to set up and empty vperfctr structure, by simply allocating space for it. So, add their existance to your toplevel/include/linux/perfctr.h file (which should be there if you patched the kernel and copied the relevant files when installing Perfctr), which should then read:

In the same file you should add some code to the perfctr_copy_task(struct task_struct *, struct pt_regs *) function. Otherwise it only contains a comment stating that nothing should be done until inheritance is implemented and sets the vperfctr structure to NULL. The code for that function should become the following:

The above will get a new structure set up (through __vperfctr_get_empty) and copy the existing settings (i.e., for the control registers etc.) to that structure (through __vperfctr_set_child_perfctr). Before we can use these functions, we need to define them, which is done in toplevel/drivers/perfctr/virtual.c.

I think the Perfctr patch uses p->thread as the first argument, so change this accordingly.

Now, to get the counter values assembled in one central spot, I’m using a second module that accumulates these values. Upon exit, each thread will pass on their values to that module through a hook. The code has protection for usage on multicore processors, through a spin_lock.

Here’s the code you should add to toplevel/drivers/perfctr/virtual.c to be able to access the accumulating module through the hook.

The final piece then is the read_counter module where the data is accumulated.

If you wish to read the data, you must create the device /dev/read_counters with major number 120 and minor number 0. Code that does the reading for you might look like this:

Method-Level Phase Behavior in Java Workloads, Andy Georges, Dries Buytaert, Lieven Eeckhout, and Koen De Bosschere

The paper abstract reads as follows.

Java workloads are becoming more and more prominent on various computing devices. Understanding the behavior of a Java workload which includes the interaction between the application and the virtual machine (VM), is thus of primary importance during performance analysis and optimization. Moreover, as contemporary software projects are increasing in complexity, automatic performance analysis techniques are indispensable. This paper proposes an off-line method-level phase analysis approach for Java workloads that consists of three steps. In the first step, the execution time is computed for each method invocation. Using an off-line tool, we subsequently analyze the dynamic call graph (that is annotated with the method invocations` execution times) to identify method-level phases. Finally, we measure performance characteristics for each of the selected phases. This is done using hardware performance monitors. As such, our approach allows for linking microprocessor-level information at the individual methods in the Java application`s source code. This is extremely interesting information during performance analysis and optimization as programmers can use this information to optimize their code. We evaluate our approach in the Jikes RVM on an IA-32 platform using the SPECjvm98 and SPECjbb2000 benchmarks. This is done according to a number of important criteria: the overhead during profiling, the variability within and between the phases, its applicability in Java workload characterization (measuring performance characteristics of the various VM components) and application bottleneck identification.

How Java Programs Interact with Virtual Machines at the Microarchitectural Level, Lieven Eeckhout, Andy Georges, and Koen De Bosschere.

The paper abstract reads as follows.

Java workloads are becoming increasingly prominent on various platforms ranging from embedded systems, over general-purpose computers to high-end servers. Understanding the implications of all the aspects involved when running Java workloads, is thus extremely important during the design of a system that will run such workloads. In other words, understanding the interaction between the Java application, its input and the virtual machine it runs on, is key to a succesful design. The goal of this paper is to study this complex interaction at the microarchitectural level, e.g., by analyzing the branch behavior, the cache behavior, etc. This is done by measuring a large number of performance characteristics using performance counters on an AMD K7 Duron microprocessor. These performance characteristics are measured for seven virtual machine configurations, and a collection of Java benchmarks with corresponding inputs coming from the SPECjvm98 benchmark suite, the SPECjbb2000 benchmark suite, the Java Grande Forum benchmark suite and an open-source raytracer, called Raja with 19 scene descriptions. This large amount of data is further analyzed using statistical data analysis techniques, namely principal components analysis and cluster analysis. These techniques provide useful insights in an understandable way.From our experiments, we conclude that (i) the behavior observed at the microarchitectural level is primarily determined by the virtual machine for small input sets, e.g., the SPECjvm98 s1 input set; (ii) the behavior can be quite different for various input sets, e.g., short-running versus long-running benchmarks; (iii) for long-running benchmarks with few hot spots, the behavior can be primarily determined by the Java program and not the virtual machine, i.e., all the virtual machines optimize the hot spots to similarly behaving native code; (iv) in general, the behavior of a Java application running on one virtual machine can be significantly different from running on another virtual machine. These conclusions warn researchers working on Java workloads to be careful when using a limited number of Java benchmarks or virtual machines since this might lead to biased conclusions.