The 2006 OLS paper Perfmon2: A flexible performance monitoring interface for Linux by Stéphane Eranian gives an overview of designing a generic interface for hardware monitoring for a diverse test of processors. Modern processors have all kinds of support to collect information about cpu cycles used, instruction pipelines, on-chip caches, etc.

Since the Performance Monitoring Unit (PMU) on different processor architectures are so different and support collecting very different sets of events designing a generic interface is pretty hard. But it looks like perfmon2 does present a nice, although somewhat complex, common interface to program the Performance Monitoring Configuration (PMC) registers and read the data collected in the Performance Monitoring Data (PMD) registers. And it provides a somewhat nicer way to program and collect data than having to go through the raw hardware. It provides things like generic 64-bit counters for events (and in gneeral makes sure that all data structures use fixed-size data types), even if the underlying hardware has smaller counters (doing emulation in software when the hardware counters overflow). And most importantly makes it available to user space in a secure manner, so one doesn’t need direct hardware access in privileged mode and prevents “data leaks” between untrusted processes.

The interface allows for both counting and profiling (sampling) events on a per-thread or per-cpu basis. But whole-process or whole-system profilling is left up to the user (through ptrace attaching a thread and tracking clone events, although exec events do automatically carry over monitoring contexts. Maybe a utrace based framework would make things simpler here, but currently it seems the perfmon2 and utrace patches don’t mix). The paper is somewhat vague on how and when one can use a mixed per-thread and per-cpu environment, which is somewhat unfortunate since it looks like if an admin is using per-cpu monitoring a self-monitoring process cannot simultaneously use the MPD counters. Self-monitoring is interesting since it means a dynamic runtime environment like hotspot that dynamically regenerates code can easily see “hot code paths”. One limitation seems to be that threads using this technique need to be tied to one processor since the monitoring context cannot migrate between CPUs.

The paper gives a good overview of the various techniques used to detect and access the various events supported by the CPU and expose the counters through new system calls, translation files in /sys/kernel from logical registers to actual register names and mapping in read only shared buffers between kernel and user space for self-monitoring threads. The event sets and multiplexing of events is interesting but very abstract. The paper doesn’t contain any code samples and one is assumed to know the kind of performance event counters modern CPUs support. Things become a little easier if one reads this paper while having access to a system with pfmon tool (and the perfmon2 kernel patch) installed or reading the pfmon manual to look at examples to make things a bit more concrete.

From Lillian on the OpenJDK distro-pkg-dev mailinglist about IcedTea 1.1

We are proud to announce the release of IcedTea 1.1. This release represents the stabilization of Crypto and SSL support.

The IcedTea project provides a harness to build the source code from OpenJDK (http://openjdk.java.net) using Free Software build tools and provides replacements libraries for the binary plugs with code from the GNU Classpath project. More information on IcedTea can be found here: http://icedtea.classpath.org

What’s new?

• GNU Crypto security providers have been merged from GNU Classpath.
• SSL support by means of the IcedTls security provider.
• DebugInfo is always generated.
• Initial graphics work: Color management and raster fixes for compatibility with LittleCMS, and work to satisfy libt2k requirements with Freetype.
• Uses OpenJDK build 15.
• Various bug fixes: http://icedtea.classpath.org/bugzilla/

The tarball and nosrc RPM can be downloaded here: http://icedtea.classpath.org/download/

The following people helped with this release:
Gary Benson, Thomas Fitzsimmons, Kyle Galloway, Andrew Haley, Francis Kung, Casey Marshall, Mark Wielaard and Lillian Angel.

We would also like to thank the bug reporters and testers!

Nice work!

Arnaldo Carvalho de Melo wrote an interesting paper for OLS: The 7 dwarves: debugging information beyond gdb. Dwarves is a DWARF debugging information library and a set of tools that uses the DWARF information inserted in ELF binaries. The tools can help you understand DWARF and the debug information available in programs (and the kernel) to do such fun things as finding holes in data structures, cacheline alignment, pack those structures (it can actually decode the debug info and generate C source code for you and explain why and how it moved the fields around) and it can analyse inline decissions made by the compiler and tell you what functions got inlined by the compiler and which were marked for inlining by the programmer. You can also get something like japitools for C/C++ with the codiff utility that inspects data structures and function changes between different versions of a binary. One interesting thing is ctracer that can use the information on structs and functions to automatically track changes in those datastructures when it moves through the code. Currently this only works for the kernel and uses raw kprobes to collect statistics. But one idea is to extend this to automatically generate systemtap scripts to gain all the safety guards that systemtap provides for statistics collection of a running kernel and with uprobes coming it will then extend to user space also.

Another nice paper about DWARF is Introduction to the DWARF Debugging Format, by Michael Eager

Fun for the whole family! GNU GPLv3 Launch. And Luis Villa finished his 4-part GPLv3, The Q/A:

• part 1 – the license
• part 2 – developers
• part 3 – companies
• part 4 – odds and ends

Update: GNU-Linux Software License Revision Praised By SIFMA. Wow. Interesting move by the large financial and securities companies. It seems these large users do actually like Free Software and the rights they get under GPLv3 much more than all those restrictions they normally get with proprietary EULAs.

At the end of this month the long awaited version 3 of the GNU General Public License will finally be released and slowly but surely various projects will adopt this new license upgrade. If you have only followed the public drafting process that has gone on since November 2005 from a distance or if you were already sold on the first draft in January 2006 you certainly want to read up on Luis Villa‘s GPL v3, the Q&A (part 1), a lot has been improved since those early days. As a small teaser here are the first and the last Q/A (and this is only part 1 of 4!):

Q: So why are we here?

A: At the end of this week, after 16 years, the Free Software Foundation should bless version three of the GNU General Public License, the sequel to what is arguably the most widely used and most impactful copyright license ever.¹ Quite literally everyone who makes software – open, proprietary, or web – needs to understand the v3 and figure out if it is evolutionary, revolutionary, or DOA, and how it impacts them as a potential contributor, consumer, cooperator or competitor. This is my small contribution towards that understanding.

Q: Bottom line- in a year, what are we going to be saying about GPL?

A: Over the next few days I’ll explore the details of how developers and companies might feel about the new license, but I think the bottom line is that within a few years many will switch over. Communities who feel particularly threatened by patents, who badly want to draw from Apache, or who are particularly inclined to protect users and disregard possible costs to developers will adopt it particularly quickly. After that, adoption will slow for a while, but as users, developers, and corporates get comfortable, the various small improvements will gradually make it the default license for a plurality of new open source code, despite the understandable reservations (some well-grounded, some not) that many people currently feel.

So go read it and make sure to catch up when parts 2, 3 and 4 are published.

Since I wanted to look a bit at the history and changes made to OpenJDK over time, but was going to be offline for a while (and the subversion server of openjdk seems a little slow even if I were online) I decided to create a read-only openjdk mercurial mirror so I could carry the whole project history with me on a laptop and do quick diffs.

Creating a mercurial mirror of a subversion repo (and keeping it up to date with a little cronjob) is pretty easy with hgsvn.

The benefits are really huge. Not only is making diffs between any two versions of any files instant once you have a local mercurial clone, it also gives you an easy way to experiment with your local patches and have them under version control, creating, merging and generating meaningful diffs between branches is much nicer than with subversion, and it is much, much more space efficient than subversion. A checkout of 1 revision of openjdk with subversion is 1.2GB, the whole mercurial repo, which includes all revisions, takes just 740MB disk space. Amazing.

Update: Saw gcc has an experimental mercurial mirror that is also amazingly small.

InfoQ had a nice article about the various GNU Classpath/OpenJDK hybrids, mentioning the work by the various teams (icedtea, ikvm, cacao, etc) doing the integration with as conclusion:

It is encouraging that the code sharing is already going both ways. Free Software projects are not just incorporating the newly freed code, they are also helping to advance the OpenJDK. It seems clear that this is only the beginning.

Funny how the same facts can lead to both screaming hell and doom about evil forking or, as this article, praise the sharing of code. Sharing, forking, and merging code bases are always intertwined in interesting ways.

IcedTea (software)

From Wikipedia, the free encyclopedia

This article is about the Java Development Kit. For the drink, see Iced tea. For other uses, see Iced tea (disambiguation).

This article or section contains information about computer software currently in development.The content may change as the software development progresses.

Linus Torvalds had an interesting comment on moving the linux kernel to GPLv3:

If Sun really _is_ going to release OpenSolaris under GPLv3, that _may_ be a good reason. I don’t think the GPLv3 is as good a license as v2, but on the other hand, I’m pragmatic, and if we can avoid having two kernels with two different licenses and the friction that causes.

I personally think GPLv3 is a massive improvement over GPLv2 and that we really need the better compatibility, internationalization, explicit patent peace provisions and the requirement for installation information that it provides. So I am hoping for a massive upgrade as soon as it comes out at the end of this month. BTW. Please add any last call comment asap at http://gplv3.fsf.org/ if you haven’t done so before.

Also Glyn Foster seems to have noticed our efforts and made the following funny remark on the OpenSolaris indiana-discuss list:

We need a plan, we need some iced tea.

Go Gman!

I have to admit I hadn’t expected a fully free boostrappable OpenJDK already, but there it is! IcedTea patches OpenJDK so that it doesn’t need any binary blobs anymore, replacing the blobs with GNU Classpath code, compiles it all using gcj and (optionally) bootstraps itself using the just build hotspot and javac. Impressive! No full gui support yet, but non-gui stuff runs fine and even the Mauve test results look pretty decent. Woot!