The 7 dwarves

Arnaldo Carvalho de Melo wrote an interesting paper for OLS: The 7 dwarves: debugging information beyond gdb. Dwarves is a DWARF debugging information library and a set of tools that uses the DWARF information inserted in ELF binaries. The tools can help you understand DWARF and the debug information available in programs (and the kernel) to do such fun things as finding holes in data structures, cacheline alignment, pack those structures (it can actually decode the debug info and generate C source code for you and explain why and how it moved the fields around) and it can analyse inline decissions made by the compiler and tell you what functions got inlined by the compiler and which were marked for inlining by the programmer. You can also get something like japitools for C/C++ with the codiff utility that inspects data structures and function changes between different versions of a binary. One interesting thing is ctracer that can use the information on structs and functions to automatically track changes in those datastructures when it moves through the code. Currently this only works for the kernel and uses raw kprobes to collect statistics. But one idea is to extend this to automatically generate systemtap scripts to gain all the safety guards that systemtap provides for statistics collection of a running kernel and with uprobes coming it will then extend to user space also.

Another nice paper about DWARF is Introduction to the DWARF Debugging Format, by Michael Eager