Mark J. Wielaard

glibc already released 2.29, but I was still on a much older version and hadn’t noticed 2.28 (which is the version that is in RHEL8) has a really nice fix for people who obsess about memory leaks.

When running valgrind to track memory leaks you might have noticed that there are sometimes some glibc data structures left.

These are often harmless, small things that are needed during the whole lifetime of the process. So it is normally fine to not explicitly clean that up. Since the memory is reclaimed anyway when the process dies.

But when tracking memory leaks they are slightly annoying. When you want to be sure you don’t have any leaks in your program it is distracting to have to ignore and filter out some harmless leaks.

glibc already had a mechanism to help memory trackers like valgrind memcheck. If you call the secret __libc_freeres function from the last exiting thread, glibc would dutifully free all memory. Which is what valgrind does for you (unless you want to see all the memory left and use --run-libc-freeres=no).

But it didn’t work for memory allocated by pthreads (libpthreads.so) or dlopen (libdl.so). So sometimes you would still see some stray “garbage” left even if you were sure to have released all memory in your own program.

Carlos O’Donell has fixed this:

Bug 23329 – The __libc_freeres infrastructure is not properly run across DSO boundaries.

So upgrade to glibc 2.28+ and really get those memory leaks to zero!

All heap blocks were freed -- no leaks are possible

Julian Seward released valgrind 3.15.0 which updates support for existing platforms and adds a major overhaul of the DHAT heap profiler.  There are, as ever, many refinements and bug fixes.  The release notes give more details.

Nicholas Nethercote used the old experimental DHAT tool a lot while profiling the Rust compiler and then decided to write and contribute A better DHAT (which contains a screenshot of the the new graphical viewer).

CORE CHANGES

• The XTree Massif output format now makes use of the information obtained when specifying --read-inline-info=yes.
• amd64 (x86_64): the RDRAND and F16C insn set extensions are now supported.

TOOL CHANGES

DHAT

• DHAT been thoroughly overhauled, improved, and given a GUI.  As a result, it has been promoted from an experimental tool to a regular tool.  Run it with --tool=dhat instead of --tool=exp-dhat.
• DHAT now prints only minimal data when the program ends, instead writing the bulk of the profiling data to a file.  As a result, the --show-top-n and --sort-by options have been removed.
• Profile results can be viewed with the new viewer, dh_view.html.  When a run ends, a short message is printed, explaining how to view the result.
• See the documentation for more details.

Cachegrind

• cg_annotate has a new option, --show-percs, which prints percentages next to all event counts.

Callgrind

• callgrind_annotate has a new option, --show-percs, which prints percentages next to all event counts.
• callgrind_annotate now inserts commas in call counts, and sort the caller/callee lists in the call tree.

Massif

• The default value for --read-inline-info is now yes on Linux/Android/Solaris. It is still no on other OS.

Memcheck

• The option --xtree-leak=yes (to output leak result in xtree format) automatically activates the option --show-leak-kinds=all, as xtree visualisation tools such as kcachegrind can in any case select what kind of leak to visualise.
• There has been further work to avoid false positives.  In particular, integer equality on partially defined inputs (C == and !=) is now handled better.

OTHER CHANGES

• The new option --show-error-list=no|yes displays, at the end of the run, the list of detected errors and the used suppressions.  Prior to this change, showing this information could only be done by specifying -v -v, but that also produced a lot of other possibly-non-useful messages.  The option -s is equivalent to --show-error-list=yes.

Since the GNU Toolchain has many shared modules it sometimes feels like you have to rebuild everything (assembler, linker, binutils tools, debugger, simulators, etc.) just to get one of the latest tools from source.

Having all this reusable shared code is fun, but it does make build times a bit long.

Luckily most of the “extras” can be disabled if all you want is a fresh new GDB. Sergio Durigan Junior added the GDB configure steps to the GDB wiki so you can build GDB in just a couple of minutes after checking it out.

git clone git://sourceware.org/git/binutils-gdb.git

Simon Marchi just announced that the next GNU Tools Cauldron will be in Montreal, Canada from Thursday September 12 till Sunday September 15.

The purpose of this workshop is to gather all GNU tools developers, discuss current/future work, coordinate efforts, exchange reports on ongoing efforts, discuss development plans for the next 12 months, developer tutorials and any other related discussions. This year, the GNU Tools Cauldron crosses the Atlantic Ocean and lands in Montréal, Canada. We are inviting every developer working in the GNU toolchain: GCC, GDB, binutils, runtimes, etc.

https://gcc.gnu.org/wiki/cauldron2019

The conference is free to attend, registration in advance is required.

David Malcolm wrote a really nice blog post on Usability improvements in GCC 9 describing nicer looking diagnostics, the JSON output format, simpler C++ errors, tracking down bad initializations, optimizer diagnostics and new fix-it hints.

Perhaps a deeper change is that we now have a set of user experience guidelines for GCC, to try to keep a focus on the programmer’s experience as we implement new diagnostics. If you’d like to get involved in GCC development, please join us on the GCC mailing list. Hacking on diagnostics is a great way to get started.

Thanks to Tom de Vries your favorite DWARF Compression tool DWZ now has a testsuite. And there is now also a buildbot CI that tests various combinations of Fedora, CentOS and Debian on armhf, aarch64, ppc64, ppc64le, s390x, x86_64 and i386 to make sure no regressions slip in during development. Happy hacking!

I have a new rsa4096 PGP key (0x1AA44BE649DE760A) created using gnupg. The master ([C]ertification) key is kept offline with the [S]igning, [E]ncryption and [A]uthentication rsa2048 subkeys held on a Nitrokey. The old one (0x8370665B57816A6A) is set to expire at the end of this year (after which I will probably revoke it).

pub   rsa4096 2019-01-19 [C]
      EC3C FE88 F6CA 0788 774F  5C1D 1AA4 4BE6 49DE 760A
 uid       Mark Wielaard mark@klomp.org
 uid       Mark Wielaard mjw@redhat.com
 uid       Mark Wielaard mjw@gnu.org
 sub   rsa2048 2019-01-20 [E]
       31D5 60E6 EC7E 13B8 8724  A568 AE58 4E5D 7893 950F
 sub   rsa2048 2019-01-20 [S]
       1276 8A96 7959 9010 7A0D  2FDF FC57 E3CC ACD9 9A78
 sub   rsa2048 2019-01-20 [A]
       0EBC BDFB 00AC 689B C7C0  FEE4 85C7 C86B C843 7776

Get the new public key at https://gnu.wildebeest.org/~mark/gnupg-pub.txt or from one of the keyservers with gpg --recv-key 0x1AA44BE649DE760A.

I’ll use the new (signing) key for release tar balls (elfutils 0.176 was signed with the new key). And I am using gpg-agent --enable-ssh-support now so I can use the new (authentication) key for ssh access. To get my current ssh pubkey simply gpg --export-ssh-key 0x1AA44BE649DE760A.

Clear signed text of the above statement.

     ____               
    /    \              
   |-. .-.|             
   (_@)(_@)             
   .---_  \             
  /..   \_/             
  |__.-^ /              
      }  |              
     |   [              
     [  ]               
    ]   |               
    |   [               
    [  ]                
   /   |        __      
  \|   |/     _/ /_     
 \ |   |//___/__/__/_   
\\  \ /  //    -____/_  
//   "   \\      \___.- 
 //     \\  __.----._/_ 
/ //|||\\ .-         __>
[        /         __.- 
[        [           }  
\        \          /   
 "-._____ \.____.--"    
    |  | |  |           
    |  | |  |           
    |  | |  |           
    |  | |  |           
    {  } {  }           
    |  | |  |           
    |  | |  |           
    |  | |  |           
    /  { |  |           
 .-"   / [   -._        
/___/ /   \ \___"-.     
    -"     "-           

strace patch.

At Fosdem we had a talk on dtrace for linux in the Debugging Tools devroom.

Not explicitly mentioned in that talk, but certainly the most exciting thing, is that Oracle is doing a proper linux kernel port:

 commit e1744f50ee9bc1978d41db7cc93bcf30687853e6
 Author: Tomas Jedlicka <tomas.jedlicka@oracle.com>
 Date: Tue Aug 1 09:15:44 2017 -0400

 dtrace: Integrate DTrace Modules into kernel proper

 This changeset integrates DTrace module sources into the main kernel
 source tree under the GPLv2 license. Sources have been moved to
 appropriate locations in the kernel tree.

That is right, dtrace dropped the CDDL and switched to the GPL!

The user space code dtrace-utils and libdtrace-ctf (a combination of GPLv2 and UPL) can be found on the DTrace Project Source Control page. The NEWS file mentions the license switch (and that it is build upon elfutils, which I personally was pleased to find out).

The kernel sources (GPLv2+ for the core kernel and UPL for the uapi) are slightly harder to find because they are inside the uek kernel source tree, but following the above commit you can easily get at the whole linux kernel dtrace directory.

Update: There is now a dtrace-linux-kernel.git repository with all the dtrace commits rebased on top of recent upstream linux kernels.

The UPL is the Universal Permissive License, which according to the FSF is a lax, non-copyleft license that is compatible with the GNU GPL.

Thank you Oracle for making everyone’s life easier by waving your magic relicensing wand!

Now there is lots of hard work to do to actually properly integrate this. And I am sure there are a lot of technical hurdles when trying to get this upstreamed into the mainline kernel. But that is just hard work. Which we can now start collaborating on in earnest.

Like systemtap and the Dynamic Probes (dprobes) before it, dtrace is a whole system observability tool combining tracing, profiling and probing/debugging techniques. Something the upstream linux kernel hackers don’t always appreciate when presented as one large system. They prefer having separate small tweaks for tracing, profiling and probing which are mostly separate from each other. It took years for the various hooks, kprobes, uprobes, markers, etc. from systemtap (and other systems) to get upstream. But these days they are. And there is now even a byte code interpreter (eBPF) in the mainline kernel as originally envisioned by dprobes, which systemtap can now target through stapbpf. So with all those techniques now available in the linux kernel it will be exciting to see if dtrace for linux can unite them all.

I did an interview with the Software Freedom Conservancy to discuss why I try to contribute to the Conservancy whenever I can. Because I believe many more free software communities deserve to have a home for their project at the Conservancy.

Please support the Software Freedom Conservancy by donating so they will be able to provide a home to many more communities. A donation of 10 US dollars a month will make you an official sponsor. Or donate directly to one of their many member projects.

Software Freedom Conservancy Member Projects