Usability improvements in GCC 9

David Malcolm wrote a really nice blog post on Usability improvements in GCC 9 describing nicer looking diagnostics, the JSON output format, simpler C++ errors, tracking down bad initializations, optimizer diagnostics and new fix-it hints.

Perhaps a deeper change is that we now have a set of user experience guidelines for GCC, to try to keep a focus on the programmer’s experience as we implement new diagnostics. If you’d like to get involved in GCC development, please join us on the GCC mailing list. Hacking on diagnostics is a great way to get started.

Buildbot CI for DWZ

Thanks to Tom de Vries your favorite DWARF Compression tool DWZ now has a testsuite. And there is now also a buildbot CI that tests various combinations of Fedora, CentOS and Debian on armhf, aarch64, ppc64, ppc64le, s390x, x86_64 and i386 to make sure no regressions slip in during development. Happy hacking!

New PGP key

I have a new rsa4096 PGP key (0x1AA44BE649DE760A) created using gnupg. The master ([C]ertification) key is kept offline with the [S]igning, [E]ncryption and [A]uthentication rsa2048 subkeys held on a Nitrokey. The old one (0x8370665B57816A6A) is set to expire at the end of this year (after which I will probably revoke it).

pub   rsa4096 2019-01-19 [C]
EC3C FE88 F6CA 0788 774F 5C1D 1AA4 4BE6 49DE 760A
uid Mark Wielaard mark@klomp.org
uid Mark Wielaard mjw@redhat.com
uid Mark Wielaard mjw@gnu.org
sub rsa2048 2019-01-20 [E]
31D5 60E6 EC7E 13B8 8724 A568 AE58 4E5D 7893 950F
sub rsa2048 2019-01-20 [S]
1276 8A96 7959 9010 7A0D 2FDF FC57 E3CC ACD9 9A78
sub rsa2048 2019-01-20 [A]
0EBC BDFB 00AC 689B C7C0 FEE4 85C7 C86B C843 7776

Get the new public key at https://gnu.wildebeest.org/~mark/gnupg-pub.txt or from one of the keyservers with gpg --recv-key 0x1AA44BE649DE760A.

I’ll use the new (signing) key for release tar balls (elfutils 0.176 was signed with the new key). And I am using gpg-agent --enable-ssh-support now so I can use the new (authentication) key for ssh access. To get my current ssh pubkey simply gpg --export-ssh-key 0x1AA44BE649DE760A.

Clear signed text of the above statement.

Raising Strauss awareness

     ____               
    /    \              
   |-. .-.|             
   (_@)(_@)             
   .---_  \             
  /..   \_/             
  |__.-^ /              
      }  |              
     |   [              
     [  ]               
    ]   |               
    |   [               
    [  ]                
   /   |        __      
  \|   |/     _/ /_     
 \ |   |//___/__/__/_   
\\  \ /  //    -____/_  
//   "   \\      \___.- 
 //     \\  __.----._/_ 
/ //|||\\ .-         __>
[        /         __.- 
[        [           }  
\        \          /   
 "-._____ \.____.--"    
    |  | |  |           
    |  | |  |           
    |  | |  |           
    |  | |  |           
    {  } {  }           
    |  | |  |           
    |  | |  |           
    |  | |  |           
    /  { |  |           
 .-"   / [   -._        
/___/ /   \ \___"-.     
    -"     "-           

strace patch.

dtrace for linux; Oracle does the right thing

At Fosdem we had a talk on dtrace for linux in the Debugging Tools devroom.

Not explicitly mentioned in that talk, but certainly the most exciting thing, is that Oracle is doing a proper linux kernel port:

 commit e1744f50ee9bc1978d41db7cc93bcf30687853e6
 Author: Tomas Jedlicka <tomas.jedlicka@oracle.com>
 Date: Tue Aug 1 09:15:44 2017 -0400

 dtrace: Integrate DTrace Modules into kernel proper

 This changeset integrates DTrace module sources into the main kernel
 source tree under the GPLv2 license. Sources have been moved to
 appropriate locations in the kernel tree.

That is right, dtrace dropped the CDDL and switched to the GPL!

The user space code dtrace-utils and libdtrace-ctf (a combination of GPLv2 and UPL) can be found on the DTrace Project Source Control page. The NEWS file mentions the license switch (and that it is build upon elfutils, which I personally was pleased to find out).

The kernel sources (GPLv2+ for the core kernel and UPL for the uapi) are slightly harder to find because they are inside the uek kernel source tree, but following the above commit you can easily get at the whole linux kernel dtrace directory.

Update: There is now a dtrace-linux-kernel.git repository with all the dtrace commits rebased on top of recent upstream linux kernels.

The UPL is the Universal Permissive License, which according to the FSF is a lax, non-copyleft license that is compatible with the GNU GPL.

Thank you Oracle for making everyone’s life easier by waving your magic relicensing wand!

Now there is lots of hard work to do to actually properly integrate this. And I am sure there are a lot of technical hurdles when trying to get this upstreamed into the mainline kernel. But that is just hard work. Which we can now start collaborating on in earnest.

Like systemtap and the Dynamic Probes (dprobes) before it, dtrace is a whole system observability tool combining tracing, profiling and probing/debugging techniques. Something the upstream linux kernel hackers don’t always appreciate when presented as one large system. They prefer having separate small tweaks for tracing, profiling and probing which are mostly separate from each other. It took years for the various hooks, kprobes, uprobes, markers, etc. from systemtap (and other systems) to get upstream. But these days they are. And there is now even a byte code interpreter (eBPF) in the mainline kernel as originally envisioned by dprobes, which systemtap can now target through stapbpf. So with all those techniques now available in the linux kernel it will be exciting to see if dtrace for linux can unite them all.

Sponsor Software Freedom Conservancy

I did an interview with the Software Freedom Conservancy to discuss why I try to contribute to the Conservancy whenever I can. Because I believe many more free software communities deserve to have a home for their project at the Conservancy.

Please support the Software Freedom Conservancy by donating so they will be able to provide a home to many more communities. A donation of 10 US dollars a month will make you an official sponsor. Or donate directly to one of their many member projects.

Software Freedom Conservancy Member Projects

Software Freedom Conservancy Member Projects

Advogato has been archived

Advogato has been archived.

When I started working on Free Software advogato was the “social network” where people would keep their diaries (I don’t believe we called them blogs yet). I still remember how proud I was when people certified me as Apprentice.

A lot of people on Planet Classpath still have their diaries imported from Advogato. robilad, audriusa, saugart, rmathew, Anthony, kuzman, jvic, jserv, aph, twisti, Ringding please let me know if you found a new home for your diary.

Fedora rpm debuginfo improvements for rawhide/f27

Hi Fedora Packagers,

rawhide rpmbuild contains various debuginfo improvements that hopefully will make various hacks in spec files redundant.

If you have your own way of handling debuginfo packages, calling find-debuginfo.sh directly, need hacks for working around debugedit limitations or split your debuginfo package by hand then please try out rpmbuild in rawhide and read below for some macros you can set to tweak debuginfo package generation.

If you still need hacks in your spec file because setting macros isn’t enough to get the debuginfo packages you want then please let us know. Also please let us know about packages that need to set debuginfo rpm macros to non-default values because they would crash and burn with the default settings (best to file a bug against rpmbuild).

The improvements have been mainly driven by the following two change proposals for f27 (some inspired by what other distros do):

https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo
https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo

The first is completely done and has been enabled by default for some months now in rawhide. The second introduces two new macros to enable separate debugsource and sub-debuginfo packages, but has not been enabled by default yet. If people like the change and no bugs are found (and fesco and releng agree) we can enable them for the f27 mass rebuild.

If your package already splits debuginfo packages in a (common) source package and/or sub-debuginfo packages, please try out the new macros introduced by the second change. You can enable the standard splitting by adding the following to your spec file:

%global _debugsource_packages 1
%global _debuginfo_subpackages 1

Besides the above two changes debuginfo packages can now (and are by default in rawhide) build by running debug extraction in parallel. This should speed up building with lots of binaries/libraries. If you do invoke find-debuginfo.sh by hand you most likely will want to add %{?_smp_mflags} as argument to get the parallel processing speedup.

If your package is invoking find-debuginfo.sh by hand also please take a look at all the new options that have been added. Also note that almost all options can be changed by setting (or undefining) rpm macros now. Using the rpm macros is preferred over invoking find-debuginfo.sh directly since it means you get any defaults and improvements that might need new find-debuginfo.sh arguments automatically.

Here is an overview of various debuginfo rpm macros that you can define undefine in your spec file with the latest rpmbuild:

#
# Should an ELF file processed by find-debuginfo.sh having no build ID
# terminate a build?  This is left undefined to disable it and defined to
# enable.
#
%_missing_build_ids_terminate_build    1

#
# Include minimal debug information in build binaries.
# Requires _enable_debug_packages.
#
%_include_minidebuginfo        1

#
# Include a .gdb_index section in the .debug files.
# Requires _enable_debug_packages and gdb-add-index installed.
#
%_include_gdb_index    1

#
# Defines how and if build_id links are generated for ELF files.
# The following settings are supported:
#
# - none
#   No build_id links are generated.
#
# - alldebug
#   build_id links are generated only when the __debug_package global is
#   defined. This will generate build_id links in the -debuginfo package
#   for both the main file as /usr/lib/debug/.build-id/xx/yyy and for
#   the .debug file as /usr/lib/debug/.build-id/xx/yyy.debug.
#   This is the old style build_id links as generated by the original
#   find-debuginfo.sh script.
#
# - separate
#   build_id links are generate for all binary packages. If this is a
#   main package (the __debug_package global isn't set) then the
#   build_id link is generated as /usr/lib/.build-id/xx/yyy. If this is
#   a -debuginfo package (the __debug_package global is set) then the
#   build_id link is generated as /usr/lib/debug/.build-id/xx/yyy.
#
# - compat
#   Same as for "separate" but if the __debug_package global is set then
#   the -debuginfo package will have a compatibility link for the main
#   ELF /usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build-id/xx/yyy
%_build_id_links compat

# Whether build-ids should be made unique between package version/releases
# when generating debuginfo packages. If set to 1 this will pass
# --build-id-seed "%{VERSION}-%{RELEASE}" to find-debuginfo.sh which will
# pass it onto debugedit --build-id-seed to be used to prime the build-id
# note hash.
%_unique_build_ids      1

# Do not recompute build-ids but keep whatever is in the ELF file already.
# Cannot be used together with _unique_build_ids (which forces recomputation).
# Defaults to undefined (unset).
#%_no_recompute_build_ids 1

# Whether .debug files should be made unique between package version,
# release and architecture. If set to 1 this will pass
# --unique-debug-suffix "-%{VERSION}-%{RELEASE}.%{_arch} find-debuginfo.sh
# to create debuginfo files which end in -<ver>-<rel>.<arch>.debug
# Requires _unique_build_ids.
%_unique_debug_names    1

# Whether the /usr/debug/src/<package> directories should be unique between
# package version, release and architecture. If set to 1 this will pass
# --unique-debug-src-base "%{name}-%{VERSION}-%{RELEASE}.%{_arch}" to
# find-debuginfo.sh to name the directory under /usr/debug/src as
# <name>-<ver>-<rel>.<arch>.
%_unique_debug_srcs     1

# Whether rpm should put debug source files into its own subpackage
#%_debugsource_packages 1

# Whether rpm should create extra debuginfo packages for each subpackage
#%_debuginfo_subpackages 1

# Number of debugging information entries (DIEs) above which
# dwz will stop considering file for multifile optimizations
# and enter a low memory mode, in which it will optimize
# in about half the memory needed otherwise.
%_dwz_low_mem_die_limit          10000000
# Number of DIEs above which dwz will stop processing
# a file altogether.
%_dwz_max_die_limit              50000000

%_find_debuginfo_dwz_opts --run-dwz\\\
--dwz-low-mem-die-limit %{_dwz_low_mem_die_limit}\\\
--dwz-max-die-limit %{_dwz_max_die_limit}

If there are settings missing that would be useful, bugs with the default settings or defaults that should be changed please do file a bug report.

Valgrind 3.13.0 for Fedora and CentOS

Valgrind 3.13.0 adds support for larger processes and programs, solidifies and improves support on existing platforms, and provides new heap-use reporting facilities. There are, as ever, many smaller refinements and bug fixes. See the release notes for details.

There are binaries for Fedora 26 (beta) for aarch64, armv7hl, i686, ppc64, ppc64le, x86_64. And Copr builds for Fedora 25 (i386, ppc64le, x86_64), CentOS 6 (i386, x86_64) and CentOS 7 (ppc64le, x86_64). I’ll keep the Copr builds up to date with any updates going into Fedora 26.

valgrind 3.12.0 and Valgrind@Fosdem

Valgrind 3.12.0 was just released with lots of exciting improvements. See the release notes for all the details. It is already packaged for Fedora 25.

Valgrind will also have a developer room at Fosdem on Saturday 4 February 2017 in Brussels, Belgium. Please join us, regardless of whether you are a Valgrind core hacker, Valgrind tool hacker, Valgrind user, Valgrind packager or hacker on a project that integrates, extends or complements Valgrind.

Please see the Call for Participation for more information on how to propose a talk or discussion topic.