Sourceware

sourceware.org is one of the original free software project hosting services. It hosts dozens of projects, including some GNU projects.

It is community operated, (mostly) Red Hat subsidized infrastructure.

Historically sourceware.cygnus.com (1998), sources.redhat.com (2000), now sourceware.org (since 2001).

Also known as cygwin.com, gcc.gnu.org, valgrind.org, elfutils.org.

Host to ~25 projects, annobin, binutils, builder, bunsen, bzip2, cgen, cygwin, debugedit, dwz, elfutils, insight, gcc, gccrs, gdb, glibc, gnu-gabi, kawa, libabigail, libffi, lvm2, newlib, poke, sid, systemtap, valgrind, ...

Plus another ~40 dormant or moved projects.

Worry-free, friendly, developer controlled, home for Free Software projects.

• mailman based mailing lists
• plus public-inbox (*)
• git hosting
• project websites, also git based
• bug tracking using bugzilla
• patch tracking via patchwork (*)
• integration testing using buildbot (*)
• bunsen test results database (*)
• others, just ask!

Trying to provide zero maintenance infrastructure. Prefer packaged software for easy automated (security) updates. Otherwise virtual environments.

Various projects have admin groups to setup their own services.

(*) New or improved - this talk!

Email is awesome, but how can we combine our discussion based patch reviews with patch tracking and test automation?

Provide infrastructure for tracking and automation of patches, testing and analyzing testresults.

And how do we keep improving and innovating as a community run free software infrastructure project?

Sourceware was started in 1998, what about the next 24 years?

An installation of Buildbot (python), along with a local community, mailinglist, git repo, container files, self-configuring.

Includes various compute resources:

• virtual machines, bare metal, containers
• distros (fedora, debian, centos, ubuntu, opensuse)
• architectures (i386, x86_64, ppc64le, s390x, ppc64, armhf, arm64)
• Thanks: Brno University, Marist University, Thomas Fitzsimmons, Mark Wielaard, Frank Eigler, IBM, The Works on Arm initiative and OSUOSL.

• Try builders - "pre-commit" - reports to author only. (binutils, elfutils, gdb and libabigail)
• CI builders - "quick regression tests", reports to author and list on regression. (bzip2, dwz, debugedit, libabigail, elfutils, glibc, gcc, gccrs, valgrind, gdb, binutils)
• Full builders - "all (slow) tests, not all commits, not all arches" (gcc x86_64/arm64/armhf, glibc x86_64/arm64, gdb-binutils x86_64, gccrust-bootstrap x86_64/arm64)

All results, reported or not, go into bunsen. 15.000+ builds a month.

• git checkout -b frob
• hack, hack, hack... OK, looks good to submit
• git commit -a -m "Awesome hack"
• git push origin frob:users/mark/try-frob
• ... wait for the emails to come in or watch buildbot logs ...
• Send in patches and mention what the try bot reported

After your patches have been accepted you can delete the branch again:

• git push origin :users/mark/try-frob

Currently enabled for binutils, elfutils, gdb and libabigail.

An installation of bunsen (git://sourceware.org/git/bunsen.git) (python), under rapid development. irc: #bunsen on irc.libera.chat.

Stores all test-related logs + metadata from each testrun into one commit of a ordinary dedicated git repo (git://sourceware.org/git/bunsendb.git).

Small python analysis scripts parse new log files from git into an sqlite database. Analysis passes build on each other. Testruns are clustered by metadata similarities.

Small command line reporting tools browse/search git + sqlite data, show diffs/regressions. More coming.

Includes a simple web frontend (https://builder.sourceware.org/testruns/).

Easy to run your own! https://sourceware.org/git/?p=bunsen.git;a=blob;f=README

An installation of patchwork (python/django).

git pw is awesome

patchwork plus CI/CD - Let's use those buildbot workers too

An installation of public-inbox (perl).

A better mail archive and so much more....

Allows people to "subscribe" to the list through atom, nntp, imap

Easy way to have mailinglist mirrors, including local (git like) mirror

Try out piem (public-inbox emacs mode) or b4 tools.

.git/config

[b4]
    midmask = https://inbox.sourceware.org/dwz/%s
    linkmask = https://inbox.sourceware.org.org/dwz/%s

 $ b4 am a2bf576d-3598-385d-2139-cae0d4f11074@suse.cz
 Looking up https://inbox.sourceware.org/dwz/a2bf576d-3598-385d-2139-cae0d4f11074%40suse.cz
 Grabbing thread from inbox.sourceware.org/dwz/a2bf576d-3598-385d-2139-cae0d4f11074%40suse.cz/t.mbox.gz
 Analyzing 1 messages in the thread
 Checking attestation on all messages, may take a moment...
 ---
   ✓ [PATCH] Use grep -E instead of egrep.
   ---
   ✓ Signed: DKIM/suse.cz
 ---
Total patches: 1
---
 Link: https://inbox.sourceware.org.org/dwz/a2bf576d-3598-385d-2139-cae0d4f11074@suse.cz
 Base: applies clean to current tree
       git am ./20220907_mliska_use_grep_e_instead_of_egrep.mbx

$ b4 ty --auto
Auto-thankanating commits in master
Found 2 of your commits since 1.week
Calculating patch hashes, may take a moment...
  Located: [PATCH] Use grep -E instead of egrep.
  Located: [PATCH] Fix executable stack warning from linker
---
Generating 2 thank-you letters
  Writing: ./mliska_suse_cz_patch_use_grep_e_instead_of_egrep_.thanks
  Writing: ./mliska_suse_cz_patch_fix_executable_stack_warning_from_linker.thanks
---
You can now run:
  git send-email ./*.thanks

Either:

• Awesome way to "close" the secure software supply chain
• Security "theater" that will exclude people and reduce code reviews to "has the submitter jumped through code signing hoops"

https://sr.ht/~sourceware/

A more webby git workflow alternative

git send-email without the email

First off: things are fine and stable.

The next 24 years of Sourceware?

Why? In case of future financial or organizational needs.

Reached out to Software Freedom Conservancy a few months ago. Wrote up an application a couple of months ago, reached out to "all" sourceware users. SFC offered project membership to Sourceware.

What is a fiscal sponsor?

Independent from any guest projects!

In particular the GNU toolchain projects have the FSF as fiscal sponsor, nothing changes about that.

Continue to have public discussions on overseers mailing list and public video chats with Conservancy.

The various GNU projects hosted on sourceware also get support for the FSF tech-team hosting some websites, releases, etc.

They have a paid staff and can also provide backups, mirrors and have offered technical assistance to sourceware.

Run lists.gnu.org from which we are learning some tricks.

bugzilla sourceware infrastructure

Offsite backup/system, BBB instance, Software Heritage, gitolite, cgit, signed release upload, dkim preserving lists...

Do this every 4 months?