The title is a bit of a lie, let's try again.
HTML created by python landslide. Press 'h' for help. For more background info see presentor notes. Press 'p'.
sourceware.org is one of the original free software project hosting services. It hosts dozens of projects, including some of the GNU toolchain components.
It is community operated, Red Hat subsidized infrastructure.
This is about infrastructure that the GNU toolchain projects use on Sourceware, together with many other projects. And it isn't directly about the Infrastructure itself, but about how you want to use it and maybe about which policies projects want to adopt that the infrastructure makes possible (setting project policies isn't the job of sourceware overseers, but we love to help making sure the infrastructure is ready).
Email is awesome, but how can we combine our discussion based patch reviews with patch tracking and test automation?
Provide zero maintenance infrastructure for tracking and automation of patches, testing and analyzing testresults.
And how do we keep improving and innovating as a community run free software infrastructure project?
Sourceware was started in 1998, what about the next 24 years?
At Red Hat we are trying to push CI testing upstream. We have to push fixes upstream if we find them in products. So try to test and find regressions earlier. Younger generation has trouble contributing to projects that primarily use email. For elfutils we got some new contributors who struggle with providing patches. gccrs frontend. project on github, but wants to integrate with gcc. There are two main things that make them productive, automated testing and easy patch tracking. Finally while setting up the new services this year we started discussing a good structure for making sure Sourceware keeps around for another 24 years. Best to have this discussion while there are no known problems or issues.
Discuss how to use the infrastructure, builder, try/ci/full buildbots, bunsen testsuite analyzis, patchwork patch tracking, CICD trybot/buildbot integration, handling patches/email with public-inbox and the sourcehut mirror and other automation to make you more productive and happy to contribute.
Discuss the future of the Sourceware infrastructure as a Software Freedom Conservancy member project (having a fiscal sponsor).
So we believe we are providing the infrastructure services for projects to automate email based patch reviews with patch tracking, but it is up to the projects to see how much they use the services and whether they want to set policies on how to use them.
After discussion with the communty the Software Freedom Conservancy voted to accepted us as member project so we should soon have a fiscal sponsor.
An installation of Buildbot (python), along with a local community, mailinglist, git repo, container files, self-configuring.
Includes various compute resources:
All results, reported or not, go into bunsen. 10.000+ builds a month.
Try builders normally run all tests as the CI builders. Full builders are only needed for those projects that have heavy/slow tests.
After your patches have been accepted you can delete the branch again: - git push origin :users/mark/try-frob
Currently enabled for binutils, elfutils, gdb and libabigail.
My favorite kind of builder!
An installation of bunsen (git://sourceware.org/git/bunsen.git) (python), under rapid development. irc: #bunsen on irc.libera.chat.
Stores all test-related logs + metadata from each testrun into one commit of a ordinary dedicated git repo (git://sourceware.org/git/bunsendb.git).
Small python analysis scripts parse new log files from git into an sqlite database. Analysis passes build on each other. Testruns are clustered by metadata similarities.
Small command line reporting tools browse/search git + sqlite data, show diffs/regressions. More coming.
Includes a simple web frontend (https://builder.sourceware.org/testruns/).
Easy to run your own! https://sourceware.org/git/?p=bunsen.git;a=blob;f=README
Sourceware testrun logs includes numerous & huge dejagnu, automake, autoconf log files, and more. Bunsen stores them in git.
Git compresses amazingly: 1TB of raw text from all 45.000 buildbot runs fits into 10GB. Use your knowledge of ordinary git to replicate, subset, garbage-collect, authenticate.
Let's flip through some screenshots of the web interface.
Each testrun uses its git commit hash as identifier.
Here we see the metadata for this particular testrun.
From the "filelist" and other tabs, one can explore what the system has found out about this testrun: all the files, all the test cases.
The table's right columns let one navigate to related testruns that share given metadata values, or else precede or follow it. Let's compare this testrun to one that came from the previous elfutils source version: "source.gitdescribe", "prev", "delta".
Here we see the differences in test results between the two builds. Nothing too worrysome. Notice the autoconf values are also differenced. We can click on each log file to instantly see the exact line context of each test case.
The regression mode limits differences to only pass->fail type transitions, omitting skipped results and other such less serious results.
Here we see the specific logfile region that accounts for the skipped run-large-elf-file.sh.trs case.
There is much more to see, clicking around. Please excuse the plain looks, it's old school HTML with simple self-contained server tech.
An installation of patchwork (python/django).
git pw is awesome
patchwork plus CI/CD - Let's use those buildbot workers too
I use it as a personal TODO for elfutils, not really a team effort. gdb used it but abandoned glibc does seem to use it as a group with weekly patch reviews some gcc hackers use it? Why does patchwork work for some, but not others?
An installation of public-inbox (perl).
A better mail archive and so much more....
Allows people to "subscribe" to the list through atom, nntp, imap
Easy way to have mailinglist mirrors, including local (git like) mirror
Try out piem (public-inbox emacs mode) or b4 tools.
All based on Message-IDs.
Make email lists like git.
Not a push, but a pull resource.
.git/config [b4] midmask = https://inbox.sourceware.org/dwz/%s linkmask = https://inbox.sourceware.org.org/dwz/%s
$ b4 am firstname.lastname@example.org Looking up https://inbox.sourceware.org/dwz/a2bf576d-3598-385d-2139-cae0d4f11074%40suse.cz Grabbing thread from inbox.sourceware.org/dwz/a2bf576d-3598-385d-2139-cae0d4f11074%40suse.cz/t.mbox.gz Analyzing 1 messages in the thread Checking attestation on all messages, may take a moment... --- ✓ [PATCH] Use grep -E instead of egrep. --- ✓ Signed: DKIM/suse.cz --- Total patches: 1 --- Link: https://email@example.com Base: applies clean to current tree git am ./20220907_mliska_use_grep_e_instead_of_egrep.mbx
$ b4 ty --auto Auto-thankanating commits in master Found 2 of your commits since 1.week Calculating patch hashes, may take a moment... Located: [PATCH] Use grep -E instead of egrep. Located: [PATCH] Fix executable stack warning from linker --- Generating 2 thank-you letters Writing: ./mliska_suse_cz_patch_use_grep_e_instead_of_egrep_.thanks Writing: ./mliska_suse_cz_patch_fix_executable_stack_warning_from_linker.thanks --- You can now run: git send-email ./*.thanks
Example showed DKIM verification, but this can also be done for gpg signed email messages or special patchatt "signed" patch emails.
A more webby git workflow alternative
git send-email without the email
Not many people seem to use it, it might not be known well enough. Was tried to submit gccrs patches, but it wasn't a success?
If it doesn't work out we can still keep the git mirrors.
First off: things are fine and stable.
The next 24 years of Sourceware?
Why? In case of future financial or organizational needs.
Reached out to Software Freedom Conservancy a few months ago. Wrote up an application last month, reached out to "all" sourceware users. SFC offered project membership to Sourceware last week.
What is a fiscal sponsor?
Does anything change? Hardly.
Independent from any guest projects!
In particular the GNU toolchain projects have the FSF as fiscal sponsor, nothing changes about that.
Continue to have public discussions on overseers mailing list and public video chats with Conservancy.
There currently isn't really a need, so now is a good time to set something up. This is just about sourceware providing free software project hosting for the next few decades. Projects still decide how or if they want to use the services. We don't want to decide any policy for any guest project. Talked to FSF to make sure this isn't in conflict with the FSF providing fiscal sponsorship to the GNU Toolchain projects or a way to undermine GNU projects leadership. We believe this is a non-event. But we want to be as transparent as possible. If this isn't a non-event we are doing it wrong. We want to know if any of this is controversial. All discussions are open and public on the overseers mailinglist. Video chats with Conservancy have been open to all on BBB.