Multi-Build Updates

We are working on this workflow and will update this page and send an announcement when it is ready or available for testing. This is a work in progress and we would love to have your input on how this should work from a user experience perspective. Please reach out to us on #fedora-ci or get in touch with any member of the team.

You can find some hints about the direction we are taking in:

What Are Multi-build Updates?

Multi-build updates contain multiple builds that are tightly coupled together.

For compiled packages that rely on a certain ABI or soname, it is easy to understand how they are tightly coupled. However, non-binary programs can also have such strong dependencies.

For example, the packages rpms/python-urllib3 and rpms/python-requests are heavily linked. Updates to python-urllib3 need to take python-requests into consideration. Sometime the update is fine, sometime it needs to wait for a new version of python-requests, in which case both builds need to be tested together as one unit.

Packages of this style are candidates for the multi-build update workflow. They need to be built and tested together.

How Does Gating Multi-Builds Updates Work?

To gate updates involving multiple builds, we need to build and test in isolation from the other builds. That is, the CI system can take the builds, test them, gate them or let them pass through, without having to consider any other changes (i.e. packages/builds).

The workflow is as follows:

  • You first need to create a side-tag via fedpkg request-side-tag. This will create a side-tag with a name such as <base-tag>-side-<id>.

  • You can then build all the packages you want in this side-tag using: fedpkg build --target=<side-tag-name>.

  • If you need to chain-build some packages and you want to be sure the previous one is available in the side-tag buildroot, use: koji wait-repo <side-tag-name> --build=<build-nvr>.

  • Once you have built all the packages in your side-tag, you can create the bodhi update for this side-tag using either:

  • the bodhi web UI, in the new update form use the Use Side Tag button

  • the bodhi CLI bodhi updates new --from-tag

  • Bodhi will then create two side-tags:

  • <your-side-tag>-signing-pending

  • <your-side-tag>-testing-pending

  • The builds in your side-tag will be moved into <your-side-tag>-signing-pending and the update will be put in the Pending status.

  • RoboSignatory is notified that your builds landed in this tag by our message bus, it will take them, sign them and move them to the <your-side-tag>-testing-pending tag.

  • Bodhi is notified that your builds were signed and moved to this tag by our message bus, it will mark the builds as signed and once all the builds have been signed, it will move the update to the Testing status.

  • Tests will be run, their results will be reported to ResultsDB which will announce these results, making Greenwave consult each gating.yaml file to see if all the required criteria of these builds have been met or not. If they have, Greenwave will send a message to the message bus announcing its decision.

  • Bodhi will listen to the bus for the decision made by Greenwave about updates. Upon receiving them it will push the corresponding update into the final Koji tag (i.e. the stable tag) and mark the update as Stable.

At this point, your builds have landed in the stable tag in Koji, it is therefore available in the buildroot for anyone to use and rely on and the corresponding Bodhi update has been marked as stable.

Note that the update being stable does not mean you will be able to install it via dnf/yum. It means the build is available in the Rawhide buildroot. It will be pushed to the mirror once the next Rawhide compose succeeds.

If you have created a side-tag and have no use for it (and did not create an update for it), please remove it so it does consume resources on the build infrastructure. You can simply remove side-tags you have created using fedpkg remove-side-tag and you can list your side tags using fedpkg list-side-tags --user=<username>.

Simplified diagram of the multi-builds updates workflow

Simplified multi builds update

Want to help? Learn how to contribute to Fedora Docs ›