The Lunar Cowboy

A powerful feature of Gentoo is the ability to pull in packages from multiple third party repos (aka overlays). Gentoo Build Publisher also takes advantage of this feature, so that a machine definition declares which repos it depends on. This is in fact one of the advantages of using GBP over the official Gentoo binhost. However there may be circumstances where one doesn't want third party repos in their primary machine definition.

Consider this scenario: a machine is defined with the official "gentoo" repo as well as a third party repo. Now perhaps that third party repo does not always adhere to as strict a standard as the "gentoo" repo, so occasionally packages from that repo fail. When a package from the third-party repo fails then so does the Gentoo Build Publisher "machine" that uses that repo, so the build does not get published, so you could be "stuck" on an old published build until the third party repo gets fixed.

This is not a great situation to be in. One way to get around it is to build the third-party repo in a separate GBP machine and have your "physical" machine depend on multiple GBP machines.

Here's an example. Let's say my laptop uses mostly packages from the official Gentoo repo, but there's one package that I want to use that's only available in an overlay. Let's say for example Wike from the GURU overlay. Now normally one would have their GBP machine definition include the GURU overlay and add gnome-extra/wike to it's world file. However here's an alternative:

Make sure you have the guru repo defined in Jenkins:

gbpbox ~ # gbp addrepo guru https://anongit.gentoo.org/git/repo/proj/guru.git
gbpbox ~ # gbp build repos/job/gentoo

Create a new machine definition, guru and have that machine definition include the guru overlay:

 gbp-machines 🢖 make guru.machine base=testing
cp -r testing/. guru/
 gbp-machines 🢖 echo guru >> guru/repos
 gbp-machines 🢖 echo '*/* X icu opengl wayland' >> guru/configs/etc-portage/package.use
 gbp-machines 🢖 echo gnome-extra/wike >> guru/configs/var-lib-portage/world
 gbp-machines 🢖 git add guru
 gbp-machines 🢖 git commit -m "New machine: guru"
 gbp-machines 🢖 git push

Now that a guru machine has been created, it can be added to Jenkins and built:

gbpbox ~ # gbp addmachine guru https://github.com/enku/my-machines.git --deps gentoo guru
gbpbox ~ # gbp build guru

After a build completes successfully, publish the build.

gbpbox ~ # gbp publish guru

Now that we have a new guru machine in GBP and a published build, the next steps are to configure our laptop to use both the original machine definition (in this case lighthouse) as well as the new guru definition.

From the laptop, create /etc/portage/repos.conf/guru.conf with the following contents:

[guru]
masters = gentoo
priority = 10
location = /var/db/repos/guru
sync-type = rsync
sync-uri = rsync://gbpbox/repos/guru/guru
sync-rsync-verify-metamanifest = False
auto-sync = yes

Note: If you instead use a single repos.conf file instead of the directory, simply append the contents to the file.

This tells the system to new new guru ebuild repo on your GBP instance. From now we should be able to sync the repo.

lighthouse ~ # emerge --sync guru
>>> Syncing repository 'guru' into '/var/db/repos/guru'...
...

Now that we our published instance of the guru overlay synced, we can emerge packages from it. However we want to use the published instance of the binary package repo as well. For this we create a binhost definition. Create /etc/portage/binrepos.conf/guru.conf with the following contents:

[gentoo-build-publisher/guru]
sync-uri = https://gbpbox/binpkgs/guru/
priority = 5

Note: If you instead use a single binrepos.conf file instead of the directory, simply append the contents to the file.

Now that we have the repo synced and the second binhost configured, we can install binary packages from the guru machine definition:

lighthouse ~ # emerge --verbose --ask --getbinpkgonly gnome-extra/wike

Local copy of remote index is up-to-date and will be used.

Local copy of remote index is up-to-date and will be used.

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 1.35 s (backtrack: 0/20).

[binary  N     ] gnome-extra/wike-3.0.1-1::guru  PYTHON_SINGLE_TARGET="python3_12 -python3_10 -python3_11" 160 KiB

Total: 1 package (1 new, 1 binary), Size of downloads: 160 KiB

Would you like to merge these packages? [Yes/No]

This is just as if I had added GURU to the lighthouse machine, however the difference is that if ever the guru machine fails to build, I can still update binary packages from the lighthouse machine. And when the GURU repo gets fixed I can build and publish guru and my laptop can get updates without holding back the gentoo repo updates.

So in summary, keeping your gentoo and overlay builds in separate GBP machine definitions is a little more work, however having separate machine definitions for each overlay can ensure that build failures in one overlay don't unnecessarily interfere with another.

Note this article is not to imply that the GURU repo is of a lesser quality than the official Gentoo repo. I only use GURU in these examples because it is a well-known Gentoo overlay.