Arch-linux building from source – and Obarun to the rescue

What if, there was a benefit in building from source, a system that is commonly used by pre-fabricated binary packages, like Arch or any of its forks and desktop flavors?  What Arch considers a “clean-chroot” is primarily of need to developers ensuring their package can be both satisfied for all dependencies AND are reproducible, as long as this can be achieved within a constantly rolling distribution.  That is open and nearly free condition for you.

Why

Building in a clean chroot prevents missing dependencies in packages, whether due to unwanted linking or packages missing in the depends array in the PKGBUILD. It also allows users to build a package for the stable repositories (core, extra, community) while having packages from [testing] installed.

Scratch most of this for several reasons.  We are not developers, we are building our own system like Gentoo-ers, k1ss-ers, Crux-ers, and others do.  We want to make sure that each of our packages fits well within the parameters of our specific machine, and it wasn’t built on another machine that may not be 100% compatible to ours. One of the aspects of Arch’s clean chroot that shouldn’t be neglected is the tmpfs space a chroot is given.  That 20G is borderline enough to build some packages … that may be compiling for half a day and end with an error message “no space left on device“.  That is a very sad waste of electricity and processing mileage.

Pacman, makepkg in specific, takes a PKGBUILD file published by your distro and builds the package from source.  pacman itself is one of the few that are sourced from Arch itself, the vast majority come from the same “upstream” source they come in most other distros.  That is what separates those few distros that claim to be “independent”, while other distros just utilize packaging already done by those independent distros.

So what we primarily need is a tmpfs large enough for the task, or a large enough partition, we need pacman, and the most elemental base to start.  What are the fewest elements we need to start.  So we start with this task, to install pacman into “nothing”, a blank space (we will call this target space /mnt).  We utilize pacman to do so, (# pacman -Syy pacman -r /mnt).  We get an error, /var/lib/pacman does not exist.  So we make this directory structure that doesn’t exist, and we proceed one obstacle at a time.  (# mkdir -p /mnt/var/lib/pacman) [1] so now we have this directory, let’s see if pacman is installable now.  Yes, it is, it synchronizes with all the repositories you have in the host, stores its databases for each repository into that newly created directory structure and is waiting for the OK to install pacman and ALL of its dependencies.

Wait, what does your pacman.conf look like?  Do you have testing and strange other repositories enabled, packages on ignore, etc.?  DO YOU NEED all those listed dependencies of pacman?  Yes, otherwise pacman and makepkg which is of interest here, will not work and will run into problems.  Here is an important decision to make, if your pacman.conf is not as it came from its source distro, you may need to consider running one specifically for this purpose.  How?  (# rm -rf /mnt/var/lib/pacman/sync/*  and then  # pacman -Syydd pacman -r /mnt) if you want to employ a separate pacman structure in the chroot.  This will not be a functional pacman, since it has no dependencies satisfied, but will create a default /mnt/etc/pacman.conf (and mirrorlist) that you can use for this early part where you can not actually chroot into the file-system /mnt.  Then edit your new /mnt/etc/pacman.conf as needed and do the same at the mirrorlist as needed for the not so immediate future.  Now, one more time, resync and install pacman, the actual thing and its dependencies ( # pacman -Sy pacman -r /mnt –config /mnt/etc/pacman.conf )  this forces your host pacman to use the target’s pacman.conf to install and configure packages, instead of the host’s.

        And here is where this tedious task of making a chroot installation gets interesting:

Can you chroot (arch-chroot is best used) to /mnt yet?  No, there is no shell to chroot to, and the filesystem in /mnt is incomplete, it is only the directories that pacman and its dependencies occupy.  So you need a shell, among other things.  Where to find what you need?  The base metapackage, or specifically core/base.  Let’s see what analytically this includes and every Arch dev will tell you “you need this complete to have true arch, as if Arch was a true scotsman”.  BS!!

# pacman -Si core/base

Evangelos Foutras says you need this: filesystem gcc-libs glibc bash coreutils file findutils gawk grep procps-ng sed tar gettext pciutils psmisc shadow util-linux bzip2 gzip xz licenses  pacman systemd systemd-sysvcompat iputils iproute2

Pacman you already have, an init system should not be needed since the system is not intended to boot, and network/internet access is provided by the host, so you don’t need those either.  Instead of install “base” we will install only what is needed to have a building-machine.  Are you going to work with bash while chrooting to it or zsh or another shell?  Bash is going to be needed by 99% of the packages that incorporate scripts for their installation, so you can’t live without bash and it becomes complicated to substitute.  So add your shell favorite to the list:

# pacman -S filesystem gcc-libs glibc bash coreutils file findutils gawk grep procps-ng sed tar gettext pciutils psmisc shadow util-linux bzip2 gzip xz licenses pacman -r /mnt –config /mnt/etc/pacman.conf

BEEP!!!!    What the F@%!&*)$_#*@)_ck is going on.  The init and its libraries are coming in through dependencies of the rest.  WOW!!!  Didn’t see that one coming DID YA:?

Why the hell do I need an init, service supervision, service management, on a system that will not run any services, will not need an init?  While you develop those critical tendencies, we have a better proposal.  Let’s see if Arch is guilty if other arch based distros follow this leap in logic.  Obarun has a similar base metapackage (that is an empty package with many dependencies, like a group but not a group, a package).  We take a look and it is: filesystem gcc-libs glibc bash coreutils file findutils gawk grep procps-ng sed tar gettext pciutils psmisc shadow util-linux bzip2 gzip xz licenses pacman  iputils iproute2 dhcpcd obsysusers skalibs execline oblibs 66 66-tools s6-linux-utils s6-portable-utils s6 s6-rc boot@-66serv dhcpcd-66serv  eudev libeudev

that is the same thing + s6-suite group (init, service supervision, service manager, libraries, execline, and udev replacement + internet/network utilities ) which we don’t need.

# pacman -S filesystem gcc-libs glibc bash coreutils file findutils gawk grep procps-ng sed tar gettext pciutils psmisc shadow util-linux bzip2 gzip xz licenses pacman -r /mnt –config /mnt/etc/pacman.conf

is the same exact command but based on Obarun repositories [2] above Arch.  Only a few of those pkgs installed come from obarun, procps-ng, util-linux, pacman, util-linux-libs, filesystem, and their dependencies, eudev/libeudev as well, which you can argue that you don’t really need for a chroot only system.  No dbus, no systemd, but nothing relating to init and service running as well.  As should be!  Maybe this is closer to what Arch used to be before the IBM trojan invaded it.  Obarun and Archlinux keyrings will come in as pacman dependencies and are necessary to proceed with installation [2].

In the first phase of this experiment all you need is a system you can chroot and operate in.  Now there are two ways to follow towards this goal if building with the most ninimal of installed software and libraries, and in fact there is very little worthwhile difference.  There is the package-group (not a metapackage) in Arch called base-devel, which is the recommended minimum you need to build packages.  But are they all necessary for building one pkg?  Not really.  The way you build in Arch so you can have a binary package that can be effectively handled by pacman, uncompressed, exploded as an archive with instructions to place its parts in the right places and able to read hooks and scripts for configuration, you get a PKGBUILD file for each package.  You should read both of these two documents from the arch-wiki

https://wiki.archlinux.org/index.php/PKGBUILD
https://wiki.archlinux.org/index.php/Arch_Build_System

There is another way to skin this cat:  You can use wget and edit the command for every package name.  For example the command to get the PKGBUILD for base from arch is:

# wget https://raw.githubusercontent.com/archlinux/svntogit-packages/packages/base/trunk/PKGBUILD

You can change …kages/base/tru… with the name of any pkg from arch and you will get the PKGBUILD for it.  There are a couple more ways that arch suggests, but all you are doing is copying a text file from Arch’s source repository.  You might say wget is not a base pkg.  You are not in chroot yet, haven’t changed your root to /mnt that is, so you can use wget from the host.  We will get back to this a little later.

# wget https://framagit.org/pkg/obcore/base/-/raw/master/trunk/PKGBUILD

this is a sample address for the Obarun base pkg.  With Obarun you have to know in which repository the pkg belongs, obcore, obextra, obcommunity, observices, obmultilib. [3]  Otherwise the procedure is the same.  Pacman is nearly identical, what changes is pacman.conf and makepkg.conf.  Makepkg in Obarun differs in respect to the compression software, it is using xz as default instead of facebook’s zstd quick and dirty compression.  We prefer pretty damn quick [4]

But will all core base packages built without the systemd-libs at least, if not the whole thing?  Try it.  Obarun rebuilds ONLY the packages having direct dependencies to systemd and libs, not all.  But some of those that come from arch have “built dependencies to systemd-libs”.  I suspect on the next phase those are going to be addressed.  If they couldn’t built without them then where does void and artix, among others, get their building done of the very same packages?  I can not answer all those questions.  There is a limit to how deep this research goes.  (we may try artix, mere-linux, void, in the near future)

Back to procedure

What I did on my test.

With pacman you can make a list of the intentionally installed sw.  The list above with the most elemental of base with as much as possible omitted.  We all know that is not enough to build anything, but a very minimal system for chroot without booting necessities is the list above.  This is a functional linux system operating inside any linux host.  You can have this obarun/arch base and chroot to it from Fedora, Centos, mUbuntu, … etc.  maybe even solaris, or OpenBSD…. I don’t know why not even MacOS or Android.  [3]

1  You have this minimal system of about 430MB on disk.  I make a root directory called /src in this system and give it user access and write abilities.

2  By default makepkg can only run by a user, not root.  So make a user, I called her make.

3  You want to work, I assume as I did, on the very core base packages, and give priority to them not their dependencies, to have as your own built replacements to those built elsewhere.  To do so run pacman -Qe  this gives you the list of intentionally installed packages.  Since this list is not very useful with scripts, since it has versions and descriptions and so on, you run the same with the q added to get just pkg names.  I call this list pkgQeq and store it in /src   ” pacman -Qeq >/src/pkgQeq ” and also keep a record of the total packages installed, that means with their dependencies that make this initial 430MB base.  Do pacman -Qsq >/src/pkgQsq as a list to work on next.  On the next steps you will see that you may want to run those commands again after you install base-devel, which you can’t escape.

4  You want to make a pkg named pkgX you make a subdirectory to /src/pkgX and download with which ever way the PKGBUILD from its source (listed above in the two wget addresses).  After you do so, you run as “make” makepkg.  The first run will tell you what you are missing as build dependencies.  Build dependencies are not for running the software but for building it.  Only when you install the final package do dependencies required will have to accompany it.  And buildependencies are no longer needed after the package has been built.  So you remove the same you just installed for building to keep the system clean.  pacman -S x y z  –> pacman -Rnsu x y z.  After you install the build dependencies “missing” from your core system, there may still be build tools needed it is assumed you have but you don’t.  The basic needed tools to build many (but not all) packages are included in a package group called base-devel.  That is another 430+ MB, for a total of 850MB +/-.   Except for pacman and util-linux which are already installed, the rest all come directly from Arch.  If not doing this you will run into building errors telling you that you tool x was not found, like strip, or fakeroot, etc.  You can torture yourself all you want and install them and uninstall them for each pkg, it is up to you and your level of masochism.  Clean-chroot means that you build each package with the core base each time, it is just that we are making it even cleaner (I think) and on a real hardware based system and not a simulated bubble, like a virtual machine or a docker container.  Real software for real hardware.

5  You may run into GPG issues, where the signature key of the packager is missing in your system.  It will give you the code for the missing signature.  As user (make) gpg –recv-keys 0000CODE000XXX  and you will get the key.  The name of the packager as her/his public key is provided and email, and should match the one on the PKGBUILD, otherwise remove it and research what is going on.

6  When you add the right key, dependencies are met, you may have missing files, that are normally stored besides the PKGBUILD, usually patches for upstream software that can’t be built otherwise or need some special installation scripts along them to be installed correctly.  Each time you will get one notice for one missing file and you either run wget with PKGBUILD replaced for the name of the file missing or look on the PKGBUILD for the list of such files, and some times it can be quite a few, and do them all at once.

7  You finally produce the desired built pkgX, it will look like /src/pkgX/pkgX-0.1.0.1-1.x86-64.pkg.tar.xz  and you use pacman -U /src/pkgX/……xz and you install/replace what you have.  Then repeat the procedure for next pkg on your pkgQeq list.  It is adviseable to remove the workarea of the pkg already built.  Next moment Arch or Obarun, may replace the pkg with next version, and sometimes this is done because it changed upstream, and this PKGBUILD is no longer valid, so useless past the moment you made it.  For future reference you may want to rename this PKGBUILD.0 to compare with a newer one and see what and if something changed.  The rest goes to trash, only /src/pkgX/PKGBUILD.0 remains.  If it was complicated making the pkg you may want to add a note or even a script that will simplify your next run, since you will not remember the obstacles you had to overcome to build it.

9  You can transfer the pkg to cache (/var/cache/pacman/pkg) but you may mix them up with other packages.  Arch packages have been ending at .zst obarun with xz.  you can add a subdirectory for your own built pkgs /var/cache/pacman/src and store them there, move them after built and installed.  Your own repository for making future installations using –cachedir /var/cache/pacman/src in pacman.

10  Say you did all this, and completed the cycle of all pkgs and dependencies for this minimal core system.  (Remember some pkgs will already need 20GB space to build, either in real disk or a tmpfs, which you can mount on /src if you will not keep anything beyond building and installing.

11  So you completed this project, use the packages to make in installation, add a kernel, a bootloader, s6-suite, you use 66 to make trees and services as per wiki, configure your boot module, and reboot.  A very light and fast system.  Does it differ from the one you made with the same base that was built elsewhere?  I haven’t realized a difference, but there can be one that I may never discover.  None of this software will run into an incompatibility….. maybe?  Does building software in a system means that it will run right?  Not necesseraly, this is how bugs are discovered and addressed.  Very few bugs relate to building environments, most are running bugs that would occur anywhere.

11.1  Then there is this other issue.  You used prefabricated/pre-compiled software to compile your own.  Checksums will never work unless you follow arch.wiki step by step, or use Obarun’s docker image and build each package in this docker without altering it.  But you trust yourself and don’t care about checsums because you know why they are different.  Even base, a metapackage will not match the official one, if you open it you will see it has a list of packages installed while building.  If you have emilia-pinball installed in the system it will be listed, we will know you play.  So will there be a difference to repeat the ENTIRE PROCEDURE again, but now the software you are using to build are the software you have already built.  Making a clean system cleaner and all of your own, like it is Linux From Scratch?  You now have a system built from source.  If you think Gentoo provides you with something better, you tell us.

Conclusion

My answer is, since you can’t audit every single line of code in all those packages you are building, you indirectly trust that packager/maintainer that did audit it, took a chance packaging it from source, and have faith that the code is not rogue, it is good and safe, open and free.  Even if it is made by a mutinational corporation with the sole goal for existence being that of  accumulating wealth for its shareholders, making all kinds of concessions to governments and armies to protect and defend their wealth.  Can you escape them?

So open and free software is a very relative term, just like using and open and free designed weapon to protect the commodities you stole from your neighbors.  I don’t think there is a patent for an AK47 anymore, maybe there never was.  It is the choice of the poor defending themselves from the rich.   They call them 3d-printers for soft plastic and balsa wood now, in the 70s and beyond they were called CNC lathes and milling machines working on hardened steel alloys, chrome, etc.  Nothing new for people with blue overalls and missing fingers.  They used cards and paper tapes to read punched data code back then, they can use a PDA now for a cad design fed to the monster machine.  Code to translate a blue-print was written in an electric typewriter type keyboard that punched little holes.   It is all good fun though, playing stupid in a world that is turning to a society of zombies.  3d-printers, discovered by hipsters to make them feel smart … HA!!  They make pen holders out of recycled polypropylene.

Nothing is really Open, nothing is really Free, as long as ………..   you know where that goes, you hear it often here!

 

Notes:

[1]  This is one of the first things pacstrap does when doing an installation to a chroot, create this /var/lib/pacman directory structure when the first update is run, the repository.db files are stored, and is up against those that pacman knows where packages are listed on the mirror.

[2]  wget https://framagit.org/pkg/obcore/pacman/-/raw/master/trunk/pacman.conf gets you a copy of obarun’s pacman.conf (no mirrorlist needed, the single server is listed in pacman.conf and arch’s mirrorlist remains as is for arch).  For this particular purpose and since this is not a bootable system, you will need to also get obarun-keyring installed and to do so you need to temporarily switch SigLevel = Required to SigLevel = Never, then update, install, run # pacman-key –populate obarun archlinux then switch back to Required.  Obarun is a bit more strict about verification of sources and packages, you can not even have a valid repository database without a valid keyring installed.

[3]  To make updates of repositories faster and more effective, since s6/66/and services are not going to be used, you can comment out [observices] which has the service scripts for booting, tty, services, daemons, modules, etc.  With ob/multilibs you are on your own, depending on what you want.

[4]  Poor XZ this trusty old friend, gets files and archives as compressed as we can safely get them.  Although it has the ability recently (some years) to utilize multithreading, it is by default sentenced to run on a single core, in the name of reproducability, as the checksums change if you use many threads/cores.  There is a configuration in xz -T$  where the number that goes next to -T is the number of cores you want used.  If you have 4 threads, you may want to use -T3 to keep one unused if you are compressing HUGE bunches of data.  For packaging work it all takes fractions of a second, or 2-3″.  You can hardly see a change in performance while building unless you are playing a high action game like racing cars, where it only takes a fraction to end up against a wall.  For using all of them I believe the option is -T0, see man xz for details, and add this to your makepkg.conf to speed things up.  If you can measure speed difference with .zstd … tell us about it, we will like to know how you measured it.  It may take a few minutes to hours to build a package, and 1.24″ to compress it.  With xz the size for sure is constantly 5-15% smaller than zstd.

Surprise, xz is not compressed as an xz archive, it is compressed by gz, since you have no xz yet.  So who built gz and how was it compressed???   😉  You can uncompress it after you install xz, and recompress it with its own self to be consistent.  The hell with Arch’s checksums, local packages can be installed without checking them or signatures.  Everything is packages by make now.