Loading...
 

shimpsblog [en]

English blog at blog.shimps.org

The Universe as Illusion? I/II

fmg Thursday April 17, 2025

Is our perception of the universe an illusion? Modern physics pushes the boundary between empirical science and philosophy in a challenging way. Is the net content (mass and energy) of the universe zero? Can it be created from nothing?

Lawrence Krauss - Illusion 1: A Universe from Nothing - Scientifically Informed Philosophy S1E1

Kinship Cafe
Jun 14, 2024

Compiling GnuPG XVI: Pinentry

fmg Tuesday April 15, 2025

Situation

Generating a public/private key pair requires a passphrase - at least it is strongly recommended not to use empty passphrases. When a key is generated with gpg --gen-key or gpg --full-gen-key the user is asked to enter the passphrase and confirm it to avoid typos. This part is handled by the program pinentry which is invoked by gpg-agent, which itself is a supporting daemon for gpg.

The Problem

When the self compiled version is installed in a non-default folder without self compiled pinentry, this process fails. The gpg binary doesn't evaluate the PATH and the lookup for pinentry is limited to the folder where it is invoked from.

Solution Methods

Self Compiled

We can compile pinentry with the same configuration prefix as gnupg.

Quick and Dirty

We can set a symlink to the distribution's version of pinentry.

Copy to clipboard
cd /tmp/canary/install/bin ln -s /usr/bin/pinentry pinentry

Configuration

We can define the configuration option pinentry-program /usr/bin/pinentry in the file gpg-agent.conf. But when the gpg-agent is running already, it doesn't know yet about the new configuration. It is possible to kill the gpg-agent, but in these cases you will lose all the settings handled by it, e.g. unlocked keys - this is quick and dirty. There is a better way.

Copy to clipboard
$ cat /tmp/canary/homedir/gpg-agent.conf pinentry-program /usr/bin/pinentry $ LD_LIBRARY_PATH="/tmp/canary/install/lib" $ cd /tmp/canary/install/bin $ ./gpg-connect-agent --homedir /tmp/canary/homedir reloadagent /bye $ ./gpg --homedir /tmp/canary/homedir --gen-key


The program gpg-agent knows about the command line option --pinentry-program ${whatsoever}, but gpg doesn't. Thus you cannot pass it by invoking the latter with this option.

Conclusion

The configuration solution method is the best choice, since it can be done on user level and it can be combined with the self compiled solution method. A user can switch between distribution's and self compiled pinentry at any time. And it doesn't matter whether pinentry is part of the self compiled suite when configured to use the distribution version.

To be continued...

GnuPG SHIMPS Signing Keys

fmg Friday April 11, 2025

Situation

Last December we started to provide GnuPG as Debian style packages with an additional external repository on software.shimps.net. The trust anchor is the repository signing key which must be deployed with

Copy to clipboard
cd /etc/apt/keyrings wget https://software.shimps.net/keys/gnupg/shimps-signing-2024.gpg


I published the fingerprint in a signed email on the gnupg-users mailing list. This way the trust anchor is shifted to the key I use to sign emails on that list. This key is available via WKD (as well as the signing keys are).

Another Key

The recommended package shimps-keyring is intended to simplify future key updates. Meanwhile there is a new signing key prepared, which is not used for the repository (or even in the keyring package) yet, but ATM intended to sign Git commits, e.g. to Codeberg diy-gnupg. It needs to be imported into the user's keyring if you want to verify the commits. Quick and dirty import:

Copy to clipboard
cd /etc/apt/keyrings wget https://software.shimps.net/keys/gnupg/shimps-signing-2025.gpg gpg --import /etc/apt/keyrings/shimps-signing-2025.gpg


Since this key will be shipped with the keyring package eventually, there is no need to deploy it below /etc/apt and you might want to import it via WKD:

Copy to clipboard
$ gpg -v --auto-key-locate clear,wkd,nodefault --locate-keys signing.2025@shimps.net gpg: using pgp trust model gpg: pub ed25519/52CAACDAA420187C 2025-04-09 Frank Guthausen (signing 2025) <signing.2025@shimps.net> gpg: key 52CAACDAA420187C: "Frank Guthausen (signing 2025) <signing.2025@shimps.net>" not changed gpg: Total number processed: 1 gpg: unchanged: 1 gpg: auto-key-locate found fingerprint E614303270D94B76EF285FFA52CAACDAA420187C gpg: automatically retrieved 'signing.2025@shimps.net' via WKD pub ed25519 2025-04-09 [SC] [expires: 2028-04-08] E614303270D94B76EF285FFA52CAACDAA420187C uid [ultimate] Frank Guthausen (signing 2025) <signing.2025@shimps.net> sub cv25519 2025-04-09 [E] [expires: 2028-04-08]

Fingerprints

Keep the trust anchor in mind. If you get the key from the website (download or WKD) you (and I) need to trust the provider to host and ship the correct key and you need to trust https encryption with an Let's Encrypt certificate.

December 2024

The signing key from December 2024 is RSA 3072.

Copy to clipboard
$ gpg -k signing.2024@shimps.net pub rsa3072 2024-12-17 [SC] [expires: 2026-12-17] 2BBD6FF068FEF7905A531348ECB53FD18ACCCF7C uid [ultimate] SHIMPS Signing Key 2024 <signing.2024@shimps.net> sub rsa3072 2024-12-17 [E] [expires: 2026-12-17]

or

Copy to clipboard
$ gpg --fingerprint --fingerprint --with-sig-list signing.2024@shimps.net pub rsa3072 2024-12-17 [SC] [expires: 2026-12-17] 2BBD 6FF0 68FE F790 5A53 1348 ECB5 3FD1 8ACC CF7C uid [ultimate] SHIMPS Signing Key 2024 <signing.2024@shimps.net> sig 3 ECB53FD18ACCCF7C 2024-12-17 SHIMPS Signing Key 2024 <signing.2024@shimps.net> sub rsa3072 2024-12-17 [E] [expires: 2026-12-17] B3E8 C9E5 48CC CDC4 1C70 0650 4D19 8A44 C9AE 8FA6 sig ECB53FD18ACCCF7C 2024-12-17 SHIMPS Signing Key 2024 <signing.2024@shimps.net>
April 2025

The signing key from April 2025 is an ECC key with curve ed25519. It is signed with the 2024 signing key to establish a chain of trust.

Copy to clipboard
$ gpg -k signing.2025@shimps.net pub ed25519 2025-04-09 [SC] [expires: 2028-04-08] E614303270D94B76EF285FFA52CAACDAA420187C uid [ultimate] Frank Guthausen (signing 2025) <signing.2025@shimps.net> sub cv25519 2025-04-09 [E] [expires: 2028-04-08]

or

Copy to clipboard
$ gpg --fingerprint --fingerprint --with-sig-list signing.2025@shimps.net pub ed25519 2025-04-09 [SC] [expires: 2028-04-08] E614 3032 70D9 4B76 EF28 5FFA 52CA ACDA A420 187C uid [ultimate] Frank Guthausen (signing 2025) <signing.2025@shimps.net> sig 3 52CAACDAA420187C 2025-04-09 Frank Guthausen (signing 2025) <signing.2025@shimps.net> sig ECB53FD18ACCCF7C 2025-04-09 SHIMPS Signing Key 2024 <signing.2024@shimps.net> sub cv25519 2025-04-09 [E] [expires: 2028-04-08] 45F2 E58A 454A 8FFA DE16 11EE A3F2 65E6 7497 B0AB sig 52CAACDAA420187C 2025-04-09 Frank Guthausen (signing 2025) <signing.2025@shimps.net>

Why a Gazillion Keys?

In principle it would be possible to work with one main key and several subkeys for different email addresses and purposes. At the end a choice has to be made. Dealing with only one key might be more convenient and less work here and there, but you cannot seperate roles on a fine grained level. Thoughts about data protection might occur, or it can become cumbersome to deal with subkeys. And finally it is a question of workflow, taste and/or personal preferences. There is no one size fits all solution.

First Anniversary

fmg Wednesday April 9, 2025

One year ago this blog started with Underpayment in Academia. There were eight blog posts published each week - distributed to three different blogs.

It's time to celebrate. Wednesday evening I attend the LUGA online Jitsi meeting (language is German).

There are still a lot of interesting scientific as well as computer topics. It is impossible to cover them all, thus I have to select how to spend the time. The major topic of the last month was compiling GnuPG on Debian. There were some pitfalls, but I guess the deed is done and the problems are understood. Building Debian packages will be a subsequent topic. ATM the Debian style SHIMPS packages of GnuPG 2.5 are still experimental.

Maybe the frequency of postings will decrease because there are a few other important things on the list - Que Sera, Sera (Whatever Will Be, Will Be).

Enjoy!

Compiling GnuPG XV: Environment II

fmg Saturday April 5, 2025

Situation

Last time we prepared the chroot-environment with debootstrap. This time we continue inside the chroot environment.

Copy to clipboard
chroot /mnt


Inside the chroot we create an user shimps and install aptitude for the other packages' installation process. YMMV.

Copy to clipboard
user="shimps" addgroup --gid 1000 ${user} adduser --gid 1000 --uid 1000 --disabled-password --gecos ${user},13,23,42,666 ${user} apt-get update apt-get install aptitude

Packages I: General

Next we do some basic preparation of the chroot, independent from the GnuPG build process later.

Copy to clipboard
aptitude install perl aptitude install libfile-fcntllock-perl aptitude install bzip2 aptitude install xz-utils aptitude install libdpkg-perl aptitude install bash-completion aptitude install git aptitude install libldap-common aptitude install libsasl2-modules aptitude install publicsuffix aptitude install openssl aptitude install ca-certificates aptitude install openssh-client aptitude install xauth aptitude install patch aptitude install etckeeper aptitude install vim update-alternatives --config editor select-editor aptitude install systemd-sysv- sysvinit-core aptitude install psmisc aptitude remove systemd aptitude install libsystemd0- libelogind0 aptitude install lsof aptitude install colordiff aptitude install wget aptitude install curl aptitude install rsync aptitude install gnupg

Packages II: Build Process

Now we install packages required for compiling GnuPG.

Copy to clipboard
aptitude install gcc aptitude install libc6-dev aptitude install libc-devtools aptitude install manpages aptitude install manpages-dev aptitude install make aptitude install zlib1g-dev aptitude install libldap-dev aptitude install libreadline-dev aptitude install libsqlite3-dev aptitude install pkgconf aptitude install gawk aptitude install libbz2-dev aptitude install bzip2-doc aptitude install gpgrt-tools aptitude install libgnutls28-dev aptitude install libtasn1-doc

Drop Privileges

Now we switch to a normal user, root privileges aren't needed anymore.

Copy to clipboard
su - ${user}

Clone Repository

As user shimps we clone the diy-gnupg repository from Codeberg and prepare the source code for compilation:

Copy to clipboard
mkdir -p codeberg/fmg cd codeberg/fmg git clone https://codeberg.org/fmg/diy-gnupg.git diy-gnupg/self-compile/bin/fetch-source.sh diy-gnupg/self-compile/bin/unpack-source.sh mkdir -p /tmp/canary/install

Compile Libraries

Now things are almost straightforward, similar to what was described earlier in Compiling GnuPG VIII: Dependencies I, but the order has changed.

Copy to clipboard
cd /tmp/canary/compile/npth-1.8 ./configure --prefix="/tmp/canary/install" make make install cd ../libgpg-error-1.51 ./configure --prefix="/tmp/canary/install" make make install cd ../libassuan-3.0.2 ./configure --prefix="/tmp/canary/install" make make install cd ../libksba-1.6.7 ./configure --prefix="/tmp/canary/install" make make install cd ../libgcrypt-1.11.0 ./configure --prefix="/tmp/canary/install" make make install

Compile GnuPG

Now we are close to run into problems with yat2m. This didn't happen before when the package pkgconf wasn't installed, but without this package it wasn't possible to build GnuPG with Sqlite3 support either. The solution is adjusting the PATH variable and to use the new compiled version of yat2m.

Copy to clipboard
PATH="/tmp/canary/install/bin:${PATH}" cd ../gnupg-2.5.5 ./configure --prefix="/tmp/canary/install" make make install LD_LIBRARY_PATH="/tmp/canary/install/lib" gpg --version

Comparison

Original
Copy to clipboard
$ ldd /usr/bin/gpg linux-vdso.so.1 (...) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (...) libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (...) libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (...) libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (...) libreadline.so.8 => /lib/x86_64-linux-gnu/libreadline.so.8 (...) libassuan.so.0 => /lib/x86_64-linux-gnu/libassuan.so.0 (...) libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (...) libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (...) /lib64/ld-linux-x86-64.so.2 (...)
Self Compiled
Copy to clipboard
$ LD_LIBRARY_PATH="/tmp/canary/install/lib" ldd /tmp/canary/install/bin/gpg linux-vdso.so.1 (...) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (...) libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (...) libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (...) libgcrypt.so.20 => /tmp/canary/install/lib/libgcrypt.so.20 (...) libreadline.so.8 => /lib/x86_64-linux-gnu/libreadline.so.8 (...) libassuan.so.9 => /tmp/canary/install/lib/libassuan.so.9 (...) libnpth.so.0 => /tmp/canary/install/lib/libnpth.so.0 (...) libgpg-error.so.0 => /tmp/canary/install/lib/libgpg-error.so.0 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (...) libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (...) /lib64/ld-linux-x86-64.so.2 (...)

Thanks

Many thanks to Andreas for pointing me to the missing pkgconf package problem on the gnupg-devel mailing list.

To be continued...

Compiling GnuPG XIV: Environment I

fmg Thursday April 3, 2025

Situation

Compiling GnuPG from the Upstream vanilla sources can lead to several surprises depending on the environment. Setting up a controlled, defined and reproducible environment seems to be mandatory to get reproducible results. One of the requirements to make the environment reproducible is the possibility to have a script doing this, and the seed method is chosen to be debootstrap. Even with Debian trixie at the horizon we stick with Debian bookworm since this release doesn't provide all the libraries in modern versions needed to compile the GnuPG devel version 2.5 tree and therefore it is a suitable laboratory. The preparation requires root privileges and we claim the namespace /srv/shimps to build the environment.

Debootstrap

Some redundant variables are introduced to keep the command line short and make it better readable. And we make a local copy of the vanilla debootstrap for later usage. Initializing the debootstrap from a local copy is much faster than network download.

Copy to clipboard
mkdir -p /srv/shimps/debootstrap/debian/bookworm mkdir -p /srv/shimps/laboratory/gnupg/bookworm mount --bind /srv/shimps/laboratory/gnupg/bookworm /mnt CODENAME="bookworm" CN=${CODENAME} ARCH="amd64" AR=${ARCH} MIRROR="http://ftp.debian.org/debian" MI=${MIRROR} LOG="/mnt/debootstrap.log" L=${LOG} { date; time debootstrap --arch ${AR} ${CN} /mnt ${MI}; date; } 2>&1 | tee -a ${L} rsync -a /mnt/ /srv/shimps/debootstrap/debian/bookworm/

Preparing apt

I consider it a bad habit to install recommended packages by apt* automatically. This hides things which might help to learn something. Therefore we switch to an explicit control setup.

Copy to clipboard
mkdir -p /mnt/etc/apt/apt.conf.available touch /mnt/etc/apt/apt.conf.available/shimps--no-recommended cd /mnt/etc/apt/apt.conf.d ln -s ../apt.conf.available/shimps--no-recommended shimps--no-recommended

The content of this file (use an editor for c&p):

Copy to clipboard
$ cat /mnt/etc/apt/apt.conf.available/shimps--no-recommended APT::Cache-Limit "67108864"; APT::Install-Recommends "0"; APT::Install-Suggests "0"; Aptitude::Recommends-Important "false";

Preparing chroot

We need awareness of the system inside the chroot environment.

Copy to clipboard
mount --bind /dev /mnt/dev mount --bind /dev/pts /mnt/dev/pts mount --bind /proc /mnt/proc mount --bind /sys /mnt/sys

The next part is due to SysV init. You might want to read zless /usr/share/doc/init-system-helpers/README.policy-rc.d.gz. Create an executable file with touch /mnt/usr/sbin/policy-rc.d, chmod +x /mnt/usr/sbin/policy-rc.d and content as shown in the next box. Some background is provided in the article ~jpetazzo/Use policy-rc.d to prevent services from starting automatically.

Copy to clipboard
$ cat /mnt/usr/sbin/policy-rc.d # shellcheck disable=SC2148 PATH="${PATH}:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" RL=$( runlevel ) if [ "${RL}" = "unknown" ] ; then exit 101 else exit 0 fi

If you are unsure, you may omit this step since we are not going to install any daemons which need to be prevented from being started automatically. But I consider it a good habit to prepare the chroot in an universal and complete manner.

It might be a good idea to prepare /mnt/root/.bashrc for bash-completion and maybe /mnt/root/.bash_logout and /mnt/root/.ssh/authorized_keys if you want to have a test environment in a virtual machine later, but this is not required. The latter one will require to have sshd configured for root login, which in itself might raise security issues.

The preparation files regarding apt and policy-rc.d are available in the Codeberg diy-gnupg repository.


Upcoming next time: we jump into the chroot and continue the preparation from inside.

To be continued...

Compiling GnuPG XIII: Libraries IV

fmg Tuesday April 1, 2025

Situation

Recently I compiled with bzip2 and gnutls support. The TLS stuff is important for dirmngr, Today I compare what is going on. However, this data set was collected before last Saturday. It indicates some problems with the environment I used to compile. It worked (in a way) for a version with GnuTLS, but it didn't work for a version with Sqlite3. I'll come back to this problem another time.

Distribution dirmngr

Copy to clipboard
$ ldd /usr/bin/dirmngr linux-vdso.so.1 (...) libassuan.so.0 => /usr/lib/x86_64-linux-gnu/libassuan.so.0 (...) libksba.so.8 => /usr/lib/x86_64-linux-gnu/libksba.so.8 (...) libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (...) libnpth.so.0 => /usr/lib/x86_64-linux-gnu/libnpth.so.0 (...) libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (...) libgcrypt.so.20 => /usr/lib/x86_64-linux-gnu/libgcrypt.so.20 (...) libldap-2.5.so.0 => /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0 (...) liblber-2.5.so.0 => /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (...) libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (...) libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (...) libunistring.so.2 => /usr/lib/x86_64-linux-gnu/libunistring.so.2 (...) libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (,,,) libnettle.so.8 => /usr/lib/x86_64-linux-gnu/libnettle.so.8 (...) libhogweed.so.6 => /usr/lib/x86_64-linux-gnu/libhogweed.so.6 (...) libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (...) /lib64/ld-linux-x86-64.so.2 (...) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (...) libffi.so.8 => /usr/lib/x86_64-linux-gnu/libffi.so.8 (...)

This is the reference to compare with.

Compiled without GnuTLS

For the next test the package libgnutls28-dev has to be deinstalled. But in this case dirmngr is not created. The ldd test seems to show dirmngr to be the only binary to link against gnutls.

Classic Compiled with GnuTLS

Copy to clipboard
$ LD_LIBRARY_PATH="/tmp/canary/install/lib" ldd /tmp/canary/install/bin/dirmngr linux-vdso.so.1 (...) libassuan.so.9 => /tmp/canary/install/lib/libassuan.so.9 (...) libksba.so.8 => /tmp/canary/install/lib/libksba.so.8 (...) libgpg-error.so.0 => /tmp/canary/install/lib/libgpg-error.so.0 (...) libnpth.so.0 => /tmp/canary/install/lib/libnpth.so.0 (...) libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (...) libgcrypt.so.20 => /tmp/canary/install/lib/libgcrypt.so.20 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (...) libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (...) libunistring.so.2 => /usr/lib/x86_64-linux-gnu/libunistring.so.2 (...) libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (...) libnettle.so.8 => /usr/lib/x86_64-linux-gnu/libnettle.so.8 (...) libhogweed.so.6 => /usr/lib/x86_64-linux-gnu/libhogweed.so.6 (...) libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (...) /lib64/ld-linux-x86-64.so.2 (...) libffi.so.8 => /usr/lib/x86_64-linux-gnu/libffi.so.8 (...)

Native Compiled with ntbTLS

Copy to clipboard
$ ldd /tmp/canary/native/bin/dirmngr linux-vdso.so.1 (0x00007ffc6ad49000) libassuan.so.9 => /tmp/canary/native/bin/../lib/libassuan.so.9 (...) libksba.so.8 => /tmp/canary/native/bin/../lib/libksba.so.8 (...) libgpg-error.so.0 => /tmp/canary/native/bin/../lib/libgpg-error.so.0 (...) libnpth.so.0 => /tmp/canary/native/bin/../lib/libnpth.so.0 (...) libntbtls.so.0 => /tmp/canary/native/bin/../lib/libntbtls.so.0 (...) libgcrypt.so.20 => /tmp/canary/native/bin/../lib/libgcrypt.so.20 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) /lib64/ld-linux-x86-64.so.2 (...)

We've got running software, but it can be improved. Setting up the environment and digging deeper into the compiling process will be the topic for another day.

To be continued...

Compiling GnuPG XII: Libraries III

fmg Friday March 28, 2025

Situation

Last time we compared the distribution binary with the self compiled one and noticed missing libraries. Now we need the packages libbz2-dev and libgnutls28-dev to be installed. To trigger usage of GnuTLS, the configuration needs to be invoked with environment variables set.

Copy to clipboard
cd /tmp/canary/compile/gnupg-2.5.5 LIBGNUTLS_CFLAGS="-I /usr/include/gnutls" LIBGNUTLS_LIBS="-L /usr/lib/x86_64-linux-gnu -l gnutls" ./configure --prefix=/tmp/canary/install make make install

The install procedure is odd. Something is wrong with the doc folder, and some kind of recursion warning is given as output message. But this is not the topic to focus on today...

The Improved Binary

Copy to clipboard
$ LD_LIBRARY_PATH="/tmp/canary/install/lib" ldd /tmp/canary/install/bin/gpg linux-vdso.so.1 (0x00007ffc86d51000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (...) libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (...) libgcrypt.so.20 => /tmp/canary/install/lib/libgcrypt.so.20 (...) libassuan.so.9 => /tmp/canary/install/lib/libassuan.so.9 (...) libnpth.so.0 => /tmp/canary/install/lib/libnpth.so.0 (...) libgpg-error.so.0 => /tmp/canary/install/lib/libgpg-error.so.0 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) /lib64/ld-linux-x86-64.so.2 (...)

The libbz2 support is visible. Support for GnuTLS is not visible. There is a replacement option with ntbTLS. This library is offered by GnuPG upstream, but there is the warning NOT WELL TESTED on the page. It is even unclear whether GnuTLS is expected to appear or not. There are a lot of questioins remaining open for further investigation.

To be continued...

Compiling GnuPG XI: Libraries II

fmg Wednesday March 26, 2025

Situation

Last time we showed information related to dynamic libraries. This time we continue and show more to compare. Again: let's see which libraries are used by the compiled binaries (some irrelevant hex numbers are removed from the output):

Distribution Compiled
Copy to clipboard
$ ldd /usr/bin/gpg linux-vdso.so.1 (...) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (...) libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (...) libgcrypt.so.20 => /usr/lib/x86_64-linux-gnu/libgcrypt.so.20 (...) libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (...) libreadline.so.8 => /lib/x86_64-linux-gnu/libreadline.so.8 (...) libassuan.so.0 => /usr/lib/x86_64-linux-gnu/libassuan.so.0 (...) libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (...) libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (...) /lib64/ld-linux-x86-64.so.2 (...)

This is considered to be the reference output.

Native Compiled & Location Changed
Copy to clipboard
$ mkdir /tmp/newlocation $ rsync -a /tmp/canary/native/ /tmp/newlocation/ $ ldd /tmp/newlocation/bin/gpg linux-vdso.so.1 (...) libz.so.1 => /tmp/newlocation/bin/../lib/libz.so.1 (...) libgcrypt.so.20 => /tmp/newlocation/bin/../lib/libgcrypt.so.20 (...) libassuan.so.9 => /tmp/newlocation/bin/../lib/libassuan.so.9 (...) libnpth.so.0 => /tmp/newlocation/bin/../lib/libnpth.so.0 (...) libgpg-error.so.0 => /tmp/newlocation/bin/../lib/libgpg-error.so.0 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) /lib64/ld-linux-x86-64.so.2 (...)

This looks good - the only thing missing for usage is the path change in the file gpgconf.ctl. We have something portable, but there are things missing.

The libsqlite3 isn't listed in the self compiled versions. But in the native compiled lib/ folder it is available. This is odd. Does this mean it is used? Or just provided? Does the native compile method need additional configuration? The libbz2 is missing, too. Maybe this can be fixed by installing the distribution package libbz2-dev.

The testing part is quite time consuming since there are some things handled by configuration options and other things by environment variables.

To be continued...

Compiling GnuPG X: Libraries I

fmg Monday March 24, 2025

Situation

Last time we compiled GnuPG in a different and more portable way. However, the library path problem isn't resolved completely. Let's see which libraries are used by the compiled binaries (some irrelevant hex numbers are removed from the output):

Classic Compiled - Wrong Way
Copy to clipboard
$ ldd /tmp/canary/install/bin/gpg linux-vdso.so.1 (...) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (...) libgcrypt.so.20 => /usr/lib/x86_64-linux-gnu/libgcrypt.so.20 (...) libassuan.so.9 => not found libnpth.so.0 => /usr/lib/x86_64-linux-gnu/libnpth.so.0 (...) libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (...) /lib64/ld-linux-x86-64.so.2 (...)

Oups...

Classic Compiled LD_LIBRARY_PATH
Copy to clipboard
$ LD_LIBRARY_PATH="/tmp/canary/install/lib" ldd /tmp/canary/install/bin/gpg linux-vdso.so.1 (...) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (...) libgcrypt.so.20 => /tmp/canary/install/lib/libgcrypt.so.20 (...) libassuan.so.9 => /tmp/canary/install/lib/libassuan.so.9 (...) libnpth.so.0 => /tmp/canary/install/lib/libnpth.so.0 (...) libgpg-error.so.0 => /tmp/canary/install/lib/libgpg-error.so.0 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) /lib64/ld-linux-x86-64.so.2 (...)

Good...

Native Compiled
Copy to clipboard
$ ldd /tmp/canary/native/bin/gpg linux-vdso.so.1 (...) libz.so.1 => /tmp/canary/native/bin/../lib/libz.so.1 (...) libgcrypt.so.20 => /tmp/canary/native/bin/../lib/libgcrypt.so.20 (...) libassuan.so.9 => /tmp/canary/native/bin/../lib/libassuan.so.9 (...) libnpth.so.0 => /tmp/canary/native/bin/../lib/libnpth.so.0 (...) libgpg-error.so.0 => /tmp/canary/native/bin/../lib/libgpg-error.so.0 (...) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (...) /lib64/ld-linux-x86-64.so.2 (...)

Even better...

Remarks

As we can see, the libraries are found as intended. This is just a snapshot of the work done so far. I'll come back to this and keep it this time short.

To be continued...

Compiling GnuPG IX: Dependencies II

fmg Saturday March 22, 2025

Situation

Last time we compiled the libraries and GnuPG itself by invoking the classic method ./configure, make and make install with an additional configure option. That resulted in a first success. Now we try a different method by using a different makefile.

Installation destination path

This time we choose the relative installation path native and the absolute path /tmp/canary/native.

Copy to clipboard
$ pwd /tmp/canary/compile/gnupg-2.5.5 $ mkdir /tmp/canary/native

Compiling

Using the speedo.mk makefile it is possible to utilize the shipped all inclusive option with the make target native. The needed libraries will be downloaded and compiled by the makefile. There is no need to resolve dependency conflicts manually. But the build environment needs the Debian package patchelf to be installed. This tool helps to generate a more portable version of the binaries.

Copy to clipboard
make -f build-aux/speedo.mk native make -f build-aux/speedo.mk install SYSROOT=/tmp/canary/native

The smoke test

Copy to clipboard
$ /tmp/canary/native/bin/gpg --version gpg (GnuPG) 2.5.5 libgcrypt 1.11.0 Copyright (C) 2025 g10 Code GmbH License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Home: /home/alice/.gnupg Supported algorithms: Pubkey: RSA, Kyber, ELG, DSA, ECDH, ECDSA, EDDSA Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH, CAMELLIA128, CAMELLIA192, CAMELLIA256 Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224 Compression: Uncompressed, ZIP, ZLIB

The environment variable LD_LIBRARY_PATH is not needed. The information about the location of libraries is taken from a configuration file which is installed next to the gpg binary.

Copy to clipboard
$ grep rootdir /tmp/canary/native/bin/gpgconf.ctl rootdir = /tmp/canary/native

The binaries are almost out of the box portable. When you move everything from /tmp/canary/native/* to another path /tmp/newlocation it should be sufficient to adjust the rootdir line in the file gpgconf.ctl to rootdir = /tmp/newlocation.

Copy to clipboard
mkdir /tmp/newlocation mv /tmp/canary/native/* /tmp/newlocation sed -i -e 's/tmp\/canary\/native/tmp\/newlocation/' \\ /tmp/newlocation/bin/gpgconf.ctl


To be continued...

Compiling GnuPG VIII: Dependencies I

fmg Thursday March 20, 2025

Situation

Last time we observed that the simple approach is not sufficient to compile GnuPG developer version 2.5 on a Debian bookworm or Devuan daedalus system. This is due to the fact that this version needs more recent versions of libraries than provided by the distribution. The libraries themselves are available and can be downloaded from the GnuPG project, but we need to make them visible where and when they are needed.

Installation destination path

Whenever a piece of software is compiled and ready to be used, it should be installed to a dedicated location. Since we do not want to interfere with the distribution setup we choose the relative installation path install and the absolute path will result into /tmp/canary/install. Remember from last time where we are and create the chosen destination:

Copy to clipboard
$ pwd /tmp/canary/compile/gnupg-2.5.5 $ mkdir /tmp/canary/install

Configuration with prefix

libassuan
Copy to clipboard
cd ../libassuan-3.0.2 ./configure --prefix="/tmp/canary/install" make make install
libgpg-error
Copy to clipboard
cd ../libgpg-error-1.51 ./configure --prefix="/tmp/canary/install" make make install

With this-+libgpg-error+- library we can compile libgcrypt.

libgcrypt-1.11.0
Copy to clipboard
cd ../libgcrypt-1.11.0 ./configure --prefix="/tmp/canary/install" make make install

Now all missing parts from the gnupg configuration feedback are available.

gnupg I
Copy to clipboard
cd ../gnupg-2.5.5 ./configure --prefix="/tmp/canary/install" echo "There is a new problem"

There is another library missing: libksba.

Copy to clipboard
configure: *** *** You need libksba to build this program. *** This library is for example available at *** https://gnupg.org/ftp/gcrypt/libksba/ *** (at least version 1.6.3 using API 1 is required). ***

The required version 1.6.3 is provided by the distribution package libksba-dev but since we changed the configuration prefix it is not found anymore. This is odd. This does not happen in a similar situation with npth later. We need to either fine tune the configuration or build a self-contained environment. We chose the latter one:

libksba
Copy to clipboard
cd ../libksba-1.6.7 ./configure --prefix="/tmp/canary/install" make make install
gnupg II
Copy to clipboard
cd ../gnupg-2.5.5 ./configure --prefix="/tmp/canary/install" echo "There is another new problem"

Again there is a library missing: npth.:

Copy to clipboard
configure: *** *** It is now required to build with support for the *** New Portable Threads Library (nPth). Please install this *** library first. The library is for example available at *** https://gnupg.org/ftp/gcrypt/npth/ *** (at least version 1.2 (API 1) is required). ***

The configuration would be fine with the distribution package libnpth0-dev installed, but we compile from the sources.

npth
Copy to clipboard
cd ../npth-1.8 ./configure --prefix="/tmp/canary/install" make make install

Back to the main software again...

gnupg III
Copy to clipboard
cd ../gnupg-2.5.5 ./configure --prefix="/tmp/canary/install" make make install

The smoke test

Copy to clipboard
$ /tmp/canary/install/bin/gpg --version /tmp/canary/install/bin/gpg: error while loading shared libraries: libassuan.so.9: cannot open shared object file: No such file or directory

This is not the intended behaviour.

Copy to clipboard
$ LD_LIBRARY_PATH="/tmp/canary/install/lib" /tmp/canary/install/bin/gpg --version gpg (GnuPG) 2.5.5 libgcrypt 1.11.0 Copyright (C) 2025 g10 Code GmbH License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Home: /home/alice/.gnupg Supported algorithms: Pubkey: RSA, Kyber, ELG, DSA, ECDH, ECDSA, EDDSA Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH, CAMELLIA128, CAMELLIA192, CAMELLIA256 Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224 Compression: Uncompressed, ZIP, ZLIB

This is a first success. But there is a lot of fine tuning left to be done.

To be continued...

Compiling GnuPG VII: First Steps

fmg Tuesday March 18, 2025

Situation

Last time I explained merging the pull request. This time we do the first compiling experiments. There are two scripts from the Codeberg diy-gnupg repository involved: fetch-source.sh and unpack-source.sh, both in the folder self-compile/bin/. Run them and swith to /tmp/canary/compile as working directory. There are several folders from the extracted tarballs:

gnupg-2.5.5
libassuan-3.0.2
libgcrypt-1.11.0
libgpg-error-1.51
libksba-1.6.7
npth-1.8

Simple Configuration

npth
Copy to clipboard
cd npth-1.8 ./configure make
libassuan
Copy to clipboard
cd ../libassuan-3.0.2 ./configure make
libksba
Copy to clipboard
cd ../libksba-1.6.7 ./configure make
libgpg-error
Copy to clipboard
cd ../libgpg-error-1.51 ./configure make
libgcrypt-1.11.0
Copy to clipboard
cd ../libgcrypt-1.11.0 ./configure echo "There is a problem"

We cannot continue with make because Debian bookworm and Devuan daedalus ship the package libgpg-error-dev with version 1.46 but a modern one is needed:

Copy to clipboard
checking for GPG Error - version >= 1.49... no configure: error: libgpg-error is needed.
gnupg
Copy to clipboard
cd ../gnupg-2.5.5 ./configure echo "There is a problem"

The distribution package libgcrypt20-dev contains version 1.10.1 and libassuan-dev contains version 2.5.5 which are additional problems. Something we don't see yet is a problem with npth:

Copy to clipboard
configure: *** *** You need libgpg-error to build this program. ** This library is for example available at *** https://gnupg.org/ftp/gcrypt/gpgrt *** (at least version 1.51 is required.) *** configure: *** *** You need libgcrypt to build this program. ** This library is for example available at *** https://gnupg.org/ftp/gcrypt/libgcrypt/ *** (at least version 1.11.0 (API 1) is required.) *** configure: *** *** You need libassuan to build this program. *** This library is for example available at *** https://gnupg.org/ftp/gcrypt/libassuan/ *** (at least version 3.0.0 (API 3) is required). *** configure: error: *** *** Required libraries not found. Please consult the above messages *** and install them before running configure again. ***

Conclusion

The developer version needs more attention with its configuration. Dealing with libraries is always a little bit tricky. But the GnuPG project provides the tools to circumvent those problems in several ways.

To be continued...

Compiling GnuPG VI: Merge a Pull Request

fmg Friday March 14, 2025

Situation

Last time I presented a first result, i.e. compiling the software and creating a Debian style package. Later in the evening a pull request came to my attention. How to deal with it? Different platforms (Codeberg, Github, Gitlab) seem to have different workflows, all integrated into the web user interface, and maybe command line support. I used Stackoverflow to figure out what to do - even if that post refers to creating rather than applying the pull request.

Solution

First I created a branch to test the method there, with several checks:

Copy to clipboard
git branch -l git log -1 git remote add ka7 https://codeberg.org/ka7/diy-gnupg.git git fetch --all git checkout -b mergepullrequest git branch -l git log -1 git cherry-pick 240ed7ed6fc0684fefe661399b5dd6f070dbeb6a git show b70ca327434814a7b2bf76acade3670cdd73f748 git push -u origin mergepullrequest git log -1 git switch main git branch -l git log -1

This worked the way I hoped, thus I could do it similarly by merging into the main branch.

Copy to clipboard
git cherry-pick 240ed7ed6fc0684fefe661399b5dd6f070dbeb6a git push

The commit is visible in the git log with timestamp Wed Mar 12 20:19:14 2025 +0100.


Thanks

Many thanks to the contributor ka7 for the pull request, Alex from C3S for explaining the problems with specific and generic workflow and Martin from LUGA for pointing me into the direction of a generic command line solution.

To be continued...

Compiling GnuPG V: Upstream Version 2.5.5 Debian Package

fmg Wednesday March 12, 2025

Latest News

Last time I discussed the second refactoring. Before I am going to focus on explaning the self compiling process itself the next days, I did a test of the new diy-gnupg repository on Codeberg with the new version GnuPG 2.5.5 from last Friday. The preparation with the scripts worked as expected and I converted the result into an update of the Debian style package shimps-gnupg-ng. This is still not the final result I aim to produce and it is still an experimental package, but it should work out of the box without path problems.

Upcoming Tonight

There is an online Jitsi meeting with friends from the Austrian LUGA (timezone CET at 7:00 pm). Language is German - feel free to join this open meeting.

To be continued...

Compiling GnuPG IV: Refactoring Reloaded

fmg Monday March 10, 2025

Latest News

Migration

Meanwhile the software project migrated from the old incubator repository to the new dedicated repository diy-gnupg. To make things easier, there is a migration script for repository cloning provided. Please adjust the localdestinationpath to your preferences or run

Copy to clipboard
$ git clone https://codeberg.org/fmg/diy-gnupg.git

from an appropriate folder of your choice.

User Settings

Now it is very easy to override the chosen GnuPG version numbers in etc/local.conf. This wil be useful if the repository is out of sync with upstram versions.

Logging

The repository provides a folder log/ which is convenient to have. The c&p examples in the README were adjusted.

Local Notes

The repository ignores a folder local/ which is convenient to have for some notes. These notes will not interfere with the Git repository setup (i.e. they do not appear as e.g. new or modified files).


Situation

Last time I presented the second script for unpacking the source code together with a refactored code structure. This allows to include (in shell terminology: source) external files, i.e. the configuration and an error library for shared usage among different scripts. The configuration file itself sourced a local configuration file with user settings - kind of an ugly construct. And to make things worse, each script was supposed to source those two files. Coming up with the idea of introducing a function library independent from the error library, this would require to source a third file and put the code into each script. Guess what I thought! Something felt wrong - time to pull the emergency brake.

Code Design

Continuing this path wasn't an option because it would pile up technical debt. This will eventually increase maintenance time to an extend that the project consumes too much time working on it at all, meaning it's dead. Even the revamp of two scripts and the configuration file was annoying, but the prospect of a flexible code structure was worth the effort. I introduced a single adapter file to be sourced by the scripts, and this can be extended to future needs without touching countless files. The habit of duplicate code usage is considered to be an anti-pattern, and this is for good reasons. It is quite obvious that this idea was inspired by the adapter pattern. The surprising part from my point of view was the fact that I didn't expect it becoming that important let alone that fast in shell programming. At the time I introduced the adapter the project consisted of four relevant files, and it was already refactored three days earlier.

The Function Library

The code is already prepared to use another library in the near future. This will make the shell scripts smaller and IMHO better readable, and it will be possible to reuse functions without code duplication. Another revamp of the two shell scripts is still necessary, together with the environment setup. This is expected to be no big deal.

First Steps

The corresponding paragraph from last time still holds unchanged. But we are approaching this milestone.


WP: Technical debt
WP: Duplicate code
WP: Anti-pattern
WP: Adapter pattern

To be continued...