Snamellit - linux

Measuring CI budgets with Guix and Grafana before setting quotas

Peter Tillemans — Fri, 09 Oct 2026 00:00:00 +0000

Emma does several jobs for me. It runs Cuirass for Guix builds and a Forgejo Actions runner, and I would like to add more services without discovering too late that CI ate the machine. My first instinct was to give each workload a quota: 11 GiB and four CPU cores for builds, 2 GiB and one core for the runner. Those are useful planning numbers, but applying them before measuring the jobs would have been guesswork.

So this is the measurement stage of setting a quota. The numbers appear as lines in Grafana; they are not memory.max, memory.high or cpu.max. A graph can show that a workload crossed a line without throttling or killing it. I will only consider limits after watching real, overlapping jobs, checking the host's spare capacity and comparing CI completion times.

Guix owns the layout

Emma runs Guix System, so I put the plumbing in its machines/emma/system.scm configuration rather than relying on a script somebody has to remember to rerun after a reboot. A one-shot Shepherd service prepares these cgroup-v2 paths:

/sys/fs/cgroup/emma/
├── ci-builds/
│   ├── cuirass/
│   └── guix-daemon/
└── emma-runner/
    ├── runner/
    └── podman/

Its Python helper, workload_cgroups.py, checks that the cpu, memory and io controllers are available, enables them on the parents and creates the leaves. It is idempotent and refuses to proceed if the required controllers are missing. The Cuirass and Guix-daemon leaves stay root-owned; only the runner's own leaves are delegated to its service account. PostgreSQL and host-management processes do not belong to either workload.

In Scheme, the dependency is explicit. The service graph requires the layout before the daemons can be started:

(define %emma-cgroup-layout
  (shepherd-service
    (provision '(emma-cgroup-layout))
    (requirement '(user-processes cgroups2-fs-owner
                   file-system-/sys/fs/cgroup))
    (one-shot? #t)
    (start #~(make-forkexec-constructor
               (list #$(file-append python "/bin/python3")
                     #$(local-file "workload_cgroups.py"))))
    (stop #~(make-kill-destructor))))

(simple-service 'emma-cgroup-layout
                shepherd-root-service-type
                (list %emma-cgroup-layout))

That local-file matters. The helper becomes part of the system build, tied to the configuration generation. The service definitions and the private Guix channel supplying the launchers are versioned together. A deployment can be built and checked before switching the running host, and the previous system generation remains a separate rollback path from Git.

Put processes in the group before they drop privileges

Creating a cgroup is easy. Accounting for the actual descendants is harder. Moving a running service supervisor into a group later can miss children it has already launched. The Guix channel therefore supplies a cgroup-launch-overlay: a file-like package tree that preserves the original package but replaces one executable with an exec-in-place launcher. The launcher writes its own PID to the leaf's cgroup.procs, drops to the normal service account where needed, then executes the original daemon. If placement fails, the service does not silently run outside accounting.

This ordering caught a real bug in the first draft. The normal Cuirass Shepherd start dropped to cuirass before my wrapper tried to enter its root-owned leaf. It failed with Permission denied. I adapted the Cuirass service type's two Shepherd starts instead, keeping the upstream account, activation sockets and other service extensions while entering the group as root and switching back to cuirass before executing the daemon. The shared Guix daemon has its own leaf because builds it starts cannot honestly be counted as Cuirass-only work.

The Forgejo runner has a separate leaf for host-mode jobs, and its rootless Podman socket service has another. The runner depends on the Podman socket and on emma-cgroup-layout. Having the socket daemon in the right group is not proof that a container process lands there; descendant placement needs to be checked during a real container job.

From `/sys/fs/cgroup` to Grafana

A second Python helper reads each parent group's memory.current, memory.swap.current, cpu.stat, memory.events and io.stat. Parent counters include their descendant leaves. Guix installs an mcron job that runs the collector every minute and atomically replaces a .prom file in node exporter's textfile directory:

(define %emma-workload-job
  #~(job '(next-minute (range 0 60))
         (string-append
          #$(file-append python "/bin/python3") " "
          #$(local-file "workload_metrics.py")
          " --output /var/lib/prometheus/node-exporter/emma-workloads.prom"
          " --group ci-builds=emma/ci-builds"
          " --budget ci-builds=11,4"
          " --group emma-runner=emma/emma-runner"
          " --budget emma-runner=2,1")))

The budget pairs mean GiB and CPU cores. The collector exports emma_workload_budget_memory_bytes and emma_workload_budget_cpu_cores as reference gauges, alongside measured memory, swap, CPU time, throttling, OOM events and I/O bytes. It also emits emma_workload_present and a collection timestamp. An incomplete or missing group gets present=0 without invented usage samples. Node exporter exposes the textfile, Prometheus scrapes it, and Grafana reads Prometheus; Grafana does not create or enforce any quota. This follows node exporter's textfile-collector pattern.

I use the existing Emma Capacity & Budgets dashboard. Here are its two workload panels from a live, two-hour Grafana view on 9 October 2026. The screenshot shows the ci-builds CPU curve crossing its four-core planning line. That is an observation, not throttling or evidence that a quota has been applied. The capture predates the completed 24-hour review.

The memory panel plots actual usage and the planning line for each workload:

emma_workload_memory_bytes{machine="emma",workload=~"ci-builds|emma-runner"} / 1024^3
emma_workload_budget_memory_bytes{machine="emma",workload=~"ci-builds|emma-runner"} / 1024^3

The CPU panel uses a five-minute rate against the core budget:

rate(emma_workload_cpu_seconds_total{machine="emma",workload=~"ci-builds|emma-runner"}[5m])
emma_workload_budget_cpu_cores{machine="emma",workload=~"ci-builds|emma-runner"}

Those are two Grafana queries per time-series panel, not a command that installs limits. I keep the host's available memory, root disk space, CPU execution versus I/O wait and OOM/swap activity alongside them. The coverage stat reads emma_workload_present; a missing series is No data, not zero usage. The dashboard was designed before the exporter existed and still contains some "proposed / not yet verified" explanatory text. Some per-workload curves are now live, but that text should only be revised once completed jobs and descendants have been checked. Until then, a pretty plot is not proof of correct attribution.

Rollout, not just reconfigure

I tested the layout and launch ordering in a disposable, network-disabled Guix VM. A read-only-store VM was enough to check placement but could not run an actual Cuirass register that wrote to the store, so I repeated that check with a guest-owned writable QCOW2 store. Both real Cuirass daemons then ran as cuirass in the intended leaf. That still was not a representative online build.

For Emma itself, I reviewed the channel and system changes on separate Git branches, ran the service regression check and guix system shepherd-graph, and dry-ran the system build. A full build on Emma used one build job and one core after an earlier attempt ran into memory pressure. Only then did an operator reconfigure the host and, later, perform a normal reboot with the previous generation available at the console. After restarting the affected services, I checked actual PIDs in /proc/<pid>/cgroup, the collector file and successive healthy Prometheus scrapes. The post-boot 24-hour observation began on 9 October 2026 at 14:39 UTC, after those checks.

That reboot exposed a separate loose end: Cuirass, the runner, its Podman socket, git-pages and node exporter did not all start unattended. We brought them up manually. A Tailscale-address race might explain some listeners, but it does not explain every stopped CI service; the boot logs still need investigation. Manual recovery is not a successful boot test.

The next pass is to match the curves against completed Cuirass builds and Forgejo jobs (including container descendants), inspect the same-window service logs and compare job duration with a baseline. The shared Guix daemon's counters measure all its clients, not just Cuirass. Until that work is done, I have a useful accounting experiment and a rollback path, not a safe quota recommendation.

The Guix system manual covers generations and reconfiguration; the kernel cgroup-v2 documentation explains the counter and control files. Grafana's visualization documentation covers building panels from Prometheus queries.

Install GUIX on Macbook 12

Peter Tillemans — Fri, 20 Mar 2026 11:01:00 +0100

Motivation

I have an old Intel macbook 12 of begin 2016, one of the so-called "Retina Macbooks". It is an ideal couch device but far too slow to run modern OS-X, provided you wanted to run that to begin with.

I wanted to revive this machine to avoid lugging the BIG laptop, especially for the friday SystemCrafters stream in my comfy chair in the living.

I saw some experience reports with Ubuntu, which worked, but then I never used it because Ubuntu is, well it is Ok, there is nothing wrong with it... Gnome works fine, KDE worked fine, it is just all different from my usual Arch/Guix machines for little added benefit.

Because I was curious about immutable distros and leveraging containers like toolbox/flatpak I installed Fedora Silverblue. Again it is fine, Gnome is really luxurious on this machine. It all works well, but again different. Distributing the software between immutable host (which you have to reboot to make packages appear after install), flatpaks and toolbox containers works fine again, but inevitably causes friction when crossing the domains, e.g. Emacs. Nothing insurmountable. Also I am not impresssed by toolbox, it is simple, but it does nothing that distrobox does not seem to do better, while being less tied to the OS. I am not a fan of the dev container workflow, and Fedora Silverblue didn't sell it to me. Again in large deployments the balance might be different.

AFAICT Guix offers (almost) the same immutability features and drawbacks but less pronounced. i.e. you do not have to reboot to make host software available in most cases. The diffferent profiles allow much more granular control with several half-way points to find the right balance of reproducability and creature comfort. Of course this is also means more opportunity to let it run out of control and create a confusing mess. In any case I can leverage the GUIX configuration of my servers and big laptop and ensure a consistent experience if I can get it on the lil' macbook and liberate it to fight another day.

Installation Process

Installing Ubuntu and Silverblue learned me that there is nothing specially difficult for the intel mac. It kinda just works with some small caveats.

A quick hardware check shows a BCM4350 wireless network adapter which is not on the open hardware list AFAIK, and since the laptop needs to do laptop duty and has only a single USB-C port WiFi is the only way to get something in and out of it. Intel graphics should not be a problem. An intel soundchip. Some Apple NVMe controller (foreshadowing)

So we probably need the nonguix version to support the firmware blobs.

Creating Installation Media

Check out on your local friendly Guix enabled device, a foreign install is fine, checkout the SystemCrafters Guix Installer Repo. In the root of the project is a script build.iso. Run it and have a walk outside or something. It takes a while. If you have no Guix enabled machine yet, you can download the last released version, which is over a year old, so the first guix pull on the macbook will be epic... .

You'll end up with a file guix-installer-202603162333.iso with of course a probably newer timestamp is spacetime is behaving consistently in your locati This can be written to a thumb drive. Ensure you check with dmesg -w which device the USB stick uses. On insertion you'll see

$ sudo dmesg -w
... lots of lines skipped
[308042.889695] usb 1-4.3.2: new high-speed USB device number 12 using xhci_hcd
[308043.000238] usb 1-4.3.2: New USB device found, idVendor=0781, idProduct=5572, bcdDevice= 1.00
[308043.000242] usb 1-4.3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[308043.000244] usb 1-4.3.2: Product: Cruzer Switch
[308043.000246] usb 1-4.3.2: Manufacturer: SanDisk
[308043.000247] usb 1-4.3.2: SerialNumber: 4C530001061011107340
[308043.038906] usb-storage 1-4.3.2:1.0: USB Mass Storage device detected
[308043.039031] scsi host2: usb-storage 1-4.3.2:1.0
[308043.039112] usbcore: registered new interface driver usb-storage
[308043.042575] usbcore: registered new interface driver uas
[308044.049756] scsi 2:0:0:0: Direct-Access     SanDisk  Cruzer Switch    1.00 PQ: 0 ANSI: 6
[308044.055625] sd 2:0:0:0: [sda] 62521344 512-byte logical blocks: (32.0 GB/29.8 GiB)
[308044.056787] sd 2:0:0:0: [sda] Write Protect is off
[308044.056790] sd 2:0:0:0: [sda] Mode Sense: 43 00 00 00
[308044.057159] sd 2:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[308044.093399]  sda: sda1 sda2
[308044.093545] sd 2:0:0:0: [sda] Attached SCSI removable disk

The '[sda]' on the last lines shows that the USB is now available as /dev/sda. Be careful as using your current harddisk by accident will cause an unscheduled backup/restore excercise. To be safe make sure you use the data which appeared on screen when inserting the drive. Pull it out and insert it again to be sure.

Also don't mount/open it if your smart desktop assistent comes asking helpfully. It only makes things slower and more complicated.

$ sudo dd if=guix-installer-202603162333.iso of=/dev/sda bs=1M
$ sync

The bs=1M is to speed things up otherwise it takes forever. Value is not critical but this is easy to type and _reasonable_™.

This will probably take a refill coffee amount of time to complete. The `sync` should ensure the buffers are written before unplugging the device.

Installation Process

One of the biggest challenges to start the installation process is to build a Jenga tower of USB dongles to ensure you can power the macbook and insert the USB stick. This device only has a single USB-C port, it dislikes thunderbolt docks or even many USB-C docks. I used a Steamdeck dock. In practice I assume most of the cheap USB extender dongles will do it, however I had to experiment before it worked.

You can probably install the OS on whatever capacity is remaining after all this time, but I would not rely on it and I would not want to do the crypto setup of the disks with the battery draining like a James Bond Villain Death Trap.

I installed it a few times. First without disk encryption. Worked fine. You follow the prompts.

Then with disk encryption. Worked fine.

I choose separate volumes for root and home formatted in ext4. When encrypting the volumes using LUKS we now get an interesting issue. During boot up we are asked 4 times to unlock the drives. Also once we selected the image to boot in Grub the kernel will ask to unlock the disks, but the internal laptop keyboard does not work. This can be solved by adding a USB keyboard to your Jenga Tower and unlock the drives. This get tired really quickly. For installation it is not a blocker so we plod on.

Also at some point the graphical installer decide that black is the new orange and show nothing else than that. However the manual installer is well documented.

Also the installation process for GUIXSD is far less important than for other distros as you can completely control the OS using `guix system reconfigure`. So if you can boot in a terminal and get on your network you're golden.

Crypto Setup

I am going to forget the keyboard issue for now and deal with the passwords. Because if that is fixed the keyboard does not matter anymore as once the kernel is fully started it works again.

If you followed along the installer will already have created the partitioning and LUKS encryption of the volumes. We can use a keyfile to unlock the partitions but we have to make sure the keyfile is not exposed on an unencrypted volume. Some people suggest a keyfile on a USB stick but this is not very practical on this device (think Jenga tower).

There is no way to pass secrets from Grub to the booting kernel. However we have access to files in the initial ramdisk. The GUIX bootloader configuration has the extra-initrd option which was originally added for this very purpose and adding the keyfile is explained there.

So the plan is:

add a key from a keyfile to a slot in the LUKS partitions
wrap the keyfile in a cpio image to merge to the initial ramdisk
configure the mapped devices to use this keyfile

add keyfile to initrd

I created a random file

$ dd if=/dev/urandom of=/keyfile.bin bs=1 count=512
$ chmod 0400 /keyfile.bin
$ echo /key-file.bin | cpio -oH newc >/key-file.cpio
$ chmod 0000 /keyfile.cpio

add the keyfile as a key to the LUKS partitions.

Update /etc/config.scm to add the extra-initrd option to make the file accessible during early boot.

(bootloader (bootloader-configuration
               (bootloader grub-efi-bootloader)
               (targets '("/boot/efi"))
               (keyboard-layout keyboard-layout)
	              (extra-initrd "/key-file.cpio")))

Tell kernel to use the keyfile

To configure the kernel to use the keyfile instead of asking for a password, we have to add the #:key-file "/key-file.bin" to the luks mapping

(mapped-devices
 (list (mapped-device
        (source (uuid "47642581-ea45-4c53-84c8-d1baf81c70de"))
        (target "my-root")
        (type luks-device-mapping)
	        (arguments '(#:key-file "/key-file.bin")))
       (mapped-device
        (source (uuid "f2d3ee87-7cc7-4a95-8f4c-1f6c0fa750ee"))
        (target "my-home")
        (type luks-device-mapping)
	        (arguments '(#:key-file "/key-file.bin")))))

Apply it

run a guix system reconfigure and reboot and now you will only have to enter 2x the password. A 50% improvement ( or 500%, 600% according to some world leader).

Now why does grub insist in decrypting the /home partition?

A quick look in /boot/grub/grub.cfg shows:

➜ cat /boot/grub/grub.cfg | head
# This file was generated from your Guix configuration.  Any changes
# will be lost upon reconfiguration.
insmod luks
insmod luks2
cryptomount -u 47642581ea454c5384c8d1baf81c70de
cryptomount -u f2d3ee877cc74a958f4c1f6c0fa750ee

# Set 'root' to the partition that contains /gnu/store.

Ok, that explains it. But why?

The manual does not give me any hints. Looking in the source code of the guix bootloader generator shows that it makes a list of all mapped devices which have a valid UUID and add it to grub. It throws a warning when a mapped device uses the regular linux devicename instead of the UUID as grub cannot find the devices with the linux name. This makes sense and we can abuse it to get rid of the 2nd password prompt:

(mapped-devices
 (list (mapped-device
        (source (uuid "47642581-ea45-4c53-84c8-d1baf81c70de"))
        (target "my-root")
        (type luks-device-mapping)
	  (arguments '(#:key-file "/key-file.bin")))
       (mapped-device
        ;;(source (uuid "f2d3ee87-7cc7-4a95-8f4c-1f6c0fa750ee"))
	  (source "/dev/nvme0n1p3")
        (target "my-home")
        (type luks-device-mapping)
	  (arguments '(#:key-file "/key-file.bin")))))

If we replace the UUID with the regular device name, it will no longer be exported to grub.cfg and the kernel kan still find the device for mapping it.

A quick reconfigure and reboot later confirms this. I can now remove the keyboard from the Jenga tower, but that would be cheating.

Fixing the keyboard

Some searching in the Guix mailing list showed that "hid_apple" was removed from the standard initrd somewhere in '24 because of issues with RISCV.

That is where the initrd-modules option is for:

(operating-system
  ...
  (initrd-modules (cons "hid_apple" %base-initrd-modules))
)

This fixes the issue. I forgot how I tested it, but it does.

Post Installation

Actually the problem is that the disk is still sleeping (remember the foreshadowing when seeing the Apple NVMe device). This is not GUIX specific as I already had to fix it in Ubuntu and Fedora. There you add

echo 0 >/sys/bus/pci/devices/0000\:01\:00.0/d3cold_allowed

in the crontab on the @reboot event.

This syntax is not supported by the herd scheduled jobs and I did not find an alternative and got no suggestions on the #guix IRC channel. So I decided to make a small shepherd service for this.

(define disk-sleep-shepherd-service
  (shepherd-service
   (provision '(disk-sleep))
   (documentation "Disallow disk to go to d3cold state to prevent crash on wake up.")
   (one-shot? #t)
   (start #~(lambda _
	       (call-with-output-file "/sys/bus/pci/devices/0000:01:00.0/d3cold_allowed"
		 (lambda (port) (display "0" port)))
	       #t))))

The only part I struggled with is that the start field takes a function with a variable number of arguments. The /var/log/messages registered the error but was not very helpful. Herd did not show the service at all until it was fixed. Comparing to other implementation gave the needed clues.

To add this to the system:

(operating-system
 ...
 (services
  ...
  (simple-service 'disk-sleep-service shepherd-root-service-type
	  (list disk-sleep-shepherd-service))
  ...
  ))

Sound : no sound from speakers.

There is no sound coming from the speakers.

There is sound coming from the headphone jack. Good enough for use in the living.

I'll call this a feature, not a bug.

Things I would do differently

Basically use btrfs to have a single encrypted LUKS volume to deal with and split the volumes in btrfs. That would also give the other goodness of btrfs and should be well supported by grub.

But for now I'll enjoy my new couch laptop.

Screenshot in Sway with GUIX

Peter Tillemans — Sat, 25 Oct 2025 14:19:00 +0200

Screenshots in Sway with Guix Home Manager

A blog post documented an elegant way to configure screenshots in wayland using grim and grimshot. After moving my sway config to use the GUIX sway-service to generate the config file Iost my configured screenshot bindings.

Output Files

The screenshots will be placed in the Pictures/screenshots folder with the timestamp as filename. To avoid duplication and ensure consistency we'll add a variable to generate the filename

(sway-configuration
  (variables (append %sway-default-variables
		     `((menu .  ,#~(string-append
                                    #$fuzzel
                                    "/bin/fuzzel"
                                    " -w 50 -x 8 -y 8 -r 3 -b 232635ff -t A6Accdff -s A6Accdff -S 232635ff -C c792eacc -m c792eacc -f \"JetBrains Mono:weight=light:size=10\" --icon-theme=\"Papirus-Dark\" --no-exit-on-keyboard-focus-loss"))
		       (term . ,(file-append alacritty "/bin/alacritty"))
		       (screenshot_out . "$(xdg-user-dir PICTURES)/screenshots/$(date +\"%Y%m%d-%H%M%S\").png"))
		     ))
  ...)

Key Bindings

I often use various screenshots pasted in chat windows or Jira tickets so the clipboard is an important destination. Alternatively saving them to move them to a target folder for inclusion whereverr they needed is an important secondary use-case.

We can now use the variable made previously and add the following keybindings:

(sway-configuration
	    ...
	    (keybindings (append
			  `(...
			    ;; Special key to take a screenshot with grim
			    ($mod+Shift+p . "exec grimshot copy anything")
			    ($mod+Shift+s . "exec grimshot save screen $screenshot_out")
			    ($mod+Shift+w . "exec grimshot save window $screenshot_out")
			    ($mod+Shift+a . "exec grimshot save area $screenshot_out")
			    ...

			    )

Deploying the configuration

To activate the configuration

$ make home
echo Loading modules from ~/.config/dotfiles/guix/src
Loading modules from /home/pti/.config/dotfiles/guix/src
guix home reconfigure -L ~/.config/dotfiles/guix/channel -c 32 ~/.config/dotfiles/guix/home-config-`hostname | cut -d. -f1`.scm
...

which quickly (for some value of quick) updates the home configuration, writing a new ~/.config/sway/config file.

When the prompt returns a quick reload of the sway config with $mod+Shift+c and testing if the modifications are active.

Testing it Out

I opened libre office writer to paste copied screenshots and made a mess to show the copy anything works as expected. It does, it highlights what it understands depending on your mouse location, a window if hovering over a window, a rectangle if you start selecting and all the screen if you move the mouse outside all windows (i.e. the boundary on a tiling wm). It is a bit fiddly so sometimes I select the wrong thing, but since it only clobbers the clipboard and I can quickly try again, this seems reasonable.

Similarly using the 3 other bindings to save different type of screenshots.

  $ ls ~/Pictures/screenshots/
20251025-141106.png  20251025-145006.png  20251025-145019.png

Cool, that made a mess in the folder and displaying them show that it are actually screenshots. Cool.

Consolidating Secrets in Pass

Peter Tillemans — Sat, 28 Jun 2025 13:15:00 +0200

Background

Like a lot of people I've had a long history managing passwords and secrets over the years. From a little black book, over an Excel sheet, using a GPG encoded secrets file (works really well with Emacs gpg support), 1password (till they racked up their prices), lastpass (till they got bought by the Evil LogMeIn Corp), KeepassXC and lately pass.

I was perfectly happy with KeepassXC for a very long time, except for the command line integration. So I kept ending up with passwords in .envrc files in folders and excluded in the global .gitignore to avoid too many red cheeks. While this does keep secrets out of harms way mostly, it kept nagging that I had them in plain text in those files. In theory there is keepassxc-cli to query the passwords from the command line, but let's say the experience does not spark joy. It has no easy way to cache the password between calls and it is optimized for interactive use. (AFAICT, just the giant size of the command to type gives me dread).

Some day I stumbled over pass and found that after setup I could just pass snamellit/website to get the password on stdout. I wrote about the setup and emacs integration in a previous post. Since it leverage gpg, password caching is handled by the gpg-agent and my .envrc files quickly were purged of blasphemous secrets, replace by pure bliss:

export MY_SECRET=$(pass my/secret)
export OTHER_SECRET=$(pass other/secret)

similarly in emacs I can consistently get my passwords and related info with:

(org-gcal-client-id (auth-source-pass-get 'secret "snamellit/org-gcal-client"))
(org-gcal-client-secret (auth-source-pass-get "id"
"snamellit/org-gcal-client"))

When needed the gpg-agent will launch the appropriate pin-entry program whether in terminal or in the GUI and the caching will not force me to login several times when entering the folder.

So I ended up with my interactive use covered by KeepassXC and automated use by pass.

However, after some time I ended up with hundreds of secrets in KeepassXC, hundreds in pass, it is not always clear whether use is interactive or automated so confusion and duplication starts and things become harder to manage. In addition KeepassXC was using historically Dropbox to make it available on all my devices, recently migrated to Nextcloud, which has issues with dealing with conflicts which occasionally bite me in the behind. On the other hand pass secrets are stored encrypted in git where conflict punch you in the face. I prefer the latter. And started contemplating whether to move everything to pass.

Thanks to the encouragement of SummerEmacs, one of the more enthusiastic SystemCrafters, ensuring the great experience in browsers and iOS mobile devices I had no more excuses to keep postponing it.

Preparation

I started out with keeping my pass passwords as part of my dotfiles. This was convenient when they were few. However this is weird so this will attract weirdness when configuring all integrations I'll need.

Also pass supports a git command to manage the password-store with git which is not really useful when it is part of something else. So the first order of the day is to move all secrets to a separate repository and update the dotfiles to check for presence and clone the repo if missing (and do a gently pull when it is). A quick visit to each of the machines in my machine park to apply this change. Everything still seems to be working.

Migration of the KeepassXC data

I used the pass-import tool which adds in import command to pass which supports a crazy amount of password managers, including keepassxc. For keepassxc it need the pykeepass. If you're running on Arch, everything is a yay -S away. However on Ubuntu and its derivatives it is the usual slog we start to get accustomed to. It's all in the pass-import README , note that on Ubunty the pykeepass library is available with apt install python3-pykeepass.

Once it is installed I tried a dry run (with the -d flag) to see if basic functionality is working

pass import -a -d keepassxc ~/Nextcloud/Apps/Keepassxc/Passwords.kdbx
Password for /home/pti/Nextcloud/Apps/Keepassxc/Passwords.kdbx:
  w  Data would be imported from keepassxc to pass
  .   Passwords imported from: /home/pti/Nextcloud/Apps/Keepassxc/Passwords.kdbx
  .   Passwords exported to: /home/pti/.password-store
  .   Number of password imported: 2035
  .   All data imported
  w  Weak password detected: eDGQqipE might be weak.  Score 2 (100000001 guesses).  This estimate is based on the sequence eDGQqipE(bruteforce)
  w  Weak password detected: eDGQqipE might be weak.  Score 2 (100000001 guesses).  This estimate is based on the sequence eDGQqipE(bruteforce)
  w  Weak password detected: eDGQqipE might be weak.  Score 2 (100000001 guesses).  This estimate is based on the sequence eDGQqipE(bruteforce)
...  large list of names of secrets

This asks for the password of the Keepass file and some remarks it has.

This all looks reasonable. So we can try the import. Since the password-store is a git repo no real damage can be done to it (as it is safely pushed somewhere else where the import tool cannot touch it) and any damage done can be reverted....

Now is a good time to check if the mooring lines of your laptop are properly secured as encrypting all the secrets will spin up the propellors if the number is large enough.

I run it again without the -d flag and after several minutes the noise dies down and I am left with a lot of additional folders in my ~/.password-store which match the grouping in KeepassXC. The files contain the secrets and the expected metadata. This looks good so I add/commit the things to complete the level.

Integration with iOS for my iPhone

Let's start with the most scary one : the iPhone.

Upon recommendation I had installed passforios which needs to be configured.

Configuring the host, repo and username to use for the git repository is straightforward enough.

I always use ssh to access my repos so we need to add an ssh keypair for this purpose. There is no support to generate key-pairs in passforios for reasons, so I have to do it externally and upload the key. A quick ssh-keygen , uploading the public key to the forge, allowing access to the repo and if I can get the private key on my phone we can access the repo.

passforios has a nice feature to load ascii armored keys via a QR code. A bit digging surfaced the asc-key-to-qr-code-gif tool which was made for this specific purpose. The ssh key is already in the appropriate format so this can be directly converted

./asc-to-gif.sh ~/.ssh/id-passforios ssh-pub.gif
display ssh-pub.gif

Then go to the repository settings, press the circled i on the SSH Key button, select the ASCII-Armor Key and click to scan the QR code. Point the camera to the QR code on the screen and it should appear in the key field in the app.

We have to repeat this 2 more times to get the private and public key for the password-store into the app. First exporting the keys

gpg --export -a 1234ABCD >gpg.pub
gpg --export-secret-key -a 1234ABCD >gpg.key

converting to a gif, displaying them and scanning them in *Settings -> PGP Key -> ASCII-Armor Key in the respective fields.

If, after synching, you go now to the Passwords you should be greeted with a listing of all folders and keys and the secrets should be visible if made visible by tapping the eye icon.

I needed to enable passforios as a source for autofill : Settings -> Passwords -> Autofill Passwords and slide the toggle for Pass. I also disabled the toggle for Strongbox which I was using for integration with the Keepass database.

Now I see the option to select the secrets from the passforios app. It does not narrow down to the right key, but that is a problem for future me.

Ok, the hard part is done. Or at least the most risky part, ... in my eyes... whatever. Moving on...

Integration with FireFox

Checking at the bottom of the pass website we find that passff is the good stuff for integration with FireFox. From previous adventures with KeepassXC and NativeMessaging I assumed there had to be a host part to be installed too.

Indeed we are directed to the passff-host github repo to get an install-script which generates the native messaging json for the different browsers and a small executable python script which contains remarkable clean and no-dependency code. Similarly the install script is straightforward. I do not understand why it support half a dozen browser, mostly chrome based as for the life of me I cannot find an extension which uses this host program. So either I need bigger glasses or there is some knowledge beyond my grasp.

Running the installer, installing the extension, restarting firefox for good luck and the extension appears and offers passwords on the sites I try.

Out of curiosity I check the configuration in ~.mozilla/native-messaging :

pti@tuxedo ~> ls .mozilla/native-messaging-hosts/
org.keepassxc.keepassxc_browser.json  passff.json  passff.py*
pti@tuxedo ~> cat .mozilla/native-messaging-hosts/passff.json
{
  "name": "passff",
  "description": "Host for communicating with zx2c4 pass",
  "path": "/home/pti/.mozilla/native-messaging-hosts/passff.py",
  "type": "stdio",
  "allowed_extensions": [ "passff@invicem.pro" ]
}
pti@tuxedo ~> cat .mozilla/native-messaging-hosts/passff.py
#!/usr/bin/python3
"""
    Host application of the browser extension PassFF
    that wraps around the zx2c4 pass script.
"""

import json
...

Nothing out of the ordinary, the passff.py python is the same as in the repo. My old keepassxc extension support is still there.

Firefox is installed natively on this machine, not with a flatpak which I assume will come with its own challenges.

Chromium Support

Time to tackle the Chrome family. Chrome is required to put food on the table so we have to get that going eventually. But Chrome is distributed as a flatpak (or a snap but I am NOT going to deal with that), and I can install Chromium natively, and apparently native installs are MUCH better supported than the versions in wrappers so let's start with that one first.

From the pass website we find that browserpass is the way to go for the chrome family. The browser extension installs from the usual places without drama and starts promptly complaining it cannot find the native host to talk to.

The native host in question is from the browsaerpass-native sister repo . As usual for all distro's there are packages ready to install but because Ubuntu-derivative I can compile from source. Downloading the source for version 3.1.0 from the releases page. Again this repo refers to all browsers including firefox although I cannot for the life of me find a Firefox Extension supporting this host app.

Then building and installing timelapse :

tar -xzvf ~/Downloads/browserpass-native-3.1.0.tar.gz
cd browserpass-native-3.1.0
ls
less README.md
PREFIX=/usr/local make configure
sudo make PREFIX=/usr/local install
which browserpass

which shows the executable lives at /usr/local/bin/browserpass and this totally went fine the first time (NOT!!!!).

The Makefile has support to install the magic json to enable native messaging for the different browsers.

PREFIX=/usr/local make hosts-chromium-user
PREFIX=/usr/local make hosts-chrome-user

The second invocation is a hail-mary because I already know the Chrome flatpak does not look in the same places and will require some additional finnagling

For now focus on Chromium and check if the configuration looks reasonable:

pti@tuxedo ~> cd .config/chromium/NativeMessagingHosts/
pti@tuxedo ~/.c/c/NativeMessagingHosts> ls
com.github.browserpass.native.json@
pti@tuxedo ~/.c/c/NativeMessagingHosts> cat com.github.browserpass.native.json
{
    "name": "com.github.browserpass.native",
    "description": "Browserpass native component for the Chromium extension",
    "path": "/usr/local/bin/browserpass",
    "type": "stdio",
    "allowed_origins": [
        "chrome-extension://naepdomgkenhinolocfifgehidddafch/",
        "chrome-extension://pjmbgaakjkbhpopmakjoedenlfdmcdgm/",
        "chrome-extension://klfoddkbhleoaabpmiigbmpbjfljimgb/"
    ]
}

Cool, the executable is looked at where it is installed (this is not obvious, don't ask how I know). The rest looks also like how these things should look. Let's try...

The extension settings page is no longer complaining the native host is missing and there are password entries visible. Checking with some website shows the password is injected. yay!.

Level complete, ready for the final boss.

Enabling Chrome Support, now with more Flatpak!

Ok, we have a working chromium support so repo access, host app, native host configuration et al are proven working. We can only focus on jumping over the Flatpak Firewall...

As a good cargo cultist I do a literature study and find that I should

find the config location of the flatpak app
use flatpak-spawn to spawn the native messaging host app
enable D-Bus Session socket access for chrome
Package up the calling of the host app in a single script to configure in the json.

Not necessarily in that order....

For the permission to access D-Bus Session start up flatseal from flathub, navigate to com.google.Chrome and enable the D-Bus Session socket. This should be possible with some additional cursing in the manifest file of Chrome. I cannot find decent reference documentation in a reasonable time, so flatseal it is.

The configuration of the flatpak app is easy too, painful experience seared in my brain that flatpaks look in ~/.var/app/ folder so for Chrome this will be ~/.var/app/com.google.Chrome . From the hail-mary install for chrome done above I know that it just creates a symbolic link to /usr/local/lib/browserpass/hosts/chromium/com.github.browserpass.native so we can start from there. We will have to edit that so copy it. We also need a wrapper to call the native host app

cd ~/.var/app/com.google.Chrome/config/google-chrome/NativeMessagingHosts
cp /usr/local/lib/browserpass/hosts/chromium/com.github.browserpass.native
ec browserpass.sh

Add the content of the wrapper

#!/bin/sh
cd ~
/usr/bin/flatpak-spawn --host /usr/local/bin/browserpass 2>/tmp/browserpass-error.log

I added the optional redirect of stderr to an error logfile because from experience I know nothing ever goes wrong if you enable error reporting beforehand.

chmod +x browserpass.sh
pwd
pwd | wl-copy
ec com.github.browserpass.native.json

Installing the browserpass extension in Chrome after restarting it (I am not superstitious, just careful) and I can bask in the glory of seeing proposals for passwords when trying to log in. Most of the proposals are pretty garbage, but that is a problem for future me.

Conclusion

I have access to my password-store secrets on my phone, my browsers on laptop and desktop, and most importantly Emacs. Narrowing of the proposed secrets is, euhmmm, sub-optimal, but since it is sub-optimal in the same way on all platforms I assume that some TLC in the password-store and cleaning of the migrated secrets will fix that in time.

In the process I gained much more confidence in configuring flatpak apps. I can decommission the keepassxc system including dealing with the sync conflicts (which was admittedly super easy with the merge database feature in KeepassXC). I no longer have to deal with giving the KeepassXC window a place on the desktop and autostarting it.

I am a bit puzzled about the host-apps referring to supporting browsers for which no extensions are available. This probably might warrant some additional investigation.

Big step forward

Samba Adventures with Guix

Peter Tillemans — Sat, 17 Aug 2024 00:00:00 +0200

Samba or CIFS file sharing is a finicky area at best, but widely used, especially since it was heavily pushed by Microsoft in the Windows ecosystem, This makes it widely used in corporate and NAS environments and even for Linux file sharing.

In this post I will look at some ways to use CIFS file sharing as a client. Personally I find hardly any uses for Samba file serving nowadays from my compute environments, with git, ssh, http(s), databases, MQTT, file synchronization services ... it hardly ever happens that I face a situation where I think : "Hey, I wish I could serve my files from my PC/VPS/VM/... with Samba". It is such a generic non-specific service I usually find a more opinionated data sharing service. Actually I find the same for NFS.

That being said, it is still ubiquitous to get enterprise data and to connect to NAS or remote hard disks.

CIFS Server Simulator

In practice setting up and connecting to a CIFS server is a frustrating experience due to a bewildering array of different protocols, security systems, credentials, ... .

To test out accessing a Samba server without having to deal with too many moving pieces at the same time I like to use a known fixed local basic samba server. This enables me to get working client - server configurations which I find easier to tweak to real life servers than starting from scratch.

Docker hub provides preconfigured images for Samba servers which are easy to use :

$ docker run --name test-smb -p 4139:139 --rm -p 4445:445 -v `pwd`/samples/:/mnt/export --rm -d dperson/samba -p -u "joe;schmoe" -s "export;/mnt/export/;yes;no;no;joe;;;Test Share"

This will start a samba server and listen on ports 4139 and 4445 so it does not clash if a server is running on the machine we're using and we do not need special privileges other than being in the docker group to run docker as a regular user. The -s ... option configures a share name export which is browsable, not readonly, not accessible by guest users and can only be accessed with the joe user account. This is to make the experience a bit more in line with usual real world configuration which are seldom as open as the default settings for shares. For more details, see the github repo for the image.

Because this is not a fun command line to type I like to put them in a Makefile in a folder with some support files

$ cd ...
$ mkdir test-smb
$ cd test-smb
$ mkdir samples
$ echo "Hello, Samba" >samples/hello.txt

and then add the Makefile

servers-start:
        docker run --name test-smb -p 4139:139 --rm -p 4445:445 -v `pwd`/samples/:/mnt/export --rm -d dperson/samba -p -u "joe;schmoe" -s "export;/mnt/export/;yes;no;no;joe;;;Test Share"

servers-stop:
        docker stop test-smb

Guix Notes for Docker

Docker (and podman too) need some OS support to talk to the kernel to create the namespaces to make the containers work. Even though podman does not need a daemon to run, it still needs some setuid helpers. I tend to mostly use docker because of habit and everyone else uses it and tends to be better supported and documented for my use cases. In any case I did not have any luck getting either to work in a local guix shell.

To enable docker on Guix-SD I have the following in my system configuration.

  ...
  (use-service-modules cups desktop docker networking ssh xorg)
  ...

(operating-system
  ...
  (users
   (cons*
    (user-account
     ...
     (supplementary-groups
      '( ... "docker")))  ;; enable access to docker daemon
    %base-user-accounts))
...
(packages
 (append
  (list
   ...
   ;; add docker command line tools
   docker
   ...
   ))))
...
(services
 (append
  (list
   ...
   ;; enable docker
   (service docker-service-type)

When using Guix on a host os, the native docker package should work fine.

Connecting using smbclient

Before mounting drives we need to ascertain our credentials are accepted by the Samba server. It makes no sense proceeding trying to mount shares if we cannot get past authentication and the overhead of configuring those shares and the Guix configuration rebuilding really impacts iteration speed when trying things out.

The most straightforward way to connect to a CIFS server is with the smbclient tool which is part of the samba package:

➜ guix shell samba

test-smb on  main via 🐃
➜ smbclient -L //localhost -p 4445 -U joe
Password for [WORKGROUP\joe]:

    Sharename       Type      Comment
    ---------       ----      -------
    export          Disk      Test Share
    IPC$            IPC       IPC Service (Samba Server)
SMB1 disabled -- no workgroup available

test-smb on  main [!] via 🐃  took 4s
➜ smbclient //localhost/export -p 4445 -U joe
Password for [WORKGROUP\joe]:
Try "help" to get a list of possible commands.
smb: \> ls
  .                                   D        0  Sat Aug 17 03:49:32 2024
  ..                                  D        0  Sat Aug 17 05:22:38 2024
  hello.txt                           N       13  Sat Aug 17 03:50:27 2024

                1912951596 blocks of size 1024. 998957000 blocks available
smb: \>

The guix shell samba creates a profile with the samba package installed which places `smbclient` on the path.

smbclient -L //servername lists the services offered by the server, in our case localhost to connect to our docker instance. The -p 4445 option selects the custom port we specified on our docker container. By default smbclient will use your login as uid, so we need to override it to the user we created when starting the docker container with -U joe .

The password is configured as schmoe.

We see the export folder is shared so we connect to it using `smbclient //localhost/export -p 4445 -U joe` and can list the contents. And of course we can upload, download and all the other terminal goodness offered by smbclient.

In practice it can be very fiddly to get the right username, password, Workgroup or Domain, port number, SMB protocol, etc dialed in. The obtuse messages often do not really help. The other methods make interpreting errors even harder, with the exception of programmatic access which is sometime surprisingly helpful in getting a connection going.

Save the credentials

The samba tools have a convention to save the credentials in a authentication file which is supported by most tools AFAICT.

To use this create a file .smbcredentials in your home folder. Well, it can be anything but I like that place as I typically only have my NAS to connect to using samba and it is for my home folder, backup folder, my Music and Movies folder and the like, i.e. stuff related to my user account, so I find it in its place in my home folder.

In it place the username, password and domain which worked with smbclient so they no longer need to be provided :

username=joe
password=schmoe
domain=WORKGROUP

Then we can use it:

$ smbclient //localhost/export -p 4445 -A ~/.smbcredentials

Try "help" to get a list of possible commands.
smb: \> ls
  .                                   D        0  Sat Aug 17 03:49:32 2024
  ..                                  D        0  Sat Aug 17 12:02:13 2024
  hello.txt                           N       13  Sat Aug 17 03:50:27 2024

                1912951596 blocks of size 1024. 998918312 blocks available
smb: \>

This save a lot of typing and we can use this file in the next stages and avoid spreading the credentials all over the disk. I am going to gloss over encrypting this info any further because I do not keep nuclear (or any other for that matter) secrets on my nas.

Mounting Shares with `mount.cifs`

Let's create a mount point in our test folder

$ mkdir mnt

and then mount the share with mount.cifs. This is part of the cifs-utils package.

ttest-smb on  main [?] via 🐃
➜ guix shell cifs-utils
The following derivation will be built:
  /gnu/store/2x7mmyrsnsf21aing02ass82899gm2yh-profile.drv

building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
building profile with 1 package...
est-smb on  main via 🐃
❮ sudo mount.cifs //localhost/export mnt -o credentials=/home/pti/.smbcredentials,port=4445

test-smb on  main [?] via 🐃
➜ ls mnt
hello.txt

test-smb on  main [?] via 🐃
➜

This mounts the share to the folder mnt. The -o option is used to point to the credentials file and the port number of our docker NAS simulator.

Checking with the regular mount command to see if it agrees we mounted the share:


❯ sudo mount -t cifs
//localhost/export on /home/pti/src/test-smb/mnt type cifs (rw,relatime,vers=3.1.1,cache=strict,username=joe,domain=WORKGROUP,uid=0,noforceuid,gid=0,noforcegid,addr=0000:0000:0000:0000:0000:0000:0000:0001,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1)

test-smb on  main [?] via 🐃
➜ sudo mount | grep cifs
/etc/auto.cifs on /nas type autofs (rw,relatime,fd=6,pgrp=1732,timeout=600,minproto=5,maxproto=5,indirect,pipe_ino=27049)
//localhost/export on /home/pti/src/test-smb/mnt type cifs (rw,relatime,vers=3.1.1,cache=strict,username=joe,domain=WORKGROUP,uid=0,noforceuid,gid=0,noforcegid,addr=0000:0000:0000:0000:0000:0000:0000:0001,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1)

Calling mount without arguments lists all mounted filesystems, however nowadays the output is so overwhelming that I prefer to use grep to narrow down the output to the filesystem type I am interested in. The same thing can be achieved by specifying the filesystem type with the -t option.

Configuring the mounts in the operating system.

The mount.cifs command is fine for testing and ad-hoc use but in practice I mostly (read 99+% of the time) want the same shares of the NAS mounted in the same place on my system. This is where the fstab file comes in. This file is read by the mount command at boot time and mounts the configured filesystems.

I do not recommend mounting CIFS shares at boot time, but configuring the shares in fstab is a good way to be able to mount them when needed with a quick mount -a command.

On my Tuxedo system I have in the /etc/fstab file

...
//nas.snamellit.com/home /home/pti/nas   cifs   rw,uid=1000,gid=100,credentials=/home/pti/.smbcredentials   0   0
//nas.snamellit.com/public /mnt/public   cifs   rw,uid=1000,gid=100,credentials=/home/pti/.smbcredentials   0   0
//nas.snamellit.com/multimedia /mnt/multimedia   cifs   rw,uid=1000,gid=100,credentials=/home/pti/.smbcredentials   0   0
...

The first column is the reference to the share as we've seen above. Then the mount point. The 3rd column indicates it is a cifs share, Then the options we've seen with smbmount. Here I have to play with the uid and gid to align my local user and group with the configuration on the nas. The last 2 columns are whether to include the mounts when dumping the filesystem and doing a filesystem check which will probably always be 0 as the CIFS server is responsible for that.

In theory this will mount the shares at boot time, and it probably does, however in my experience it seems they are unmounted on suspend and not remounted after resume. Or there is some timeout. I never investigated I must admit. I just do a quick mount -a before I need them and this works wonders.

On Guix-SD the file-systems are specified in the operating-system section of the system configuration :

(operating-system
 ...

(file-systems
 (cons*
  ...
  (file-system
    (device "//nas.snamellit.com/public")
    (options "uid=1000,gid=1000,credentials=/home/pti/.smbcredentials")
    (mount-point "/mnt/public")
    (type "cifs")
    (mount? #f)
    (create-mount-point? #t))
  (file-system
    (device "//nas.snamellit.com/multimedia")
    (options "uid=1000,gid=1000,credentials=/home/pti/.smbcredentials")
    (mount-point "/mnt/multimedia")
    (type "cifs")
    (mount? #f)
    (create-mount-point? #t))
  (file-system
    (device "//nas.snamellit.com/home")
    (options "uid=1000,gid=1000,credentials=/home/pti/.smbcredentials")
    (mount-point "/home/pti/nas")
    (type "cifs")
    (mount? #f)
    (create-mount-point? #t))
  %base-file-systems)))
 )

Which is just a straightforward translation of the fstab columns in Guix-ese.

Automatic Mounting with AutoFs

Of course the inefficiency of having to type mount -a almost on a weekly basis is unbearable and this inefficiency has to be addressed even if this means we cannot expect net positive effect in the coming millenia.

Enter autofs which is an awesome system to dynamically mount and unmount filesystems on an as needed basis. The kernel will notice when a mounted folder is accessed and ask the autofs daemon to mount the configured mount and unmount it after some timeout occurs without any activity.

This is transparant for the user other than a slight delay when accessing the folder the first time when it is unmounted.

This system is very flexible at it was clearly intended for far more ambitious use-cases than accessing your personal music library from your nas, but that does not mean we cannot strip it down and use it for our purposes.

What is a bit confusing is the configuration. It uses 3 different file types to configure the system.

the /etc/autofs.conf file which configures the daemon operating options, like the timeouts and the location of the main master file.
the master file(s), there is always a main file, but this can delegate to directory with additional parts of the file which can be assumed to be imported in the main file. Its main purpose is to map a parent folder of mount points to a map file.
the map files which map a subfolder in the folder from the line in the master file to a mount specification which will be used when that folder is accessed.

Example on the Tuxedo:

On my Tuxedo (running Tuxedo OS which is an Ubuntu 22.04 derivate) I have the following configuration:

/etc/autofs.conf:

...
[ autofs ]
#
# master_map_name - default map name for the master map.
#
master_map_name = /etc/auto.master
...

so the master map is in the file /etc/auto.master.

/etc/auto.master:
```
...
/-  /etc/autofs.direct  -ro
```

The "/-" is a special case which means "any" directory. In this case the folder name in the map file is assumed to be a fully qualified directory name instead of a subfolder in the folder mentioned in the first column. In this case the map file is /etc/autofs.direct and the default options are -ro .

/etc/autofs.direct:
```
/home/pti/nas -fstype=cifs,rw,noperm,vers=3.0,credentials=/home/pti/.smbcredentials    ://nas.snamellit.com/home
```
This shows that the mount point /home/pti/nas will mount the share //nas.snamellit.com/home of file system type cifs with the options (the rest of the 2nd column) as mount options.

Example on the Guix-SD desktop

Unfortunately there is not (yet) packaged support for an autofs service-type, so we have to make it ourselves.

First we define a configuration record type for our new service:

(define-record-type* <autofs-configuration>
  autofs-configuration make-autofs-configuration
  autofs-configuration?
  (pid-file autofs-configuration-pid-file
            (default "/var/run/autofs.pid"))
  (autofs-direct autofs-configuration-autofs-direct
                 (default (plain-file "autofs.direct" "/home/pti/autonas -fstype=cifs,rw,noperm,vers=3.0,credentials=/home/pti/.smbcredentials  ://nas.snamellit.com/home"))))

I should probably not put my actual configuration in the default, but it makes my life easier at the moment and I'll refactor it later. (Yeah, sure...)

Since the autofs service needs some boilerplate configuration files I generate them with an activation function:

(define (autofs-activation config)
  "Return the activation GEXP to create the config files for autofs"
  (with-imported-modules '((guix build utils))
                         #~(begin
                             (use-modules (guix build utils)
                                          (ice-9 textual-ports))

                             (define (touch file-name)
                               (call-with-output-file file-name (const #t)))

                             ;; 'sshd' complains if the authorized-key directory and its parents
                             ;; are group-writable, which rules out /gnu/store.  Thus we copy the
                             ;; authorized-key directory to /etc.
                             (call-with-output-file "/etc/autofs.conf"
                               (lambda (f) (put-string f "
[ autofs ]
master_map_name = /etc/auto.master
timeout = 300
")))
                             (call-with-output-file "/etc/auto.master"
                               (lambda (f) (put-string f (format #f "/-  ~a -ro\n" #$(autofs-configuration-autofs-direct config)))))

                             )))

You'll recognize the absolutely stripped down versions of the files mentioned above being generated. The /etc/auto.master file maps the directfs map file to the file generated in the configuration record.

This should tie up nicely the links between the 3 files.

Then I need a shepherd service to start the autofs daemon:

(define (autofs-shepherd-service config)
  (define pid-file (autofs-configuration-pid-file config))

  (list
   (shepherd-service
    (provision '(autofs))
    (documentation "AutoFS Service.")
    (requirement '(networking))
    (start #~(make-forkexec-constructor
              (list
               #$(file-append autofs "/sbin/automount")
               "-f" ;; run in foreground to give shepherd more control
               "-p" #$(autofs-configuration-pid-file config)
               "/etc/auto.master")
              #:pid-file #$(autofs-configuration-pid-file config)))
    (stop #~(make-kill-destructor))
    )))

We point it to the generated auto.master file from the activation function. (I could probably refactor this to use guix instantiated files but at this stage that extra level of indirection would only confuse me and make debugging harder. I find jumping through 3 files already hard enough to follow).

Then we need to put a bow around it and define an autofs-service-type.

(define autofs-service-type
  (service-type (name 'autofs)
                (description "Run the autofs daemon to automount folders on access.")
                (extensions
                 (list
                  (service-extension activation-service-type autofs-activation)
                  (service-extension shepherd-root-service-type autofs-shepherd-service)))
                (compose concatenate)
                (default-value (autofs-configuration)))
  )

This just extends the activation-service-type and the shepherd-root-service-type to run our initialisation code and ensure the autofs daemon gets started.

Then we can add this to our system configuration:

(operating-system
 ...
 (packages
  (append
   (list
    ...
    autofs)))
 (services
  (append
   (list
    ...
    (service autofs-service-type))
   ...)))

i.e. add the autofs package to install the daemon and support stuff and enable the service of type autofs-service-type.

Conclusion

I can now use files on my nas transparantly. I was a bit flippant on my reasons to enable autofs. The real reason was that I want to keep automatic backup copies of my forge running on an VPS somewhere on my NAS with a cron job, which means I would not be there to run a mount -a at the time. (I now realize I could do that as part of the cron job : 20/20 hindsight). In any case this is a major quality of life improvement. As a side effect, my music library now gets properly indexed and is made available on the default music player. Apparently I still use CIFS more than I care to admit.

The big problem with SMB/CIFS is getting the initial connection going. Once that is achieved and the credentials are safely stored away in credentials file, they can be easily reused with a lot less surprising things along the way.

It also pays to test things out in the smallest possible meaningful scope, in this case smbclient before adding more obfuscation layers on top of it, as that just adds more complexity, pitfalls and rabbit holes to get lost in. By going step by step, building on previous result I often get results faster (or at all) even if I have to do additional steps which turn out to no longer be needed in the final solution.

Workflows for Unix Password Store with Emacs and Shell

Peter Tillemans — Tue, 25 Jun 2024 00:00:00 +0000

Managing Secrets using Pass

On UNIX there is a well known tool to manage secrets called password-store or pass for short.

It is a very minimal tool, more to facilitate workflows that to do real work, very much in the UNIX philosophy. It stores all secrets in plain files in a folder structure. It does not care about what is in the files and encrypts them using a GPG public key so only the owner of the private key can decrypt them. It does offer special access to the first line so a password can be quickly fetched and copied to the clipboard or stdout or wherever some pass aware integration needs it.

Certain folders can be configured to use a different public keys to allow pragmatic secret delegation to different systems without exposing all secrets.

It leverages the gpg infrastructure for key management, distribution, unlocking with pinentry, caching with gpg-agent.

Installation

GUIX

Add password-store, gpg and pinentry to your package list.

The pinentry program provides a client to securely unlock your keys in a GUI and terminal environment. There are other options to make it better fit your environment, but this is fine for me.

Debian, Ubuntu et al

Add pass, gpg, gpg-agent and pinentry-gnome of pinentry-qt using

$ apt-get install pass gpg gpg-agent pinentry-qt

Both the gnome and qt versions at least fall back gracefully if no graphical environment are available.

The current selected pinentry program is provided as /usr/bin/pinentry and this link is managed by the usual alternatives machinery in debian based distros.

Emacs Integration

Emacs auth-source infrastructure supports pass out of the box.

Enable the Unix Password Store.

We have to make sure the password-store is added to the auth-sources. There is a handy function for that:

;; enable unix password-store
(auth-source-pass-enable)

We add that somewhere in the init.el before any secrets are needed.

Helper Function

Auth-sources is a flexible system and works like a database, i.e. you can query, browse through results and have multiple fields per secret.

Example:

(defun snam-password (host user)
  "Get password from the unix password store.

Searches the password file for a secret in the folder corresponding to
the HOST name given, which is the folder with the '/' replaced by a '.'.
The filename in the folder corresponds to the USER argument with a
'.pgp' extension."
  (auth-info-password
   (car (auth-source-search
         :max 1
         :host host
         :user user))))

`auth-source-search` returns a list of results, so we have to get the first entry with `car`. The secret is lightly obfuscated, hence the need to decode it with the `auth-info-password` function.

Luckily there is a function auth-source-pass-get to get a password from the password store which also follows the recommended conventions for multiple fields in a pass file.

(auth-source-pass-get 'secret "snamellit/znc")

The pseudo key `'secret` returns the first line of the password store entry which contains the password, per pass conventions.

Using in Emacs Configuration

For single secrets, like connecting to my znc IRC bouncer:

(erc-tls :id 'znc :server "********"
         :port "****" :user "xyz" :nick
         "foobar" :password (auth-source-pass-get 'secret "foobar/znc")))

For multifield secrets, like for google authentication, we can leverage the multiple fields in the password store. Here is my org-gcal configuration:

(setq org-gcal-client-id (auth-source-pass-get 'secret "snamellit/org-gcal-client")
      org-gcal-client-secret (auth-source-pass-get "id" "snamellit/org-gcal-client")
      org-gcal-fetch-file-alist '(("xyz@foobar.com" .  "~/org/schedule.org")))

Shell integration

DirEnv integration

When developping 12-factor or similar inspired apps, the configuration is passed using environment variables. Using .envrc files with direnv integration in the shell is a very smooth way to work in multiple projects.

It is of course less than ideal to have the secrets exposed in the .envrc files in your project tree even if it is in the .gitignore file, although that is infinitely better than having secrets end up in the git repository.

Secrets in the .envrc files can be easily moved to the password store by entering the folder. The following snippet prints the current value to the screen and then inserts it in the password store, and verifies it actually is entered correctly.

$ echo $FOO_BAR_PASSWORD
$ echo $FOO_BAR_PASSWORD | pass add -e foo/bar
$ pass foo/bar

and then editing the .envrc file from

...
FOO_BAR_PASSWORD=<some secret>
...

...
FOO_BAR_PASSWORD=$(pass foo/bar)
...

After modification you'll be asked to allow to read the new .envrc file content and you can check if it still works by comparing the password with the one printed previously.

Printing the passwords allows to fix any typos. If this are the only copies you have of them you might rug-pull yourself. Ideally it should be possible to quickly recreate secrets if you lose any, but reality is often far from ideal. Echoing all these passwords is also not ideal, but preferable over keeping a little black book of secrets. If these passwords are coming from another password manager it is not needed of course.

Do not forget to clear your terminal scroll back history with

$ clear

To help migration of .envrc files I created a bash script I stored in ~/.local/bin/envrc-to-pass:

#!/bin/bash

if [ "$#" -ne 2 ]; then
    echo "Usage: $0 <variable_name> <namespace>" >&2
    exit 1
fi

VAR_NAME=$1
SLUG=$(echo $VAR_NAME | sed 's/_/-/g' | tr '[:upper:]' '[:lower:]')
NAMESPACE=$2


echo "Moving variable $VAR_NAME to $NAMESPACE/$SLUG in password store"

SECRET=$(grep "$VAR_NAME=" .envrc | cut -d'=' -f2)
if (echo $SECRET | grep '^\$(pass.*)'); then
    echo "secret already migrated"
else
    echo $SECRET | pass insert -e $NAMESPACE/$SLUG
fi

sed -i "s#$VAR_NAME=.*#$VAR_NAME=\\\$(pass $NAMESPACE\/$SLUG)#" .envrc

This makes short work of migrating projects to use the password-store.

Tips

Entering passwords on the terminal

Usually invoking `pass add foobar/baz` will ask to enter the password and confirm it in the shell.

However when piping a secret into the password-store with

echo FOO_BAR_PASSWORD | pass add foo/bar

will silently fail although the man pages tell that `pass add` will read from stdin. It is not clear IMO that you have to specify the -e flag like :

echo FOO_BAR_PASSWORD | pass -e add foo/bar

to suppress the confirmation and make it work as expected.

Getting fields from multi field secrets

Often secrets come in multiple parts which are nice to be stored in a single entry in order not to complicate the tree. The pass documentation suggests:

<password or main secret>
field1: <some data>
field2: <more data>

the main secret is just the first line, and can have the same structure as the other lines, if that makes more sense.

e.g. for a google integration:

service-account: 1234567-abcdefghijklm@developer.gserviceadmin.com(some-project)
email: xyz@foobar.com
private-key: ...

In this case it is useful to document which kind of secret it is as it could also be an api-key, or a refresh-token, or whatever part of the authentication menagerie that Google offers.

To get these fields individually I use:

GOOGLE_EMAIL=$(pass foo/bar | awk '/^email:/ {print $2})

in emacs this syntax is supported directly and the same info can be fetched with

(let
    ((google-email (auth-source-pass-get "email" "foo/bar")))
  ... )

KeePassXC from flatpak

Peter Tillemans — Sun, 12 May 2024 00:00:00 +0000

KeePassXC from flatpak

More and more browsers are coming packaged in containers like flatpak or snaps. Especially the ones of the chromium variety.

Since browsers are at the best of times scary things with arbitrary code being executed in every webpage runnning them in a sandbox makes a lot of sense.

However the sandbox also impacts communication with helpers like the password manager KeepassXC.

In order to keep things reproducible on many machines, if we're going to fight with flatpak we might as well run keepassxc as a flatpak.

$ flatpak install org.keepassxc.KeePassXC

Allow access to the socket file

Allow read-only access to the keepassxc socket at $XDG_RUNTIME_DIR/org.keepassxc.KeePassXC.BrowserServer:ro :

$ flatpak override --user \
  --filesystem=xdg-run/app/org.keepassxc.KeePassXC:ro \
  com.google.Chrome

this needs to be done for each flatpak application needing access to keepassxc.

keepassxc-proxy

There is a statically compiled rust based socket proxy.

rust based keepassxc proxy

It can be compiled with MUSL for a fully static executable.

Place it together with the configuration file

Configure Chrome

The Chrome flatpak can be configured in the folder *~/.var/app/com.google.Chrome :

❯ ls .var/app/com.google.Chrome/config/google-chrome/NativeMessagingHosts 
keepassxc-proxy  org.keepassxc.keepassxc_browser.json

the native messaging config file needs to be configured for the local user:

{
    "allowed_origins": [
        "chrome-extension://pdffhmdngciaglkoonimfcmckehcpafo/",
        "chrome-extension://oboonakemofpalcgghocfoadofidjkkk/"
    ],
    "description": "KeePassXC integration with native messaging support",
    "name": "org.keepassxc.keepassxc_browser",
    "path": "/home/pti/.var/app/com.google.Chrome/config/google-chrome/NativeMessagingHosts/keepassxc-proxy",
    "type": "stdio"
}

and do not forget to give access to the socket

$ flatpak override --user --filesystem=xdg-run/app/org.keepassxc.KeePassXC:ro com.google.Chrome

Notes;

the path field needs to contain the full path including the username AFAICT.
The extension id can be stolen from the URL box when going into the settings pane of the extension. It has changed in the past.

Configure Brave

we need again an org.keepassxc.keepassxc_browser.json file in ~/.var/app/com.brave.Browser/config/BraveSoftware/Brave-Browser/NativeMessagingHosts/ folder together with an executable version of the proxy compiled above.

{
    "allowed_origins": [
        "chrome-extension://pdffhmdngciaglkoonimfcmckehcpafo/",
        "chrome-extension://oboonakemofpalcgghocfoadofidjkkk/"
    ],
    "description": "KeePassXC integration with native messaging support",
    "name": "org.keepassxc.keepassxc_browser",
    "path": "/home/pti/.var/app/com.brave.Browser/config/BraveSoftware/Brave-Browser/NativeMessagingHosts/keepassxc-proxy",
    "type": "stdio"
}

You can copy it from the Google Chrome config but do not forget to update the path to the proxy.

Now we still need to give access to brave to the keepassxc socket:

$ flatpak override --user --filesystem=xdg-run/app/org.keepassxc.KeePassXC:ro com.brave.Browser

Configure Firefox

I have firefox running native on my laptop so these shenanigans are not needed. However the example I based most of was for firefox so I assume it works similar. Note that the native messaging file is different for firefox.

{
    "allowed_extensions": [
        "keepassxc-browser@keepassxc.org"
    ],
    "description": "KeePassXC integration with native messaging support, workaround for flatpaked Firefox, see https://is.gd/flatpakFirefoxKPXC",
    "name": "org.keepassxc.keepassxc_browser",
    "path": "/home/pti/.var/app/org.mozilla.firefox/.mozilla/native-messaging-hosts/keepassxc-proxy",
    "type": "stdio"
}

it uses allowed_extensions iso allowed_origin

And give access to the socker file to firefox

$ flatpak override --user --filesystem=xdg-run/app/org.keepassxc.KeePassXC:ro org.mozilla.firefox

references

Intercepting Bluetooth Keyboard Events

Peter Tillemans — Thu, 14 Dec 2023 00:00:00 +0000

Intercepting Bluetooth Keyboard Events

I like keyboards and I like custom keyboards. However many do not allow sufficient configuration to let a key be Ctrl on hold and Esc on top. Notably the HHKB.

USB devices are nicely listed in /dev/input/by-id/ folder but on my system the event device is anonymously placed in /dev/input/.

Figuring out the name

❯ for dev in /dev/input/*; do udevadm info --attribute-walk $dev; done | grep -A10 -B10 Keyboard
Unknown device "/dev/input/by-id": No such device
Unknown device "/dev/input/by-path": No such device
    ATTRS{capabilities/led}=="1f"
    ATTRS{capabilities/msc}=="10"
    ATTRS{capabilities/rel}=="0"
    ATTRS{capabilities/snd}=="0"
    ATTRS{capabilities/sw}=="0"
    ATTRS{id/bustype}=="0005"
    ATTRS{id/product}=="0021"
    ATTRS{id/vendor}=="04fe"
    ATTRS{id/version}=="0001"
    ATTRS{inhibited}=="0"
    ATTRS{name}=="HHKB-Hybrid_1 Keyboard"
    ATTRS{phys}=="04:33:c2:f4:e9:70"
    ATTRS{power/control}=="auto"
    ATTRS{power/runtime_active_time}=="0"
    ATTRS{power/runtime_status}=="unsupported"
    ATTRS{power/runtime_suspended_time}=="0"
    ATTRS{properties}=="0"
    ATTRS{uniq}=="cb:7a:7c:03:2d:f2"

for other keyboards a bit of finagling for searching with grep might be needed or pipe into less and scroll till you find it.

Now I can add it to my */etc/interception/udevmon.d/ctrl_tap_esc.yaml :

- JOB: "intercept -g $DEVNODE | dual-function-keys -c /etc/interception/dual-function-keys.yaml | uinput -d $DEVNODE"
  DEVICE:
    NAME: "PFU Limited HHKB-Hybrid"
    EVENTS: 
      EV_KEY: [KEY_LEFTCTRL]
- JOB: "intercept -g $DEVNODE | dual-function-keys -c /etc/interception/dual-function-keys.yaml | uinput -d $DEVNODE"
  DEVICE:
    NAME: "PFU Limited HHKB-Hybrid Keyboard"
    EVENTS: 
      EV_KEY: [KEY_LEFTCTRL]
- JOB: "intercept -g $DEVNODE | dual-function-keys -c /etc/interception/dual-function-keys.yaml | uinput -d $DEVNODE"
  DEVICE:
    NAME: "HHKB-Hybrid_1 Keyboard"
    EVENTS: 
      EV_KEY: [KEY_LEFTCTRL]

The first 2 are for USB but that is still a work in progress

Snamellit - linux

Measuring CI budgets with Guix and Grafana before setting quotas

Guix owns the layout

Put processes in the group before they drop privileges

From /sys/fs/cgroup to Grafana

Rollout, not just reconfigure

Install GUIX on Macbook 12

Motivation

Installation Process

Creating Installation Media

Installation Process

Crypto Setup

add keyfile to initrd

Tell kernel to use the keyfile

Apply it

Fixing the keyboard

Post Installation

System does not allow to login after sleep

Sound : no sound from speakers.

Things I would do differently

Screenshot in Sway with GUIX

Screenshots in Sway with Guix Home Manager

Output Files

Key Bindings

Deploying the configuration

Testing it Out

Consolidating Secrets in Pass

Background

Preparation

Migration of the KeepassXC data

Integration with iOS for my iPhone

Integration with FireFox

Chromium Support

Enabling Chrome Support, now with more Flatpak!

Conclusion

Samba Adventures with Guix

CIFS Server Simulator

Guix Notes for Docker

Connecting using smbclient

Save the credentials

Mounting Shares with mount.cifs

Configuring the mounts in the operating system.

Automatic Mounting with AutoFs

Example on the Tuxedo:

Example on the Guix-SD desktop

Conclusion

Workflows for Unix Password Store with Emacs and Shell

Table of Contents

Managing Secrets using Pass

Installation

GUIX

Debian, Ubuntu et al

Emacs Integration

Enable the Unix Password Store.

Helper Function

Using in Emacs Configuration

Shell integration

DirEnv integration

Tips

Entering passwords on the terminal

Getting fields from multi field secrets

KeePassXC from flatpak

Table of Contents

KeePassXC from flatpak

Allow access to the socket file

keepassxc-proxy

Configure Chrome

Configure Brave

Configure Firefox

references

Intercepting Bluetooth Keyboard Events

Intercepting Bluetooth Keyboard Events

Figuring out the name

From `/sys/fs/cgroup` to Grafana

Mounting Shares with `mount.cifs`