Workflows for Unix Password Store with Emacs and Shell

Peter Tillemans — Tue, 25 Jun 2024 00:00:00 +0000

Managing Secrets using Pass

On UNIX there is a well known tool to manage secrets called password-store or pass for short.

It is a very minimal tool, more to facilitate workflows that to do real work, very much in the UNIX philosophy. It stores all secrets in plain files in a folder structure. It does not care about what is in the files and encrypts them using a GPG public key so only the owner of the private key can decrypt them. It does offer special access to the first line so a password can be quickly fetched and copied to the clipboard or stdout or wherever some pass aware integration needs it.

Certain folders can be configured to use a different public keys to allow pragmatic secret delegation to different systems without exposing all secrets.

It leverages the gpg infrastructure for key management, distribution, unlocking with pinentry, caching with gpg-agent.

Installation

GUIX

Add password-store, gpg and pinentry to your package list.

The pinentry program provides a client to securely unlock your keys in a GUI and terminal environment. There are other options to make it better fit your environment, but this is fine for me.

Debian, Ubuntu et al

Add pass, gpg, gpg-agent and pinentry-gnome of pinentry-qt using

$ apt-get install pass gpg gpg-agent pinentry-qt

Both the gnome and qt versions at least fall back gracefully if no graphical environment are available.

The current selected pinentry program is provided as /usr/bin/pinentry and this link is managed by the usual alternatives machinery in debian based distros.

Emacs Integration

Emacs auth-source infrastructure supports pass out of the box.

Enable the Unix Password Store.

We have to make sure the password-store is added to the auth-sources. There is a handy function for that:

;; enable unix password-store
(auth-source-pass-enable)

We add that somewhere in the init.el before any secrets are needed.

Helper Function

Auth-sources is a flexible system and works like a database, i.e. you can query, browse through results and have multiple fields per secret.

Example:

(defun snam-password (host user)
  "Get password from the unix password store.

Searches the password file for a secret in the folder corresponding to
the HOST name given, which is the folder with the '/' replaced by a '.'.
The filename in the folder corresponds to the USER argument with a
'.pgp' extension."
  (auth-info-password
   (car (auth-source-search
         :max 1
         :host host
         :user user))))

`auth-source-search` returns a list of results, so we have to get the first entry with `car`. The secret is lightly obfuscated, hence the need to decode it with the `auth-info-password` function.

Luckily there is a function auth-source-pass-get to get a password from the password store which also follows the recommended conventions for multiple fields in a pass file.

(auth-source-pass-get 'secret "snamellit/znc")

The pseudo key `'secret` returns the first line of the password store entry which contains the password, per pass conventions.

Using in Emacs Configuration

For single secrets, like connecting to my znc IRC bouncer:

(erc-tls :id 'znc :server "********"
         :port "****" :user "xyz" :nick
         "foobar" :password (auth-source-pass-get 'secret "foobar/znc")))

For multifield secrets, like for google authentication, we can leverage the multiple fields in the password store. Here is my org-gcal configuration:

(setq org-gcal-client-id (auth-source-pass-get 'secret "snamellit/org-gcal-client")
      org-gcal-client-secret (auth-source-pass-get "id" "snamellit/org-gcal-client")
      org-gcal-fetch-file-alist '(("xyz@foobar.com" .  "~/org/schedule.org")))

Shell integration

DirEnv integration

When developping 12-factor or similar inspired apps, the configuration is passed using environment variables. Using .envrc files with direnv integration in the shell is a very smooth way to work in multiple projects.

It is of course less than ideal to have the secrets exposed in the .envrc files in your project tree even if it is in the .gitignore file, although that is infinitely better than having secrets end up in the git repository.

Secrets in the .envrc files can be easily moved to the password store by entering the folder. The following snippet prints the current value to the screen and then inserts it in the password store, and verifies it actually is entered correctly.

$ echo $FOO_BAR_PASSWORD
$ echo $FOO_BAR_PASSWORD | pass add -e foo/bar
$ pass foo/bar

and then editing the .envrc file from

...
FOO_BAR_PASSWORD=<some secret>
...

...
FOO_BAR_PASSWORD=$(pass foo/bar)
...

After modification you'll be asked to allow to read the new .envrc file content and you can check if it still works by comparing the password with the one printed previously.

Printing the passwords allows to fix any typos. If this are the only copies you have of them you might rug-pull yourself. Ideally it should be possible to quickly recreate secrets if you lose any, but reality is often far from ideal. Echoing all these passwords is also not ideal, but preferable over keeping a little black book of secrets. If these passwords are coming from another password manager it is not needed of course.

Do not forget to clear your terminal scroll back history with

$ clear

To help migration of .envrc files I created a bash script I stored in ~/.local/bin/envrc-to-pass:

#!/bin/bash

if [ "$#" -ne 2 ]; then
    echo "Usage: $0 <variable_name> <namespace>" >&2
    exit 1
fi

VAR_NAME=$1
SLUG=$(echo $VAR_NAME | sed 's/_/-/g' | tr '[:upper:]' '[:lower:]')
NAMESPACE=$2


echo "Moving variable $VAR_NAME to $NAMESPACE/$SLUG in password store"

SECRET=$(grep "$VAR_NAME=" .envrc | cut -d'=' -f2)
if (echo $SECRET | grep '^\$(pass.*)'); then
    echo "secret already migrated"
else
    echo $SECRET | pass insert -e $NAMESPACE/$SLUG
fi

sed -i "s#$VAR_NAME=.*#$VAR_NAME=\\\$(pass $NAMESPACE\/$SLUG)#" .envrc

This makes short work of migrating projects to use the password-store.

Tips

Entering passwords on the terminal

Usually invoking `pass add foobar/baz` will ask to enter the password and confirm it in the shell.

However when piping a secret into the password-store with

echo FOO_BAR_PASSWORD | pass add foo/bar

will silently fail although the man pages tell that `pass add` will read from stdin. It is not clear IMO that you have to specify the -e flag like :

echo FOO_BAR_PASSWORD | pass -e add foo/bar

to suppress the confirmation and make it work as expected.

Getting fields from multi field secrets

Often secrets come in multiple parts which are nice to be stored in a single entry in order not to complicate the tree. The pass documentation suggests:

<password or main secret>
field1: <some data>
field2: <more data>

the main secret is just the first line, and can have the same structure as the other lines, if that makes more sense.

e.g. for a google integration:

service-account: 1234567-abcdefghijklm@developer.gserviceadmin.com(some-project)
email: xyz@foobar.com
private-key: ...

In this case it is useful to document which kind of secret it is as it could also be an api-key, or a refresh-token, or whatever part of the authentication menagerie that Google offers.

To get these fields individually I use:

GOOGLE_EMAIL=$(pass foo/bar | awk '/^email:/ {print $2})

in emacs this syntax is supported directly and the same info can be fetched with

(let
    ((google-email (auth-source-pass-get "email" "foo/bar")))
  ... )

Issue with SBCL console output in Sly

Peter Tillemans — Sat, 13 Apr 2024 00:00:00 +0000

Inconsistent output in REPL

When doing the examples in the Practical Common Lisp book I had confusing output in the repl when prompting the user for CDs using SBCL.

; Dedicated output stream setup (port 46185)
; Redirecting all output to this MREPL
; SLY 1.0.43 (#<MREPL mrepl-1-1>)
CL-USER> (prompt-for-cd)
Title: Unplugged
Eric Clapton 
8
y

Apparently a similar issue was first found on MacOSX and later confirmed on Arch.

The suggestion was given to disable passing the output via a socket to have higher performance that via the protocol by adding

(setf slynk:*use-dedicated-output-stream* nil)

Since I rather have something working than it being highly performant and not-working I followed the suggestion which fixed the issue:

CL-USER> (prompt-for-cd)
Title: Bar 
Artist: Foo Fighters
Rating: 8
Ripped: y

y (y or n) y
(:TITLE "Bar " :ARTIST "Foo Fighters" :RATING 8 :RIPPED T)
CL-USER>

so I can continue with the chapter.

Snamellit - lisp