How to get the best out of your Yubikey with GPG

I’ve been using a set of Yubikeys for some time now, not just for 2FA but for SSH authentication, remote code signing and password storage too. It’s worked so well (and required such significant effort to get working) that I thought I’d write about how I did it and what makes my approach secure.

GPG, or GnuPG – now in version 2.x implements the OpenPGP email encryption standard and allows the use of hardware smartcards such as the Yubikey 5.

This guide is applicable to most Linux distributions and MacOS. I’m currently using Ubuntu 20.04 and MacOS Mojave.

A reference implementation of a system that supports this all automatically can be found in my dotfiles.

Initialisation

To start, you’ll need to install ykman and gnupg together with the right interface software; pcscd and scdaemon on Ubuntu.

In order to use your Yubikey for everything in this article, it has to contain the only copy of your GPG private keys; GPG must have the public parts and be configured to use the Yubikey as an OpenPGP smartcard.

With OpenPGP/gnupg there are many different possible key topologies and crypto settings. What you want will depend on what you need the configuration for.

OpenPGP setup

I recommend you follow this setup process in livecd session in order to mitigate the risk of the private keys being intercepted by a bad actor before they are copied to the yubikey.¹

GnuPG2 classifies keys by their capability; you’ll notice (some of) the characters [CSEA] alongside listed secret keys with GnuPG2. Whilst fundamentally the private keys are just a number of bits, GnuPG2 can earmark each key with one or more capabilities upon generation such that it can only ever be used for the allocated purpose thereafter. The key capabilities are: [C]ertificaiton, [S]igning, [E]ncryption and [A]uthentication.

As I understand, it is simply good cryptographic practice to have single-purpose keys; you should not, for instance, use the same key to authenticate as you should to encrypt. The reasoning behind this is presumably to mitigate some kind of theoretical attack.

GnuPG2 historically has in interface designed for (unusual) humans, not machines; until recently it was difficult to script. Since version 2.2, there is an interface to generate and set up keys in non-interactive mode. As such I created a script called gpg-keygen to generate the following topology for you:

1. One master key (CS) (4096 bit RSA)
2. S subkey (4096 bit RSA)
3. E subkey (4096 bit RSA)
4. A subkey (4096 bit RSA)

To use it, run the script in an empty directory. The master key has a longer validity and can be used to revoke compromised keys² as well as create new, certified subkeys. Store it encrypted (with all the other generated files) on a flash drive in a physical safe a with a backup copy in another safe; it’s sensitive.

The remaining 3 keys are for signing, encryption and authentication respectively. They are as big as a Yubikey 5 will allow, and are of the RSA type as the Yubikey 5 does not support elliptic curves.³

Next, you’ll need to copy the contents of gnupghome to ~/.gnupg – on the live session. GnuPG2 should then be able to load and use the keys, though before use the private subkeys need to be moved to the Yubikey.

Now is a good time to change the default OpenPGP user and admin pin from 123456 and 123435678 respectively. Steps:

1. Run gpg2 --card-edit <email>
2. Enter admin
3. Enter passwd – change PIN and admin PIN as prompted

To move the keys, use gpg --edit-key <email> and enter the following commands:

1. toggle
2. For each key (S,E,A) select one by toggling the asterisk one at a time with the key command (1,2,3 respectively)
3. When a key is selected, run keytocard to copy the key to the correct slot on the yuibkey (there are 3, one each for S,E,A)
4. save then exit. If you don’t save, the private keys will persist locally which defeats the purpose of using the yubikey.

The private keys are now on your yubikey, and no longer exist in ~/.gnupg. The next step is to harvest the public parts of the key to initialise your target machine.

In order to use your new identity on the target machine, you’ll need to import the public keys and give the master key “ultimate trust.” This might sound odd but this means GPG will implicitly trust any sub-key certified by your master key so you can use them.

I prefer a clean-room approach in this case to ensure that not private keys are copied to the target machine; in addition it makes it clear what steps are necessary to set up new machines, too. Bear in mind that if you use the web-of-trust and intend to keep other peoples public keys in GnuPG, you’ll need to transfer them separately.

To make GPG trust your master key ultimately, you’ll need to use the trusted-key setting in ~/.gnupg/gpg.conf. On the live session, you’ll notice a file called gpg-keyid.txt which contains your master key ID⁴. You can append it to your GPG config file with this scriptable example from my dotfiles, user-configuration-naggie.sh:

cat <<EOF >> ~/.gnupg/gpg.conf
trusted-key 9D37503A7DA6F9B6
EOF

Of course you should substitute my key ID for yours.

Next, you’ll need to import the public portions of your newly generated keys. These can be found in gpg-pubkeys.asc on the live session in ASCII-guarded format. It was exported by my gpg-keygen script using gpg2 --export --armor > gpg-pubkeys.asc. The keys can be piped into gpg2 --quiet --import – see https://github.com/naggie/dotfiles/blob/master/user-configuration-naggie.sh#L43 for reference.

Finally, tt’s important to install a hardened gpg.conf to ~/.gnupg/gpg.conf. GnuPG2, by default, will use some relatively weak ciphers presumably for the sake of compatibility.

General Yubikey settings

I recommend disabling OTP mode (unless you like injecting random OTP passwords into chat or the terminal all the time). ykpersonalise can do this, but can become locked out as it relies on OTP mode. To disable OTP mode: ykman mode FIDO+CCID.

It’s important to enable touch-to-everything; this is a fundamental concept – malicious programs are incapable of physically pressing a button.

1. ykman openpgp set-touch enc on – decrypting passwords will now require a touch
2. ykman openpgp set-touch aut on – after this you’ll have to touch the yubikey to use SSH
3. ykman openpgp set-touch sig on – signing commits will now require a touch

SSH authentication

There are now actually three ways of using your Yubikey to authenticate to an OpenSSH server:

1. Using it as an OpenPGP Smartcard with gnupg2 as an SSH-agent (covered here)
2. Using it in PIV mode⁵
3. Using it as a U2F device with OpenSSH 8.2+

In terms of convenience, if you control the server U2F mode might be the best; certainly the least expensive as U2F devices are prolific. Using the PIV mode is a good option of you don’t want to mess with GPG.⁶

In order to use option (1), after setting up GnuPG as per the previous section you’ll have to:

1. Export your authentication public key in SSH format. You can do this with gpg2 --export-ssh-key <email> and copy it to ~/.ssh/authorized_keys on your remote machines as normal.
2. Configure your environment to use GnuPG2 as an SSH agent (only locally, overriding system agent)

To set up GPG as an ssh agent, I recommend use of the following function in your .bashrc/ or .zshrc. It can even be set as a precmd hook to automatically update the environment – see the section on Tmux below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

function _update_agents {
    # take over SSH keychain (with gpg-agent soon) but only on local machine, not remote ssh machine
    # keychain used in a non-invasive way where it's up to you to add your keys to the agent.
    if [ ! "$SSH_CONNECTION" ] && which gpg-connect-agent &>/dev/null; then
        export SSH_AUTH_SOCK=$(gpgconf --list-dirs | grep agent-ssh-socket | cut -f 2 -d :)
        # start GPG agent, and update TTY. For the former only, omit updatestartuptty
        # ssh-agent protocol can't tell gpg-agent/pinentry what tty to use, so tell it
        # if GPG agent has locked up or there is a stale remote agent, remove
        # the stale socket and possible local agent.
        if ! timeout -k 2 1 gpg-connect-agent updatestartuptty /bye > /dev/null; then
            echo "Removing stale GPG agent"
            socket=$(gpgconf --list-dirs | grep agent-socket | cut -f 2 -d :)
            test -S $socket && rm $socket
            killall -KILL gpg-agent 2> /dev/null
            # try again
            timeout -k 2 1 gpg-connect-agent updatestartuptty /bye > /dev/null
        fi
    fi
}

This function overrides the system SSH agent and replaces it with a GPG agent – but only if the current session isn’t a remote one in order to make sure agent forwarding is not clobbered. Running any GnuPG2 command will start a daemonised agent if one is not running.

As described in the comments, there’s also a bit of code to automatically recover an hung agent; unfortunately this is necessary to recover from dead sockets that happen time to time.

Note that GPG can ingest regular SSH keys into its own store with ssh-add – assuming you’re running a GPG agent. However, these keys won’t end up on the Yubikey.

GPG agent forwarding

If you’re developing on a remote machine, you’re probably familiar with forwarding your SSH agent to allow the use of Git with local SSH credentials. GPG is capable of a similar mechanism, allowing commits on a remote machine to be signed using your local Yubikey. GPG uses a unix socket for the agent connection, and a special restricted “extra” socket for remote use. The official guide suggests 2 methods, depending on whether your version of OpenSSH supports unix socket forwarding or not. Both methods are flawed:

1. The first method (recommended for OpenSSH older than 6.7) suggests using netcat to forward the unix socket over a TCP socket. This is insecure, because anyone else on the remote host could connect to the forwarded TCP port and use your forwarded agent. This might be OK if you’re the only use on a given server but it clearly isn’t generally.
2. OpenSSH 6.7 and above can forward unix sockets, so the next recommendation is to do so natively. However, the location of the local and remote sockets is dependent on the version of GPG, your OS, and your username. As such, a common workaround is to inspect every remote host and add a configuration to the SSH config per host. It also requires server configuration to delete the stale remote socket automatically. Both are manual steps.

Worse, if you use the remote machine locally too (I do for my workshop PC) you’ll find that the stale socket can interfere with any local GPG agent.

A better solution

My solution (after a fair bit of experimentation) was to create an SSH wrapper that probes the remote host to discover the correct location of the socket before killing any remote GPG agent and connecting the intended session. Combined with a pre-command hook that wires up the correct SSH, GPG and X sockets, the result is a seamless experience. See the section on tmux for more information.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

function gssh {
    echo "Preparing host for forwarded GPG agent..." >&2

    # prepare remote for agent forwarding, get socket
    # Remove the socket in this pre-command as an alternative to requiring
    # StreamLocalBindUnlink to be set on the remote SSH server.
    # Find the path of the agent socket remotely to avoid manual configuration
    # client side. The location of the socket varies per version of GPG,
    # username, and host OS.
    remote_socket=$(cat <<'EOF' | command ssh -T "$@" bash
        set -e
        socket=$(gpgconf --list-dirs | grep agent-socket | cut -f 2 -d :)
        # killing agent works over socket, which might be dangling, so time it out.
        timeout -k 2 1 gpgconf --kill gpg-agent || true
        test -S $socket && rm $socket
        echo $socket
EOF
)
    if [ ! $? -eq 0 ]; then
        echo "Problem with remote GPG. use ssh -A $@ for ssh with agent forwarding only." >&2
        return
    fi

    if [ "$SSH_CONNECTION" ]; then
        # agent on this host is forwarded, allow chaining
        local_socket=$(gpgconf --list-dirs | grep agent-socket | cut -f 2 -d :)
    else
        # agent on this host is running locally, use special remote socket
        local_socket=$(gpgconf --list-dirs | grep agent-extra-socket | cut -f 2 -d :)
    fi

    if [ ! -S $local_socket ]; then
        echo "Could not find suitable local GPG agent socket" 2>&1
        return
    fi

    echo "Connecting..." >&2
    ssh -A -R $remote_socket:$local_socket "$@"
}

$SSH_CONNECTION is set only when the session is remote; in which case the special remote socket is used, else the full local one. This allows chaining of agent forwarding so you can still use your local GPG via a bastion host.

I call my wrapper gssh rather than wrapping the ssh command. Being separate from the normal ssh command, it allow allows you to be mindful about when you forward your mgent – forwarding your GPG agent carries similar risks to forwarding your SSH agent – an administrator on the remote host could use your local GPG agent to further mitigate the risk, enable touch settings on your yubikey as documented below.

Configuration for code signing

To enable code signing with Git + Github:

1. Upload the output of gpg --export --armor <email> (your GPG public keys) to https://github.com/settings/keys.
2. Tell git which key to use: git config --global user.signingkey <email>
3. Tell git to sign always: git config --global commit.gpgsign true

Password management

I use the standard unix password manager which is compatible with GnuPG2 out of the box. It is powered by git so syncing passwords is easy. I use browserpass to reduce the effort required when used with Firefox. Documentation is well written and usage is straightforward, so I won’t detail the steps here.

My dotfiles copy the passwords over to my PCs when provisioning and synchronising – so long as my yubikey is connected, even new PCs can clone and decrypt my password repository.

One piece of advice: make sure touch to decrypt is enabled, such that passwords cannot be decrypted without your human approval. To do this, see the first section on initialisation.

Seamless tmux integration

When using tmux you’ll find that it’s necessary to re-spawn existing shells in order to use SSH agent / X forwarding when reconnecting. This is irritating, so I’ve written bash and ZSH precmd hooks to take care of updating the environment transparently before every command.

precmd hooks allow the execution of shell commands immediately before each interactive command is run. This gives us the opportunity to modify the environment of the command to be run. Helpfully this means we can make sure the GPG, SSH and X forwarding is set up always, regardless of how old a particular tmux shell is.

Zsh has a precmd hook by default, bash does not – though it’s possible with bash-preexec.

To enable this behaviour, simply set _update_agents (defined above) to be called by the precmd hook after a new function _tmux_update_env:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

    function _tmux_update_env {
        # tmux must be running
        [ "$TMUX" ] || return

        # update current shell to parent tmux shell (useful for new SSH connections, x forwarding, etc)
        eval $(tmux show-environment -s | grep 'DISPLAY\|SSH_CONNECTION\|SSH_AUTH_SOCK')
    }

    function preexec() {
        _tmux_update_env
        _update_agents
    }

This new function inspects the environment of the new parent tmux client and relays the appropriate X and SSH variables.

Conclusion

Using GnuPG2 with a Yubikey can be secure and convenient. The set up is rather involved, but scripting it all as part of managing dotfiles means it becomes quick and easy.

Thanks to my dotfiles I’m able to get a new installation running and configured within 30 minutes unattended – packages are installed, followed by system configuration and finally user configuration. As part of user configuration, GPG is set up to use my identity with the private keys on my Yubikey. See user-configuration-naggie.sh to see how this is done.

Historically I had to set up a new or existing SSH/GPG identity after the provisioning; this was a rather tedious step considering the number of servers and services I have and use; as a compromise the SSH key could be moved on a flash drive, but that’s not ideal. The solution is to automate the association of the private keys on my Yubikey with their respective public keys within my gnupg2 database.

I now have a convenient way of securely accessing secrets upon provisioning a new host. Now, my dotfiles will detect my yubikey and initialise the following:

• My notes, a git repository in ~/notes (I have a script to push/pull to prevent commit fatigue)
• My task management database to ~/.dstask (powered by dstask)
• My passwords to ~/.password-store (using the standard unix password manager detailed above)

See also: yubikey.md for some older, more general notes on the uses of yubikeys. In addition, my dotfiles repository implements all of this automatically so can be used as a reference.

Thanks for reading! If you have comments or like this article, please post or upvote it on Hacker news, Twitter, Hackaday, Lobste.rs, Reddit and/or LinkedIn.

Please email me with any corrections or feedback.