Two hardware tokens, one wrong guess: why SSH keeps asking for your PIN twice

A developer running two YubiKeys, one on a keyring and one in a drawer, both enrolled everywhere that matters, spent months watching `git push` ask for a PIN, reject it, and then ask again ^[1][2]. The variable was not the network or the repo but which of the two tokens happened to be in the USB port, and the root cause was an absence: no `~/.ssh/config` at all ^[2][5].

Without a config, ssh offers the agent's keys in the order the agent hands them over, and the server takes the first one it recognises ^[5]. Both credentials were `sk-ssh-ed25519` keys loaded automatically into the gnome-keyring agent, listed as `yk1-12345678` and `yk2-12345679` ^[4]. GitHub had both enrolled, so it accepted the first offer every time, which means ssh committed to yk1 before anything had checked whether the attached token could satisfy that commitment ^[6]. With yk1 in the port the guess is right and you get one prompt; with yk2 in the port, ssh asks the plugged-in token to sign for a credential it has never held, so you get a dialog, type your PIN, watch it fail, and then succeed on the fallback ^[7].

The reason this is visible rather than silent is a flag. Both keys are resident credentials with user verification required, and the flags byte in the private key blob is `0x25`, which decodes with the constant names from OpenSSH's `sk-api.h` as `SSH_SK_USER_PRESENCE_REQD | SSH_SK_USER_VERIFICATION_REQD | SSH_SK_RESIDENT_KEY` ^[8]. Before ssh can request an assertion at all, the client must obtain a PIN/UV auth token from the authenticator, and that is the step that puts the dialog on screen; only after the PIN is typed does the assertion request go down the wire and the token answer that it has never seen this credential ^[9]. Strip `USER_VERIFICATION_REQD` and the wrong-key attempt fails quietly, which is why most people with two enrolled tokens never learn the ordering exists ^[10].

The tax is small and constant: five seconds, about fifteen times a day ^[3], which works out to roughly 75 seconds a day and around five hours over a 250-day working year ^[17].

The fix uses `Match exec`, which runs a shell command and applies the block if it exits zero ^[12]. Two blocks, each testing for a serial number and pinning one `IdentityFile` plus `IdentitiesOnly yes`, resolve it ^[12]. The serial mapping was never missing: the author had already written each token's serial into the key comment when generating it, so `ssh-add -l` had been printing the answer for months ^[11].

Two details decide whether this helps or hurts. First, the `host` criterion is not optional. `Match exec` on its own applies to every destination, and the first matching block then pins `IdentitiesOnly` and a single identity for whatever you are connecting to, so every other host with every other key stops authenticating ^[13]. Second, `IdentitiesOnly yes` is the load-bearing line and the one people omit. It does not mean "ignore the agent"; `-v` shows the agent still returning both keys. It restricts which of them are eligible to be offered, to those whose public half matches an `IdentityFile` in scope ^[14].

Watch the fresh-machine case. Because these are resident credentials, you can arrive on a new laptop, pull the keys off the token with `ssh-add -K`, and have nothing in `~/.ssh` at all, at which point the blocks match nothing and do nothing ^[15]. If neither serial matches, no block applies and you fall back to trying everything, which is the failure mode you want from a config that does not understand its situation ^[16].