Rotating an SSH key means adding the new public key to the server's authorized_keys, proving the new key works, and only then removing the old line. Do it in the other order and a single mistake locks you out of the server.
Automation keys, deploy keys and service-account keys are often generated once and never changed. They sit in authorized_keys files on dozens of hosts, copied to laptops and CI variables. This article describes a rotation process that is safe to automate, and the failure modes you need to design for.
The sequence
- Connect with the current key. You are editing the very file that grants you access, so the current key's session is your safety net. Keep it open until the end.
- Back up the file next to itself before any change, and keep the last few backups.
- Append the new public key. Add the missing final newline if the file lacks one, or the new line will merge with the last one and break both.
- Verify with a new connection using the new key, and run a command to prove it can actually use the account, not just authenticate.
- Store the new private key in a vault or secrets store.
- Remove the old line after a grace period, so consumers still holding the old key have time to switch.
- Roll back if needed during the grace period: restore the old key as current and remove the new line.
At every step, remove a line by its key blob, not by its position or comment. That way you only ever delete the line you added or the old one you own.
Pin the host key, and never trust on first use in automation
A rotation tool that connects to a host must be sure it is talking to the right server. If it accepts whatever host key the server presents, an attacker who can intercept the connection can capture credentials or feed it a rogue server.
- Pin the host key's fingerprint and refuse to connect when it differs.
- Set the pin deliberately. A person should compare the fingerprint to one obtained out of band (from the console of the machine, or from a trusted record) and confirm it, rather than the tool silently probing and trusting the answer in the same call.
- Treat a changed host key as an event. It can mean a rebuilt server or an attack. The tool should refuse to rotate, show the host key status as changed, and require a conscious, recorded re-pin.
- Use the pin you confirmed. If a server is rebuilt while a grace window is open, you should only need a confirmed re-pin, not to wait for the window to end.
Failure modes and how to handle them
| Failure | What happens | Safe behaviour |
|---|---|---|
| New key type refused by the server (for example a server that only accepts RSA) | The new key cannot authenticate | Verification fails after its time budget, the new line is removed again, and the old key keeps working |
authorized_keys has loose permissions | With StrictModes yes, sshd ignores the file | Stop before changing anything and say why |
| File has Windows line endings or no final newline | A naive append merges two lines | Read as bytes, preserve everything you do not own byte for byte, add a newline only when needed |
| Someone adds a line during the rotation | A rewrite by position would delete it | Edit by key blob so foreign lines are kept |
| The rotation process crashes after the append | An orphan new line is left behind | Write a recovery record before the append, and have a worker clean up orphans after a delay |
| Host key changed | Possible rebuild or interception | Refuse before any login and leave the file untouched |
| SFTP not available, or a forced command or chroot | The file cannot be edited as the user | Use an admin-assisted path with a privileged account instead |
| Current key lost | Self-rotation has nothing to log in with | Use admin mode to recover, and record that it was used |
Self-rotation or admin-assisted
- Self-rotation logs in as the target user with the current key and edits that user's own file over SFTP. It needs no admin credential, no shell and no sudo, so a compromise of the rotation tool exposes one user's key, not the host. This should be the default.
- Admin-assisted rotation uses a privileged account, with a handful of fixed
sudocommands. It is needed for first provisioning, for a lost key, and for locked-down hosts. It is root-equivalent on the host, so document it, restrict it, and prefer a narrow helper script over broad sudo where you can.
Server requirements checklist
- The user can sign in with a key, and owns a
~/.sshdirectory (mode 700) andauthorized_keysfile (mode 600). - The server accepts the key algorithm you chose (check
PubkeyAcceptedAlgorithms). - SFTP is enabled, or you plan to use admin mode.
- The new key's options (for example
from=restrictions) are carried over from the old line unless you change them deliberately.
Choosing an algorithm
Ed25519 is a good default: short keys, fast, widely supported. Use RSA with a long modulus (3072 or 4096 bits) only where a server or an older client requires it. Whatever you pick, make sure the target accepts it before the rotation starts, because the failure appears only at the verification step.
How ZenXPAM supports this
ZenXPAM generates the new key pair, installs it in authorized_keys, verifies it with a fresh connection that runs a command, stores the private key in the vault, and removes the old line when the grace window ends. It pins host keys only after a person confirms the fingerprint, refuses a changed host key before any login, writes a backup before every change, preserves lines it does not own byte for byte, and cleans up an orphaned line if the rotation process is interrupted. Self mode and admin mode are both supported, with the egress allow-list limiting which targets can be reached. Self mode was tested live against a throwaway OpenSSH server, including loose permissions, missing newlines, Windows line endings, a rebuilt host key, a refused key type and a killed rotation process. Admin mode (with sudo), production servers and Windows OpenSSH targets have not yet been tested live. See non-human identity governance or book a demo.
FAQ
Can I just replace the line in authorized_keys? That works until it does not: if the new key fails you have no way back in. Add first, verify, then remove.
Why verify with a command, not just a login? A key that authenticates but cannot run a command (a forced command, a restricted shell) is not a usable credential for automation.
How long should the grace window be? Long enough for every consumer of the old key to switch. For keys used by scheduled jobs, a day is a reasonable start.
What if the same key is on many servers? Treat each target as its own rotation, and prefer one key per host so a single compromise has a small blast radius.
Should I use SSH certificates instead? Short-lived certificates avoid long-lived keys entirely and are worth considering where your infrastructure supports them. Rotation is for the keys you still have.
