SSH Keys for Cloud Servers: Generate, Add, and Rotate

How SSH key authentication works

A key pair has two halves. The private key stays on your workstation and is never sent anywhere. The public key is copied to the server and stored in the authorized keys file of the user you log in as. When you connect, the server sends a challenge that only the holder of the private key can answer, so the server learns that you hold the key without ever seeing it.

Losing a private key means generating a new pair and replacing the public half everywhere it was installed. Sharing a private key with a colleague means you can no longer tell who logged in. Leaving password login enabled alongside keys means the weaker method is still available to brute force.

Generating a key pair the right way

Use the ssh-keygen tool that ships with OpenSSH on Linux, macOS, and current Windows. Choose the Ed25519 algorithm, which produces short keys, is fast, and is supported by every maintained SSH server; fall back to RSA with a 4096 bit length only when you must talk to legacy systems that do not accept Ed25519. Give the key a comment that identifies the person and the device, because that comment is what you will see in the authorized keys file a year from now when deciding which entries to remove.

Protect the private key with a passphrase. The passphrase encrypts the key file at rest, so a stolen laptop or a leaked backup does not immediately hand over server access. Use the SSH agent so you type the passphrase once per session rather than on every connection. Keep the file with owner-only permissions; OpenSSH refuses to use a private key that other users can read.

• One key pair per person per device. Do not copy private keys between machines.
• Ed25519 by default, RSA 4096 only for legacy targets.
• Passphrase on every private key, with the SSH agent to avoid retyping it.
• A descriptive comment so the key can be identified in authorized keys lists later.
• Keep a separate deploy key for automation, stored in your CI system’s secret store, never in a repository.

Adding public keys on Linode, UpCloud, Kamatera, and others

Every major provider lets you register a public key with the account and then select it when creating a server, which places it in the root or default user’s authorized keys before the first boot. On Linode, keys are stored under the profile’s SSH Keys tab and offered as a checkbox in the Create Linode form; the Linode CLI and API accept the same list. UpCloud keeps SSH keys in the account settings and attaches them at deployment through the control panel, the API, or its Terraform provider. Kamatera’s console asks for a public key during server creation and also supports a password, so uncheck or clear the password option once key login works. On AWS EC2 the equivalent is the key pair object in each region; on Google Compute Engine keys are attached through project or instance metadata or, better, through OS Login tied to IAM; on Azure VMs the public key is passed at creation and stored in the VM configuration.

For a server that already exists, the ssh-copy-id utility appends a public key to the remote authorized keys file over an existing session. If you have no session yet, most providers offer a browser based console (Lish on Linode, the VNC console on UpCloud and Kamatera) where you can log in with the initial root password, paste the key, and then disable password login.

Team access and least privilege

When more than one person needs access, resist the temptation to share a single “ops” key. Each person registers their own public key with the provider account, and each server’s authorized keys file lists exactly the people who should be on it. Remove a key the same day someone leaves the team or changes roles. Where SSH access is tied to identity, as with Google Cloud OS Login or AWS Systems Manager Session Manager, access is granted and revoked through IAM without touching the servers.

Restrict what a key can do where the tooling allows it. The authorized keys file accepts options that limit a key to a specific source address or a single forced command, which is useful for backup and deployment keys. Give automation its own key with the narrowest permissions the job needs, and keep the private half in a secrets manager or the CI platform’s encrypted variables rather than in a repository. The Cloud Library article on API keys and secrets management covers where those secrets should live.

Rotating a key without locking yourself out

The order of operations matters. Generate the new pair first. Add the new public key to every server and to the provider account, keeping the old key in place. Test a fresh login with the new key from a second terminal on every server that matters. Only then remove the old public key from the authorized keys files and from the provider console, and finally delete the old private key from your workstation and any agent that has it loaded.

Plan to rotate on a schedule that matches your risk, and always rotate immediately after a suspected leak, a lost device, or a team member’s departure. Keep an inventory of where each public key is installed; without one, removing an old key from “every server” turns into guesswork. A short script that lists the key fingerprints on each server, run from your configuration management tool, is enough for most small teams.

Step 1

Generate the new key pair with a fresh passphrase

Step 2

Add the new public key everywhere the old one lives, servers and provider consoles alike

Step 3

Verify a new login on each server from a second terminal

Step 4

Remove the old public key, then delete the old private key

Where to go next

The providers discussed here all support key based login from the first boot. The marketplace lists Linode accounts, UpCloud accounts, and Kamatera accounts with their exact variant labels on each page.

To put keys in context, start with the first cloud server checklist, then continue to securing a VPS with firewalls and Fail2ban. The Cloud Library hub has the full list of developer workflow and security guides.