Lost Your SSH Key to an Azure VM? Don’t Panic. Here’s the Fix.

Check it...all code referenced in this blog is for you to peruse here!

There is a moment that almost every cloud engineer eventually experiences. You sit down at a new computer, try to SSH into a VM you built months ago, and realize something uncomfortable. The SSH key you originally used to create that machine is nowhere to be found. Maybe it lived on a laptop that was replaced. Maybe it was tucked away in a .ssh folder that never got backed up. Maybe it was generated automatically by the Azure CLI and you never thought about it again.

It happens all the time. In fact, I'm the reason I'm making a blog post...not a customer conversation in this instance! ;)

The important thing to understand is that this situation is not a disaster. Azure does not store private SSH keys when you deploy a Linux VM or K8s cluster with --generate-ssh-keys. The platform only stores the public key inside the virtual machine so it knows which identities are allowed to log in. The private key lives wherever it was generated. If that computer disappears, the key disappears with it.

The good news is that Azure gives you a very straightforward way to recover access. Instead of rebuilding the VM or doing something drastic, you simply generate a new SSH key on your current machine and inject the new public key into the VM. Once the VM trusts that new key, you can log in again as if nothing ever happened.

Let’s walk through the process.

Step 1: Generate a New SSH Key

The first thing you need to do is generate a new SSH key pair on your current computer. This works the same way whether you are using macOS, Linux, Windows, or WSL.

Run the following command in a terminal.

ssh-keygen -t ed25519 -C "myVm"

This creates a new key pair consisting of a private key and a public key. By default they will be placed in your .ssh directory.

You should now see something like this.

~/.ssh/id_ed25519~/.ssh/id_ed25519.pub

The .pub file is the public key that will be added to the VM. The private key remains on your machine and should never be shared.

If you want to create a dedicated key specifically for this VM or environment, you can also name the key explicitly.

ssh-keygen -t ed25519 -f ~/.ssh/myVm

This creates:

~/.ssh/azure-ts-router~/.ssh/myVm.pub

Using named keys for different environments can make your life much easier later when you rotate or revoke access.

Step 2: Inject the New Key into the VM

Now comes the step that makes this recovery workflow so easy.

Azure provides a command that allows you to inject a new SSH public key directly into an existing VM user account.

Run the following command from your terminal.

az vm user update \  
--resource-group myRg \  
--name myVm \  
--username azureuser \  
--ssh-key-value ~/.ssh/id_ed25519.pub

What this command does is surprisingly simple. Azure connects to the VM agent running inside the virtual machine and updates the user's SSH configuration.

Behind the scenes, the command updates this file inside the VM:

/home/azureuser/.ssh/authorized_keys

That file contains the list of public keys that are allowed to authenticate as that user. Once your new public key is added to that file, the VM immediately trusts the private key that lives on your machine.

Microsoft documentation for this command can be found here:

https://learn.microsoft.com/cli/azure/vm/user#az-vm-user-update

Step 3: Connect to the VM

Once the new key has been injected, connecting to the VM is exactly the same as before.

ssh [email protected]

If you created a named key, specify it explicitly.

ssh -i ~/.ssh/myVm [email protected]

Within seconds you should be back inside your VM.

No redeployment. No rebuild. No emergency console access.

Just a quick key injection and you're back in business.

Why This Happens So Often

This scenario is incredibly common in real environments. Engineers replace laptops. Development machines get rebuilt. People move from local terminals to WSL. Sometimes the original SSH key was generated by automation and nobody documented where it lived.

Eventually someone tries to SSH into a machine and discovers the key is gone.

Once you understand that Azure never stores private keys, this behavior makes perfect sense. The platform cannot recover the key because it never had it in the first place. The only option is to inject a new public key into the VM.

Microsoft explains this design in their SSH documentation:

SSH access to Linux VMs

https://learn.microsoft.com/azure/virtual-machines/linux/ssh-from-windows

Creating SSH keys for Azure Linux VMs

https://learn.microsoft.com/azure/virtual-machines/linux/mac-create-ssh-keys

Protecting Your SSH Keys

While recovering access is easy, it is still smart to protect your keys so you do not end up in this situation repeatedly. SSH private keys should be treated like credentials. Anyone who has the private key can authenticate as you. That means storing them safely and backing them up properly.

One simple strategy is creating an encrypted backup of your .ssh directory.

tar -czf ssh-backup.tar.gz ~/.sshgpg -c ssh-backup.tar.gz

This creates an encrypted archive that you can store somewhere safe. If your laptop ever disappears, you can restore the keys quickly.

Another good option is storing SSH keys in a secrets manager or password manager that supports file storage. Many engineers store keys in tools such as:

1Password

https://1password.com

Bitwarden

https://bitwarden.com

Azure Key Vault

https://learn.microsoft.com/azure/key-vault/general/overview

The goal is simple: Do not let a single laptop be the only place your keys exist.

A Small Best Practice That Helps a Lot

Instead of relying on the default id_rsa or id_ed25519 keys everywhere, it is much cleaner to create environment specific keys.

For example:

~/.ssh/azure-router~/.ssh/github~/.ssh/lab-environment

This makes it easy to rotate keys, revoke access, or move machines without impacting unrelated environments.

Rotating keys is also simple in Azure. You can inject a new key at any time using the same az vm user update command we used earlier.

Final Thoughts

If you spend enough time managing cloud infrastructure, eventually you will lose an SSH key. It's inevitable and practically a rite of passage (you might even get knighted along the way). A laptop gets replaced, a development environment gets rebuilt, or a key quietly disappears into the digital void. All happen way too often in this cloudy world.

The good news is Azure was designed with this reality in mind. Injecting a new public key into a running VM takes only a few minutes, and once you know the command, the entire process feels refreshingly straightforward. Instead of rebuilding servers or scrambling for emergency access, you simply generate a new key, update the VM, and move on with your day.

This is one of those operational tasks that sounds stressful until you realize it is just another small piece of the cloud toolkit. Once you know how this works, restoring access becomes routine (and so very boring).