SSH Key Authentication & Hardening on Ubuntu

Password-based SSH logins are the single most attacked surface on any internet-facing Linux server. Bots hammer port 22 around the clock guessing credentials. Key-based authentication eliminates that entire class of attack: instead of a password, you prove your identity with a cryptographic key pair that is effectively impossible to brute-force. This guide sets up SSH key authentication on Ubuntu 24.04 and then hardens the SSH daemon so password attacks simply cannot work.

We will generate a modern ed25519 key, install the public key on the server, test it, and only then disable password login and root login. The order matters: we keep a working session open and verify key login before turning anything off, so there is always a safe rollback path.

Prerequisites

• A server running Ubuntu 24.04 LTS with SSH access (password login working for now).
• A local machine (Linux, macOS, or Windows with OpenSSH) to generate and hold your private key.
• A user with sudo on the server.

Step 1: Generate an ed25519 key pair

Run this on your local machine, not the server. The ed25519 algorithm is fast, compact, and more secure than older RSA keys:

ssh-keygen -t ed25519 -C 'you@example.com'

Press ENTER to accept the default location (~/.ssh/id_ed25519). When prompted for a passphrase, set one—it encrypts the private key on disk so a stolen laptop does not equal a stolen server. You will see:

Generating public/private ed25519 key pair.
Your identification has been saved in /home/you/.ssh/id_ed25519
Your public key has been saved in /home/you/.ssh/id_ed25519.pub

Two files now exist: id_ed25519 (private—never share it) and id_ed25519.pub (public—safe to distribute).

Step 2: Copy the public key to the server

The easy way is ssh-copy-id, which appends your public key to the server's ~/.ssh/authorized_keys and fixes permissions:

ssh-copy-id user@your-server-ip

It will prompt for your password one last time. If ssh-copy-id is unavailable, do it manually:

cat ~/.ssh/id_ed25519.pub | ssh user@your-server-ip 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'

Step 3: Test key-based login (keep the old session open)

Open a new terminal and connect. Do not close your existing password session yet:

ssh user@your-server-ip

If your key has a passphrase, you are prompted for the passphrase (not the server password). Once you land on the shell without entering the account password, key authentication works. Confirm with verbose output if needed:

ssh -v user@your-server-ip 2>&1 | grep -i 'Authentication succeeded'

Step 4: Harden the SSH daemon

Now we lock the door. On Ubuntu 24.04, drop-in config files in /etc/ssh/sshd_config.d/ override the main /etc/ssh/sshd_config and survive package upgrades cleanly. Create one:

sudo nano /etc/ssh/sshd_config.d/99-hardening.conf

Add:

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no
MaxAuthTries 3
X11Forwarding no

What each line does: PermitRootLogin no stops direct root logins (use sudo instead). PasswordAuthentication no and KbdInteractiveAuthentication no together disable every password path, so only keys work. MaxAuthTries 3 drops the connection after three failed attempts. X11Forwarding no removes an unneeded feature on a headless server.

Step 5: Validate and apply the config

Before restarting, check the configuration for syntax errors—a typo here can break SSH entirely:

sudo sshd -t

If it prints nothing, the config is valid. Apply it by restarting the service:

sudo systemctl restart ssh

If your changes do not seem to take effect, note that Ubuntu 24.04 uses SSH socket activation; restart the socket unit as well:

sudo systemctl restart ssh.socket

Step 6: Verify hardening from a new session

With your original session still open as a safety net, open another new terminal and connect again to confirm key login still works:

ssh user@your-server-ip

Then prove that password auth is truly off by trying to force it:

ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@your-server-ip

Permission denied (publickey).

That Permission denied (publickey) is exactly what you want: the server refuses to even offer password authentication. Only now is it safe to close your original password session.

Optional: use an SSH config alias

To avoid typing the full command each time, add an entry to ~/.ssh/config on your local machine:

Host myserver
    HostName your-server-ip
    User user
    IdentityFile ~/.ssh/id_ed25519

Now you connect with just ssh myserver.

Common pitfalls and troubleshooting

• Locked out after disabling passwords. Always keep one session open and verify key login in a second session before restarting sshd. If you are locked out, recover via your provider's console or recovery mode.
• Wrong permissions. SSH refuses keys if permissions are too open. The server needs ~/.ssh at 700 and authorized_keys at 600, owned by the user. Fix with chmod as shown above.
• Key not offered. If the client has many keys, the server may hit MaxAuthTries before reaching the right one. Specify the key with ssh -i ~/.ssh/id_ed25519 or an IdentityFile in the config.
• Editing the wrong file. On Ubuntu 24.04, settings in /etc/ssh/sshd_config.d/*.conf can override your edits to the main file. Check for conflicting drop-ins.
• Forgot the passphrase. There is no recovery for a lost private-key passphrase—generate a new key pair and re-deploy the public key.

Conclusion

Key-based authentication plus a hardened sshd turns SSH from your biggest liability into a non-event: with passwords disabled and root login off, brute-force bots have nothing to attack. The whole change takes a few minutes and, done in the order above, carries no risk of lockout.

SSH hardening is one pillar of a secure baseline. Add a host firewall with our guide to setting up UFW on Ubuntu, and auto-ban repeat offenders with fail2ban. These are the same controls clouditiv applies by default: our sovereign cloud is hardened to ISO 27001 and BSI C5 standards, with key-only access and data residency in Germany—so you inherit a secure-by-default posture instead of building it host by host.