SSH Hardening on Linux: Keys, Settings and Mistakes

SSH is often the main administrative entry point to a Linux server. SSH hardening reduces the risk of password attacks, stolen credentials, unauthorised forwarding and configuration errors without preventing legitimate administrators from connecting.

This guide uses OpenSSH, the standard SSH implementation on most Linux distributions. Test every change in a lab or a second session before applying it to a production server.

What does SSH hardening on Linux involve?

SSH hardening on Linux means reducing unnecessary access methods, requiring strong authentication, limiting who can connect and monitoring failed or suspicious logins. It also requires a safe change process because an incorrect setting can lock every administrator out of the server.

A hardened connection can be pictured as a sequence of gates:

Administrator
    |
    | 1. Network firewall permits the source
    v
TCP port 22 on the Linux server
    |
    | 2. sshd checks allowed users and groups
    v
Public-key authentication
    |
    | 3. Optional MFA or access policy
    v
Restricted shell and authorised privileges
    |
    | 4. Logs record the session
    v
Linux system

Hardening one gate does not replace the others. For example, moving SSH to another port may reduce basic scanning noise, but it does not provide the protection of keys, firewall restrictions and login monitoring.

How should SSH keys be created and installed?

Generate a modern key pair on the administrator's trusted workstation, protect the private key with a passphrase and install only the public key on the server. Ed25519 is a suitable default for current OpenSSH clients, while RSA may be required for compatibility with older or regulated environments.

Create an Ed25519 key:

ssh-keygen -t ed25519 -a 100 -C admin-laptop

The command normally creates these files:

The -a 100 option increases the key derivation rounds used to protect the private key file. It strengthens passphrase-based protection but does not change the server-side authentication algorithm.

Install the public key while password login is temporarily available:

ssh-copy-id -i ~/.ssh/id_ed25519.pub admin@192.0.2.20
ssh -i ~/.ssh/id_ed25519 admin@192.0.2.20

For manual installation, log in to the server as the target user and run:

mkdir -p ~/.ssh
chmod 700 ~/.ssh
cat admin-laptop.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
chown -R "$USER":"$(id -gn)" ~/.ssh

Do not use the cat example unless admin-laptop.pub is actually present on the server and contains only the intended public key. A safer alternative is to paste the public key through a controlled provisioning process.

Linux ownership and mode rules are important because sshd may reject files that other users can modify. See Linux file permissions and ownership explained for a deeper explanation of modes and ownership.

Which sshd configuration options should be hardened?

Start by disabling direct root login and unused authentication methods, then restrict access to approved users or groups. Forwarding features should be disabled only when they are not required because bastion hosts, development tools and automation may depend on them.

The main server configuration is usually /etc/ssh/sshd_config. Modern distributions may also load files from /etc/ssh/sshd_config.d/ through an Include directive.

A practical baseline is:

# /etc/ssh/sshd_config or an included configuration file
Protocol 2
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitEmptyPasswords no
UsePAM yes

AllowGroups sshusers
MaxAuthTries 3
LoginGraceTime 30
MaxSessions 5

X11Forwarding no
AllowAgentForwarding no
AllowTcpForwarding no
PermitTunnel no
GatewayPorts no

LogLevel VERBOSE

Add authorised administrators to the allowed group before enabling AllowGroups:

sudo groupadd -f sshusers
sudo usermod -aG sshusers admin
id admin

The options have different operational effects:

UsePAM yes does not automatically enable password login. On common Linux distributions, PAM also manages account restrictions, session setup and optional MFA, so it may remain enabled while PasswordAuthentication is disabled.

Do not blindly copy lists of ciphers, message authentication codes or key exchange algorithms from an old checklist. Supported algorithms change across OpenSSH releases, and manually pinned lists can prevent upgrades or break current clients. Review the installed version and effective defaults first:

sshd -V 2>&1
ssh -Q cipher
ssh -Q mac
ssh -Q kex

The legacy ssh-rsa signature algorithm based on SHA-1 is different from an RSA key that uses modern RSA-SHA2 signatures. Replacing all RSA keys may therefore be unnecessary; inspect the negotiated algorithm and compatibility requirements.

How can SSH changes be applied without causing a lockout?

Keep an existing privileged SSH session open, validate the configuration and test a second login before closing the first session. Production systems should also have a recovery path such as a cloud serial console, virtual machine console or physical console.

Use this safe rollout procedure:

1. Confirm that key-based login already works.
2. Keep the current session connected.
3. Back up the active configuration.
4. Validate syntax with sshd -t.
5. inspect the effective configuration with sshd -T.
6. Reload rather than restart the service where possible.
7. Open a second terminal and test a new connection.
8. Close the original session only after the test succeeds.

sudo cp -a /etc/ssh/sshd_config /etc/ssh/sshd_config.backup
sudo sshd -t
sudo sshd -T | grep -E 'permitrootlogin|passwordauthentication|kbdinteractiveauthentication|allowgroups'

Reload commands differ by distribution:

# Ubuntu and Debian
sudo systemctl reload ssh

# RHEL, Rocky Linux, AlmaLinux and similar systems
sudo systemctl reload sshd

OpenSSH generally uses the first obtained value for many configuration keywords. An included file named 99-hardening.conf does not necessarily override an earlier value. Check the position of the Include directive and use sshd -T to verify what the daemon will actually apply.

Match blocks add another complication. Settings below a Match User, Match Group or Match Address line apply conditionally until another Match line or Match all appears. Test a conditional configuration for a specific connection:

sudo sshd -T -C user=admin,addr=192.0.2.50,host=server1

Should SSH be protected with network controls and rate limiting?

Yes. Permit SSH only from required management networks, VPN ranges or bastion hosts whenever operations allow it, and use rate limiting as an additional control rather than as a replacement for key authentication.

For a server using UFW, an example rule is:

sudo ufw allow from 192.0.2.0/24 to any port 22 proto tcp
sudo ufw status numbered

For firewalld, create a source-specific rich rule:

sudo firewall-cmd --permanent \
  --add-rich-rule='rule family="ipv4" source address="192.0.2.0/24" port port="22" protocol="tcp" accept'
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

Test firewall rules from the intended administration network before removing an existing broad rule. Cloud servers may also require an AWS security group, Azure network security group or another provider firewall rule. Both the cloud control and the Linux host firewall must permit the connection.

Tools such as Fail2ban can temporarily block sources that repeatedly fail authentication. They reduce automated login noise, but poor filters or aggressive thresholds can block legitimate administrators. Key authentication, source restrictions and monitoring remain the primary controls.

Security engineers practise these layered controls in the Cybersecurity and SOC course, including Linux access review, log analysis and incident investigation. Administrators responsible for cloud-hosted Linux systems may also benefit from the identity and network security topics in the AWS Solutions Architect course.

What are the most common SSH hardening mistakes?

The most serious mistakes are disabling passwords before testing keys, applying copied configuration without validation and restricting access without a recovery route. Other errors create a false sense of security instead of reducing meaningful risk.

Hardware-backed FIDO2 keys can reduce the risk of private-key extraction when supported by the client and operational process. For example, OpenSSH can generate an Ed25519 security-key credential with ssh-keygen -t ed25519-sk, but enrolment, backup and recovery must be planned before deployment.

How do you troubleshoot SSH after hardening?

Troubleshoot from the client, server configuration, file permissions, account policy and network layers in that order. Verbose client output and server logs usually identify whether the failure occurs before the TCP connection, during key selection or after authentication policy checks.

Start with client diagnostics:

ssh -vvv -i ~/.ssh/id_ed25519 admin@192.0.2.20

Look for these messages:

• Connection timed out: routing, firewall, security group or an incorrect address may be responsible.
• Connection refused: the host is reachable, but sshd may be stopped or not listening on that port.
• Offering public key: the client found and presented a key.
• Server accepts key: the public key matched and the server accepted that authentication step.
• Permission denied (publickey): the key, account, permissions or server policy did not match.

On the server, run:

sudo systemctl status sshd --no-pager
sudo ss -lntp | grep sshd
sudo sshd -t
sudo sshd -T | grep -E 'port|listenaddress|pubkeyauthentication|authorizedkeysfile'

Use systemctl status ssh instead of sshd on systems where the service is named ssh. Review logs with the appropriate command:

# systemd journal
sudo journalctl -u sshd --since '15 minutes ago'
sudo journalctl -u ssh --since '15 minutes ago'

# Common distribution-specific files
sudo tail -f /var/log/auth.log
sudo tail -f /var/log/secure

Check account and key permissions:

namei -l /home/admin/.ssh/authorized_keys
sudo ls -ld /home/admin /home/admin/.ssh
sudo ls -l /home/admin/.ssh/authorized_keys
sudo passwd -S admin

On SELinux-enabled systems, correct labels may be required even when Unix permissions are correct:

sudo restorecon -Rv /home/admin/.ssh

If a problem spans services, storage or account configuration, follow the structured checks in this Linux troubleshooting guide rather than changing several settings at once.

SSH hardening checklist

A reliable SSH baseline combines strong authentication, minimal exposure, tested configuration and useful logging. The exact policy must still reflect whether the host is a bastion, automation node, cloud instance or general administration server.

• Use a separate named account for each administrator.
• Prefer Ed25519 or an approved hardware-backed key.
• Add a passphrase to private keys.
• Disable direct root login.
• Disable passwords only after key login is verified.
• Restrict users or groups when practical.
• Limit network access to management sources.
• Disable unused forwarding features.
• Run sshd -t and inspect sshd -T before reloading.
• Keep an active session and a console recovery method.
• Review logs and remove obsolete authorised keys.
• Document exceptions required by automation or legacy clients.

Summary

SSH hardening is not a single configuration switch. Secure keys, controlled accounts, source restrictions, validated sshd settings, protected private keys and continuous log review work together to reduce risk while preserving reliable administration.

Reviewed by Network Rhinos Linux and cybersecurity trainers.

To practise Linux access hardening, authentication analysis and security monitoring in guided labs, enquire about upcoming batches for the Cybersecurity and SOC course.

Related reading: Linux Administration Basics: Users, Services, Cron and Packages