SSH – Key Management and Best Practices

Secure Shell (SSH) is a network protocol that uses public-key cryptography to authenticate users and encrypt data flowing to and from remote systems. SSH is the primary mechanism for remote server administration, system-to-system file transfers, and automated access in CI/CD pipelines. Because SSH keys have no built-in expiry, no Certificate Authority, and no revocation mechanism, they accumulate silently in authorized_keys files across infrastructure, creating persistent access paths that are rarely reviewed. The recommended action: run a discovery scan to inventory all SSH keys across your environment, implement a key rotation policy with a maximum key age of no more than one to two years, disable password authentication on all SSH servers, enforce role-based access controls for key management, and eliminate hard-coded keys from application code.

Quick Answer: What Is SSH Key Management and Why Does It Matter?

SSH key management is the process of discovering, inventorying, controlling, rotating, and auditing SSH key pairs across an organization’s infrastructure. It matters because SSH keys are powerful authentication credentials with no built-in expiry or revocation mechanism: an authorized_keys entry grants access to anyone holding the corresponding private key, indefinitely, unless the key is explicitly rotated or removed. Large organizations accumulate hundreds of thousands of authorized_keys entries, many associated with former employees, contractors, or service accounts that should have been revoked years earlier. Poor SSH key management creates persistent backdoors, enables lateral movement across servers, and violates compliance requirements under PCI DSS, NIST SP 800-53, and HIPAA.

What Is SSH and How Does It Work?

SSH (Secure Shell), defined in RFC 4251-4254, is a cryptographic network protocol for secure remote access, file transfer, and tunneling. It operates on the principle of public-key cryptography, using key pairs where the public key is placed on servers in the authorized_keys file and the private key is held by the user or system requesting access. When an SSH client connects to a server, the server challenges the client to prove it holds the private key corresponding to an authorized public key, without the private key ever being transmitted across the network.

Typical SSH use cases include remote login to servers and network devices, system-to-system file transfers (SCP, SFTP), automated CI/CD pipeline access to build and deployment servers, port forwarding and tunneling, and programmatic API access where SSH is used as the transport.

SSH Keys vs. X.509 Certificates: Key Differences

Property	SSH keys	X.509 certificates (TLS)
Governing authority	None; keys are self-governing within the organization	Certificate Authority (CA) issues and signs certificates; CA/Browser Forum governs public CAs
Built-in expiry	No expiry; authorized_keys entries persist indefinitely unless manually removed	Certificates have NotBefore and NotAfter validity dates enforced by clients
Revocation mechanism	No equivalent of CRL or OCSP; revocation requires removing the public key from authorized_keys on every server	CRL (Certificate Revocation List) and OCSP (Online Certificate Status Protocol) enable revocation
Identity validation	No CA validation; the server trusts any key in authorized_keys regardless of who generated it	CA validates the certificate holder’s identity before issuance (for public CAs: domain validation or organization validation)
Remote access capability	Enables remote login and command execution on servers	Provides authentication for network connections but does not enable remote shell access on its own
Inventory challenge	No central registry; keys must be discovered by scanning servers and systems	Certificates can be discovered through Certificate Transparency logs; PKI tools track issued certificates

Risks of Unmanaged SSH Keys

The absence of built-in expiry and revocation makes SSH key governance a distinct challenge from certificate management. Specific risks that unmanaged SSH keys create:

• Persistent backdoors from former employees and contractors: when a developer, system administrator, or contractor leaves, their SSH public key remains in authorized_keys files on every server they had access to unless explicitly removed. A former employee who still holds their private key retains access to those servers indefinitely.
• Key sprawl enabling lateral movement: in large organizations, a single service account or build system may have its public key in authorized_keys on hundreds of servers. An attacker who compromises one system with an authorized SSH private key can use that key to authenticate to every other server that trusts the corresponding public key, enabling rapid lateral movement across the environment.
• Stale keys with weak algorithms: keys generated years ago may use RSA-1024 or DSA (both deprecated) because the policy and tooling at the time did not enforce stronger algorithms. Without a discovery and inventory process, these weak keys remain in authorized_keys indefinitely.
• Hard-coded keys in applications: SSH keys embedded in application code, Docker images, or configuration files are difficult to rotate, may be exposed through source code repositories, and persist in version history even after removal from the current codebase.
• Key rotation is often skipped: without automated management, key rotation requires manually updating authorized_keys on every server that trusts the key. In environments with hundreds of servers, this is operationally burdensome and is frequently deferred indefinitely.

Algorithm and Key Type Selection for SSH

Key type	Algorithm	Security strength	Recommendation	Notes
Ed25519	Edwards-curve DSA (EdDSA)	128-bit	Preferred for new SSH key generation	Fast, compact (32-byte public key), resistant to timing side-channel attacks, supported by OpenSSH 6.5+ (released 2014)
ECDSA P-256	Elliptic Curve DSA	128-bit	Acceptable; use Ed25519 instead if available	Wider compatibility with older systems; less timing-attack resistant than Ed25519
RSA-3072	RSA	128-bit	Acceptable where Ed25519/ECDSA not supported	Much larger key size (384 bytes vs. 32 bytes for Ed25519); slower operations; compatible with the widest range of legacy systems
RSA-2048	RSA	112-bit	Minimum; plan migration to stronger key types	Proposed for deprecation after 2030 per NIST IR 8547; use RSA-3072 or Ed25519 for new keys
RSA-1024	RSA	80-bit (broken)	Do not use	NIST disallowed for new keys; computationally breakable
DSA	DSA (original)	80-bit (deprecated)	Do not use	Fixed 1024-bit key size; formally deprecated by NIST; removed from OpenSSH 7.0+ by default

SSH Key Management Best Practices

1. Gain complete visibility through discovery: the first step in SSH key management is knowing what you have. Run periodic discovery scans across all servers and network devices to locate and inventory all authorized_keys entries and all SSH private keys on the network. Map each authorized public key to the user or system account that owns it, the servers it grants access to, when it was generated, and what algorithm and key size it uses. Without this inventory, no other governance control is meaningful.
2. Enforce a maximum key age and rotation policy: define a policy that requires SSH keys to be rotated on a defined schedule. For interactive user keys, a maximum age of one to two years is common. For service accounts and automated systems, shorter rotation intervals are appropriate for higher-risk access. Rotation requires generating a new key pair, distributing the new public key to all authorized servers, verifying access, and removing the old public key from all authorized_keys files. Automate this process in large environments.
3. Disable password authentication on SSH servers: configure all SSH servers with PasswordAuthentication no and ChallengeResponseAuthentication no in sshd_config. Require key-based authentication exclusively. Password authentication is vulnerable to brute-force attacks and phishing; SSH key authentication is immune to both when properly managed.
4. Enforce audit trails and policy: log all SSH authentication events, including which key was used and from which source IP, to a centralized SIEM. These logs are essential for detecting unauthorized access (for example, an old key used from an unexpected location after a former employee’s departure) and for demonstrating compliance with PCI DSS, NIST SP 800-53, and HIPAA audit requirements. Align logging configuration with your organization’s key usage policy.
5. Implement role-based permissions for key management: restrict who can add, modify, or remove entries in authorized_keys files. In environments where administrators can arbitrarily add authorized_keys entries without oversight, unauthorized or undocumented access grants accumulate. Use directory services or a centralized SSH key management platform to enforce that all key grants follow an approval workflow and are recorded in inventory.
6. Eliminate hard-coded keys from application code: SSH private keys embedded in application code, container images, or configuration files are dangerous: they are difficult to rotate, may be exposed through source code repositories (including repository history after deletion), and often have broad access. Replace hard-coded keys with dynamically provisioned credentials retrieved from a secrets management system at runtime.
7. Use passphrase-protected private keys for interactive use: SSH private keys used for interactive login should be protected with a strong passphrase. The passphrase encrypts the private key file so that an attacker who obtains the key file cannot use it without knowing the passphrase. For automated systems, the private key should be stored in a secrets management system with access controls rather than in plaintext on the filesystem.

Deployment Example: Enterprise SSH Key Governance

An organization with 500 Linux servers and a DevOps environment using SSH for CI/CD pipeline access implements the following SSH key governance program:

1. Initial discovery: run an SSH key discovery scan across all 500 servers. The scan enumerates all entries in /root/.ssh/authorized_keys and /home/*/.ssh/authorized_keys, cataloguing each public key’s fingerprint, algorithm, key size, and the accounts and servers it appears on. The initial scan reveals 47,000 authorized_keys entries, of which 12,000 have no corresponding user account in the current Active Directory, indicating former employee or contractor keys that were never revoked.
2. Emergency revocation of orphaned keys: the 12,000 orphaned authorized_keys entries are removed from all affected servers. The 35,000 remaining entries are assigned to current user or service accounts and added to the key inventory with a review status of pending.
3. Algorithm remediation: the inventory identifies 3,200 RSA-1024 keys and 800 DSA keys that must be replaced. New Ed25519 keys are generated for the affected users and service accounts, distributed to all relevant servers, and the old keys are removed. RSA-2048 keys are flagged for rotation to RSA-3072 or Ed25519 on their next scheduled rotation.
4. Rotation schedule implementation: a policy is established requiring all SSH keys to be rotated annually for interactive users and every six months for service accounts with access to production systems. An SSH key management platform automates rotation: generating new keys, updating authorized_keys across affected servers, and removing old keys after confirmed successful authentication with the new key.
5. CI/CD pipeline credential replacement: SSH private keys hard-coded in CI/CD pipeline configuration files are identified and replaced with dynamically provisioned short-lived credentials retrieved from a secrets management system at build time. The old hard-coded keys are revoked and removed from authorized_keys.
6. Ongoing audit logging: SSH authentication events are forwarded to the SIEM. Alerts are configured for authentication events using keys older than the policy’s maximum age, authentication from unexpected source IPs using service account keys, and any new authorized_keys entries added outside the approved workflow.

Limitations of SSH Key Management Controls

• No native central authority: SSH lacks a CA model, so there is no single source of truth for what keys exist. Every authorized_keys file on every server is an independent store of trust. Maintaining a consistent inventory requires active discovery scanning rather than querying a central registry.
• Manual authorized_keys management at scale is impractical: organizations with more than a few dozen servers cannot maintain SSH key governance manually. Automated SSH key management platforms are a practical requirement, not an optimization, for enterprise environments.
• SSH certificate authority (SSH CAs) as an alternative model: OpenSSH supports an alternative to authorized_keys using SSH Certificate Authorities: the SSH CA signs a user’s public key to create an SSH certificate with a built-in expiry and optional revocation. This model provides the expiry and revocation capabilities that plain SSH keys lack, but requires updating all SSH servers to trust the SSH CA and deploying the certificate issuance workflow. For organizations willing to invest in the infrastructure, SSH CAs address most of the governance challenges of plain SSH keys.
• Quantum computing vulnerability: RSA SSH keys are quantum-vulnerable. Ed25519 and ECDSA keys are also quantum-vulnerable but have a longer projected safe window. NIST is working on post-quantum algorithms for asymmetric cryptography; organizations with SSH infrastructure that must remain secure into the 2030s should plan for algorithm migration as PQC standards mature.

How Encryption Consulting Can Help

• SSH Secure: our SSH Secure solution provides automated SSH key lifecycle management, including discovery scanning, centralized inventory, rotation automation, policy enforcement, and audit logging. It eliminates the manual burden of SSH key governance at scale and ensures compliance with PCI DSS, NIST SP 800-53, and HIPAA requirements for authentication credential management.
• Encryption Advisory Services: our Encryption Advisory Services assess your current SSH key infrastructure, identify orphaned keys, weak algorithms, hard-coded credentials, and policy gaps, and provide a remediation roadmap aligned to compliance requirements.
• CBOM Secure: CBOM Secure discovers all cryptographic assets in your environment, including SSH keys, and provides the inventory needed to identify weak algorithms (RSA-1024, DSA) and plan migration to stronger key types.

Conclusion

SSH keys are powerful authentication credentials that grant direct server access. Without built-in expiry, revocation, or a central authority, they accumulate silently in authorized_keys files, creating persistent access paths long after the intended use has ended. The consequences of poor SSH key management range from compliance violations to significant data breaches enabled by lateral movement through a network of servers that each trust the same compromised private key.

The governance program is straightforward: discover, inventory, remediate weak algorithms, enforce rotation, disable password authentication, restrict who can manage keys, and audit access continuously. Automation is not optional at enterprise scale: the manual effort of maintaining authorized_keys across hundreds of servers without tooling virtually guarantees that governance breaks down. If you want to assess your SSH key posture or implement automated lifecycle management, contact Encryption Consulting. For related reading, see our Key Management in Cryptography guide.

Frequently Asked Questions

What is SSH and how does it differ from TLS certificates?

SSH uses public-key cryptography to authenticate users and encrypt remote access sessions. SSH keys are self-governing with no CA, no built-in expiry, and no revocation mechanism. TLS certificates are issued by Certificate Authorities, have validity dates, and can be revoked via CRL or OCSP. SSH uniquely enables remote shell access and command execution, which TLS certificates alone cannot provide.

What is SSH key sprawl and why is it a security risk?

SSH key sprawl is the uncontrolled accumulation of SSH key pairs across infrastructure without inventory or lifecycle management. Each authorized_keys entry is a persistent access path that remains active until explicitly removed. Former employee and contractor keys that were never revoked, and service account keys distributed across hundreds of servers, create backdoors that attackers can exploit indefinitely.

What algorithms should be used for SSH keys?

Ed25519 is the current preferred algorithm: 128-bit security, fast, compact, and resistant to timing side-channel attacks. ECDSA P-256 is also acceptable. RSA-3072 is acceptable for legacy compatibility; RSA-2048 is the minimum. RSA-1024 and DSA must not be used. Plan migration to PQC algorithms as NIST standards for SSH key types mature.

How should SSH keys be rotated?

Generate a new key pair, add the new public key to authorized_keys on all target servers, verify access, then remove the old public key from all authorized_keys files and destroy the old private key. Define a maximum key age policy (one to two years for interactive keys; shorter for service accounts) and automate rotation in environments with more than a few dozen servers.

What compliance frameworks require SSH key management controls?

PCI DSS Requirements 8.2 and 8.6 address authentication credential and non-interactive account management, covering SSH keys. NIST SP 800-53 AC-2 and IA-5 require inventory, rotation, and revocation of authentication credentials. HIPAA Technical Safeguards (164.312(d)) require entity authentication controls for systems handling ePHI. ISO 27001 Annex A.9.3 and A.9.4 require controls over cryptographic authentication mechanisms.

What is the difference between SSH keys and password authentication?

SSH key authentication uses a cryptographic challenge-response; no secret is transmitted over the network. Password authentication transmits a password (within the encrypted SSH session). SSH keys are immune to brute-force and phishing attacks, use 256-4096 bit secrets (vs. typical human-memorable passwords), and should be the exclusive authentication method on SSH servers (PasswordAuthentication no in sshd_config).