7 Cryptography in Secure Protocols
Learn how cryptographic primitives work together to authenticate peers, establish keys, and protect data in TLS, secure messaging, and SSH.
How cryptographic primitives work together
Secure protocols combine cryptographic primitives to establish keys, verify identities, and protect data in transit. No single primitive provides all of these properties. A common design authenticates public keys, agrees on secret material, derives traffic keys, and protects each message with AEAD. Public-key operations establish trust and keys; efficient symmetric cryptography protects bulk data.
What each primitive contributes
Hash functions map data to fixed-length digests. Protocols use hashes to summarize handshakes or as input to key-derivation functions, but a plain hash neither encrypts data nor proves who created it.
Digital signatures let a verifier check that a message was signed by the holder of a private signing key. They can authenticate identities and bind protocol messages together.
connect a public key to an identity. A certificate authority (CA) signs a certificate, and a client checks the certificate chain and whether the certificate is valid for the intended identity. A certificate carries a public key and identity information; it does not contain the corresponding private key or itself encrypt application data.
, commonly ephemeral Diffie–Hellman, lets parties derive shared secret material over a public channel without sending that secret directly.
Key-derivation functions (KDFs) turn shared secret material into separate keys of appropriate sizes. HKDF is widely used for this purpose.
encrypts data while detecting tampering. Associated data is authenticated but remains visible.
HPKE is a standardized example of combining a key-encapsulation mechanism (KEM), a KDF, and AEAD.
TLS handshake and data protection
In a typical TLS 1.3 handshake, a browser and server negotiate protocol parameters and exchange ephemeral key shares. Each side uses to derive shared secret material, then HKDF derives distinct handshake and application traffic secrets.
The server sends a certificate chain. The browser validates it for the requested server name and uses the certificate’s public key to verify the server’s CertificateVerify signature over the handshake transcript. A message, computed with a key derived from the shared secrets, confirms that the peer has the same handshake state and keys. After the handshake, AEAD protects application records.
These checks have different roles: certificate validation connects the server’s key to its claimed identity; the signature proves possession of the corresponding private key and binds that proof to the handshake; confirms the derived keys and transcript; and AEAD provides confidentiality and tamper detection for data.
With ephemeral Diffie–Hellman, later compromise of a long-term certificate-signing key does not by itself reveal previously recorded session traffic, assuming the ephemeral secrets were securely erased.
Secure messaging and changing keys
End-to-end messaging often needs to work while a recipient is offline and to limit the damage if a device key is later exposed. In Signal’s X3DH design, a sender uses the recipient’s published identity and prekeys to establish shared secret material asynchronously.
The recipient’s identity key must still be authenticated, for example by comparing fingerprints or scanning a QR code. If users do not authenticate identity keys, they lack a cryptographic guarantee that the keys belong to the intended person.
Evolving message keys
A ratchet derives fresh message keys over time. The combines symmetric-key chains with new Diffie–Hellman contributions: each message uses a distinct key, and ratchet steps can provide and, after fresh secret input, recovery from some key compromises.
Applications must protect and correctly update ratchet state. Reusing a message key or losing state can undermine security.
For group messaging, uses authenticated group , a key schedule, and per-sender message-key ratchets. Membership changes advance the group to a new epoch with new key material, so removed members are excluded from subsequent epoch secrets when the protocol is correctly run.
SSH host authentication
SSH parties negotiate algorithms and perform key exchange. The client verifies the server’s host key, commonly against a previously trusted known_hosts entry. The resulting session keys protect the connection with encryption and integrity protection.
Unlike the usual public-CA model for websites, SSH commonly relies on users or administrators to trust host keys directly. Accepting an unexpected changed key without checking it can defeat server authentication.
Practical security principles
Use well-reviewed protocol implementations and current, compatible algorithms rather than designing a custom combination of primitives. Keep private keys secret, use strong randomness, and prevent key or nonce reuse with AEAD.
Always consider whether the identity behind a public key has been authenticated. Encryption to an unauthenticated public key can protect data from outsiders while still allowing an active attacker to impersonate the intended peer.
Protocol context matters: transcript hashes, labels, and associated data bind keys and ciphertexts to the appropriate handshake, identity, or message. Understand what a protocol protects. TLS protects data in transit between endpoints, while end-to-end messaging aims to keep message contents inaccessible to the service relaying them; neither automatically hides all metadata or protects a compromised endpoint.