VPNs — IPsec and the Tunnel Model

A VPN carries private traffic across a network you don't trust by tunneling — wrapping each packet inside another packet and encrypting it, so anyone on the path sees only ciphertext addressed between the two tunnel endpoints. IPsec is the long-standing standard for this: site-to-site tunnels that join two networks as if they shared a wire, or remote-access tunnels that put a single user's laptop logically inside a network. Because IPsec operates at layer 3, it secures everything above it transparently — TCP, UDP, every application — without any of them knowing a tunnel exists.

That transparency is IPsec's strength and the source of its reputation for being fiddly. It runs below the application, so it must negotiate keys, cipher suites, and which traffic to protect entirely on its own, through a separate protocol with its own phases and its own failure modes. Get a single parameter mismatched between the two sides and the tunnel simply never comes up, with logs that name the phase but rarely the fix.

Tunneling and Encapsulation

Tunneling means packet-in-packet: the original IP packet becomes the payload of a new outer IP packet addressed between the two gateways, with IPsec's own header and encryption in between. The inner packet, with its private source and destination, is invisible on the wire — what a router on the path forwards is the outer packet between the public gateway addresses.

The unavoidable consequence is that the outer headers and the ESP overhead eat into the MTU. A standard 1500-byte link now has to carry your original packet plus roughly 50–60 bytes of tunnel headers, so the inner packet's usable size shrinks. If you don't account for that — by lowering the inner MTU or clamping TCP MSS — large packets that can't fragment get silently dropped, and you get the classic "ping works, big transfers hang" failure described in the MTU topic.

Transport Mode versus Tunnel Mode

IPsec has two encapsulation modes. Transport mode encrypts only the payload of the original packet and keeps its original IP header, so it secures host-to-host traffic between two endpoints that already route to each other. Tunnel mode encrypts the entire original packet, header and all, and prepends a new outer header — this is what joins two networks, because the real source and destination are hidden inside and only the gateways are visible outside.

The two also differ in protocol. AH (Authentication Header) authenticates a packet but does not encrypt it, and it breaks through NAT because it covers the IP header; almost nobody uses it now. ESP (Encapsulating Security Payload) both encrypts and authenticates and is what every real deployment uses. Site-to-site VPNs are ESP in tunnel mode, essentially always.

IKE and the Two Phases

IKE (Internet Key Exchange) is the negotiation protocol that sets up the tunnel before any data flows, and it works in two phases. Phase 1 authenticates the two peers — by pre-shared key or certificate — and builds a secure channel for negotiation itself, agreeing on encryption, hash, Diffie-Hellman group, and lifetime. Phase 2 then negotiates the actual security association for user traffic: which cipher protects the data, and which subnets the tunnel covers.

Every parameter in both phases must match on both ends. A Phase 1 mismatch — different DH group, different lifetime — means the tunnel never authenticates; a Phase 2 mismatch means Phase 1 succeeds but no traffic passes, which is the more confusing case because the tunnel looks "up." Reading IKE logs by phase is how you localize which parameter disagrees.

# strongSwan: a phase-1 (IKE) mismatch names the stage, not the fix
ipsec statusall
# site-a[1]: IKE SA established, 198.51.100.7...203.0.113.9
# site-a{1}: INSTALLED, TUNNEL, ESP   <- phase 2 child SA is up
# NO_PROPOSAL_CHOSEN  <- the other side rejected every offered cipher set

Site-to-Site versus Remote-Access

The two deployment shapes solve different problems. A site-to-site tunnel runs gateway to gateway and joins two whole networks — a branch office to headquarters, a data center to a cloud VPC — so every host on each side reaches the other without running any VPN software itself. The tunnel is permanent and the routing is the hard part: the two sides must have non-overlapping address space, or there is no way to route the inner packets.

A remote-access tunnel runs from one user's device to a gateway, putting that single laptop logically inside the network. It is set up on demand when the user connects, often combined with NAT traversal because the user is behind a home router. The same IKE negotiation underlies both; what differs is whether the far end is a peer network or one roaming client.