Reverse Proxy vs API Gateway — System Design Explained
Proxies mediate traffic. Gateways add routing, auth, throttling, protocol translation, and observability.
A proxy is a middleman. When you use a forward proxy, your outbound requests go through it first — the proxy can filter content, cache responses, or hide your IP address. When a server uses a reverse proxy, incoming client requests hit the proxy first — the proxy can terminate TLS, balance load, cache, and route to the right backend service.
API gateways are reverse proxies with intelligence added: they know about your services and can enforce authentication, rate limiting, request transformation, and routing rules at the edge — before any request reaches your backend. One API gateway can serve as the single entry point for dozens of microservices.
In production, the typical stack looks like: clients → CDN → load balancer (L4) → Nginx/Envoy reverse proxy (L7, terminates TLS) → API gateway (auth, rate limiting, routing) → microservices. Each layer adds a capability: the CDN caches, the load balancer distributes, the reverse proxy handles protocol, and the gateway enforces policy.
Key concepts
Step-by-step approach
- 1
Client request arrives at the edge (CDN or L4 load balancer) — routes TCP traffic by IP:port without decryption.
- 2
L7 reverse proxy (Nginx, Envoy) terminates TLS: decrypts HTTPS, validates the certificate chain, and establishes a new HTTP connection to the backend.
- 3
API gateway enforces authentication (JWT validation, API key check), rate limiting (per-user or per-IP quotas), and routing rules (which path goes to which microservice).
- 4
Request reaches the target microservice — it only ever sees internal HTTP, never raw TLS or client IPs directly.
- 5
Response travels back through the same layers; the reverse proxy re-encrypts and the CDN caches the response if the headers allow it.
Key trade-offs
One gateway vs. service mesh
An API gateway is a centralized chokepoint — simple to reason about but a single point of failure and potential latency bottleneck. A service mesh (Envoy sidecars) distributes the proxy to each service, eliminating the single chokepoint at the cost of operational complexity.
Sidecar vs. shared proxy
Sidecar proxies (one per service instance) provide per-service observability and circuit breaking with zero cross-service blast radius. Shared proxies reduce resource usage but create shared fate.
Gateway auth vs. in-service auth
Validating JWTs at the gateway prevents any unauthenticated traffic from reaching services. In-service validation gives each service full context but duplicates logic. Use both for defense-in-depth.
Common pitfalls
Using a single API gateway as the only layer of defense: the gateway should enforce policy, but individual services should still validate auth tokens — defense in depth.
Not considering latency added by the gateway: each auth check, rate limit check, and routing decision adds latency. A poorly configured gateway can add 10-50ms to every request.
Forgetting to set timeouts at the gateway level: without timeouts, a slow backend service can hold open gateway connections indefinitely, eventually exhausting the gateway's connection pool.
Interview questions on this topic
Practice answering these with AI feedback → Start on CrackLab
Continue learning — explore 207 more topics on CrackLab DesignHub
From Classic HLD designs (Twitter, YouTube, Uber) to LLD patterns, distributed systems, databases, and company case studies.
Related topics
Distribute traffic across healthy backends using L4/L7 algorithms and health checks.
Define contracts between clients and services with clear resources, errors, pagination, versioning, and idempotency.
A request moves from browser to DNS, TCP, TLS, HTTP, load balancer, app, storage, and back.