An API gateway is a single entry point that sits in front of a collection of backend services, routing each incoming request to the right one while handling cross-cutting concerns like authentication, rate limiting, and logging in a single place instead of duplicating that logic across every service. It is one of the first pieces of infrastructure teams reach for once a system grows past a handful of services, and one of the most commonly over-engineered pieces of infrastructure in systems that have not.
What changed in 2026
- Managed gateways closed the gap with self-hosted ones. Cloud provider gateways (API Gateway, API Management, Apigee) now support most of what teams used to need Kong or custom Envoy configs for, reducing operational burden for teams that do not need the extra flexibility.
- Gateway-level LLM proxying became a real use case. Many gateways now ship built-in support for routing and rate-limiting requests to AI model APIs, treating them as just another upstream with their own cost and latency profile.
- eBPF-based gateways emerged as a lower-overhead alternative to traditional reverse-proxy architectures for teams running extremely high request volumes.
What an API gateway actually does
At its core, a gateway performs a small set of jobs, repeatedly and reliably:
- Routing — mapping a path or hostname to the correct backend service.
- Authentication and authorization — validating tokens or API keys before a request ever reaches application code.
- Rate limiting and throttling — protecting backend services from being overwhelmed by a single client, complementary to any rate limiting your services do internally.
- Request/response transformation — reshaping payloads, aggregating multiple backend calls into one response, or stripping internal headers before a response leaves the system.
- Observability — centralized logging, metrics, and tracing for every request that crosses the boundary.
API gateway vs load balancer vs reverse proxy
These three terms get used interchangeably, but they solve different problems.
| Component |
Primary job |
Operates at |
| Load balancer |
Distribute traffic across identical instances |
Network/transport layer |
| Reverse proxy |
Forward requests to a backend, optionally caching or terminating TLS |
Application layer |
| API gateway |
Route, authenticate, rate-limit, and transform requests across many distinct services |
Application layer, API-aware |
A gateway is best understood as a reverse proxy with API-specific policy logic layered on top — it knows about routes, scopes, and quotas, not just "forward this connection somewhere healthy."
When you actually need one
A gateway earns its keep once you have multiple services that need consistent authentication, rate limiting, or client-facing versioning, and duplicating that logic in every service has become a real maintenance burden. It is also valuable when external partners or third-party developers consume your API and you need a policy enforcement point that is not application code.
A gateway is usually premature if you have a single service, a monolith, or two or three internal services with no external API surface — in that case, a plain reverse proxy or even direct routing is simpler and has fewer moving parts to operate.
Common pitfalls
Putting business logic in the gateway. Gateways should enforce policy — auth, quotas, routing — not make decisions specific to your domain. Business logic in the gateway layer is hard to test and creates a second place every rule has to be maintained.
Making the gateway a single point of failure. A gateway that is not deployed redundantly, with proper health checks and failover, turns one component into an outage risk for every service behind it.
Under-provisioning for the added latency hop. A gateway adds a network hop to every request. For latency-sensitive paths, that overhead needs to be measured, not assumed to be negligible.
FAQ
Do microservices always need an API gateway?
No. Many small microservice systems run fine with a simpler reverse proxy or service mesh sidecar handling routing, adding a dedicated gateway once the number of services and the need for centralized policy grows.
Is a service mesh the same thing as an API gateway?
No — a service mesh handles service-to-service traffic inside the system, while a gateway typically handles north-south traffic from external clients into the system. Some architectures use both.
What is the biggest performance cost of an API gateway?
The extra network hop and any transformation work it performs. Well-configured gateways add low single-digit milliseconds of overhead; poorly configured ones with heavy transformation logic can add much more.
Can an API gateway replace authentication logic in each service?
It can handle the initial token validation, but services should still enforce their own authorization rules for what an authenticated caller is allowed to do — never trust the gateway as the only layer of defense.
Where to go next