An API gateway and a service mesh both route traffic, enforce policy, and add observability, which is why they get confused constantly — but they operate on different traffic and solve different problems. A gateway sits at the edge of your system, handling north-south traffic: requests coming in from external clients — browsers, mobile apps, third-party integrators. A service mesh sits inside your system, handling east-west traffic: requests between your own services. The short version of where each fits: gateway at the boundary, mesh in the interior, and most architectures past a certain size eventually run both.
The core idea
A gateway's job is client-facing: authenticate external callers, enforce rate limits per API key or customer, version and route public-facing endpoints, and often transform requests into whatever shape internal services expect. It is the single, deliberate front door — one entry point that external traffic must pass through.
A mesh's job is internal: give every service-to-service call mutual TLS, consistent retries and timeouts, fine-grained traffic shifting for canary releases, and uniform telemetry — all without any single call being a designated "entry point," because every service in the mesh talks to every other service it needs to directly.
Side-by-side comparison
| Dimension |
API Gateway |
Service Mesh |
| Traffic direction |
North-south (external client to system) |
East-west (service to service) |
| Typical deployment |
Single logical entry point (often HA-clustered) |
Sidecar proxy per service instance |
| Primary concerns |
Client auth, rate limiting, versioning, transformation |
mTLS, retries, circuit breaking, internal observability |
| Common tools |
Kong, Apigee, AWS API Gateway, cloud-native gateways |
Istio, Linkerd, Consul Connect, Cilium |
| Failure mode if down |
External clients cannot reach the system at all |
Internal calls may fall back to direct networking, degraded but not always fully down |
The overlap that causes confusion: both can do routing, both can do rate limiting, both can do observability. The difference is scope and audience — a gateway's rate limit protects your system from external abuse; a mesh's retry policy protects one internal service from another's transient blips.
Common mistakes
- Using an API gateway for internal service-to-service calls. Routing every internal call through a centralized gateway adds latency and a scaling bottleneck that a mesh's distributed sidecar model does not have. Reserve the gateway for the system's actual edge.
- Running a full service mesh with no real internal traffic complexity. If you have three or four internal services with simple, stable call patterns, mesh operational overhead often costs more than the consistency it buys — see when a service mesh is worth it.
- Duplicating policy in both layers with no clear ownership. Rate limiting configured independently at the gateway and inside mesh policy, disagreeing with each other, is a common source of confusing, hard-to-debug throttling. Decide which layer owns which concern.
- Ignoring the added latency of a double hop. A request that passes through a gateway and then several mesh-proxied service calls accumulates hops. Measure end-to-end latency budget with both layers in the path, not just one at a time.
FAQ
Can I use an API gateway and a service mesh together?
Yes, and it is the common setup at scale — the gateway handles the external boundary, the mesh handles everything behind it. They are complementary layers, not competing choices.
Which one should I adopt first?
Almost always the gateway, since it solves an immediate, visible problem — a single entry point for external clients — while a mesh's value only shows up once you have enough internal services for inconsistent networking code to hurt.
Does a service mesh replace API authentication?
No. A mesh secures service-to-service calls with mTLS; it does not replace validating an external client's identity or API key at the gateway. Those are different trust boundaries.
Is Kong an API gateway or a service mesh?
Kong is primarily an API gateway, though its Kubernetes Ingress and mesh-adjacent offerings blur the line. For genuine internal service-to-service mesh features — sidecar-based mTLS and traffic shifting between internal services — teams still typically reach for Istio, Linkerd, or Consul Connect.
Where to go next
For a deeper look at what a mesh specifically adds over plain internal networking, see service mesh explained. If your gateway is starting to accumulate client-specific transformation logic, the backend-for-frontend pattern is often the cleaner place for that to live. And for the REST fundamentals both layers ultimately route traffic to, how to build a REST API in Node is a solid baseline.