Distributed tracing follows a single request as it travels across every service, queue, and database call it touches, stitching the whole journey back together so an engineer can see exactly where time was spent and where something failed. In a monolith, a stack trace and a profiler get you most of the way to understanding a slow request. In a microservices architecture, the request might cross a dozen services, and none of them alone can tell you the full story — that is the gap distributed tracing fills.
What changed in 2026
- OpenTelemetry is now the default, not one option among many. Nearly every major tracing backend — Jaeger, Zipkin, Honeycomb, Datadog, Grafana Tempo — consumes OpenTelemetry data natively, which means instrumentation is no longer locked to a single vendor.
- Auto-instrumentation got significantly better. For common frameworks and languages, OpenTelemetry agents now capture spans for HTTP calls, database queries, and queue operations with little to no manual code changes.
- Trace-based sampling decisions moved closer to the edge. Tail-based sampling — deciding whether to keep a trace after seeing how it ended, not before — became more accessible via collector-level configuration rather than custom infrastructure.
The core concepts: spans and traces
A span represents a single unit of work — an HTTP request, a database query, a function call you chose to instrument. Each span records a start time, a duration, and metadata (tags) about what happened. A trace is the full tree of spans generated by one originating request, all sharing the same trace ID.
trace_id: 8f3a... (one request, start to finish)
span: api-gateway (12ms)
span: auth-service (3ms)
span: order-service (140ms)
span: inventory-service (95ms)
span: postgres query (80ms)
span: payment-service (30ms)
The visualization this produces — often called a "waterfall" or "flame graph" — makes it immediately obvious that in this example, the inventory service's database query is the dominant cost, not the order service itself.
Why context propagation is the hard part
Building the trace tree above requires every service in the chain to receive the trace ID (and its own parent span ID) from the caller, and pass it along to whatever it calls next — over HTTP headers, message queue metadata, or gRPC context. If any single hop drops that context — a service that does not forward a header, a queue message that strips metadata — the trace breaks into disconnected fragments at exactly the point that mattered most. Getting propagation right, consistently, across every service and every protocol in use, is usually the majority of the real engineering effort in adopting tracing, not the dashboards.
Tracing tools compared
| Tool |
Type |
Notes |
| OpenTelemetry |
Instrumentation standard |
Vendor-neutral; the collection layer most teams standardize on |
| Jaeger |
Open-source backend |
Self-hosted, CNCF project, integrates with OpenTelemetry |
| Grafana Tempo |
Open-source backend |
Cost-efficient storage, pairs with Grafana dashboards |
| Datadog APM |
Commercial |
Fully managed, strong correlation with logs and metrics |
| Honeycomb |
Commercial |
Built around high-cardinality querying of trace data |
Sampling: what you can actually afford to keep
Tracing every single request at high scale is expensive to store and mostly redundant — thousands of identical successful requests do not need individual traces. Common strategies:
- Head-based sampling decides whether to trace a request at the start, based on a fixed percentage. Simple, cheap, but can miss rare slow or failed requests.
- Tail-based sampling waits until a request completes, then decides whether to keep the trace based on outcome — always keeping errors and outliers, dropping routine fast successes. More useful, more infrastructure to run.
- Priority sampling biases toward specific high-value paths (checkout, login) regardless of outcome.
Common pitfalls
Instrumenting services but not queues or async work. Traces that stop at a message queue boundary lose the ability to follow work that happens asynchronously, which is often where the interesting latency lives.
Sampling too aggressively too early. Teams new to tracing often set sampling rates low from day one and never see the rare, expensive traces that would have justified the investment.
Treating tracing as separate from logs and metrics. The most value comes from correlating a trace ID with logs and metrics for the same request, not from tracing in isolation.
FAQ
Do I need distributed tracing if I only have three or four services?
It helps sooner than most teams expect — even a handful of services can produce confusing cross-service latency that logs alone will not explain. It becomes close to mandatory well before you reach a dozen services.
Is distributed tracing the same as an api gateway's request logging?
No. Gateway logging typically shows one hop — client to gateway. Tracing follows the request through every subsequent hop, which the gateway alone cannot see.
How much does tracing cost to run at scale?
It depends heavily on sampling strategy and retention. Storing full traces for every request at high volume is expensive; a well-tuned sampling strategy keeps costs proportional to what is actually useful to debug.
Can tracing help during a chaos engineering experiment?
Yes — it is one of the primary tools for observing the actual effect of an injected failure, showing exactly which downstream calls slowed down or failed as a result.
Where to go next