{"post":{"slug":"how-to-monitor-a-service-2026","title":"How to monitor a service in 2026: metrics, logs, traces, and alerts that work","excerpt":"A service without observability is a black box you only look inside after something breaks. Here is the 2026 guide to metrics, logs, traces, and alerts.","category":"code","date":"April 1, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">M</span>onitoring is the difference between \"we noticed before the customer did\" and \"we found out from Twitter.\" Good observability does not mean having dashboards — it means being able to answer \"what is broken, where, and why\" in minutes. The 2026 stack is more standardised than ever, with OpenTelemetry as the single collection layer and managed backends that have gotten cheaper and more capable.\n\n## What changed in 2026\n\n- **OpenTelemetry reached GA for all three signals** — metrics, logs, and traces are all stable, and most major frameworks auto-instrument via zero-code agents.\n- **SLO tooling went mainstream.** Sloth, OpenSLO, and built-in SLO features in Datadog, Grafana Cloud, and Google Cloud Monitoring mean teams can define error budgets without custom alerting math.\n- **AI-assisted anomaly detection** is now standard in most managed observability platforms — it reduces the need to hand-tune thresholds on low-signal metrics.\n- **eBPF-based observability** (Cilium, Grafana Beyla) can capture traces and metrics from unmodified binaries at the kernel level — useful for legacy services you cannot recompile.\n\n## The three pillars\n\n| Signal | What it answers | Storage | Query |\n| --- | --- | --- | --- |\n| Metrics | \"Is the system healthy right now?\" | Time-series DB (Prometheus, Mimir) | PromQL |\n| Logs | \"What exactly happened?\" | Log store (Loki, Elasticsearch) | LogQL / Lucene |\n| Traces | \"Where did the latency go?\" | Trace store (Tempo, Jaeger) | TraceQL |\n\n## Instrument with OpenTelemetry\n\nOne SDK, all three signals, any backend:\n\n```typescript\n// Node.js — zero-code auto-instrumentation\nimport { NodeSDK } from '@opentelemetry/sdk-node';\nimport { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';\nimport { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';\n\nconst sdk = new NodeSDK({\n  traceExporter: new OTLPTraceExporter({\n    url: 'http://otel-collector:4318/v1/traces',\n  }),\n  instrumentations: [getNodeAutoInstrumentations()],\n});\n\nsdk.start();\n```\n\nAdd manual spans for business operations that auto-instrumentation misses:\n\n```typescript\nimport { trace } from '@opentelemetry/api';\nconst tracer = trace.getTracer('order-service');\n\nasync function processOrder(orderId: string) {\n  return tracer.startActiveSpan('processOrder', async (span) => {\n    span.setAttribute('order.id', orderId);\n    try {\n      const result = await doWork(orderId);\n      span.setStatus({ code: SpanStatusCode.OK });\n      return result;\n    } catch (err) {\n      span.recordException(err as Error);\n      span.setStatus({ code: SpanStatusCode.ERROR });\n      throw err;\n    } finally {\n      span.end();\n    }\n  });\n}\n```\n\n## The four golden signals\n\nAlert on these, in roughly this priority order:\n\n| Signal | Description | PromQL example |\n| --- | --- | --- |\n| **Error rate** | % requests returning 5xx | `rate(http_requests_total{status=~\"5..\"}[5m])` |\n| **Latency** | p99 response time | `histogram_quantile(0.99, rate(http_duration_seconds_bucket[5m]))` |\n| **Saturation** | How full is the system? | `sum(rate(http_requests_total[5m])) / limit` |\n| **Traffic** | Requests per second | `rate(http_requests_total[5m])` |\n\n## Structured logging\n\nUnstructured logs are hard to query at scale. Emit JSON:\n\n```typescript\nimport pino from 'pino';\nconst logger = pino();\n\nlogger.info(\n  { orderId, userId, durationMs: 142 },\n  'Order payment completed'\n);\n// {\"level\":30,\"time\":...,\"orderId\":\"123\",\"userId\":\"u456\",\"durationMs\":142,\"msg\":\"Order payment completed\"}\n```\n\nAlways include: `trace_id` (from the active span context), `service.name`, `environment`, and the relevant business entity IDs. This makes log-to-trace correlation instant.\n\n## SLOs and alerts\n\nDefine what \"good\" looks like before writing any alert:\n\n```yaml\n# OpenSLO format\napiVersion: openslo/v1\nkind: SLO\nmetadata:\n  name: orders-api-availability\nspec:\n  service: orders-api\n  indicator:\n    metadata:\n      name: success-rate\n    spec:\n      ratioMetric:\n        good:\n          metricSource:\n            type: Prometheus\n            spec:\n              query: sum(rate(http_requests_total{status!~\"5..\"}[5m]))\n        total:\n          metricSource:\n            type: Prometheus\n            spec:\n              query: sum(rate(http_requests_total[5m]))\n  objectives:\n    - displayName: \"99.9% availability\"\n      target: 0.999\n      timeWindow: 30d\n```\n\nAlert when you are burning your error budget too fast — not on raw error count.\n\n## How to pick observability tools\n\n1. **Greenfield, cloud-native?** Grafana Cloud (Prometheus + Loki + Tempo) or Datadog — both are mature, OpenTelemetry-native, and have free tiers.\n2. **AWS-centric?** CloudWatch + X-Ray is the path of least resistance; add Managed Prometheus/Grafana if you need richer querying.\n3. **Self-hosted required?** Prometheus + Grafana + Loki + Tempo is the canonical open-source stack.\n4. **Small team, low ops budget?** Axiom or Better Stack (Logtail) — generous free tiers, zero infrastructure.\n5. **Enterprise with compliance needs?** Datadog or New Relic — richer RBAC, audit logs, and SOC 2 certifications.\n\n## Common mistakes\n\n**Alerting on infrastructure metrics first.** High CPU is not an incident; high error rate is. Alert on symptoms; use metrics for diagnosis.\n\n**No runbooks.** Every alert should link to a runbook that answers \"what do I do right now?\" without waking someone up to think from scratch.\n\n**Missing trace IDs in logs.** Without `trace_id` in log lines, correlating a log event to a trace requires guesswork. Inject it automatically via your logging middleware.\n\n**Alert thresholds set on gut feel.** Use historical data to set thresholds. If an alert fires every deploy, it is tuned wrong.\n\n**No on-call rotation documentation.** The monitoring stack is only as good as the humans who respond to it.\n\n## What to skip\n\n- **A dashboard for every metric.** Build dashboards for the four golden signals per service; create others on demand, not speculatively.\n- **Alerting on every 5xx.** Alert on error *rate* over a window, not individual errors — a brief spike during a deploy should not page anyone.\n- **Logging sensitive data.** PII, tokens, and passwords in log lines are compliance and security liabilities. Scrub before logging.\n\n## FAQ\n\n**Do I need all three observability pillars for a small service?**\nMetrics and structured logs get you far. Traces pay off most in microservices or anywhere latency is hard to attribute. Start with metrics + logs; add tracing when you have multi-service calls.\n\n**What is the difference between monitoring and observability?**\nMonitoring answers known questions (\"is the error rate too high?\"). Observability lets you ask new questions without redeploying (\"why is this specific user's request slow?\"). You need both.\n\n**How many alerts is too many?**\nIf the average alert requires no action more than 10% of the time, it should be removed or downgraded. A handful of high-quality alerts beats dozens of noisy ones.\n\n**How do I monitor a serverless function?**\nUse structured logging with the platform logger (CloudWatch Logs, Vercel Logs), instrument with OpenTelemetry via a layer, and track cold start rate and duration alongside error rate and latency.\n\n## Where to go next\n\nSee [How to profile slow code in 2026](/blog/how-to-profile-slow-code-2026), [Logging explained in 2026](/blog/logging-explained-2026), and [How to set up CI/CD in 2026](/blog/how-to-set-up-ci-cd-2026).","hue":319,"metaTitle":"How to Monitor a Service in 2026: Metrics, Logs, Traces","metaDescription":"Monitor your service in 2026 with OpenTelemetry, SLOs, structured logs, and distributed traces. Set alerts that fire on symptoms and avoid alert fatigue.","keywords":["how to monitor a service 2026","service observability 2026","opentelemetry guide 2026","metrics logs traces","slo sli error budget 2026","structured logging best practices","distributed tracing 2026","alerting best practices","prometheus grafana 2026","alert fatigue prevention"],"tldr":["**The three pillars are metrics, logs, and traces** — each answers different questions; you need all three for production-grade observability.","**Instrument with OpenTelemetry** — vendor-neutral, widely supported, and the 2026 standard for all three signal types.","**Alert on symptoms, not causes** — page on high error rate and high latency, not on CPU usage or memory.","**Define SLOs before writing alerts** — what error rate is acceptable over what window? Your alerts should defend that target.","**Skip:** alerting on everything — alert fatigue means real incidents get ignored; ruthlessly prune noisy alerts."],"readTime":"5 min read","wordCount":1025},"related":[{"slug":"best-observability-tools-2026","title":"Best observability tools in 2026: see what your system is doing","excerpt":"A practical comparison of observability tools in 2026 - Datadog, Grafana, OpenTelemetry, and more - covering logs, metrics, traces, and cost control.","category":"code","date":"May 27, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":212},{"slug":"observability-vs-monitoring-2026","title":"Observability vs monitoring in 2026: what you actually need to run production","excerpt":"Monitoring tells you something broke; observability tells you why. In 2026 the distinction has real tooling and cost consequences every engineering team faces.","category":"code","date":"April 7, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":319},{"slug":"logging-explained-2026","title":"Logging explained in 2026: structured logs that help you debug production","excerpt":"Good logging is the difference between fixing a production bug in 10 minutes and 10 hours. This 2026 guide covers structured logging, log levels, and observability.","category":"code","date":"April 20, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":310}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"how-to-mock-an-api-2026","title":"How to mock an API in 2026","excerpt":"Mocking APIs lets you develop and test in isolation without hitting real services. This 2026 guide covers MSW, local stubs, and contract testing.","category":"code","date":"April 1, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":321},"older":{"slug":"how-to-open-a-high-yield-savings-account-2026","title":"How to open a high-yield savings account in 2026: a step-by-step guide","excerpt":"A high-yield savings account can pay many times more than a traditional bank account. Here is how to pick one and open it in under 10 minutes in 2026.","category":"finance","date":"April 1, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":282}}