{"post":{"slug":"webhook-retry-and-idempotency-design-2026","title":"Webhook Retry and Idempotency Design in 2026","excerpt":"Webhooks are delivered at-least-once, which means every consumer must handle retries and duplicates safely. This guide covers event ID dedup, fast acknowledgment, and idempotent handler design for 2026.","category":"code","date":"August 10, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">W</span>ebhook consumers fail in production not because the webhook itself is complicated, but because engineers assume delivery is exactly-once when every major provider documents it as at-least-once. Stripe, GitHub, Shopify, and Twilio all retry on timeout, on connection failure, and on any non-2xx response — often for a day or more with exponential backoff. If your handler is not idempotent, a slow database call or a dropped connection turns one event into a duplicate charge, a duplicate email, or a duplicate order. The fix is straightforward: acknowledge fast, deduplicate on event ID, and make every state change safe to apply twice.\n\n## What changed in 2026\n\n- **Standard `Webhook-ID` and `Webhook-Timestamp` headers spread beyond Stripe.** The emerging Standard Webhooks spec, adopted by Svix-backed providers and a growing list of platforms, standardizes signature and ID headers so consumers write one verification path instead of one per vendor.\n- **Providers extended retry windows.** Several major platforms now retry failed deliveries for three to five days with exponential backoff instead of giving up after a few hours, so dedup retention needs to match.\n- **Queue-backed ingestion became the default pattern.** Teams increasingly return 200 immediately and push the raw payload onto a queue, see [job queue comparison](/blog/job-queue-comparison-2026), rather than processing inline in the HTTP handler.\n- **Replay tooling matured.** Most provider dashboards now let you replay a specific event ID on demand, which makes idempotent handlers a hard requirement rather than a nice-to-have, since replays are routine during incident recovery.\n\n## How webhook retries actually behave\n\n| Provider | Retry trigger | Backoff | Window |\n| --- | --- | --- | --- |\n| Stripe | Non-2xx or timeout | Exponential | Up to 3 days |\n| GitHub | Non-2xx, timeout, DNS failure | Exponential | Up to 24 hours, redelivery available anytime after |\n| Shopify | Non-2xx or timeout | Exponential | Up to 48 hours, 19 attempts |\n| Twilio | Non-2xx | Exponential | Configurable, default around 4 attempts |\n\nThe pattern is consistent: anything other than a fast 2xx response is treated as failure, and failure means retry. Two consequences follow directly. First, your handler must respond within the provider's timeout, typically five to fifteen seconds, or it will be retried even if the work eventually succeeds. Second, because retries are keyed to the HTTP outcome and not to whether you actually processed the event, you will receive genuine duplicates whenever your 2xx response itself is lost to a network blip — the work succeeded, the acknowledgment did not arrive, and the provider resends the same event.\n\n## Designing the consumer\n\n1. **Verify the signature before anything else.** HMAC-verify the payload against the provider's signing secret and reject invalid or replayed (stale timestamp) requests with a 4xx before touching the database.\n2. **Deduplicate on the event ID.** Insert the provider's event ID into a `processed_webhook_events` table with a unique constraint, inside the same transaction as your side effect. A constraint violation means \"already handled\" — return 200 and stop.\n3. **Acknowledge fast, process async.** Once the event is verified and recorded as received, enqueue the actual business logic to a background worker and return 200 immediately. This decouples your response time from downstream latency.\n4. **Make the handler itself idempotent, not just deduplicated.** Dedup catches exact repeats; idempotency protects against partial failures mid-processing, see [idempotency explained](/blog/idempotency-explained-2026) for the underlying pattern. If a worker crashes after step one of three, retrying the whole job should not double-apply step one.\n5. **Guard against out-of-order delivery.** Providers do not guarantee ordering across events. Compare a version, sequence number, or `updated_at` timestamp in the payload against your stored state, and discard the event if it is older than what you already applied.\n6. **Set a dedup retention window that matches the provider's retry window.** A 24-hour retry window needs at least seven days of retained event IDs to stay safe against delayed redelivery and manual replays.\n\n## Common mistakes\n\n**Processing synchronously inside the HTTP handler.** A slow downstream call, such as an email send or a third-party API request, inside the webhook handler risks a timeout, which the provider reads as failure and retries — creating duplicates for work that already started. Acknowledge first, process second.\n\n**Deduplicating on payload hash instead of event ID.** Two legitimately different events can hash differently even when your business logic should treat them as the same operation, such as a corrected retry with an updated field. Use the provider's stable event ID instead.\n\n**Trusting arrival order.** Concurrent delivery and provider-side retries mean a `payment.updated` event can arrive before `payment.created`. Handlers that assume order silently corrupt state under load.\n\n**No dead-letter path.** When a webhook consistently fails to process because of bad data, a bug, or a downstream outage, it needs to land somewhere a human can inspect and replay it, not vanish after the provider gives up retrying.\n\n## FAQ\n\n**Is deduplicating on event ID enough, or do I still need idempotency keys?**\nEvent ID dedup catches exact redeliveries. You still need idempotent handler logic for partial failures, such as a worker crash mid-job that gets retried by your own queue rather than the provider.\n\n**How long should I keep processed event IDs?**\nAt least as long as the provider's documented retry window, plus a buffer for manual replays. Seven to thirty days is typical.\n\n**What HTTP status code should a webhook handler return on success?**\nAny 2xx. Most teams standardize on 200 with an empty or minimal body; returning 202 is also common when work is handed off to a queue and not yet complete.\n\n**Should I process the webhook payload directly or re-fetch the object from the provider's API?**\nFor anything security- or money-sensitive, re-fetch the current object state from the provider's API using the ID in the payload rather than trusting the payload body verbatim. This protects against stale or tampered data.\n\n## Where to go next\n\n- [Idempotency explained in 2026](/blog/idempotency-explained-2026)\n- [Job queue comparison in 2026](/blog/job-queue-comparison-2026)\n- [Distributed locks explained in 2026](/blog/distributed-locks-explained-2026)","hue":304,"metaTitle":"Webhook Retry and Idempotency Design 2026","metaDescription":"Webhook retry and idempotency design for 2026: dedupe by event ID, handle out-of-order delivery, and return the right status codes to stop redundant retries.","keywords":["webhook retry idempotency design 2026","idempotent webhook consumer","webhook deduplication event id","handling duplicate webhook events","webhook retry backoff strategy","webhook signature verification hmac","at least once delivery webhooks","webhook queue processing","out of order webhook events"],"tldr":["**Treat every webhook as at-least-once delivery** — providers retry on timeout, non-2xx responses, and network failure, so the same event can arrive two or more times.","**Deduplicate on the provider's event ID, not a payload hash** — store processed event IDs with a unique constraint and skip reprocessing on a repeat.","**Return 2xx fast, then process async** — acknowledge receipt inside the provider's timeout window and hand the real work to a queue.","**Handle out-of-order delivery explicitly** — compare a sequence number or timestamp in the payload before applying state changes; never assume arrival order.","**Skip:** verifying the signature after you have already mutated state — verify first, and reject unsigned or replayed payloads before touching your database."],"readTime":"5 min read","wordCount":1005},"related":[{"slug":"webhooks-deep-dive-explained-2026","title":"Webhooks deep dive in 2026: delivery, security, and reliability","excerpt":"Webhooks are HTTP callbacks that push events to your server instead of making you poll. This 2026 guide covers delivery guarantees, signature verification, and the failure modes teams hit in production.","category":"code","date":"April 21, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":314},{"slug":"dead-letter-queues-explained-2026","title":"Dead Letter Queues: Where Failed Messages Should Go","excerpt":"A message that cannot be processed will retry forever unless you catch it. Here is how dead letter queues work, what to put in one, and the replay discipline.","category":"code","date":"August 12, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":278},{"slug":"webhooks-vs-polling-2026","title":"Webhooks vs polling in 2026: which push model wins?","excerpt":"Choosing between webhooks and polling shapes your app's latency, cost, and complexity. Here is the honest 2026 tradeoff guide with real numbers.","category":"code","date":"March 3, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":307}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"webauthn-explained-2026","title":"WebAuthn Explained in 2026: How Passwordless Login Works","excerpt":"WebAuthn replaces passwords with public-key cryptography, binding each credential to the exact site it was registered for. Here is how the registration and login ceremonies work, and why passkeys build on it.","category":"code","date":"August 10, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":318},"older":{"slug":"whole-life-insurance-as-an-investment-2026","title":"Whole Life Insurance as an Investment in 2026: Does It Work","excerpt":"Whole life insurance is pitched as a way to get tax-free growth and a death benefit in one product, but the internal rate of return rarely beats a simple index fund. Here is the honest math.","category":"finance","date":"August 10, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":313}}