An idempotency key is a token the client generates and attaches to a request so the server can recognize a retry and return the original result instead of executing the operation again. The concept is simple to state; the implementation has several details that determine whether it actually works under concurrency and partial failure. How you store the key, what you hash alongside it, and how you lock against simultaneous duplicate requests are what separate a correct implementation from one that only looks correct in testing.
How it works
The client generates a unique key before the first attempt and sends it on every retry of that same logical request.
1. Client generates key K (UUID or ULID) before the first attempt.
2. Client -> POST /charges Idempotency-Key: K
3. Server checks storage for K.
- Not found: process the charge, store (K, request_hash, response), return response.
- Found, same request_hash: return the stored response without reprocessing.
- Found, different request_hash: reject with 422 -- same key, different payload.
4. Network failure or timeout -> client retries with the same key K.
5. Server sees K already stored -> returns the original response.
The request hash is what separates a correct implementation from a naive one. Without it, a client that accidentally reuses a key for a genuinely different request gets the wrong response replayed back silently.
Building the storage layer
CREATE TABLE idempotency_keys (
key TEXT PRIMARY KEY,
request_hash TEXT NOT NULL,
response_status INT NOT NULL,
response_body JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
locked_at TIMESTAMPTZ
);
import hashlib, json
def request_hash(body: dict) -> str:
canonical = json.dumps(body, sort_keys=True)
return hashlib.sha256(canonical.encode()).hexdigest()
def handle_request(key, body):
h = request_hash(body)
existing = db.query("SELECT * FROM idempotency_keys WHERE key = %s", key)
if existing:
if existing.request_hash != h:
return error(422, "Idempotency key reused with a different request")
return existing.response_body # replay, do not reprocess
# Insert first, relying on the primary key to block concurrent duplicates
try:
db.execute(
"INSERT INTO idempotency_keys (key, request_hash, response_status, response_body) "
"VALUES (%s, %s, %s, %s)",
key, h, 202, {"status": "processing"},
)
except UniqueViolation:
return error(409, "Request with this key is already being processed")
result = do_the_actual_work(body)
db.execute(
"UPDATE idempotency_keys SET response_status = %s, response_body = %s WHERE key = %s",
result.status, result.body, key,
)
return result.body
The insert-first pattern is what handles concurrency: two simultaneous requests with the same key race on the same primary key insert, and the database's uniqueness guarantee ensures only one wins.
Storage backend comparison
| Backend |
Concurrency guarantee |
TTL/expiry |
Best for |
| Postgres unique constraint |
Strong (ACID) |
Manual cleanup job or pg_cron |
Systems already on Postgres; strongest correctness |
| Redis with SETNX |
Strong, single-node; needs care in cluster mode |
Native EXPIRE |
High-throughput APIs already using Redis |
| DynamoDB conditional write |
Strong |
Native TTL attribute |
AWS-native serverless stacks |
| In-memory cache only |
None across replicas |
N/A |
Never use for idempotency across multiple servers |
Whichever backend you pick, the requirement is the same: the check-and-store step must be atomic, or two concurrent requests can both slip through before either has written its key.
Common mistakes
Checking for the key, then inserting it, as two separate non-atomic steps. This race condition lets two concurrent requests both pass the "not found" check before either writes, defeating the entire mechanism. Use a single atomic insert or conditional write.
Not hashing the request body. Storing only the key means a client that mistakenly reuses a key for a different request gets the old response silently replayed instead of a clear error.
Storing the key only in application memory or a single-node cache. Any deployment with more than one server instance needs a shared store, such as Postgres, Redis, or DynamoDB, or a retry can land on a different instance that has never seen the key.
No expiry policy. An idempotency key table with no TTL grows forever and slows down lookups. Set a TTL that covers your realistic retry window, hours to days rather than months, and clean up on a schedule.
FAQ
Who generates the idempotency key, the client or the server?
The client, always, and before the first attempt. If the server generates it, a request that times out before the client receives a response has no key to retry with.
What should the server do if the same key arrives with a different request body?
Reject it, typically with 422 or 409, rather than either silently processing the new payload or silently returning the old response. A changed payload under a reused key indicates a client bug.
How long should an idempotency key be stored?
Long enough to cover the realistic retry window for your system, often 24 hours for a payment API, longer for asynchronous workflows. Expire and clean up on a schedule rather than storing indefinitely.
Is a database unique constraint alone enough for idempotency?
It prevents a duplicate row, but you also need to store and return the original response, and ideally hash the request body to catch a reused key with a different payload. The constraint alone only solves the concurrency half of the problem.
Where to go next
For the broader concept, which HTTP methods are idempotent by nature and when you need a key at all, see idempotency explained in 2026. If you are rolling out idempotency keys as part of a version change, API deprecation strategy for 2026 covers keeping old and new clients working during the transition, and how to write a database migration in 2026 covers adding the storage table safely.