Idempotency and Exactly-Once Semantics

In distributed systems, things fail in the most inconvenient ways. Not clean failures where you know something went wrong -- partial failures where you have no idea what happened. A client sends a payment request, the server processes it successfully, but the response gets lost in the network. The client, seeing a timeout, does the sensible thing: retry. And now the customer has been charged twice.

This is not a hypothetical edge case. It happens constantly in production systems. Every time a mobile app hits a flaky cell connection, every time a load balancer times out, every time a TCP connection drops mid-response -- the client needs to retry. And without idempotency, every retry is a potential duplicate.

The fix is straightforward in concept: make your operations idempotent. An idempotent operation produces the same result whether you execute it once or ten times. GET requests are naturally idempotent -- fetching a page twice gives you the same page. But POST requests (create a payment, send an email, place an order) are not idempotent by default. You have to make them so.

Idempotency Keys#

The most common approach is the idempotency key pattern. The client generates a unique identifier (typically a UUID) and attaches it to every request. The server uses this key to detect retries and return the previously computed result instead of re-executing the operation.

Here is how it works step by step:

1. Client generates a key -- a UUID like abc-123 -- and sends it as a header: Idempotency-Key: abc-123
2. Server checks the dedup store -- has this key been seen before? If yes, return the stored response immediately
3. Server processes the request -- executes the business logic, stores the result keyed by abc-123, and returns the response

If the client retries with the same key, the server finds the cached result and returns it. The operation executes exactly once regardless of how many times the request is sent.

The dedup store entry should include the idempotency key, a hash of the request body (to detect misuse where different requests reuse the same key), the response body, the HTTP status code, and a creation timestamp. Set a TTL on entries -- 24 to 72 hours is typical -- so the store does not grow unbounded.