Once a system splits into services with their own databases, a transaction that touches more than one of them loses the guarantee a single database gave you for free. There is no shared commit log across services, so "all of this happens or none of it does" has to be built deliberately. You have three real options: block every participant until they all agree, let each step commit locally and undo it later if something downstream fails, or redesign the flow around asynchronous events so no cross-service transaction is needed at all. None of the three eliminates the coordination problem; each one just decides where the resulting risk gets absorbed — in blocked locks, in compensating business logic, or in a temporary window of visible inconsistency.
The core idea
Inside one database, a transaction is trustworthy because the engine enforces it: every write either commits together or rolls back together, and the database handles the coordination invisibly. Split that data across two services and you lose the shared engine. Each service's own local transaction is still perfectly reliable, the problem is coordinating several of those local transactions into one outcome, across a network, where any participant can be slow, unavailable, or owned by a different team entirely. This is also why the choice rarely stays purely technical for long: whichever option a team picks changes what an on-call engineer has to reason about when one of those services is degraded.
The three approaches
| Approach |
How it works |
Cost |
| Two-phase commit |
A coordinator holds every participant until all agree, then commits everyone at once |
Blocks on any slow or failed participant; only realistic when you control every participant |
| Compensating steps |
Each service commits locally; a failure later in the chain triggers compensating actions on the earlier steps |
No global lock, but compensations are real business logic that must be designed and tested |
| Event-driven with an outbox |
Each service commits its own change and publishes an event in the same local transaction; other services react asynchronously |
No cross-service coordination at all, but consumers must tolerate a window of eventual consistency |
None of these approaches is strictly better in isolation; each shifts the cost somewhere else. Two-phase commit moves the cost onto availability. Compensating steps move it onto engineering effort, since every compensating action is bespoke business logic with its own edge cases. Event-driven design moves it onto the consumer, which must handle data that is temporarily behind the source of truth.
How to decide
- Check whether it truly needs to span services. Sometimes a boundary is drawn wrong, and the data actually belongs together in one service.
- Identify who owns every participant. If you do not operate all of them, blocking coordination is not realistically viable.
- Decide what the business can tolerate. A visible pending state is often acceptable; a hard guarantee before responding to the user sometimes is not.
- Default to eventual consistency with an outbox before reaching for heavier coordination, and only escalate if the business genuinely cannot accept the delay.
- Write down the failure mode you are accepting, whatever you choose. A blocked lock, a failed compensation, or a stale read are all recoverable if someone designed for them in advance, and all three are outages if nobody did.
Common mistakes
- Drawing service boundaries around data that must stay transactionally consistent. If two things always update together, they may belong in one service and one database, not two.
- Assuming a message queue alone solves the dual-write problem. Writing to the database and publishing to the queue as two separate steps can fail between them, which is exactly the gap an outbox closes.
- Reaching for two-phase commit across services you do not own. A partner's slow API becomes your outage the moment you hold a lock open waiting on it.
- Skipping idempotency on the consuming side. Any retry-based recovery, whether compensation or event redelivery, needs consumers that can safely process the same message twice.
FAQ
Can I just avoid this by using one big database?
Sometimes, and it is often the simplest fix if the service split is not paying for itself yet. Not every system needs to be this distributed.
Is eventual consistency safe for money-related operations?
It is common in practice, typically ledger-style with reconciliation jobs, but it requires deliberate design: visible pending states and a clear recovery path for mismatches.
How is this different from the saga pattern specifically?
Compensating steps are one concrete pattern within this space. This is the wider decision: whether to block, compensate, or redesign around events in the first place.
Do I need a message broker for the event-driven approach?
Typically yes, or a pipeline that reads committed events off an outbox table and forwards them, which is how many teams implement this reliably.
Does a bigger, shared database make this problem go away?
It shrinks rather than disappears. Even a shared database can have logically separate schemas per team facing the same coordination question, just inside one engine instead of across a network.
Where to go next
See two-phase commit explained for the blocking option in detail, the outbox pattern explained for making the event-driven option safe, and what a race condition is for the shared-state failure mode all three approaches are ultimately designed to prevent.