A message broker is a piece of infrastructure that sits between services, accepting messages from producers and routing them to consumers without the two sides needing a direct connection. That indirection is the entire point: a service publishing an event does not need to know who is listening, how many consumers exist, or whether they are online right now. It is one of the standard tools for building systems that need to scale independently or tolerate parts of the system being temporarily down.
What changed in 2026
- Managed broker services matured further, with more teams defaulting to hosted Kafka, SQS, or Pub/Sub-equivalent offerings instead of operating brokers themselves, shifting the operational burden to the provider.
- Exactly-once semantics became easier to get right across major broker platforms, reducing (though not eliminating) the amount of manual idempotency handling application code needs.
- Event schema management tightened, with schema registries and enforced contracts becoming a default expectation rather than an advanced add-on for teams running broker-based architectures at scale.
How a message broker actually works
A producer sends a message to the broker instead of directly to a consumer. The broker stores it (briefly or durably, depending on configuration) and delivers it to one or more consumers based on the pattern in use — a queue or a topic. The consumer processes the message and, in most configurations, acknowledges receipt so the broker knows not to redeliver it. This decoupling means the producer can keep working even if a consumer is temporarily down, and new consumers can be added without changing the producer at all.
Queues vs topics
| Pattern |
Behavior |
Typical use |
| Queue (point-to-point) |
Each message is delivered to exactly one consumer |
Distributing work across a pool of workers |
| Topic (pub/sub) |
Each message is delivered to every subscriber |
Broadcasting an event to multiple independent services |
| Log (e.g. Kafka-style) |
Messages persist in order; consumers track their own position |
Event streaming, replay, multiple readers at different speeds |
Queues are for work distribution — you want each task handled once. Topics are for event broadcasting — you want every interested service to know something happened, independently.
Delivery guarantees, and why they matter
At-most-once: message may be lost, never duplicated
At-least-once: message may be duplicated, never lost
Exactly-once: message delivered once, no loss or duplication (hardest to guarantee)
Most systems use at-least-once delivery because it is achievable without exotic infrastructure — but it means your consumer must be idempotent, since the same message can arrive twice. Designing for that upfront (using message IDs to detect duplicates, for example) avoids a class of bugs that only show up under real production load.
Kafka, RabbitMQ, and managed alternatives
Kafka is built for high-throughput event streaming with replay and long retention — a good fit for event-driven architectures and analytics pipelines. RabbitMQ is built around flexible routing and traditional queueing, often simpler to reason about for classic task-queue use cases. Managed options (cloud provider queues and pub/sub services) trade some flexibility for near-zero operational overhead, a reasonable default unless you have a specific reason to self-host.
When not to use a message broker
If two services can call each other directly with an ordinary API request and the failure modes are acceptable, a broker adds infrastructure and failure modes of its own — message ordering, redelivery, dead-letter handling — without a corresponding benefit. Reach for a broker when you need to decouple services in time (one is down, the other keeps working), decouple in scale (one producer, many consumers), or need an audit trail of events.
FAQ
Is a message broker the same as a message queue?
A queue is one pattern a broker can implement. "Broker" is the broader term for the infrastructure; queues and topics are patterns it supports.
Do I need Kafka for a small application?
Usually not. Kafka's strengths — high throughput, long retention, replay — matter at a scale most small applications never reach. A simpler managed queue is often the better starting point.
What happens if a consumer crashes before acknowledging a message?
With at-least-once delivery, the broker redelivers the message after a timeout, which is why consumers need to handle duplicate processing safely.
Can a message broker guarantee message order?
Often within a partition or queue, yes, but guaranteeing global order across a distributed system usually requires giving up some throughput or parallelism.
Where to go next