What are message queue delivery semantics?

In this article

1. Before you start
2. Why interviewers ask this
3. Delivery semantics from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page belongs to the distributed systems interview questions path.

Why interviewers ask this

A queue that loses an order or charges a card twice costs real money, so any design with a queue between two services has to say what happens when one step fails halfway. Interviewers use delivery semantics to see whether a candidate reasons about failure or only about the happy path.

A weak answer says "we use a queue, so messages are reliable" or "we use exactly once." A strong answer names the three semantics, says which one the default is, and shows where a crash can lose or repeat a message. A Staff-level answer adds the boundary: exactly once is real only inside the system that offers it, so every external side effect still needs an idempotent write.

The same commit-order logic applies to acknowledgment-based queues: in RabbitMQ or Amazon Simple Queue Service (SQS), acknowledging or deleting the message before processing gives at most once, and after processing gives at least once. The same reasoning appears in payment system designs, where a repeated message can mean a repeated charge, and in the broader backend interview questions.

Delivery semantics from first principles

Delivery semantics

Delivery semantics is the set of promises a message queue makes about how many times a sent message reaches its consumer when something fails, so it states what a failure is allowed to cost: a lost message, a repeated message, or neither of the two.

Idempotent operation

An idempotent operation is an operation that has the same effect whether it runs once or many times, such as writing a record under a primary key so that a repeated write overwrites the same record instead of adding a second one.

In plain words

Think of a courier who needs your signature. If you do not answer the door, he cannot tell whether you are out or the bell is broken, so he either leaves or comes back. Leaving risks the parcel never arriving. Coming back risks you receiving it twice. The analogy breaks in one place: a courier can see the parcel left behind, but a producer often cannot see whether the broker stored its message.

An acknowledgment is a short reply from the receiver that says the message arrived and was stored. At-least-once delivery is a promise that no message is lost, paid for by letting a message arrive more than once. At-most-once delivery is the opposite trade: nothing arrives twice, but a message may never arrive. The Confluent Kafka documentation (updated September 2025) describes exactly these two, plus exactly once, and says the choice trades latency against durability.

In one precise sentence, delivery semantics are decided by what a sender or reader does when it cannot tell whether the last step succeeded.

Step 1: The producer retries until the broker confirms

The producer sends a message and waits for an acknowledgment. If the acknowledgment never comes, the producer cannot know whether the broker stored the message or the reply was lost on the way back. Retrying gives at-least-once delivery: if the first copy was stored, the retry adds a second. Never retrying gives at-most-once delivery. The Confluent documentation says Kafka provides at-least-once delivery by default, and that at-most-once means disabling producer retries.

Input: a message and a lost acknowledgment. Output: either a possible duplicate in the log or a possible gap. The fix for the duplicate is the idempotent producer, available since Kafka version 0.11.0.0, which attaches a producer identifier and a sequence number to each message so the broker discards a repeat. The Strimzi write-up says that since Kafka 3.0 the producer defaults to acks=all and enable.idempotence=true. A webhook sender that retries after a timeout has the same problem.

Step 2: The consumer's commit order picks at most once or at least once

The consumer reads a message, does its work and saves its position by committing the offset. The order of those two actions is the whole story. Commit first, process second: a crash between them means the restarted consumer starts after the message, so the message is skipped, which is at-most-once. Process first, commit second: a crash between them means the restarted consumer starts at the old offset and handles the message again, which is at-least-once. Both descriptions follow the Confluent documentation.

Input: a crash between two steps. Output: a lost message or a repeated message, never both. This is why "the consumer commits its offset before or after processing" is the first thing to ask about any queue consumer. You now know that two of the three semantics come from one ordering choice. The next step shows how the third is built.

Step 3: Exactly once inside the system uses idempotence and transactions

Exactly once means no loss and no repeat. Inside Kafka it is built from two pieces. The idempotent producer removes duplicates caused by retries. A transaction, a group of writes that either all become visible or none do, lets a read-process-write job write its output messages and its input offsets together. A consumer configured with isolation.level=read_committed reads only messages from committed transactions, per the Confluent documentation.

The current Kafka producer configuration reference (Kafka 4.0 documentation, read 26 October 2026) says that when enable.idempotence is true the producer ensures exactly one copy of each message is written in the stream, and that with it false, retries after broker failures may write duplicates. Input: messages read from one topic. Output: results in another topic, each appearing once. The scope matters: this covers Kafka-to-Kafka work, which leads to the last step.

Step 4: Exactly once ends at the edge of the system

Confluent's documentation warns that many systems claim exactly-once semantics, but the claim may not cover failures outside the system. When a consumer writes to a database or calls an email or payment service, the broker cannot undo that side effect. A crash after the effect and before the commit produces a repeat.

The answer is an idempotent operation at the sink. The Confluent page suggests giving messages a primary key so that an update overwrites the same record, and the sink then ignores the repeat. This is the pattern behind idempotent workers on an at-least-once queue: the queue repeats, the worker's write does not. The sink must ignore a repeated message, so payment designs attach an idempotency key to each charge, as in how to design a payment system, and counting systems such as an ad click aggregator deduplicate by event identifier.

1 · Producer retry2 · Commit order3 · Transaction4 · Sink dedupeReplay

Step-by-step description

1. A lost acknowledgment makes the producer retry.
2. Commit before or after processing.
3. Output and offset commit together.
4. The external sink ignores repeats.

• The producer retries when an acknowledgment is lost.
• The consumer commits its offset before or after processing.
• Output and offset are written together in one transaction.
• The external sink must ignore a repeated message.

The producer retries when an acknowledgment is lost. The consumer commits its offset before or after processing. Output and offset are written together in one transaction. The external sink must ignore a repeated message.

Worked example

Take one message and crash the consumer at each point. The outcome follows from the order of the steps, not from luck. With commit first, a crash between the commit and the processing leaves zero effects, so the message is lost. With process first, a crash between the processing and the commit leaves two effects after the restart, because the broker redelivers the message.¹ With process first and an idempotent write, the same crash still delivers the message twice but leaves one effect.¹

Now scale it up with illustrative numbers: 1,000 messages and a 1% chance (illustrative) that the consumer crashes between its two steps. The expected number of messages that meet the crash is 10.¹ In that illustrative run, commit first loses 10 messages and process first repeats 10. Process first with an idempotent write still redelivers 10 messages but leaves 0 duplicate effects.¹ Apply the same 1% (illustrative) to lost producer acknowledgments: a retry without an idempotent producer adds 10 duplicate log entries, and with the idempotent producer adds 0.¹

Text description of the diagram

• Commit first loses 10 messages in the illustrative run.
• Process first repeats 10 messages in the illustrative run.
• Process first with an idempotent write leaves 0 duplicate effects.

The data are a counting exercise, not a benchmark: real crash rates differ, and you can size a dedupe store the same way as in back-of-the-envelope estimation.

Trade-offs and edge cases

At most once never repeats a message but may lose one. At least once never loses a message but may repeat one. Exactly once inside a system needs the output and the offset written together. The choice is made per application, and Confluent frames it as a trade between latency and durability, so at-most-once is a deliberate choice for data where a loss is cheaper than a delay, not a bug.

Text description of the diagram

• At most once never repeats a message but may lose one.
• At least once never loses a message but may repeat one.
• Exactly once inside a system needs the output and the offset written together.

Edge cases to name. First, an idempotent producer deduplicates only its own retries into the log; it does not make the consumer's side effects safe. Second, a dedupe check that sits outside the sink can race with the effect: store the processed identifier in the same transaction as the effect where the sink allows it. Third, a sink with no key and no transaction, such as a plain email call, cannot be made exactly once by the queue alone. Fourth, waiting for acknowledgments adds latency to every send, which shows up in tail latency under load.

How to answer it in an interview

The spoken structure: name the three promises, show the commit order, choose at least once with an idempotent write, then state the boundary of exactly once.

Common mistakes

• "At least once means messages can be lost." (wrong) It means a message is never lost but may arrive more than once. (right) "At least once: no loss, possible duplicates."
• "Exactly once is a switch I turn on." (wrong) The claim has a boundary and does not cover side effects outside the system. (right) "Exactly once inside the queue, an idempotent write at the sink."
• "The idempotent producer makes the whole pipeline exactly once." (wrong) It only removes duplicates from producer retries. (right) "The consumer side still needs idempotence or transactions."
• "At most once is just a weaker version." (wrong) It is a deliberate choice, made by committing first or disabling retries. (right) "I would use it where a loss is cheaper than the added latency."
• "Kafka's default is exactly once." (wrong) The documented default is at least once; newer producers add idempotence by default. (right) "Default at least once, exactly once by configuration and scope."

Follow-up questions

What is the difference between at-most-once, at-least-once and exactly-once delivery?

At most once may lose a message but never repeats it. At least once never loses a message but may repeat it. Exactly once neither loses nor repeats within the system's boundary. See the distributed systems interview path for neighboring topics.

How do idempotent workers make at-least-once delivery safe?

The queue may deliver a message twice, so the worker makes its effect idempotent: write under a primary key, or record the message identifier with the effect in one transaction, so the second run changes nothing. The queue provides no loss, the worker provides no duplicate effect.

Is exactly-once delivery possible?

Inside one system, yes, with idempotent producers and transactions that write output and offsets together. Across a boundary to an external service, the receiving side must take part through idempotent writes or transactions, so the honest phrase is exactly-once processing within a stated scope.

What happens if a consumer crashes before it commits its offset?

After the restart it reads from the last saved offset and handles the same message again. That is at-least-once behavior, so the processing step must tolerate a repeat.

When would you choose at-most-once delivery?

Choose it when a lost message is cheaper than a delay or a duplicate, for example a frequent reading that the next message replaces. It is made by committing before processing or by disabling producer retries.

Sources

1. Kafka Message Delivery Semantics (Confluent documentation) — Confluent (vendor documentation; accessed 2026-10-26), 26 Sep 2025
2. Exactly-once semantics with Kafka transactions — Strimzi (CNCF project blog; accessed 2026-10-26), 3 May 2023
3. Producer configs (Apache Kafka 4.0 documentation) — Apache Kafka (the page carries no date, so the read date is given), 26 Oct 2026