Reliable RabbitMQ: Preventing Message Loss, Duplicates, and Ordering Issues

Building a Reliable Messaging System

Posted by Martín Peveri on October 31, 2025 · 9 mins read

“Duplicate messages aren’t a bug, they’re the price of reliability.”

Introduction

In distributed systems, “never lose a message” sounds simple, until you try to guarantee it. Networks fail, consumers crash, brokers restart, and suddenly you’re facing duplicated or lost events.

In this article, we’ll explore how to build a reliable, idempotent, and ordered message pipeline using RabbitMQ. We’ll go through real-world failure scenarios and the patterns that keep your system consistent, even under chaos.

1. Understanding the Message Lifecycle

A message travels across three stages:

  1. Producer → Broker (publishing)
  2. Broker → Consumer (delivery)
  3. Consumer → Database / Side-effect (processing)

Each stage can fail independently. Let’s see how to protect all of them.

2. Producer Safety, The Outbox Pattern

Publishing a message outside of your main transaction is dangerous:

  • The transaction could be committed, but the message never gets sent.
  • Or the message is sent, but the transaction rollback.

To solve this, we use the Outbox Pattern: store outgoing messages in your database, within the same transaction that triggers them.

# app/services/orders/create_order.rb
class CreateOrder
  def call(order_params)
    ActiveRecord::Base.transaction do
      order = Order.create!(order_params)

      OutboxEvent.create!(
        event_type: "OrderCreated",
        payload: { id: order.id, total: order.total }
      )
    end
  end
end

Then, a background job (dispatcher) safely publishes these events to RabbitMQ with publisher confirms:

# app/jobs/outbox_dispatcher.rb
class OutboxDispatcher
  def call
    connection = Bunny.new.start
    channel = connection.create_channel
    exchange = channel.direct('events', durable: true)

    OutboxEvent.pending.find_each do |event|
      channel.confirm_select
      exchange.publish(event.payload.to_json,
                       routing_key: event.event_type,
                       persistent: true)

      channel.wait_for_confirms
      event.mark_as_sent!
    end
  ensure
    connection.close
  end
end

NOTE: Another popular alternative is to use tools like Debezium to send the messages from the outbox table to RabbitMq.

Guarantees:

  • No messages are lost between your app and RabbitMQ.
  • Everything is transactional and recoverable.

3. Broker Safety, Durability and Persistence

RabbitMQ won’t magically persist everything, you need to configure it properly.

Make sure:

  • The queue is durable
  • The message is persistent
channel.queue('orders', durable: true)

exchange.publish(payload.to_json, persistent: true)

Guarantees:

  • Messages survive broker restarts.
  • No loss even if RabbitMQ crashes mid-flight.

4. Consumer Safety, Acks and Idempotency

Once RabbitMQ delivers a message, it expects an acknowledgement (ack). If the consumer crashes before sending the ack, RabbitMQ re-delivers the message.

That means:

  • RabbitMQ guarantees at-least-once delivery, not exactly-once.

So you must design your consumers to be idempotent, able to process the same message multiple times without side effects.

Example: Idempotent Consumer in Ruby

class OrderCreatedConsumer
  def call(delivery_info, properties, body)
    payload = JSON.parse(body)
    message_id = properties.message_id

    return if ProcessedMessage.exists?(message_id: message_id) # Idempotency check

    ActiveRecord::Base.transaction do
      ProcessedMessage.create!(message_id: message_id)
      OrderProcessor.new.call(payload)
    end

    channel.ack(delivery_info.delivery_tag)
  rescue => e
    channel.reject(delivery_info.delivery_tag, requeue: true)
  end
end

NOTE: Another valid approach is to use a create_or_update operation based on a unique key in the payload. The key idea is that processing the same message twice must produce the same result, without creating duplicates or inconsistent state.

Guarantees:

  • Duplicates are harmless.

Messages should be acknowledged only after the database transaction commits successfully. This ensures that if the consumer crashes during processing, RabbitMQ will re-deliver the message, and thanks to idempotency checks, re-processing it will not cause duplicate effects.

5. Message Ordering: Beyond "One Consumer per Queue"

RabbitMQ guarantees FIFO only within a single queue and a single consumer.

Let’s look at a real example:

Use case 1 → publishes message M1
Subscriber → runs use case 2 → publishes message M2
Subscriber → runs use case 3 → publishes message M3

Now imagine a single queue with multiple consumers:

M1, M2, and M3 are enqueued in order.

RabbitMQ distributes them round-robin across consumers.

If Consumer 1 is slower than Consumer 2, M3 may finish before M1.

The observable order of effects breaks, generating bugs.

This happens because ordering guarantees stop once messages are processed concurrently, even if RabbitMQ delivered them in order.

How to Fix It

Option 1: One Consumer per Queue (Strict FIFO)

The simplest solution: one queue + one consumer. For the next reasons:

  • RabbitMQ delivers messages in order.
  • Your consumer processes sequentially.
  • Each message is acked only after successful processing.

You can also limit concurrency explicitly:

channel.prefetch(1)

This ensures only one unacknowledged message is in flight per consumer.

If you can allow one consumer per queue, you won't have order problems.

Option 2: One Queue per Side-Effect (Parallel + Safe)

If you need parallelism and each queue represents an independent side-effect, you can create one queue per workflow. Each message in a queue triggers a specific side-effect.

exchange (direct)
├── queue_use_case_1 → consumer_use_case_1
├── queue_use_case_2 → consumer_use_case_2
└── queue_use_case_3 → consumer_use_case_3

Each queue represents a specific side-effect. Messages are routed to the queue corresponding to the side-effect they trigger. Multiple consumers can process messages in parallel within the same queue without breaking the process logic, because each message is an independent side-effect.

Note: Setting prefetch = 1 limits concurrency per consumer but does not guarantee strict ordering. In this model, strict ordering is not required because each message represents an independent side-effect.

Example:

Queue receives messages: M1a, M1b, M1c
Consumer A processes M1a → publishes M2a
Consumer B processes M1b → publishes M2b
Consumer A processes M1c → publishes M2c

Even if M2a is published after M2b and M2c, the system works correctly because each message represents an independent side-effect. The logical flow of the process is preserved, regardless of the order in which messages are processed within a queue.

This approach is very good option if you need one or multiple consumers per queue. It's pretty scalable, you can monitorize each queue independently and adjust resources as needed.

In my experience, this model works well for a variety of use cases, especially when dealing with complex workflows that require high availability and fault tolerance.

6. Network Failures, The Real Enemy

Common scenarios

Scenario What happens Risk
Consumer processes message, loses connection before ack RabbitMQ re-delivers Duplicate
Consumer receives message but crashes before ack RabbitMQ re-delivers Duplicate
Producer publishes but loses connection before confirm Message may or may not have arrived Retry needed

How to survive

Some strategies:

  • Publisher confirms: ensures the producer knows whether the message reached the broker.
  • Manual acks on consumers: prevents marking a message as processed before it has actually been completed.
  • Retries and DLQs: handle errors without losing messages.
  • Idempotency: key to safely processing duplicate messages without side effects.

Guarantees:

  • No lost messages even under network instability.
  • Only duplicates, which are safely ignored.

7. Putting It All Together

Common Risks and Solutions

Risk Solution
Lost messages on publish Outbox Pattern + Publisher Confirms
Lost messages in RabbitMQ Durable Queues + Persistent Messages
Lost messages on consume manual Ack after commit
Duplicates Idempotency
Out-of-order processing One consumer per queue or queue per side-effect
Network failures Retries + DLQ + Confirm mode

8. The Reality of Distributed Systems

You can’t have exactly-once semantics in an unreliable network. What you can have is at-least-once delivery with idempotent consumers, and that’s good for real-world systems.

By combining:

  • Outbox pattern
  • Durable queues
  • Persistent messages
  • Manual acks
  • Idempotency
  • Proper ordering strategies

you achieve no message loss and predictable, consistent behavior, even under failure.

Final Thoughts

Building reliable message-driven systems isn’t about eliminating failure, it’s about making failure safe. With RabbitMQ, that means embracing duplicates, ensuring order where it matters, and designing every step to survive a crash, a retry, or a network glitch.

Reliability isn’t magic, it’s discipline.

← Previous Post