Message queues sound like advanced architecture. They are not. The idea is simple, and once you see the problem it solves, it becomes obvious when to use one.
Step 1: The problem
A user signs up. You need to save them, send a welcome email, create their profile in a third-party CRM, and generate a report entry.
app.post("/signup", async (req, res) => {
const user = await db.createUser(req.body);
await sendWelcomeEmail(user); // 900ms
await crm.createContact(user); // 1200ms, sometimes down
await reports.record(user); // 400ms
res.json(user);
});
Two problems here:
- The user waits about 2.5 seconds for a signup that really only needed the first line.
- If the CRM is down, the whole signup fails — even though the account was already created. The user sees an error for something that actually worked.
Step 2: The idea
Write down what needs to happen and let something else do it.
app.post("/signup", async (req, res) => {
const user = await db.createUser(req.body);
await queue.publish("user.created", { userId: user.id });
res.json(user); // responds in ~50ms
});
Now three separate workers listen for user.created and each does its own
job. The signup is fast, and if the CRM is down its worker retries later while everything
else carries on.
That is a message queue. RabbitMQ is the thing holding that list.
Step 3: The three words you need
- Producer — the code that publishes a message. Your API.
- Queue — where messages wait. RabbitMQ.
- Consumer — the worker that picks messages up and does the work.
One extra idea that matters: a message is only removed from the queue once the consumer acknowledges it. If your worker crashes halfway, no ack is sent, and RabbitMQ gives that message to someone else. Nothing is lost.
Step 4: Three settings that save you
1. Acknowledge after the work, not before
This is the mistake almost everyone makes once.
// Wrong — message is gone even if the work fails
channel.ack(msg);
await doTheWork(msg);
// Right
await doTheWork(msg);
channel.ack(msg);
2. Dead-letter queue (DLQ)
Some messages will never succeed — bad data, a deleted record. Without a DLQ they retry forever, blocking the queue. A DLQ is a second queue where a message goes after failing a few times, so you can inspect it later while everything else keeps moving.
3. Retry with a delay
If a third-party API is down, retrying instantly 500 times does not help anyone — including them. Wait a bit, and wait longer each time: 1s, 2s, 4s, 8s.
Step 5: The circuit breaker
One step further. If a service has failed 20 times in a row, stop calling it for a minute. Just fail fast and try again after that.
This sounds harsh but it protects both sides: you stop wasting workers on calls that will fail, and you stop hammering a service that is already struggling. I used this pattern across every third-party and on-chain call on a trading platform, and it is the difference between one slow dependency and a full outage.
Step 6: When you do not need a queue
Be honest about this. You do not need RabbitMQ if:
- The work is fast and must finish before you can answer the user anyway.
- You have one small app and no background work worth separating.
- A simple database table with a
statuscolumn and a cron job would do. For low volume, this genuinely works and is far less to operate.
A queue is another piece of infrastructure to run, monitor and debug. Add it when slow or unreliable background work is actually hurting you — not before.
Short version: a queue lets your API answer immediately and hand slow or fragile work to a worker. Acknowledge after the work, add a dead-letter queue, back off between retries — and only add one when you have a real problem it solves.