RabbitMQ - Reprocessing messages in a dead letter queue

Before getting into dead letters, let's first look at how a message typically gets transmitted from a producer to a consumer. Typically, the producer puts the message onto an exchange with a routing key. The routing key is then used to determine what queue the message should be routed to, the messages gets onto the queue, and the consumer gets the message from the queue.

 

This is fine and dandy, but let's say the consumer is unable to get messages off the queue. This could result in a situation where messages accumulate in the queue. Fortunately, messages can be moved to another queue when:

Typically, the queue gets bound to a dead letter exchange via queue arguments or a policy, and then the dead letter exchange places the message on a dead letter queue.

 

If dead-letter-routing-key is used, then the message will have the dead-letter-routing key. For example, if the dead-letter-routing-key is deadletter.key, then the message will have routing key deadletter.key.

 

So, how can messages on the dead letter queue be reprocessed? It is important to first recognize that according to https://www.rabbitmq.com/dlx.html, the message will be dropped when the message is moved to a dead letter exchange without a dead letter routing key if the message was not rejected by a consumer. To put this another way.

 

Dropping messages, or having messages endlessly cycle, is almost always a situation we want to avoid, and I do not see any built in features in RabbitMQ that can be easily applied to reprocess messages while ensuring the messages are not dropped or put into an endless cycle.

It is also important to consider the scenario where there are two (or more) exchanges or two (or more) queues in a virtual host. In this scenario, if there is a single (shared) dead letter queue, the dead letter queue could contain messages that were intended for different exchanges, different queues. I have not been able to figure out how to get messages in a shared dead letter queue to go back to different original target exchanges.

 

For this reason, I think it makes sense to have a dedicated exchange and queue for each original queue, so that you know for sure that the messages get reprocessed back to the correct original queue.

 

A slightly different version of this is to have the messages in each dead letter queue reprocessed to the original exchange. The only reason I do not prefer this approach is because then you have to take into consideration the scenario where messages were originally published with different routing keys. By instead reprocessing the messages back to the original queue, this bypasses the need to account for different routing keys.

 

The solution that I came up with was to run a home grown script once every hour, where the script checks for messages in a dead letter queue. If there are messages in a dead letter queue, then the script creates dynamic shovels to move the messages from the dead letter queue to the original exchange.




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee

Add a Comment




We will never share your name or email with anyone. Enter your email if you would like to be notified when we respond to your comment.





Please enter 93f95 in the box below so that we can be sure you are a human.




Comments

Web design by yours truely - me, myself, and I   |   jeremy.canfield@freekb.net   |