RabbitMQ - Limit queue depth message-ttl argument (Time to Live dead letter)

Before getting into Time to Live (TTL), let's first look at how a message typically gets transmitted from a producer to a consumer. Typically, the producer puts the message onto an exchange, a routing key is then used to determine what queue the message should be routed to, the messages gets onto the queue, and the consumer gets the message from the queue.

 

This is fine and dandy, but let's say the consumer is unable to get messages off the queue. This could result in a situation where messages accumulate in the queue. Fortunately, RabbitMQ has some options for dealing with this type of situation. One such option is the message-ttl argument, which can be used to do something with a message after x seconds, such as moving the message to a dead letter queue.

There are numerous ways to set the message-ttl argument.

  • Set message-ttl as a queue argument
  • Set message-ttl as a policy and apply the policy to one or more queues and/or exchanges
  • Set the x-message-ttl argument in an app (such as a Java or .NET app).

 

AVOID TROUBLE

It is important to recognize that once a queue or exchange has been defined, if the queue or exchange does not have the message-ttl argument, it is not possible to update the queue or exchange to have the message-ttl argument. However, you can create a message-ttl policy and then apply the policy to one or more queues/exchanges. For this reason, this article will focus on only the message-ttl policy.

 

The rabbitmqctl set_policy command can be used to create a policy for message-ttl. In this example, a policy named TTL is created in virtual host vhost001, the policy is applied to all of the queues (.*) in vhost001, so that a message is considered "dead" after 60000 milliseconds (60 seconds).

  • A TTL value of 1000 would be 1 second
  • A TTL value of 60000 would be 1 minute
  • A TTL value of 3600000 would be 1 hour
rabbitmqctl set_policy  
--vhost vhost001
--priority 0
--apply-to queues
TTL
".*" 
'{
  "message-ttl": 60000
}'

 

The rabbitmqctl list_policies command can be used to display the policies.

rabbitmqctl list_policies -p vhost001

 

Which should return something like this.

Listing policies for vhost "vhost001" ...
vhost     name    pattern    apply-to   definition                priority
vhost001  TTL     .*         queues     {"message-ttl": 60000 }   0

 

dead letter exchange and dead letter queue can be setup to move messages that reach the TTL to the dead letter queue. Let's say deadletter.exchange is bound to deadletter.queue with routing key * (wildcard, match anything).

 

AVOID TROUBLE

Each queue can have only 1 policy. For this reason, it probably makes sense to delete the TTL policy and then to create a new TTL policy.

 

Let's use the rabbitmqctl set_policy command again. Notice that this dead letter policy contains both the message-ttl and dead letter arguments, and contains the exact name of our dead letter exchange (deadletter.exchange) and dead letter routing key (dlrk).

rabbitmqctl set_policy
--vhost vhost001 
--priority 0
--apply-to queues 
deadletter
".*"
'{
  "message-ttl": 60000, 
  "dead-letter-exchange": "deadletter.exchange",
  "dead-letter-routing-key": "*" 
 }'

 

Let's say you to publish a message to one of the queues in vhost001 using the web console, and the message is not consumed. In this example, we can see 1 message entered the queue, remained in the queue for a short while, and then got moved to the dead letter queue.




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee

Add a Comment




We will never share your name or email with anyone. Enter your email if you would like to be notified when we respond to your comment.





Please enter cbf84 in the box below so that we can be sure you are a human.




Comments

Web design by yours truely - me, myself, and I   |   jeremy.canfield@freekb.net   |