RabbitMQ - Resolve low disk space (mnesia msg_stores)

Typically, on a Linux system, a dedicated partition or volume is used for RabbitMQ. In this example, the /opt/rabbitmq directory has reached 95% used.

~]# df -h
Filesystem                                           Size  Used Avail Use% Mounted on
/dev/mapper/datavg-lv_rabbitmq                        20G   19G  1.2G  95% /opt/rabbitmq


The du -hs command can be used. Often, the mnesia directory will be large. mnesia is a database used by RabbitMQ where messages are stored.

~]# du -hs /opt/rabbitmq/*
18G     /opt/rabbitmq/mnesia


And often the msg_stores directory will be large. Messages are stored in the msg_stores directory.

~]# du -hs /opt/rabbitmq/mnesia/node1/*
18G     /opt/rabbitmq/mnesia/node1/msg_stores


And often the msg_stores_persistent and/or msg_stores_transient directories will be large.

~]# du -hs /opt/rabbitmq/mnesia/node1/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/*
13G     /opt/rabbitmq/mnesia/node1/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent
4.2G    /opt/rabbitmq/mnesia/node1/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient


And the msg_stores_persistent and/or msg_stores_transient directories will contain a large collection of .rdq files.


A lazy queue will always store messages on disk, in the mnesia database.

Do NOT edit, move, delete or in any way try to tamper with the .rdq files, as these files contain messages.

~]# ls -l /opt/rabbitmq/mnesia/node1/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent
-rw-r----- 1 rabbitmq rabbitmq 14581574 Sep 23 10:52 0.rdq
-rw-r----- 1 rabbitmq rabbitmq 16755385 Sep 23 09:14 1016.rdq
-rw-r----- 1 rabbitmq rabbitmq 16749183 Oct  1 11:33 10183.rdq
-rw-r----- 1 rabbitmq rabbitmq 16596144 Oct 26 17:33 10689.rdq
-rw-r----- 1 rabbitmq rabbitmq 15955144 Oct 26 17:33 11214.rdq
-rw-r----- 1 rabbitmq rabbitmq 16720815 Oct  5 22:18 11771.rdq
-rw-r----- 1 rabbitmq rabbitmq 16750081 Oct  6 15:14 12350.rdq
-rw-r----- 1 rabbitmq rabbitmq 16712827 Oct  7 10:43 12630.rdq
-rw-r----- 1 rabbitmq rabbitmq 15322101 Oct  8 07:51 13019.rdq


The rabbitmqctl list_queues command can be used to count the number of messages in each queue, as a way to determine which queues contain the most messages. Or the same could be done with the rabbitmqadmin list queues command, or with the RabbitMQ REST API.

In this example, you would want to determine why messages on a.queue and foo.queue and bar.queue are not being consumed.

~]# rabbitmqctl list_queues -p / messages name | sort -n -r | head
619221  a.queue
48882   foo.queue
27561   bar.queue


Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee

Add a Comment

We will never share your name or email with anyone. Enter your email if you would like to be notified when we respond to your comment.

Please enter 9e3e9 in the box below so that we can be sure you are a human.


Web design by yours truely - me, myself, and I   |   jeremy.canfield@freekb.net   |