RabbitMQ - Create dynamic shovel using the REST API

A shovel is used to move a message from a queue or exchange to a different queue or exchage. The queues or exchanges can reside on the same RabbitMQ node or on different RabbitMQ nodes.

 

AVOID TROUBLE

Before you can create a shovel, the rabbitmq_shovel plugin will need to be enabled, and you may want to also enable the rabbitmq_shovel_management plugin so that you can use Shovel Status and Shovel Management in the RabbitMQ web browser console and the RabbitMQ REST API to list/create/delete dynamic shovels. The rabbitmq-plugins list command can be used to determine if the plugins are enabled. If not, the rabbitmq-plugins enable command can be used to enable the plugins.

If the RabbitMQ node is not using SSL, the src-uri or dest-uri should start with amqp://. If SSL is being used, the src-uri or dest-uri should start with amqps://.

 

There are two types of shovels.

 

curl can be used to create a dynamic shovel. In this example, a dynamic shovel named shovel001 will be created that can be used to move messages from queue001 on rabbit001.example.com to queue002 on rabbit002.example.com. Refer to the RabbitMQ REST API documentation.

AVOID TROUBLE

If the RabbitMQ node is not using SSL, the src-uri or dest-uri should start with amqp://. If SSL is being used, the src-uri or dest-uri should start with amqps://.

src-queue must be used for amqp091

src-address must be used for amqp10

curl 
--user john.doe:itsasecret 
--request PUT
--url http://hostname:15671/api/parameters/shovel/vhost001/shovel001
--header 'content-type: application/json'
--data
'{
    "component": "shovel",
    "vhost": "/vhost001",
    "value": {
        "src-uri": "amqps://john.doe:itsasecret@server001:5671/vhost001",
        "src-queue": "queue001",
        "src-protocol": "amqp091",
        "src-prefetch-count": 1000,
        "src-delete-after": "never",
        "dest-protocol": "amqp091",
        "dest-uri": "amqps://john.doe:itsasecret@server001:5671/vhost001",
        "dest-add-forward-headers": false,
        "dest-queue": "queue002",
        "ack-mode": "on-confirm"
    }
}'

 

Or messages can be moved from a queue to an exchange with a routing key.

curl 
--user john.doe:itsasecret 
--request PUT
--url http://hostname:15671/api/parameters/shovel/vhost001/shovel001
--header 'content-type: application/json'
--data
'{
    "component": "shovel",
    "vhost": "/vhost001",
    "value": {
        "src-uri": "amqps://john.doe:itsasecret@server001:5671/vhost001",
        "src-queue": "queue001",
        "src-protocol": "amqp091",
        "src-prefetch-count": 1000,
        "src-delete-after": "queue-length",
        "dest-protocol": "amqp091",
        "dest-uri": "amqps://john.doe:itsasecret@server001:5671/vhost001",
        "dest-add-forward-headers": false,
        "dest-exchange": "exchange002",
        "dest-exchange-key": "key001",
        "ack-mode": "on-confirm"
    }
}'

 

If dest-add-forward-headers is set to true, then when the message is moved to the target queue, the header will contain the x-shovelled header, like this.

[
    {
        "exchange": "foo.exchange",
        "message_count": 10,
        "payload": "",
        "payload_bytes": 10257,
        "payload_encoding": "string",
        "properties": {
            "content_type": "text/plain",
            "delivery_mode": 2,
            "headers": {
                "timestamp_in_ms": 1625660278601,
                "x-death": [
                    {
                        "count": 1,
                        "exchange": "bar.exchange",
                        "queue": "bar.queue",
                        "reason": "maxlen",
                        "routing-keys": [
                            "key001"
                        ],
                        "time": 1625660278
                    }
                ],
                "x-first-death-exchange": "bar.exchange",
                "x-first-death-queue": "bar.queue",
                "x-first-death-reason": "maxlen",
                "x-shovelled": [
                    {
                        "dest-exchange": "foo.exchange",
                        "dest-exchange-key": "key001",
                        "dest-uri": "amqps://server1.example.com:5671/foo",
                        "shovel-name": "temp.foo.exchange.shovel",
                        "shovel-type": "dynamic",
                        "shovel-vhost": "foo",
                        "shovelled-by": "rabbit@server1.example.com",
                        "src-queue": "deadletter.queue",
                        "src-uri": "amqps://server1.example.com:5671/foo"
                    }
                ]
            },
            "priority": 0,
            "timestamp": 1625660278
        },
        "redelivered": false,
        "routing_key": "key001"
    }
]

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee

Add a Comment




We will never share your name or email with anyone. Enter your email if you would like to be notified when we respond to your comment.





Please enter 0c042 in the box below so that we can be sure you are a human.




Comments


July 19th, 2022 by patrick
Good article, just note that "src-queue" is not more recognized and it should be named "src-address" instead.


July 20th, 2022 by Patrick
in fact I will correct my last comment: src-queue is the proper field for amqp091 but in amqp10 it must be src-address instead.


July 20th, 2022 by Jeremy (moderator)
Awesome, thanks for sharing this Patrick. I've updated this article to have src-queue for amqp091 and src-address for amqp10.

Web design by yours truely - me, myself, and I   |   jeremy.canfield@freekb.net   |