Bootstrap FreeKB - RabbitMQ - Create dynamic shovel using the REST API
RabbitMQ - Create dynamic shovel using the REST API

Updated:   |  RabbitMQ articles

A shovel is used to move a message from a queue or exchange to a different queue or exchage. The queues or exchanges can reside on the same RabbitMQ node or on different RabbitMQ nodes.

 

AVOID TROUBLE

Before you can create a shovel, the rabbitmq_shovel plugin will need to be enabled, and you may want to also enable the rabbitmq_shovel_management plugin so that you can use Shovel Status and Shovel Management in the RabbitMQ web browser console and the RabbitMQ REST API to list/create/delete dynamic shovels. The rabbitmq-plugins list command can be used to determine if the plugins are enabled. If not, the rabbitmq-plugins enable command can be used to enable the plugins.

If the RabbitMQ node is not using SSL, the src-uri or dest-uri should start with amqp://. If SSL is being used, the src-uri or dest-uri should start with amqps://.

 

There are two types of shovels.

 

curl can be used to create a dynamic shovel. In this example, a dynamic shovel named shovel001 will be created that can be used to move messages from queue001 on rabbit001.example.com to queue002 on rabbit002.example.com. Refer to the RabbitMQ REST API documentation.

AVOID TROUBLE

If the RabbitMQ node is not using SSL, the src-uri or dest-uri should start with amqp://. If SSL is being used, the src-uri or dest-uri should start with amqps://.

src-queue must be used for amqp091

src-address must be used for amqp10

curl 
--user john.doe:itsasecret 
--request PUT
--url http://hostname:15671/api/parameters/shovel/vhost001/shovel001
--header 'content-type: application/json'
--data
'{
    "component": "shovel",
    "vhost": "/vhost001",
    "value": {
        "src-uri": "amqps://john.doe:itsasecret@server001:5671/vhost001",
        "src-queue": "queue001",
        "src-protocol": "amqp091",
        "src-prefetch-count": 1000,
        "src-delete-after": "never",
        "dest-protocol": "amqp091",
        "dest-uri": "amqps://john.doe:itsasecret@server001:5671/vhost001",
        "dest-add-forward-headers": false,
        "dest-queue": "queue002",
        "ack-mode": "on-confirm"
    }
}'

 

Or messages can be moved from a queue to an exchange with a routing key.

curl 
--user john.doe:itsasecret 
--request PUT
--url http://hostname:15671/api/parameters/shovel/vhost001/shovel001
--header 'content-type: application/json'
--data
'{
    "component": "shovel",
    "vhost": "/vhost001",
    "value": {
        "src-uri": "amqps://john.doe:itsasecret@server001:5671/vhost001",
        "src-queue": "queue001",
        "src-protocol": "amqp091",
        "src-prefetch-count": 1000,
        "src-delete-after": "queue-length",
        "dest-protocol": "amqp091",
        "dest-uri": "amqps://john.doe:itsasecret@server001:5671/vhost001",
        "dest-add-forward-headers": false,
        "dest-exchange": "exchange002",
        "dest-exchange-key": "key001",
        "ack-mode": "on-confirm"
    }
}'

 

If dest-add-forward-headers is set to true, then when the message is moved to the target queue, the header will contain the x-shovelled header, like this.

[
    {
        "exchange": "foo.exchange",
        "message_count": 10,
        "payload": "",
        "payload_bytes": 10257,
        "payload_encoding": "string",
        "properties": {
            "content_type": "text/plain",
            "delivery_mode": 2,
            "headers": {
                "timestamp_in_ms": 1625660278601,
                "x-death": [
                    {
                        "count": 1,
                        "exchange": "bar.exchange",
                        "queue": "bar.queue",
                        "reason": "maxlen",
                        "routing-keys": [
                            "key001"
                        ],
                        "time": 1625660278
                    }
                ],
                "x-first-death-exchange": "bar.exchange",
                "x-first-death-queue": "bar.queue",
                "x-first-death-reason": "maxlen",
                "x-shovelled": [
                    {
                        "dest-exchange": "foo.exchange",
                        "dest-exchange-key": "key001",
                        "dest-uri": "amqps://server1.example.com:5671/foo",
                        "shovel-name": "temp.foo.exchange.shovel",
                        "shovel-type": "dynamic",
                        "shovel-vhost": "foo",
                        "shovelled-by": "rabbit@server1.example.com",
                        "src-queue": "deadletter.queue",
                        "src-uri": "amqps://server1.example.com:5671/foo"
                    }
                ]
            },
            "priority": 0,
            "timestamp": 1625660278
        },
        "redelivered": false,
        "routing_key": "key001"
    }
]

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


July 19 2022 by patrick
Good article, just note that "src-queue" is not more recognized and it should be named "src-address" instead.

July 20 2022 by Patrick
in fact I will correct my last comment: src-queue is the proper field for amqp091 but in amqp10 it must be src-address instead.

July 20 2022 by Jeremy (moderator)
Awesome, thanks for sharing this Patrick. I've updated this article to have src-queue for amqp091 and src-address for amqp10.

Add a Comment


Please enter d8ea62 in the box below so that we can be sure you are a human.