Bootstrap FreeKB - Amazon Web Services (AWS) - Get Kinesis Records using the AWS CLI
Amazon Web Services (AWS) - Get Kinesis Records using the AWS CLI


This assumes you have already configured the aws command line tool. If not, check out my article on Getting Started with the AWS CLI.

This also assumes you are familiar with Kinesis. If not, check out my article Getting Started with Kinesis.

The aws kinesis list-streams command can be used to list your Kinesis Data Stream. Let's say you have a Kinesis Data Stream named my-kinesis-data-stream.

~]$ aws kinesis list-streams
{
    "StreamNames": [
        "my-kinesis-data-stream"
    ],
    "StreamSummaries": [
        {
            "StreamName": "my-kinesis-data-stream",
            "StreamARN": "arn:aws:kinesis:us-east-1:123456789012:stream/my-kinesis-data-stream",
            "StreamStatus": "ACTIVE",
            "StreamModeDetails": {
                "StreamMode": "ON_DEMAND"
            },
            "StreamCreationTimestamp": "2024-05-15T01:13:06+00:00"
        }
    ]
}

 

And then the aws kinesis get-shard-iterator command can be used to get the Shard Iterator of one of your Shards.

A shard iterator represents the position of the stream and shard from which the consumer will read.

~]$ aws kinesis get-shard-iterator --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON --stream-name my-kinesis-data-stream
{
    "ShardIterator": "AAAAAAAAAAEfW0Zybg2XGNf7C/j9I25NCC0egOxYd8RDXGqJMVJ0+zfxz2lmIfsWu+x+P0IZystt75j5cGBtVW1ZFzIJfEXUtEgStUJlv+0A/I1zQR/73yhSzLdaXRFFjsYIljI9jg6a14qJ3b6LMRvwarEvVPYVe7CiPOW6Z8QeihnQ1m/TxbHFiJK3g9eiuJSPNt5YoAJPnpb5nUYng9QcyJYG0Du4cJlMAx3RZMhqkw70zagOoItQ5Hb9v76zq9ns7ojlCb4="
}

 

And then the aws kinesis get-records command with the Shard Iterator can be used to get the records in the shard.

~]$ aws kinesis get-records --shard-iterator AAAAAAAAAAEmAWYxVHzUk+h7JE+iB6JIYJVj8TM7ofvcCJI5FiF8h0YIhTLQYt6AIMvKmE5GyU3gFxaEr0dTJfPvaO2ymyNWOKNCR6xHACar5gvGCZBfItRUr6PFjr/QPaUSRpzTof/9OTeiqGBhTHCBf1l8zUxnO02Q3mBRAu3RZuQNglzapuaHm7rWE6U+qTUiGUwoUTfaN8eNzdRKVCs/Wuv9WBx5kZsJXoOwH6mPP10Buc4kRaKgNCq0SYTFUas31VopWsY=
{
    "Records": [
        {
            "SequenceNumber": "49652152523783921918411693299885165842964966611036930050",
            "ApproximateArrivalTimestamp": "2024-05-18T07:20:03.309000+00:00",
            "Data": "SGVsbG8gV29ybGQK",
            "PartitionKey": "123"
        }
    ],
    "NextShardIterator": "AAAAAAAAAAFppvgEIER5FI4JB0krnt5SwsT4Pqd1dVKrBlAvlT9tbccL1proIDY1JI1e/cwADHnCD8gndEctHyBeUWJgAxoTp0mYx6W1GSK+aTJ3EbGMpH3LSlBTP7ghCIcmyokZ7tYzLoGpoOCKAq1T96ptKoWNO3rCXM1/LSOGivjPEcPB5cUgtrOREmpVb/hubIL/UDmzaIyR/yHEXnlguIoCO9Nj9HEP73M3+1PME7qjL7XOFdEsMPB6yQOs49+SUUzONG0=",
    "MillisBehindLatest": 0
}

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter 339410 in the box below so that we can be sure you are a human.