
This assumes you are familiar with Kinesis. If not, check out my article Getting Started with Kinesis.
There are a few different Kinesis Agents. For example, in the AWS console, after selecting your Kinesis Data Stream, the Applications tab lists a few Kinesis Agents. These are known as the Producers.
There are a few different ways to install and setup the Amazon Kinesis Java Agent.
- Install and setup Amazon Kinesis Java Agent from GitHub
- Install and setup Amazon Kinesis Java Agent using yum or dnf
- Install and setup Amazon Kinesis Java Agent using Docker (this article)
This assumes you already have a system setup with Docker. Let's create a tmp directory.
mkdir /tmp/aws-kinesis-agent
And move into the tmp directory.
cd /tmp/aws-kinesis-agent
Create a file named Dockerfile.
touch Dockerfile
Add the following to the Dockerfile.
FROM amazonlinux
RUN yum install -y aws-kinesis-agent which findutils
COPY agent.json /etc/aws-kinesis/agent.json
CMD ["start-aws-kinesis-agent"]
Notice this expects agent.json. Let's create agent.json in the same directory as your Dockerfile, replacing kinesisStream with the name of your AWS Kinesis Data Stream.
There are much better options rather that having your AWS Access Key and Secret Key in agent.json, but again, I'm just trying to keep this simple just to get Kinesis up and working.
]$ cat agent.json
{
"awsAccessKeyId": "ABK349fJFM499FMDFD23",
"awsSecretAccessKey": "Ab587Cd234Ef98gG9834Aa345908bB23409234Cc",
"flows": [
{
"filePattern": "/tmp/app.log*",
"kinesisStream": "my-kinesis-data-stream"
}
]
}
Use the docker build command to create an image using the Dockerfile.
sudo docker build --file Dockerfile --tag aws-kinesis-agent:latest .
The docker images command can be used to display the images.
]$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
aws-kinesis-agent latest 507358e24973 2 seconds ago 791MB
Then the docker run command can be used to create and start the container.
sudo docker run --detach --name aws-kinesis-agent aws-kinesis-agent:latest
The Docker container should now be up and running.
~]$ sudo docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
253e80a77ac5 aws-kinesis-agent:latest "start-aws-kinesis-a…" 7 seconds ago Up 6 seconds aws-kinesis-agent
Next you probably will want to Send log data to Kinesis Data Stream using Kinesis Agent.
Did you find this article helpful?
If so, consider buying me a coffee over at