Bootstrap FreeKB - Amazon Web Services (AWS) - Install and setup Amazon Kinesis Java Agent using Docker
Amazon Web Services (AWS) - Install and setup Amazon Kinesis Java Agent using Docker


This assumes you are familiar with Kinesis. If not, check out my article Getting Started with Kinesis.

There are a few different Kinesis Agents. For example, in the AWS console, after selecting your Kinesis Data Stream, the Applications tab lists a few Kinesis Agents. These are known as the Producers.

 

There are a few different ways to install and setup the Amazon Kinesis Java Agent.

This assumes you already have a system setup with Docker. Let's create a tmp directory.

mkdir /tmp/aws-kinesis-agent

 

And move into the tmp directory.

cd /tmp/aws-kinesis-agent

 

Create a file named Dockerfile.

touch Dockerfile

 

Add the following to the Dockerfile.

FROM amazonlinux

RUN yum install -y aws-kinesis-agent which findutils
COPY agent.json /etc/aws-kinesis/agent.json

CMD ["start-aws-kinesis-agent"]

 

Notice this expects agent.json. Let's create agent.json in the same directory as your Dockerfile, replacing kinesisStream with the name of your AWS Kinesis Data Stream.

There are much better options rather that having your AWS Access Key and Secret Key in agent.json, but again, I'm just trying to keep this simple just to get Kinesis up and working.

]$ cat agent.json
{
  "awsAccessKeyId": "ABK349fJFM499FMDFD23",
  "awsSecretAccessKey": "Ab587Cd234Ef98gG9834Aa345908bB23409234Cc",
  "flows": [
    {
      "filePattern": "/tmp/app.log*",
      "kinesisStream": "my-kinesis-data-stream"
    }
  ]
}

 

Use the docker build command to create an image using the Dockerfile.

sudo docker build --file Dockerfile --tag aws-kinesis-agent:latest .

 

The docker images command can be used to display the images.

]$ sudo docker images
REPOSITORY                         TAG          IMAGE ID       CREATED         SIZE
aws-kinesis-agent                  latest       507358e24973   2 seconds ago   791MB

 

Then the docker run command can be used to create and start the container.

sudo docker run --detach --name aws-kinesis-agent aws-kinesis-agent:latest

 

The Docker container should now be up and running.

~]$ sudo docker container ls -a
CONTAINER ID   IMAGE                                       COMMAND                  CREATED         STATUS                    PORTS     NAMES
253e80a77ac5   aws-kinesis-agent:latest                    "start-aws-kinesis-a…"   7 seconds ago   Up 6 seconds                        aws-kinesis-agent

 

Next you probably will want to Send log data to Kinesis Data Stream using Kinesis Agent.




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter 130cb9 in the box below so that we can be sure you are a human.