Bootstrap FreeKB - Amazon Web Services (AWS) - Install and setup Amazon Kinesis Java Agent using yum or dnf
Amazon Web Services (AWS) - Install and setup Amazon Kinesis Java Agent using yum or dnf


This assumes you are familiar with Kinesis. If not, check out my article Getting Started with Kinesis.

There are a few different Kinesis Agents. For example, in the AWS console, after selecting your Kinesis Data Stream, the Applications tab lists a few Kinesis Agents. These are known as the Producers.

 

There are a few different ways to install and setup the Amazon Kinesis Java Agent.

dnf install or yum install can be used to install the Kinesis Agent.

dnf install aws-kinesis-agent

 

The /etc/aws-kinesis/agent.json file should have something like this. Let's update "kinesisStream" to have the name of your Kinesis Data Stream.

There are much better options rather that having your AWS Access Key and Secret Key in agent.json, but again, I'm just trying to keep this simple just to get Kinesis up and working.

~]$ cat /etc/aws-kinesis/agent.json
{
  "awsAccessKeyId": "ABK349fJFM499FMDFD23",
  "awsSecretAccessKey": "Ab587Cd234Ef98gG9834Aa345908bB23409234Cc",
  "flows": [
    {
      "filePattern": "/tmp/app.log*",
      "kinesisStream": "my-kinesis-data-stream"
    }
  ]
}

 

Start and enable aws-kinesis-agent.

sudo systemctl start aws-kinesis-agent
sudo systemctl enable aws-kinesis-agent

 

And verify that aws-kinesis-agent is running.

~]$ sudo systemctl status aws-kinesis-agent
● aws-kinesis-agent.service - LSB: Daemon for Amazon Kinesis Agent.
     Loaded: loaded (/etc/rc.d/init.d/aws-kinesis-agent; generated)
     Active: active (running) since Fri 2024-05-17 00:56:20 UTC; 8s ago

 

Assuming there are no issues, /var/log/aws-kinesis-agent/aws-kinesis-agent.log should contain something like this.

 ~]$ cat /var/log/aws-kinesis-agent/aws-kinesis-agent.log
2024-05-17 01:00:22.323+0000  (FileTailer[kinesis:my-kinesis-data-stream:/tmp/app.log*].MetricsEmitter RUNNING) com.amazon.kinesis.streaming.agent.tailing.FileTailer [INFO] FileTailer[kinesis:my-kinesis-data-stream:/tmp/app.log*]: Tailer Progress: Tailer has parsed 0 records (0 bytes), transformed 0 records, skipped 0 records, and has successfully sent 0 records to destination.
2024-05-17 01:00:22.329+0000  (Agent.MetricsEmitter RUNNING) com.amazon.kinesis.streaming.agent.Agent [INFO] Agent: Progress: 0 records parsed (0 bytes), and 0 records sent successfully to destinations. Uptime: 240046ms
2024-05-17 01:00:22.327+0000  (FileTailer[fh:yourdeliverystream:/tmp/app.log*].MetricsEmitter RUNNING) com.amazon.kinesis.streaming.agent.tailing.FileTailer [INFO] FileTailer[fh:yourdeliverystream:/tmp/app.log*]: Tailer Progress: Tailer has parsed 0 records (0 bytes), transformed 0 records, skipped 0 records, and has successfully sent 0 records to destination.

 

Next you probably will want to Send log data to Kinesis Data Stream using Kinesis Agent.

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter 311578 in the box below so that we can be sure you are a human.