This assumes you are familar with the basic configurations needed to connect to Amazon Web Services (AWS) using Python boto3. If not, check out my article Python (Scripting) - Getting Started with Amazon Web Services (AWS) boto3.

The query here is basically the same query you would perform in the Amazon Web Services Cloudwatch Log Insights console. This will return results where @message contains "foo".

#!/usr/bin/python3
import boto3
from datetime import datetime, timedelta
import time

client    = boto3.client('logs')
query     = "fields @timestamp, @message | sort @timestamp desc | limit 20"
log_group = "my-log-group"

start_query_response = client.start_query(
    logGroupName=log_group,
    startTime=int((datetime.today() - timedelta(hours=5)).timestamp()),
    endTime=int(datetime.now().timestamp()),
    queryString=query,
)

query_id = start_query_response['queryId']

response = None

while response == None or response['status'] == 'Running':
    print('Waiting for query to complete ...')
    time.sleep(1)
    response = client.get_query_results(
        queryId=query_id
    )

for result in response['results']:
  for dict in result:
    if dict['field'] == "@timestamp":
      print(f"time = {dict['value']}")
    if dict['field'] == "@message":
      print(f"message = {dict['value']}")

Which should return something like this.

Waiting for query to complete ...
Waiting for query to complete ...
Waiting for query to complete ...
Waiting for query to complete ...

time = 2024-02-01 02:12:16.154
message = 172.31.19.227 - - [01/Feb/2024:02:12:16 +0000] "GET /base_images/favicon.ico HTTP/1.1" 304 0 "https://www.example.come/foo" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" "115.163.124.49"

time = 2024-02-01 02:12:16.154
message = [pid: 13|app: 0|req: 10470/16308] 172.31.19.227 () {60 vars in 1198 bytes} [Thu Feb  1 02:12:16 2024] GET /base_images/favicon.ico => generated 0 bytes in 1 msecs (HTTP/1.1 304) 4 headers in 185 bytes (0 switches on core 0)

time = 2024-02-01 02:12:15.879
message = 172.31.19.227 - - [01/Feb/2024:02:12:15 +0000] "GET /base_images/favicon.ico HTTP/1.1" 200 1150 "https://www.example.come/bar" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" "115.163.124.49"

time = 2024-02-01 02:12:15.878
message = [pid: 13|app: 0|req: 10469/16307] 172.31.19.227 () {56 vars in 1086 bytes} [Thu Feb  1 02:12:15 2024] GET /base_images/favicon.ico => generated 1150 bytes in 1 msecs via sendfile() (HTTP/1.1 200) 7 headers in 283 bytes (0 switches on core 0)

If you want to parse the logs for a particular string.

query = = "fields @timestamp, @message, @logStream | filter @message ~= /foo/ | sort @timestamp desc | limit 20"

If you want to parse the logs for a particular log stream.

query = "fields @timestamp, @message, @logStream | filter @logStream == 'my-log-stream' | sort @timestamp desc | limit 20"