Bootstrap FreeKB - Amazon Web Services (AWS) - Download file in S3 Bucket using Python boto3
Amazon Web Services (AWS) - Download file in S3 Bucket using Python boto3


This assumes you are familar with the basic configurations needed to connect to Amazon Web Services (AWS) using Python boto3. If not, check out my article Python (Scripting) - Getting Started with Amazon Web Services (AWS) boto3.

Here is the minimal boilerplate code without any error handling to list your S3 Buckets

#!/usr/bin/python3
import boto3
import os

client = boto3.client('s3')
s3_buckets = client.list_buckets()
print(f"s3_buckets = {s3_buckets}")

 

Or, os.environ['AWS_PROFILE'] can be used to specify the profile in /home/john.doe/.aws/config and /home/john.doe/.aws/credentials to use.

import boto3
import os

os.environ['AWS_PROFILE'] = 'johndoe'

client = boto3.client('s3')
s3_buckets = client.list_buckets()
print(f"s3_buckets = {s3_buckets}")

 

Which should return something  that contains something like this.

'Buckets': [
    {'Name': 'my-bucket-foo', 
     'CreationDate': datetime.datetime(2023, 7, 21, 23, 17, 2, tzinfo=tzutc())
    }, 
    {'Name': 'my-bucket-bar', 
     'CreationDate': datetime.datetime(2023, 8, 21, 11, 9, 20, tzinfo=tzutc())
    }
  ], 
}

 

And here is how you can list the objects in a bucket.

client = boto3.client('s3')
s3_buckets = client.list_buckets()

for s3_bucket in s3_buckets['Buckets']:
  print(f"s3_bucket name = {s3_bucket['Name']}")

  if s3_bucket['Name'] == 'my-bucket-bar':
    objects = client.list_objects(Bucket='my-bucket-bar')
    for object in objects['Contents']:
      print(f"object = {object['Key']}")

 

Something like this should returned.

s3_bucket name = my-bucket-foo
s3_bucket name = my-bucket-bar
object = hello.html
object = index.html
object = main.js

 

Or, get_object can be used to determine if a specific object exists in an S3 Bucket.

#!/usr/bin/python3
import boto3
from botocore.exceptions import ClientError
import os

client = boto3.client('s3')
s3_buckets = client.list_buckets()

objects = ['foo.txt', 'bar.txt']

for s3_bucket in s3_buckets['Buckets']:
  print(f"s3_bucket name = {s3_bucket['Name']}")

  if s3_bucket['Name'] == 'my-bucket-abc123':
    for object in objects:
      try:
        response = client.get_object(Bucket='my-bucket-abc123', Key=object)
      except Exception as exception:
        if exception.response['Error']['Code'] == "NoSuchKey":
          print(f"object {object} does NOT exist in S3 Bucket my-bucket-abc123")
      else:
          print(f"object {object} exists in S3 Bucket my-bucket-abc123")

 

Something like this should returned.

foo.txt does NOT exist in my-bucket-abc123
bar.txt does exists in my-bucket-abc123



Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter 0bb47a in the box below so that we can be sure you are a human.