
This assumes you are familar with the basic configurations needed to connect to Amazon Web Services (AWS) using Python boto3. If not, check out my article Python (Scripting) - Getting Started with Amazon Web Services (AWS) boto3.
Here is the minimal boilerplate code without any error handling to list your S3 Buckets
#!/usr/bin/python3
import boto3
import os
client = boto3.client('s3')
s3_buckets = client.list_buckets()
print(f"s3_buckets = {s3_buckets}")
Or, os.environ['AWS_PROFILE'] can be used to specify the profile in /home/john.doe/.aws/config and /home/john.doe/.aws/credentials to use.
import boto3
import os
os.environ['AWS_PROFILE'] = 'johndoe'
client = boto3.client('s3')
s3_buckets = client.list_buckets()
print(f"s3_buckets = {s3_buckets}")
Which should return something that contains something like this.
'Buckets': [
{'Name': 'my-bucket-foo',
'CreationDate': datetime.datetime(2023, 7, 21, 23, 17, 2, tzinfo=tzutc())
},
{'Name': 'my-bucket-bar',
'CreationDate': datetime.datetime(2023, 8, 21, 11, 9, 20, tzinfo=tzutc())
}
],
}
And here is how you can list the objects in a bucket.
client = boto3.client('s3')
s3_buckets = client.list_buckets()
for s3_bucket in s3_buckets['Buckets']:
print(f"s3_bucket name = {s3_bucket['Name']}")
if s3_bucket['Name'] == 'my-bucket-bar':
objects = client.list_objects(Bucket='my-bucket-bar')
for object in objects['Contents']:
print(f"object = {object['Key']}")
Something like this should returned.
s3_bucket name = my-bucket-foo
s3_bucket name = my-bucket-bar
object = hello.html
object = index.html
object = main.js
Or, get_object can be used to determine if a specific object exists in an S3 Bucket.
#!/usr/bin/python3
import boto3
from botocore.exceptions import ClientError
import os
client = boto3.client('s3')
s3_buckets = client.list_buckets()
objects = ['foo.txt', 'bar.txt']
for s3_bucket in s3_buckets['Buckets']:
print(f"s3_bucket name = {s3_bucket['Name']}")
if s3_bucket['Name'] == 'my-bucket-abc123':
for object in objects:
try:
response = client.get_object(Bucket='my-bucket-abc123', Key=object)
except Exception as exception:
if exception.response['Error']['Code'] == "NoSuchKey":
print(f"object {object} does NOT exist in S3 Bucket my-bucket-abc123")
else:
print(f"object {object} exists in S3 Bucket my-bucket-abc123")
Something like this should returned.
foo.txt does NOT exist in my-bucket-abc123
bar.txt does exists in my-bucket-abc123
Did you find this article helpful?
If so, consider buying me a coffee over at