This assumes you are familar with the basic configurations needed to connect to Amazon Web Services (AWS) using Python boto3. If not, check out my article Python (Scripting) - Getting Started with Amazon Web Services (AWS) boto3.

Here is the minimal boilerplate code without any error handling to upload a file to one of your S3 Buckets. This will upload /tmp/foo.txt on your local system to the root directory in your S3 Bucket.

#!/usr/bin/python3
import boto3
import os

client = boto3.client('s3')
client.upload_file("/tmp/foo.txt", "my-bucket-abc123", "foo.txt")

This will upload /tmp/foo.txt on your local system to the bar directory in your S3 Bucket.

#!/usr/bin/python3
import boto3
import os

client = boto3.client('s3')
client.upload_file("/tmp/foo.txt", "my-bucket-abc123", "bar/foo.txt")

Here is a more practical example, with try/except/else error handling.

#!/usr/bin/python3
import boto3
import sys

try:
  client = boto3.client('s3')
except Exception as exception:
  print(exception)
  sys.exit(1)

try:
  client.upload_file("/tmp/foo.txt", "my-bucket-abc123", "foo.txt")
except Exception as exception:
  print(exception)
else:
  print(f"successfully uploaded {full_path_to_local_file} to {bucket_dest} in my-bucket-abc123")

And here is an example using os.walk to using os.walk to upload all of the files at and below a directory.

#!/usr/bin/python3
import boto3
import os
import re
import sys

try:
  client = boto3.client('s3')
except Exception as exception:
  print(exception)
  sys.exit(1)

for root, dirs, files in os.walk("/tmp"):
  for filename in files:
    full_path_to_local_file = os.path.join(root, filename)
    bucket_dest = re.sub('/tmp', 'backups/', full_path_to_local_file)

    try:
      client.upload_file(full_path_to_local_file, "my-bucket-abcdefg", bucket_dest)
    except Exception as exception:
      print(f"got the following exception when attempting to upload {full_path_to_local_file} to {bucket_dest} in my-bucket-abcdefg - {exception}")
    else:
      print(f"successfully uploaded {full_path_to_local_file} to {bucket_dest} in my-bucket-zplamkinsh")

If you want to upload the file using your own KMS Customer Managed Keys (CMK), include ExtraArgs={"ServerSideEncryption": "aws:kms", "SSEKMSKeyId", "your key id"}.

At a high level, there are 3 types of keys.

• Server Side Encryption Amazon Managed (sse-amz)
• Server Side Encryption Key Management Service (sse-kms)
• Server Side Encryption Customer (sse-c)

Here is how you can upload a file using your own Server Side Encryption Key Management Service (sse-kms).

#!/usr/bin/python3
import boto3
import os

client = boto3.client('s3')
client.upload_file("/tmp/foo.txt", 
                   "my-bucket-abc123", 
                   "foo.txt",
                   ExtraArgs={"ServerSideEncryption": "aws:kms", 
                              "SSEKMSKeyId": "4802df3b-1b8b-4f7b-af98-61bbf207468d"})

In this scenario, the file in the S3 bucket should contain the KMS key you used.

And here is an example of how you can upload a file using your own Server Side Encryption Customer (sse-c).

client.upload_file("/tmp/bar.txt",
                   "my-bucket-abc123",
                   "bar2.txt",
                   ExtraArgs={
                       "ServerSideEncryption": "AES256",
                       "SSECustomerAlgorithm": "AES256",
                       "SSECustomerKey": "4s6iQXekYL6BxzCZX8Zn3Kr4djK42BSLgb1nP3C7qp0=",
                       "SSECustomerKeyMD5":"tAasKToBgkFA3Sy43tQjSA=="})

Be aware that if you upload a file to an S3 Bucket with an sse-c, you'll most likely see the following when viewing the object in the S3 console, which is probably OK since you manage your own sse-c keys.