Data backup is a critical part of any IT infrastructure. Whether you are managing personal files, application data, or business documents, maintaining regular backups helps protect against accidental deletion, hardware failures, ransomware attacks, and other unexpected incidents.<\/p>\n
Amazon S3 (Simple Storage Service) is a highly durable, secure, and scalable cloud storage service provided by Amazon Web Services (AWS). By storing backups in Amazon S3, organizations can ensure their data remains accessible and protected from local system failures.<\/p>\n
In this tutorial, we will create a Python script that automatically uploads files from a local Windows folder to an Amazon S3 bucket. The script organizes backups into date-based folders, making it easy to manage historical backups and restore files when required. We will also discuss how to schedule the script to run automatically using Windows Task Scheduler.<\/p>\n
Before proceeding, ensure the following requirements are met:<\/p>\n
The script is designed to run on a Windows system.<\/p>\n
Install Python 3 on your Windows machine and verify the installation:<\/p>\n
python --version<\/code><\/pre>\n3. AWS Account<\/h3>\n
You need an active AWS account with access to Amazon S3.<\/p>\n
4. S3 Bucket<\/h3>\n
Create an S3 bucket where backups will be stored.<\/p>\n
Example bucket:<\/p>\n
my-backup-bucket<\/code><\/pre>\n5. IAM User Credentials<\/h3>\n
Create an IAM user with the necessary S3 permissions and obtain:<\/p>\n
\n- AWS Access Key ID<\/li>\n
- AWS Secret Access Key<\/li>\n<\/ul>\n
A sample IAM policy is shown below:<\/p>\n
{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Action\": [\n \"s3:PutObject\",\n \"s3:GetObject\",\n \"s3:ListBucket\"\n ]<\/span>,\n \"Resource\": [\n \"arn:aws:s3:::my-backup-bucket\",\n \"arn:aws:s3:::my-backup-bucket\/*\"\n ]<\/span>\n }\n ]\n}<\/code><\/pre>\nInstall Required Python Package<\/h2>\n
Install the AWS SDK for Python (boto3):<\/p>\n
pip install boto3<\/code><\/pre>\nVerify installation:<\/p>\n
pip show boto3<\/code><\/pre>\nCreate the Backup Script<\/h2>\n
Create a file named:<\/p>\n
s3backup.py<\/code><\/pre>\nAdd the following code:<\/p>\n
import boto3\nimport os\nfrom datetime import datetime\n\n# AWS Configuration\nAWS_ACCESS_KEY_ID = \"YOUR_ACCESS_KEY\"\nAWS_SECRET_ACCESS_KEY = \"YOUR_SECRET_KEY\"\nAWS_REGION = \"us-east-1\"\n\n# S3 Configuration\nBUCKET_NAME = \"my-backup-bucket\"\n\n# Local Folder to Backup\nSOURCE_DIR = r\"C:\\backup\"\n\n# Create Date-Based Folder\ndate_folder = datetime.utcnow().strftime(\"%Y%m%d\")\n\n# Create S3 Client\ns3 = boto3.client(\n \"s3\",\n aws_access_key_id=AWS_ACCESS_KEY_ID,\n aws_secret_access_key=AWS_SECRET_ACCESS_KEY,\n region_name=AWS_REGION\n)\n\n# Upload Files\nfor root, dirs, files in os.walk(SOURCE_DIR):\n for file in files:\n local_file = os.path.join(root, file)\n\n # Preserve Folder Structure\n relative_path = os.path.relpath(local_file, SOURCE_DIR)\n s3_key = f\"{date_folder}\/{relative_path.replace(os.sep, '\/')}\"\n\n print(f\"Uploading {local_file}\")\n\n try:\n s3.upload_file(local_file, BUCKET_NAME, s3_key)\n print(f\"Uploaded to s3:\/\/{BUCKET_NAME}\/{s3_key}\")\n except Exception as e:\n print(f\"Failed: {e}\")\n\nprint(\"Backup completed successfully.\")<\/code><\/pre>\nHow the Script Works<\/h2>\nStep 1: Connect to AWS<\/h3>\n
The script creates a connection to Amazon S3 using your IAM credentials.<\/p>\n
s3 = boto3.client(...)<\/code><\/pre>\nStep 2: Generate a Date-Based Folder<\/h3>\n
The current date is generated in the format:<\/p>\n
20260818<\/code><\/pre>\nThis helps organize backups by date.<\/p>\n
date_folder = datetime.utcnow().strftime(\"%Y%m%d\")<\/code><\/pre>\nStep 3: Scan the Local Directory<\/h3>\n
The script recursively scans all files under:<\/p>\n
C:\\backup<\/code><\/pre>\nusing:<\/p>\n
os.walk()<\/code><\/pre>\nStep 4: Upload Files to S3<\/h3>\n
Each file is uploaded to the S3 bucket while maintaining the original folder structure.<\/p>\n
Example:<\/p>\n
C:\\backup\\documents\\report.pdf<\/code><\/pre>\nbecomes:<\/p>\n
s3:\/\/my-backup-bucket\/20260818\/documents\/report.pdf<\/code><\/pre>\nRunning the Script<\/h2>\n
Execute the script from PowerShell:<\/p>\n
python s3backup.py<\/code><\/pre>\nExample output:<\/p>\n
Uploading C:\\backup\\documents\\report.pdf\nUploaded to s3:\/\/my-backup-bucket\/20260818\/documents\/report.pdf\n\nUploading C:\\backup\\images\\logo.png\nUploaded to s3:\/\/my-backup-bucket\/20260818\/images\/logo.png\n\nBackup completed successfully.<\/code><\/pre>\nVerify Backup in Amazon S3<\/h2>\n
Log in to the AWS Management Console and navigate to:<\/p>\n
Amazon S3 \u2192 Your Bucket<\/code><\/pre>\nYou should see a date-based folder:<\/p>\n
20260818\/<\/code><\/pre>\nInside the folder, all files and subdirectories from the source folder will be available.<\/p>\n
Automating Daily Backups<\/h2>\n
Windows Task Scheduler can be used to automate the backup process.<\/p>\n
Step 1: Open Task Scheduler<\/h3>\n
Press:<\/p>\n
Windows + R<\/code><\/pre>\nand run:<\/p>\n
taskschd.msc<\/code><\/pre>\nStep 2: Create a New Task<\/h3>\n
Select:<\/p>\n
Create Basic Task<\/code><\/pre>\nStep 3: Configure Trigger<\/h3>\n
Choose:<\/p>\n
Daily<\/code><\/pre>\nand specify the desired execution time.<\/p>\n
Step 4: Configure Action<\/h3>\n
Program:<\/p>\n
C:\\Python312\\python.exe<\/code><\/pre>\nArguments:<\/p>\n
C:\\scripts\\s3backup.py<\/code><\/pre>\nStep 5: Save the Task<\/h3>\n
The backup process will now execute automatically every day.<\/p>\n
Best Practices<\/h2>\nUse IAM Roles and Policies<\/h3>\n
Grant only the required permissions to the backup user.<\/p>\n
Avoid Hardcoding Credentials<\/h3>\n
Instead of storing credentials in scripts, use:<\/p>\n
AWS CLI Credentials File<\/code><\/pre>\nor<\/p>\n
IAM Roles<\/code><\/pre>\nwhere possible.<\/p>\n
Enable S3 Versioning<\/h3>\n
Versioning provides additional protection against accidental deletions and overwrites.<\/p>\n
Use Lifecycle Policies<\/h3>\n
Configure lifecycle policies to move older backups to:<\/p>\n
\n- S3 Standard-IA<\/li>\n
- S3 Glacier<\/li>\n
- S3 Glacier Deep Archive<\/li>\n<\/ul>\n
to reduce storage costs.<\/p>\n
Enable Encryption<\/h3>\n
Use:<\/p>\n
\n- SSE-S3<\/li>\n
- SSE-KMS<\/li>\n<\/ul>\n
to protect backup data.<\/p>\n
Alternative Approach: AWS CLI<\/h2>\n
For simple backup requirements, AWS CLI can be used instead of Python.<\/p>\n
Install AWS CLI:<\/p>\n
aws configure<\/code><\/pre>\nSync a folder:<\/p>\n
aws s3 sync C:\\backup s3:\/\/my-backup-bucket\/backup\/<\/code><\/pre>\nCreate date-based backups:<\/p>\n
$today = Get-Date -Format \"yyyyMMdd\"\naws s3 sync C:\\backup s3:\/\/my-backup-bucket\/$today\/<\/code><\/pre>\nAWS CLI is often the preferred choice for straightforward backup tasks because it is easy to configure and maintain.<\/p>\n