Creating Subfolders in S3 Bucket Using AWS CLI

Introduction

Amazon Simple Storage Service (Amazon S3) is a scalable and durable object storage service used to store files, backups, logs, media assets, and application data. A common requirement when working with S3 is organizing data using folders and subfolders. This guide explains how to create subfolders in an S3 bucket using AWS CLI in a clear and practical way.

Understanding the Folder Concept in Amazon S3

How Amazon S3 Stores Data

Amazon S3 does not use a traditional file system. Instead, it stores data as objects inside buckets. Each object consists of:

• Bucket name
• Object key (full path)
• Object data

Example of an S3 Object Key

• my-bucket/logs/2025/january/application.log

In this example, logs, 2025, and january appear as subfolders even though they are part of the object key.

What Is AWS CLI?

Overview

AWS Command Line Interface (AWS CLI) is a powerful tool that allows you to interact with AWS services using terminal commands. It is widely used for automation, scripting, and managing cloud resources efficiently.

Why Use AWS CLI for S3?

• Efficient bulk file operations
• Easy automation and scripting
• Better control for DevOps workflows
• Faster uploads and downloads

Prerequisites

Before You Begin

• An active AWS account
• An existing S3 bucket
• AWS CLI installed
• AWS CLI configured with IAM credentials

Verify AWS CLI Installation

📄
aws --version

Configure AWS CLI

📄
aws configure

How to Create Subfolders in S3 Bucket Using AWS CLI

Method 1: Create an Empty Subfolder

This method creates a folder-like structure by uploading an empty object with a trailing slash.

📄
aws s3api put-object --bucket my-bucket --key projects/2025/january/

Explanation

• The trailing slash represents a folder
• An empty object is created in S3

Method 2: Create Subfolders While Uploading a File

You can create subfolders automatically by uploading a file to a specific path.

📄
aws s3 cp report.pdf s3://my-bucket/reports/2025/january/report.pdf

Result

• Folders are created automatically
• The file is stored in the specified subfolder

Method 3: Using aws s3 sync

This method is ideal for uploading directories with multiple subfolders.

📄
aws s3 sync ./local-folder s3://my-bucket/data/

Advantages

• Preserves directory structure
• Supports incremental uploads
• Efficient for large datasets

Real-World Use Cases

Application Log Storage

• s3://logs-bucket/app-name/year/month/

Media File Organization

• s3://media-bucket/images/products/
• s3://media-bucket/videos/tutorials/

Backup and Archival

• s3://backup-bucket/database/daily/
• s3://backup-bucket/database/monthly/

Common AWS CLI Commands for S3

Best Practices for S3 Subfolder Management

• Use meaningful and consistent folder names
• Avoid unnecessary deep nesting
• Apply lifecycle rules for cost optimization
• Use IAM policies to control access

Common Mistakes to Avoid

• Assuming S3 folders behave like local directories
• Forgetting trailing slashes for empty folders
• Using special characters in folder names
• Ignoring permission-related errors

Frequently Asked Questions

1. Does Amazon S3 support real folders?

No. S3 uses object key prefixes to simulate folders.

2. Can I create an empty folder in S3?

Yes. You can create an empty folder by uploading an empty object with a trailing slash.

3. Do I need to create folders before uploading files?

No. S3 automatically creates the folder structure when files are uploaded.

4. What is the best command to upload multiple folders?

The aws s3 sync command is best for uploading entire directory structures.

5. How do I delete a subfolder using AWS CLI?

📄
aws s3 rm s3://my-bucket/folder-name/ --recursive

Creating subfolders in an S3 bucket using AWS CLI is based on understanding object keys and prefixes. While S3 does not provide real folders, AWS CLI makes it simple to organize data using logical structures. With proper commands and best practices, you can efficiently manage files for backups, logs, and application data.