How is data organised in S3?

Data in S3 is organized using a flat structure with buckets as top-level containers, where each object is identified by a unique key, and logical hierarchy is created through key name prefixes and delimiters like forward slashes.

Amazon S3 uses a fundamentally flat data model for storage, but provides powerful mechanisms to organize data logically. At the top level, you create buckets which serve as containers for your objects. Within these buckets, each object is identified by a unique key (essentially the full path and filename). While there are no actual folders or subdirectories in the underlying storage, you can create a logical hierarchy using key name prefixes and delimiters to organize objects in a familiar folder-like structure .

Key Organizational Concepts

1
Buckets: The top-level containers for all data stored in S3. Each bucket name must be globally unique across all AWS accounts and exist within a specific AWS Region .
2
Objects and Keys: Each object is uniquely identified within a bucket by a key name. The key is a sequence of Unicode characters with UTF-8 encoding up to 1,024 bytes long, and serves as the complete identifier for retrieving the object .
3
Prefixes: Strings at the beginning of object key names used to group related objects. For example, keys like 'photos/vacation/image.jpg' use 'photos/vacation/' as a prefix to group vacation photos together .
4
Delimiters: Characters (commonly forward slash '/') that help organize objects hierarchically. When listing objects, you can specify a delimiter to roll up all keys sharing a common prefix into a single summary result .

Example of Prefix-Based Organization

The S3 console enhances this organizational model by visually representing prefixes as folders. When you create a folder in the console, it actually creates a zero-byte object with the folder name as the key, which helps maintain the folder illusion. Behind the scenes, you can also add additional organization through object tags (up to 10 key-value pairs per object) for more flexible categorization, access control, and lifecycle management .

Additional Organization Methods

1
Object Tags: Up to 10 key-value pairs per object that can be created, updated, and deleted throughout an object's lifecycle. Useful for fine-grained access control (ABAC) and lifecycle rule management .
2
Object Metadata: User-defined metadata stored as key-value pairs within the object. Limited to 2KB total size (including both keys and values) and all keys are converted to lowercase .
3
S3 Inventory: Reports that list objects and their metadata, tags, and encryption status, configurable on a daily or weekly basis for comprehensive tracking .
4
S3 Metadata: A newer feature that delivers queryable object metadata in near real-time for accelerated data discovery, storing information in Apache Iceberg tables .

Difficulty: 5/10

Topics: Buckets and objects, Key naming and prefixes, Versioning & storage classes

Scenario Questions

0-2 years experience

1
If you need to store user profile pictures in S3, how would you name the objects to make it easy to retrieve a specific user's picture?
2
What happens if you upload two objects with the same key into the same bucket without versioning enabled?
3
How would you list all objects that belong to a particular project using the S3 console or CLI?

2-5 years experience

1
Your application is failing to find a file in S3 even though you see it in the console. What are possible reasons related to key naming or prefixes?
2
We need to store logs that are partitioned by date. How would you structure the S3 key hierarchy, and what trade‑offs does that introduce for lifecycle policies?
3
Explain why enabling versioning changed the way our delete operation works and how you would adjust the code to permanently delete an object.

5-8 years experience

1
Design a data lake on S3 that supports both raw ingestion and curated datasets, considering how you organize buckets, prefixes, and storage classes for cost and performance.
2
Our team is experiencing high request latency when listing objects under a common prefix that contains millions of files. How would you redesign the organization to mitigate this?
3
We need to enforce different retention policies for regulatory data versus analytics data in the same S3 account. How would you structure buckets/prefixes and use S3 features to achieve this at scale?

8+ years experience

1
Across multiple business units, we want a unified S3 data architecture that balances security, cost, and data discoverability. Walk me through how you would define the bucket hierarchy, naming conventions, and cross‑account access patterns.
2
We are migrating a legacy on‑prem file system with deep directory trees into S3. What strategy would you use to map the existing hierarchy to S3 keys while minimizing impact on downstream processes?
3
How would you evolve the S3 organization strategy over time as we adopt event‑driven processing and need to integrate S3 events with many downstream services?

Follow-up Questions

❓Can you give an example of a key naming pattern you used for a real project?
❓How would you prevent accidental overwrites of critical objects?
❓What monitoring or alerts would you set up to detect drift in your S3 organization?