Amazon S3 Fundamentals
AWS's object storage service — buckets, objects, storage classes, and the access control model that keeps your data secure by default.
Want a visual for this topic?
Generate a diagram tailored to Amazon S3 Fundamentals — the AI picks whichever visual (architecture, flowchart, ER diagram, etc.) best fits this specific AWS concept.
Sign in to generate a visual →🎓 Learning objectives
- •Create a bucket and upload/retrieve objects via the Console and CLI
- •Explain the difference between bucket policies and IAM policies for S3 access
- •Choose an appropriate S3 storage class for a given access pattern
- •Explain why S3 buckets are private by default and how that protects against accidental exposure
What is it?
Amazon S3 (Simple Storage Service) is AWS's core object storage service — you create buckets (top-level containers) and store objects (files, up to 5TB each) inside them, identified by a key (essentially a file path string). It's designed for durability and virtually unlimited scale, and is the default place to store anything from user-uploaded files to application logs to static website assets on AWS.
Why it exists
Applications need somewhere reliable to store files that isn't tied to a single server's local disk — a local disk disappears the moment that server is terminated, and doesn't scale beyond that one machine's capacity. S3 exists to give applications durable, effectively infinite storage accessible over a simple HTTP API from anywhere, decoupled entirely from any specific compute instance's lifecycle.
Problem it solves
It solves the durable-storage problem (S3 objects survive independently of any EC2 instance, engineered for 99.999999999% durability), the scale problem (no practical limit on total storage or object count), and the access problem (any authorized client, anywhere, can read/write via a simple REST API, not just something physically attached to one server).
Intuition
Think of an S3 bucket as an enormous, infinitely expandable filing cabinet drawer that lives outside any single office (server) — anyone with the right key (credentials/permissions) can put a file in or take one out from anywhere in the world, and the drawer itself never runs out of space or burns down, because AWS maintains many redundant copies behind the scenes.
Analogy
A bank's safety deposit vault system: you don't store your valuables in your own house (a local disk that could be lost with the house); you store them in a professionally secured, redundant vault (S3) that's accessible whenever you present the right credentials, regardless of what happens to your house.
Technical explanation
S3 provides strong read-after-write consistency for all operations (PUTs of new objects and overwrites of existing ones are immediately visible to subsequent reads) — a guarantee that wasn't always true historically but is now standard. Buckets are Regional resources but have a single global namespace for their names. By default, every new bucket and object is fully private — S3 Block Public Access settings (enabled by default on new buckets) provide an account/bucket-level safety net that prevents public access even if an individual bucket policy or ACL would otherwise allow it, specifically to guard against accidental misconfiguration.
Architecture
A typical application architecture uses S3 as the durable store for user-uploaded content (photos, documents, videos), with the application server generating pre-signed URLs so the client's browser uploads/downloads directly to/from S3 rather than proxying large files through the application tier. Static website assets (HTML/CSS/JS) are often served from an S3 bucket configured for static website hosting, typically fronted by CloudFront for both performance and to keep the bucket itself private.
Workflow
- Create a bucket with a globally unique name in your target Region. 2) Leave Block Public Access enabled unless you have a specific, deliberate reason to serve public content directly from S3 (prefer fronting with CloudFront instead). 3) Set a bucket policy or rely on IAM policies on your application's role to control access. 4) Upload objects via the Console, CLI, SDK, or pre-signed URLs from your application. 5) Choose an appropriate storage class per object or via lifecycle rules based on expected access frequency.
Example
A photo-sharing app has users upload directly to an S3 bucket via a pre-signed URL generated by the backend (so large photo uploads never pass through the application server), stores each photo at a key like users/{user_id}/photos/{photo_id}.jpg, keeps the bucket fully private with Block Public Access enabled, and serves photos to viewers through a CloudFront distribution with Origin Access Control — no direct public S3 access exists at any point.
Real-world usage
S3 is used as the storage layer under an enormous fraction of the internet's applications — from small startups storing user uploads to Netflix storing video assets to Pinterest storing image data at exabyte scale; its durability guarantee and simple HTTP API have made it the default 'just put it in S3' answer for unstructured data storage across the industry.
Trade-offs
Storing data in S3 versus a database or block storage is a tradeoff between simplicity/durability/cost at scale (S3) and low-latency structured querying or in-place random access (databases, EBS) — S3 is the right choice for whole-file storage accessed as a unit, not for data you need to query, index, or partially update in place. Making a bucket public (disabling Block Public Access) is simpler for serving content directly but loses the ability to enforce access logging, geographic restrictions, or DDoS protection that fronting with CloudFront provides.
Visual explanation
Picture a bucket as a labeled box (globally unique name across all of AWS, like a domain name) containing objects, each with a key like photos/2024/vacation.jpg — there's no real folder structure underneath; the '/' characters in keys are just part of the string, displayed as folders in the Console UI for convenience. Access is controlled by a combination of the bucket's own policy, IAM policies on the calling identity, and (rarely used today) per-object ACLs, evaluated together to decide Allow or Deny.
Advantages
- —
Extremely durable (99.999999999% annual durability) and virtually unlimited in total storage capacity
- —
Simple, well-documented REST API supported by every major SDK and language
- —
Private by default, with Block Public Access providing a strong safety net against accidental exposure
- —
Multiple storage classes let you optimize cost based on actual access patterns without changing application code
Disadvantages
- —
Not a file system — no true directory operations, no in-place partial file edits (you replace the whole object or use multipart upload for large ones)
- —
Higher per-request latency than local disk or block storage, unsuitable for latency-critical random-access workloads like a database's data files
- —
Misconfigured bucket policies remain one of the most common sources of real-world data breaches when Block Public Access is deliberately disabled
- —
Data transfer OUT of S3 to the internet has a real cost that can add up at scale, unlike storage cost alone
Common mistakes
- —
Disabling Block Public Access and setting an overly permissive bucket policy 'to make it work,' unintentionally exposing sensitive data to the entire internet — a leading real-world cause of data breaches
- —
Treating S3 like a traditional file system and trying to do partial in-place edits, when the correct pattern is replacing (or multipart-uploading a new version of) the whole object
- —
Not setting up lifecycle policies, leaving old/cold data in the expensive Standard storage class indefinitely
- —
Proxying large file uploads/downloads through the application server instead of using pre-signed URLs for direct client-to-S3 transfer, unnecessarily loading the application tier
- —
Forgetting that bucket names are globally unique across ALL of AWS, not just your account — a seemingly available name can be taken by another AWS customer entirely
In the AWS Console
- 1
AWS Console → S3 → Buckets → Create bucket
Enter a globally unique bucket name, choose your Region, and leave 'Block all public access' checked unless you have a specific, deliberate reason to disable it.
If a bucket name is rejected as 'already exists,' remember bucket names are unique across all of AWS globally, not just your own account.
- 2
S3 → [your bucket] → Upload → Add files
Upload one or more files directly through the console, or use 'Add folder' to preserve a folder-like key prefix structure.
The 'folders' shown in the console are a UI convenience built from '/' characters in object keys — there's no real directory structure underneath.
- 3
S3 → [your bucket] → Permissions → Bucket policy
Edit the bucket policy JSON to grant specific access (e.g. allow a specific CloudFront distribution via OAC, or a specific IAM role) rather than a blanket public-read policy.
Bucket policies and IAM policies on the calling identity are evaluated together — an explicit Deny in either one blocks the request regardless of what the other allows.
- 4
S3 → [your bucket] → Properties → scroll to Static website hosting
Enable static website hosting and specify an index document if serving a static site directly from the bucket (note: this requires public access, so pair it with caution or prefer fronting with CloudFront instead).
Static website hosting endpoints don't support HTTPS directly — for a production site, front the bucket with CloudFront to get SSL, custom domains, and caching.
🎤 Interview questions
Why are S3 buckets private by default, and what is Block Public Access? (Listen for: prevents accidental exposure — private-by-default plus Block Public Access is a deliberate safety net requiring explicit, conscious steps to make anything public.)
What's the difference between a bucket policy and an IAM policy for controlling S3 access? (Listen for: bucket policy is attached to the resource (the bucket) and can grant cross-account access; IAM policy is attached to the identity (user/role); both are evaluated together, explicit Deny in either wins.)
Why would you use a pre-signed URL instead of proxying a file upload through your application server? (Listen for: offloads bandwidth/CPU from the app server, lets the client upload directly to S3 with a time-limited, scoped permission.)
S3 is described as 'not a file system' — what does that mean practically? (Listen for: no true directories, no in-place partial edits, objects are replaced wholesale or via multipart upload, higher latency than local/block storage for random access.)
What real-world security mistake commonly exposes S3 data publicly? (Listen for: disabling Block Public Access combined with an overly permissive bucket policy or ACL, often done to 'quickly fix' an access problem without understanding the exposure.)