Design an Image Upload & Processing Service on AWS
A worked system-design example for an image-heavy application — direct-to-S3 upload via pre-signed URLs, asynchronous thumbnail/resize processing via Lambda, and content moderation.
Want a visual for this topic?
Generate a diagram tailored to Design an Image Upload & Processing Service on AWS — the AI picks whichever visual (architecture, flowchart, ER diagram, etc.) best fits this specific AWS concept.
Sign in to generate a visual →🎓 Learning objectives
- •Design the direct-to-S3 upload flow that keeps large image files off the application server
- •Explain how S3 event notifications drive asynchronous image processing
- •Design for multiple output sizes (thumbnail, medium, full) without blocking the upload response
- •Incorporate content moderation into the pipeline without adding synchronous latency
What is it?
An image upload and processing service's AWS system design uses pre-signed S3 URLs for direct client-to-S3 upload (keeping the application server out of the large-file data path), S3 event notifications to asynchronously trigger a Lambda function that generates multiple resized variants (thumbnail, medium, full) and optionally runs content moderation (via Amazon Rekognition), writing outputs back to S3 for eventual serving via CloudFront.
Why it exists
This pattern exists because image upload and processing has the same two core problems as any large-file-handling design — keeping big binary transfers off the application server's own compute path, and not making users wait synchronously for genuinely slow processing work — and S3's event-notification-plus-Lambda combination is AWS's standard, low-effort answer to both.
Problem it solves
It solves efficiently and scalably handling user-uploaded images — keeping large uploads off the application server, generating the multiple size variants a real application typically needs without blocking the uploader, and screening for inappropriate content — using managed, event-driven AWS services rather than custom-built processing infrastructure.
Intuition
The consistent principle across this design: never make the uploader wait for anything that doesn't strictly need to happen before they get a response — the only synchronous step is the original upload itself; every derived artifact (resized variants, moderation result) happens asynchronously afterward, triggered automatically by the upload event.
Analogy
This design is like a photo-printing kiosk that takes your original photo instantly (direct upload to S3), then quietly produces wallet-size, standard, and poster-size prints in the back room (asynchronous Lambda processing) while you're handed a receipt immediately, rather than making you stand at the counter until every print size is physically ready.
Technical explanation
S3 event notifications can trigger Lambda directly or route through SNS/SQS first for fan-out to multiple independent processing consumers (e.g., one Lambda for resizing, a separate one for moderation) without either needing to know about the other, following the same SNS-fan-out decoupling pattern used elsewhere in event-driven AWS architectures. Amazon Rekognition's DetectModerationLabels API returns a confidence-scored list of potentially inappropriate content categories, letting the application set its own confidence threshold for what counts as 'flagged,' rather than getting a single binary safe/unsafe answer — this threshold is a real product decision balancing false positives (legitimate content incorrectly flagged) against false negatives (inappropriate content that slips through).
Architecture
The client requests and receives a pre-signed S3 PUT URL from the application (a cheap API call), then uploads the original image directly to an S3 'originals' bucket/prefix. An S3 event notification configured on that prefix triggers a processing Lambda, which reads the original object, generates each required size variant using an image-processing library (or, for heavier processing needs, could instead pass the job to a container-based service), and writes outputs to predictable, derived S3 keys — often in a separate 'derived'/'processed' bucket or prefix, kept separate from originals for lifecycle-policy and access-control reasons. A parallel or chained step calls Amazon Rekognition's DetectModerationLabels API on the original image, with the result determining the image's visibility status in the application's own database. CloudFront serves the final processed variants to end users.
Workflow
- Application generates a pre-signed S3 PUT URL and returns it to the client. 2) Client uploads the original image directly to S3. 3) An S3 event notification (on object creation in the 'originals' prefix/bucket) triggers a processing Lambda. 4) The Lambda downloads the original, generates the required size variants using an image-processing library, and writes each to a predictable output key (e.g.,
thumb/{imageId}.jpg,medium/{imageId}.jpg). 5) The same or a separate Lambda invocation calls Amazon Rekognition's content moderation API on the original image. 6) The application's database record for the image is updated to 'ready'/'pending review' based on the moderation result, and the client is notified (via polling or a push mechanism) that processing is complete.
Example
A user uploads a profile photo directly to S3 via a pre-signed URL the application generated; an S3 event notification triggers a Lambda function that generates a 150px thumbnail and a 800px medium variant using an image-processing library, writes both back to S3 under predictable derived keys, and separately calls Amazon Rekognition's content moderation API — if Rekognition flags the image, the application marks it 'pending review' rather than immediately showing it publicly; otherwise it becomes visible once the resize step completes.
Real-world usage
This exact pre-signed-upload-plus-S3-event-plus-Lambda-processing pattern is extremely common in production for any application handling meaningful volumes of user-uploaded images (social apps, marketplaces, content platforms), and Amazon Rekognition's content moderation API specifically is widely used precisely because building and maintaining a custom content-moderation ML model is a significant undertaking most product teams reasonably choose to avoid.
Trade-offs
Asynchronous processing means there's a real (if usually short) delay between 'upload complete' and 'all size variants and moderation result available' — the application needs a 'processing' state and a way to communicate that to the client, adding a bit of product/UX complexity compared to a (much slower and more resource-intensive) synchronous approach. Running content moderation asynchronously means there's a brief window where an image could theoretically be visible before moderation completes unless the application explicitly gates visibility on moderation status — a deliberate design decision, not an oversight, but one that needs to be made consciously.
Visual explanation
Picture a package drop-off box (pre-signed S3 upload) that immediately alerts a back-office team (Lambda) the moment something's dropped in — the person dropping off the package gets an instant receipt and walks away, while the back office quietly repackages the item into several standard sizes and runs a quick safety inspection, updating a status board (the application's database) once everything's done.
Advantages
- —
Pre-signed direct upload keeps large image transfers entirely off the application server, reducing its bandwidth/memory burden and improving scalability
- —
S3 event-driven Lambda processing requires no polling infrastructure — each upload automatically and immediately triggers its own processing pipeline
- —
Content moderation via Rekognition requires no custom ML model training or hosting — a managed API call handles it
- —
The 'processing' status pattern generalizes cleanly to adding more asynchronous steps later (watermarking, additional size variants, format conversion) without changing the upload path itself
Disadvantages
- —
Introduces a real processing delay between upload and full availability of all size variants and moderation results, requiring explicit application-level status tracking
- —
Image-processing Lambda functions need enough memory/timeout configured for the largest expected original image, and very large images (or unusual formats) can push against Lambda's execution limits, sometimes requiring Fargate/ECS for exceptionally heavy processing instead
- —
Storing multiple derived size variants for every uploaded image multiplies S3 storage cost relative to storing just the original
- —
A forgotten or delayed moderation step (e.g., a Lambda failure) could leave an image stuck in 'pending review' indefinitely unless the pipeline has retry/dead-letter-queue handling for that failure case
Common mistakes
- —
Routing image uploads through the application server instead of using a pre-signed URL for direct-to-S3 upload, adding unnecessary server load and network transfer
- —
Making image resizing/moderation synchronous within the upload request, needlessly slowing down the upload response for processing that doesn't need to block it
- —
Making an uploaded image publicly visible immediately, before an asynchronous content-moderation step has actually completed and cleared it
- —
Not configuring a dead-letter queue or retry/alerting for the processing Lambda, risking images silently stuck unprocessed if a transient failure occurs
In the AWS Console
- 1
S3 → Buckets → Create bucket, then Properties → Event notifications
Create the S3 bucket for originals with an event notification configured to trigger a Lambda function on object creation.
- 2
Lambda → Create function, then Configuration → Permissions
Create the processing Lambda function, attaching an image-processing library layer and granting S3 read/write and Rekognition permissions.
- 3
Rekognition → (Console demo) Content moderation
Test the Rekognition content moderation API against a sample image.
🎤 Interview questions
Why should image upload go directly from the client to S3 via a pre-signed URL, rather than through the application server? (Listen for: routing the full image bytes through the application server doubles network transfer (client→app, then app→S3) and consumes application server memory/bandwidth for something that doesn't need application logic in the data path at all; a pre-signed URL lets the client upload directly to S3, with the application server only involved in the cheap step of generating that URL)
How would you generate a thumbnail and a few other sizes from an uploaded image without making the uploader wait for processing? (Listen for: an S3 event notification on object creation triggers a Lambda function asynchronously, which reads the original image, generates the needed size variants (using a library like Sharp/ImageMagick/Java's ImageIO/Thumbnailator), and writes them back to S3 — the upload response to the client returns immediately after the original upload completes, with size variants becoming available moments later, often reflected via a 'processing' status the client polls or is notified of)
Where would content moderation (detecting inappropriate images) fit into this pipeline, and should it block the upload? (Listen for: content moderation (e.g., via Amazon Rekognition's content moderation API) fits as another asynchronous step triggered off the same S3 event, run in parallel with or after resizing — it generally shouldn't block the synchronous upload response, but the application should mark the image as 'pending review' or hide it from public view until moderation completes, rather than making it publicly visible before moderation has run)
How would you avoid generating duplicate size variants if the same image is somehow uploaded or the Lambda is triggered twice? (Listen for: use a deterministic, idempotent output key naming scheme (e.g., derived from the original object key plus a fixed suffix per size) so a re-triggered Lambda invocation simply overwrites the same output key rather than creating duplicates, and/or check for the output object's existence before regenerating it if the processing is expensive)
What would you change about this design for an application needing near-instant thumbnail availability right after upload (not even a brief 'processing' delay)? (Listen for: consider generating a very cheap, fast client-side or Lambda-generated low-quality placeholder/blurhash immediately as part of the upload response, while the full-quality resized variants continue processing asynchronously in the background — trading a small amount of initial quality for zero perceived wait, a common pattern in image-heavy consumer apps)