Amazon SQS: Standard and FIFO Queues
A fully managed message queue for decoupling services — buffering work between producers and consumers so neither depends on the other being available at the same instant.
Want a visual for this topic?
Generate a diagram tailored to Amazon SQS: Standard and FIFO Queues — the AI picks whichever visual (architecture, flowchart, ER diagram, etc.) best fits this specific AWS concept.
Sign in to generate a visual →🎓 Learning objectives
- •Explain why a queue decouples a producer from a consumer
- •Explain the difference between Standard and FIFO queues
- •Explain visibility timeout and how it prevents duplicate processing
- •Explain the role of a Dead Letter Queue
What is it?
Amazon SQS (Simple Queue Service) is a fully managed message queuing service — producers send messages to a queue, and consumers poll the queue to retrieve and process them, with the queue durably storing messages until they're successfully processed and deleted.
Why it exists
Directly calling one service from another (synchronous coupling) means both must be available and fast at the same moment — if the receiving service is down or slow, the calling service is blocked or fails too. SQS exists to decouple that relationship: a producer can send a message and move on immediately, regardless of whether the consumer is currently available, fast, or even running at all right now — the queue durably holds the message until a consumer is ready.
Problem it solves
It solves the tight-coupling problem (producer and consumer don't need to be simultaneously available), the traffic-spike problem (a queue absorbs a burst of incoming work, letting consumers process it at their own sustainable pace instead of being overwhelmed), and the reliability problem (a message isn't lost if a consumer crashes mid-processing — it becomes visible again for another consumer to retry).
Intuition
A queue is like a restaurant's order ticket rail: the server (producer) posts an order and immediately moves on to the next table, without waiting for the kitchen (consumer) to be free right that second. The kitchen works through tickets at its own sustainable pace, and if a cook has to step away mid-order, the ticket stays on the rail for someone else to pick up — the order is never simply lost.
Analogy
A physical mailbox: you drop a letter in and walk away immediately, with no need for the recipient to be home at that exact moment. The letter waits safely in the box until someone comes to collect and read it — the sender and reader are fully decoupled in time.
Technical explanation
When a consumer receives a message, it isn't deleted immediately — it becomes temporarily invisible to other consumers for a configured Visibility Timeout, giving the consumer time to process it. If the consumer successfully finishes, it explicitly calls DeleteMessage; if it crashes or times out without deleting, the message automatically becomes visible again after the timeout expires, and another consumer (or the same one) will receive it again — this is what makes SQS 'at-least-once' delivery, meaning a message might occasionally be processed more than once, so consumer logic should be idempotent. Standard queues offer nearly unlimited throughput but only best-effort ordering and at-least-once (occasionally more than once) delivery. FIFO queues guarantee strict ordering and exactly-once processing within a Message Group, at a lower throughput ceiling (though High Throughput FIFO mode raises this significantly) and require a MessageGroupId on every message.
Architecture
An order-processing system has a web server (producer) place a message onto an SQS queue the instant an order is placed, returning an immediate response to the customer without waiting for order fulfillment logic to run. A fleet of worker instances (or Lambda functions triggered by the queue) consume messages at their own pace, processing each order's fulfillment steps — if the fulfillment logic briefly fails or the worker crashes, the message becomes visible again automatically for a retry, with a Dead Letter Queue catching any message that fails repeatedly so it doesn't block the queue indefinitely.
Workflow
- Create a queue — Standard for maximum throughput and simplicity where strict ordering/exactly-once isn't required, FIFO where message order and exactly-once processing within a group genuinely matter (e.g. financial transactions). 2) Configure an appropriate Visibility Timeout based on how long your consumer actually needs to process one message/batch. 3) Configure a Dead Letter Queue with a maxReceiveCount so messages that repeatedly fail processing are automatically routed there instead of blocking the main queue forever. 4) Write idempotent consumer logic, since at-least-once delivery means occasional duplicate processing is a normal, expected possibility, not an edge case.
Example
A payment processing system uses a FIFO queue with the customer's account ID as the MessageGroupId, ensuring all of one customer's transactions are processed in the exact order they were submitted (critical for correct balance calculations), while different customers' transactions can still be processed in parallel across different message groups. A separate, much higher-volume image-processing pipeline uses a Standard queue (strict ordering doesn't matter for independent image uploads) with a Dead Letter Queue catching any image that fails processing 3 times, routing it for manual review instead of endlessly retrying.
Real-world usage
SQS is one of AWS's oldest and most foundational services, used extremely widely as the default decoupling mechanism between microservices; the combination of SQS plus Lambda (using SQS as a Lambda event source, automatically triggering function invocations as messages arrive) is a very common serverless async-processing pattern documented extensively in AWS's own architecture guidance.
Trade-offs
Standard queues maximize throughput and simplicity at the cost of ordering guarantees and occasional duplicate delivery; FIFO queues guarantee order and effectively-once processing within a group at a throughput cost, though High Throughput FIFO mode has narrowed that gap substantially. A shorter visibility timeout risks a message becoming visible again (and re-processed by another consumer) while the original consumer is still legitimately working on it; too long a timeout delays legitimate retries when a consumer genuinely crashes without deleting the message.
Visual explanation
Picture a Producer service sending messages into an SQS queue (a simple arrow in). One or more Consumer instances poll the queue (an arrow out), each retrieving a batch of messages, processing them, and explicitly deleting each one only after successful processing — if a consumer crashes before deleting a message, that message automatically becomes visible again after its visibility timeout expires, ready for another consumer to pick up.
Advantages
- —
Fully decouples producers and consumers in both time and availability — one can be down without blocking the other
- —
Absorbs traffic spikes naturally, letting consumers process at a sustainable pace rather than being directly overwhelmed
- —
Built-in retry via visibility timeout and Dead Letter Queues for messages that repeatedly fail, without custom retry logic
- —
Scales automatically with no capacity provisioning needed — both Standard and FIFO handle high throughput without pre-configuration
Disadvantages
- —
At-least-once delivery on Standard queues means occasional duplicate processing, requiring consumer logic to be idempotent
- —
Standard queues don't guarantee ordering, unsuitable for workloads where processing order genuinely matters
- —
FIFO queues have a lower throughput ceiling than Standard (though High Throughput mode narrows this gap significantly)
- —
Adds asynchronous complexity and eventual (not immediate) processing — inappropriate for anything needing a synchronous, immediate response
Common mistakes
- —
Writing non-idempotent consumer logic that breaks when a message is (rarely but validly) delivered and processed more than once under Standard queue's at-least-once guarantee
- —
Setting the visibility timeout too short relative to actual processing time, causing a message to become visible again and be processed twice by two different consumers simultaneously, even though the first was still legitimately working
- —
Using a Standard queue for a workload that genuinely requires strict ordering, then trying to work around the lack of ordering guarantee with fragile application-level sequencing logic instead of just using FIFO
- —
Not configuring a Dead Letter Queue, so a single malformed or unprocessable message retries forever, potentially blocking visibility of other messages behind it or generating endless error noise
- —
Forgetting that FIFO queues require a MessageGroupId on every message, and choosing a group ID that's too coarse (e.g. one single group for the entire queue) unintentionally serializing all processing that could have run in parallel across different groups
In the AWS Console
- 1
AWS Console → SQS → Create queue
Choose Standard or FIFO type (note: FIFO queue names must end in .fifo), and configure the Visibility Timeout based on your expected processing time per message.
The queue type cannot be changed after creation — choose deliberately based on whether ordering and exactly-once processing are genuinely required.
- 2
SQS → [your queue] → Dead-letter queue → Edit
Create or select a separate queue as the Dead Letter Queue, and set the 'Maximum receives' threshold after which a repeatedly-failing message is routed there instead of the main queue.
Monitor the Dead Letter Queue actively — messages landing there represent processing failures that need investigation, not just a place to silently discard problems.
- 3
SQS → [your queue] → Lambda triggers → Configure Lambda function trigger
Select a Lambda function to automatically invoke as messages arrive in the queue, configuring batch size (how many messages per invocation).
Lambda automatically deletes successfully processed messages from the queue; a function error causes the message(s) in that batch to become visible again for retry, per standard SQS behavior.
🎤 Interview questions
Why would you put a queue between two services instead of having them call each other directly? (Listen for: decouples availability and timing — producer doesn't need consumer to be up/fast right now; absorbs traffic spikes; enables retry without custom logic.)
What's the difference between Standard and FIFO SQS queues? (Listen for: Standard = higher throughput, best-effort ordering, at-least-once (possible duplicates); FIFO = strict ordering and exactly-once processing within a message group, lower throughput ceiling.)
Explain visibility timeout and what happens if it's set too short. (Listen for: hides a received message from other consumers while being processed; too short means the message reappears and gets processed again by a different consumer while the original is still legitimately working on it.)
Why does SQS's at-least-once delivery guarantee require consumer logic to be idempotent? (Listen for: a message can occasionally be delivered and processed more than once — e.g. after a visibility timeout expiry before deletion — so processing it twice must produce the same correct end result, not a duplicated side effect.)
What's a Dead Letter Queue for? (Listen for: catches messages that have failed processing repeatedly (past a configured max receive count), preventing them from blocking the main queue or retrying forever, and surfacing them for investigation.)