Amazon MQ & MSK: Managed Message Brokers
The managed-broker alternative to SQS/SNS — Amazon MQ for lift-and-shift of existing ActiveMQ/RabbitMQ workloads, and MSK for fully-managed Apache Kafka.
Want a visual for this topic?
Generate a diagram tailored to Amazon MQ & MSK: Managed Message Brokers — the AI picks whichever visual (architecture, flowchart, ER diagram, etc.) best fits this specific AWS concept.
Sign in to generate a visual →🎓 Learning objectives
- •Explain why Amazon MQ exists given that SQS/SNS already cover queuing and pub/sub
- •Describe what MSK manages versus what a self-hosted Kafka cluster requires you to manage yourself
- •Distinguish when to reach for MSK over SQS/Kinesis for a streaming use case
- •Understand MSK Serverless as the lower-operational-overhead alternative to provisioned MSK
What is it?
Amazon MQ is a managed message broker service supporting industry-standard protocols (JMS, AMQP 0-9-1/1.0, MQTT, STOMP, OpenWire) via managed ActiveMQ or RabbitMQ engines — it exists specifically for migrating existing applications built against those standard protocols without rewriting them against SQS/SNS's AWS-proprietary APIs. Amazon MSK (Managed Streaming for Apache Kafka) is a fully-managed Apache Kafka service — AWS provisions, patches, and operates the broker and controller infrastructure, while you design topics, partitions, and consumer groups exactly as you would with open-source Kafka, and existing Kafka client libraries and tooling (Kafka Connect, Kafka Streams, Schema Registry-compatible tools) work against it unchanged.
Why it exists
Both services exist to solve the same underlying problem from two different angles: a huge number of existing enterprise applications and data platforms were already built against specific messaging standards (JMS/AMQP for traditional enterprise messaging, Kafka for streaming data platforms) years before AWS's own proprietary services existed. Rewriting that code against SQS/SNS/Kinesis APIs to move to AWS would be expensive and risky — Amazon MQ and MSK exist so that migration can be 'point your existing client at a new managed endpoint' instead of a rewrite.
Problem it solves
It solves the migration problem: moving an application that already depends on a specific open-source messaging protocol or platform to AWS, without a rewrite, while removing the operational burden of patching and managing broker infrastructure yourself.
Intuition
The decision tree is really about what already exists, not which service is 'better' in the abstract. If you already have code written against JMS/AMQP/MQTT (Amazon MQ) or against the Kafka client API and ecosystem (MSK), those existing protocol/API investments are exactly what these services let you keep using unmodified on AWS-managed infrastructure. If you're building something new with no existing protocol commitment, SQS/SNS or Kinesis are usually the simpler, more AWS-native starting point.
Analogy
SQS and SNS are AWS's own proprietary courier service with its own pickup/delivery rules. Amazon MQ is a managed depot that still speaks the universal shipping-container standard (JMS/AMQP/MQTT) your existing trucks were already built for, so you don't have to retrofit your fleet. MSK is a managed version of a specific, extremely popular third-party freight network (Apache Kafka) that many companies already standardized their entire logistics around.
Technical explanation
Amazon MQ for ActiveMQ's active/standby mode uses a shared EFS-backed message store between the active and standby broker, so failover doesn't lose in-flight messages — the standby simply takes over the same underlying storage. MSK's newer KRaft mode replaces Zookeeper-based cluster metadata management with Kafka's own built-in Raft-based controller quorum, removing an entire separate service (Zookeeper) that self-managed Kafka clusters historically had to operate. MSK Connect runs Kafka Connect as a fully-managed service, handling connector worker scaling and restart-on-failure the same way MSK handles broker management — both extend the 'managed infrastructure, unmodified open-source API' pattern to the connector layer.
Architecture
Amazon MQ runs the actual open-source ActiveMQ or RabbitMQ engine on AWS-managed EC2 infrastructure, in single-instance or active/standby (for ActiveMQ) / cluster (for RabbitMQ) configurations, with automatic failover to a standby on failure. MSK provisions a cluster of Kafka brokers across multiple Availability Zones, each broker running actual Apache Kafka, backed by EBS storage per broker and using either Zookeeper or the newer KRaft controller mode for cluster metadata — both entirely managed by AWS, with broker replacement/patching handled transparently. Both services expose the standard connection endpoints (AMQP/JMS endpoint for MQ, Kafka bootstrap-broker list for MSK) that unmodified client libraries connect to directly.
Workflow
For Amazon MQ: 1) Create a broker (single-instance or active/standby pair for high availability), choosing the ActiveMQ or RabbitMQ engine. 2) Point your existing application's connection configuration at the broker's endpoint, keeping its existing JMS/AMQP client code unchanged. 3) AWS handles patching, failover, and monitoring. For MSK: 1) Create an MSK cluster (provisioned, sized by broker count/type, or Serverless), 2) Create topics with your desired partition count and replication factor, exactly like open-source Kafka, 3) Point existing Kafka producer/consumer client code at the cluster's bootstrap brokers, 4) Optionally attach Kafka Connect via MSK Connect for managed source/sink connectors.
Example
A company migrating a legacy Java application built on JMS and ActiveMQ moves it to Amazon MQ for ActiveMQ with only a connection-string change, avoiding a rewrite to SQS. Separately, a data platform team already standardized on Kafka Connect and Kafka Streams for its event pipelines adopts MSK to get the exact same Kafka semantics and tooling, but without operating Zookeeper/broker patching themselves.
Real-world usage
Enterprises with a large existing investment in JMS-based messaging (common in traditional banking/insurance Java stacks) commonly use Amazon MQ specifically as a lift-and-shift step, sometimes as a stepping stone toward eventually re-architecting onto SQS/SNS later. MSK is heavily used by data platform and analytics teams that already standardized their event pipelines on Kafka Streams/Connect and want managed infrastructure without abandoning that tooling investment — often alongside Kinesis for newer, AWS-native-only pipelines built after the Kafka migration.
Trade-offs
Amazon MQ trades SQS's near-infinite auto-scaling and pay-per-request simplicity for standard-protocol compatibility — a broker still has a defined instance size and needs capacity planning, closer to RDS than to SQS. MSK trades Kinesis's fully-serverless simplicity for full Kafka ecosystem compatibility — provisioned MSK still requires partition/broker sizing decisions, though MSK Serverless closes much of that gap at a different cost model.
Visual explanation
Picture three lanes for moving events between systems. Lane one (SQS/SNS): AWS's own proprietary lane, simplest on-ramp, no capacity planning. Lane two (Amazon MQ): a lane built to the exact same width as the JMS/AMQP trucks you already own, so they drive straight on without modification. Lane three (MSK): a lane built to Kafka's exact specification, so every Kafka-shaped vehicle and every Kafka-specific tool (Connect, Streams) already built for that shape works immediately.
Advantages
- —
Zero rewrite for existing JMS/AMQP/MQTT (MQ) or Kafka (MSK) applications — the exact protocol/API surface is preserved
- —
AWS manages broker patching, failover, and (for MSK) Zookeeper/KRaft controller operations that are otherwise a real operational burden
- —
MSK Connect provides managed Kafka Connect, and MSK integrates with IAM for authentication as an alternative to SASL/mTLS
- —
Full compatibility with the existing open-source ecosystem's tooling — monitoring dashboards, client libraries, and third-party integrations built for ActiveMQ/RabbitMQ/Kafka work unmodified
Disadvantages
- —
Amazon MQ still requires instance-size capacity planning like RDS — it isn't a scale-to-zero or infinitely-elastic service the way SQS is
- —
Provisioned MSK requires the same partition-count and broker-sizing decisions as self-managed Kafka — the operational simplification is in patching/replication, not in eliminating capacity planning entirely
- —
Running two protocol-compatible services (MQ, MSK) in addition to SQS/SNS/Kinesis means more services to understand and choose between for a new project
- —
Cross-AZ data transfer costs apply to MSK broker replication the same way they would for a self-managed multi-AZ Kafka cluster
Common mistakes
- —
Choosing MSK for a brand-new project with no existing Kafka dependency, when SQS/SNS or Kinesis would be simpler and require no partition/broker capacity planning at all
- —
Under-provisioning MSK broker count/size and hitting partition-level throughput limits under load, the same failure mode as an under-sized self-managed Kafka cluster
- —
Not enabling Amazon MQ's active/standby (high-availability) mode for a production workload, leaving a single point of failure
- —
Assuming MSK Serverless has zero configuration decisions — topic-level settings and IAM-based ACLs still need deliberate design even without broker sizing
In the AWS Console
- 1
Amazon MQ → Brokers → Create brokers
Create an Amazon MQ broker, choosing the ActiveMQ or RabbitMQ engine and single-instance or active/standby deployment mode.
Active/standby deployment costs roughly double a single instance but removes the single point of failure — use it for anything production.
- 2
MSK → Clusters → Create cluster
Create an MSK cluster, choosing Provisioned (broker count/type) or Serverless.
MSK Serverless removes broker sizing entirely but has its own throughput quotas per topic — check they fit your expected load.
- 3
(Kafka CLI, not console) — copy bootstrap broker string from MSK → Clusters → [cluster] → View client information
Create a Kafka topic on the new MSK cluster using the standard Kafka CLI tools pointed at the cluster's bootstrap brokers.
🎤 Interview questions
Why would you use Amazon MQ instead of SQS/SNS? (Listen for: Amazon MQ exists specifically for migrating existing applications already built against a standard broker protocol — JMS, AMQP, MQTT, STOMP, OpenWire — via ActiveMQ or RabbitMQ, without rewriting the application to use the SQS/SNS APIs; SQS/SNS are AWS-proprietary APIs, MQ speaks industry-standard protocols)
What is Amazon MSK, and how does it differ from self-managed Kafka on EC2? (Listen for: MSK is fully-managed Apache Kafka — AWS handles broker provisioning, patching, replication, and Zookeeper/KRaft controller management; you still design topics, partitions, and consumer groups exactly like open-source Kafka, and existing Kafka client code/tooling works unchanged)
When would you choose MSK over Kinesis Data Streams for a streaming workload? (Listen for: MSK when you need Kafka-specific ecosystem compatibility — Kafka Connect, Kafka Streams, existing Kafka-based tooling/client libraries — or need to migrate an existing Kafka workload as-is; Kinesis when you want a simpler, fully-serverless AWS-native stream with tighter integration into other AWS services like Lambda/Firehose and no cluster concept to manage at all)
What is MSK Serverless? (Listen for: a deployment mode of MSK with no brokers to size or provision — capacity scales automatically with throughput, billed per data volume rather than per broker-hour — trading some configuration flexibility for zero cluster-capacity-planning overhead, similar to how DynamoDB on-demand relates to provisioned DynamoDB)
A team has an existing on-premises RabbitMQ-based application they want to move to AWS with minimal code changes. What service fits? (Listen for: Amazon MQ for RabbitMQ — it speaks the exact AMQP/RabbitMQ protocol the application already uses, so client code and connection strings need minimal changes, unlike a rewrite to SQS)