AWS X-Ray & Health Dashboard: Tracing & Service Health
Distributed tracing for pinpointing exactly which service/call in a request chain is slow or failing, plus the two dashboards for knowing whether an issue is AWS's fault or yours.
Want a visual for this topic?
Generate a diagram tailored to AWS X-Ray & Health Dashboard: Tracing & Service Health — the AI picks whichever visual (architecture, flowchart, ER diagram, etc.) best fits this specific AWS concept.
Sign in to generate a visual →🎓 Learning objectives
- •Explain what a trace and a segment/subsegment represent in X-Ray
- •Describe how X-Ray helps diagnose latency in a microservices architecture that CloudWatch metrics alone can't pinpoint
- •Distinguish AWS Health Dashboard (account-specific operational issues) from AWS Service Health Dashboard (public, AWS-wide status)
- •Understand the X-Ray SDK instrumentation model and its relationship to CloudWatch
What is it?
AWS X-Ray is a distributed tracing service that follows a single request as it travels across every service, function, and downstream call involved in producing its response, visualizing the full chain as a 'service map' and breaking down exactly where time was spent. The AWS Health Dashboard (formerly Personal Health Dashboard) is an account-specific view of operational issues and scheduled changes that affect YOUR resources specifically — instance retirements, API throttling tied to your account, region-specific issues affecting services you actually use. The (public) AWS Service Health Dashboard shows AWS-wide service availability status visible to anyone, regardless of what you personally run.
Why it exists
Before distributed tracing tools existed, diagnosing latency in a multi-service architecture meant manually correlating timestamps across separate logs from every service the request touched — slow, error-prone, and often impossible once a request crossed more than two or three service boundaries. X-Ray exists to automate that correlation using a shared trace ID, turning a manual log-hunting exercise into a single visual timeline. The Health Dashboards exist because a generic 'AWS is having issues somewhere' status page doesn't tell an individual account whether THEIR specific resources are actually affected — account-specific operational awareness needed its own dedicated, personalized channel.
Problem it solves
X-Ray solves the 'which specific service/call in this chain is actually slow' problem in distributed architectures, where aggregate metrics alone can tell you something is wrong but not where. The Health Dashboards solve the 'is this issue on my end or AWS's end, and does it specifically affect resources I run' problem — distinguishing a genuine AWS-side operational issue from an application bug, and distinguishing an issue that affects your account from unrelated AWS-wide noise.
Intuition
Metrics answer 'how much/how often,' logs answer 'what exactly happened at this one point,' and traces answer 'where in the whole chain did this individual request's time actually go' — three different, complementary questions, which is why X-Ray is described as the third pillar of observability alongside CloudWatch metrics and logs, not a replacement for either.
Analogy
CloudWatch metrics are like knowing a package took three days to arrive overall. X-Ray is the full tracking history showing it sat in a specific regional sorting facility for two of those three days — you now know exactly where the delay happened, not just that a delay happened. The Health Dashboards are the difference between a general 'the postal service is experiencing nationwide delays' notice (Service Health Dashboard) and a specific 'your particular package is stuck, here's why' notification addressed just to you (AWS Health Dashboard).
Technical explanation
The X-Amzn-Trace-Id header format encodes the trace ID, the parent segment ID, and a sampling decision flag, so every downstream service in a call chain both contributes to the same trace and respects the same sampling decision made upstream — this is why enabling tracing at the entry point (e.g., API Gateway) is usually enough to propagate sampling consistently through the whole chain, as long as every hop is instrumented. X-Ray's default sampling rule records the first request each second and 5% of additional requests thereafter, per service, which can be overridden with custom sampling rules to capture more (or less) detail for specific high-value routes. AWS Health events are delivered as structured JSON via EventBridge with an event-type-category field (issue, accountNotification, scheduledChange), letting automated handling distinguish, for example, an urgent security issue from a routine scheduled maintenance notice.
Architecture
For EC2/ECS/on-premises, the X-Ray daemon runs as a local UDP listener that receives segment data emitted by the X-Ray SDK embedded in application code, batches it, and forwards it to the X-Ray API — the daemon exists so applications don't make a network call per segment directly to AWS. For Lambda, Active Tracing removes the need for a separate daemon; the Lambda service itself handles segment collection. Every segment carries the shared trace ID (propagated via the X-Amzn-Trace-Id HTTP header across service calls), and X-Ray's backend groups all segments sharing a trace ID into one trace, computing the service map and per-node latency statistics by aggregating traces over a time window. Health Dashboard events are generated by AWS's internal operational systems and delivered per-account, filtered to resources/services your account actually uses, and can be subscribed to via EventBridge for programmatic handling.
Workflow
- Instrument each service with the X-Ray SDK (for Lambda, this can be as simple as enabling 'Active Tracing' with zero code changes; for EC2/ECS, the X-Ray daemon runs as a sidecar/agent collecting segment data and forwarding it to the X-Ray API). 2) Incoming requests get a trace ID either generated fresh or propagated from an upstream caller via the
X-Amzn-Trace-Idheader. 3) Each service's segment (and any subsegments for specific calls like a DB query or HTTP request) gets reported to X-Ray, tagged with that shared trace ID. 4) X-Ray stitches every segment sharing a trace ID into one end-to-end trace, viewable as a timeline and a service map showing average/p99 latency per node. 5) Separately, check the AWS Health Dashboard periodically (or subscribe its events to EventBridge) for account-specific operational notices, and check the public Service Health Dashboard when suspecting a broader AWS outage.
Example
A checkout API's p99 latency spikes according to CloudWatch, but every individual service's CPU/memory looks normal. Pulling up the X-Ray trace for a slow request reveals the actual chain: API Gateway → Lambda (fast) → a downstream payment microservice (fast) → a DynamoDB query inside that service taking 1.8 of the request's 2 total seconds — pointing straight at a missing index rather than a compute or scaling issue anywhere else in the chain.
Real-world usage
Teams running Lambda-based or ECS-based microservices architectures commonly enable X-Ray specifically to debug intermittent latency issues that are hard to reproduce and don't show up as a clear resource-utilization problem in CloudWatch. Enterprises with Business or Enterprise support plans commonly wire AWS Health Dashboard events into their incident-management tooling (via EventBridge) so an account-specific AWS-side issue (like a scheduled instance retirement) automatically creates a tracked ticket instead of relying on someone noticing a console banner.
Trade-offs
X-Ray adds a small amount of latency and requires instrumenting every service in a request chain to get full end-to-end visibility — a chain with one un-instrumented service creates a visibility gap in the middle of the trace. It's most valuable specifically in microservices/distributed architectures; for a single monolithic service, CloudWatch metrics and logs alone often already answer 'where is the time going' well enough that X-Ray's incremental value is smaller.
Visual explanation
Picture a relay race with a shared stopwatch (the trace ID) handed baton-to-baton between runners (services). Each runner's individual lap time is a segment. X-Ray's service map lays out the whole race as a diagram with each runner's average lap time labeled, instantly showing which runner is the slow leg of the race — rather than you having to time each runner separately and manually compare notes afterward.
Advantages
- —
Automatically stitches together a full request's journey across every instrumented service using one shared trace ID, no manual log correlation needed
- —
The service map visualization immediately highlights which node in a chain has the highest latency or error rate
- —
Lambda's 'Active Tracing' requires zero code changes to get basic tracing — a checkbox in the function configuration
- —
AWS Health Dashboard notifications can be routed to EventBridge/SNS for automated alerting on account-specific operational issues, not just manually checked
Disadvantages
- —
Requires instrumenting every service in a chain — partial instrumentation leaves gaps in the trace that can be as confusing as having no tracing at all for that segment
- —
Adds a small amount of overhead per request and a real (if usually modest) cost per trace recorded, especially at very high request volumes without sampling configured
- —
Service maps can get visually cluttered in architectures with many services, requiring filtering/searching to find the trace you actually care about
- —
AWS Health Dashboard events are AWS-generated and can't be customized or triggered manually for testing your own alerting pipeline
Common mistakes
- —
Instrumenting only some services in a chain and being confused by gaps in the resulting trace, rather than recognizing it as a coverage gap
- —
Never sampling trace collection at high request volumes, driving up X-Ray cost unnecessarily when a representative sample would answer the same latency questions
- —
Only checking the public Service Health Dashboard during an incident and missing account-specific detail that was actually available in the AWS Health Dashboard the whole time
- —
Treating X-Ray as a replacement for CloudWatch logs/metrics rather than a complementary third view — traces don't replace the need for detailed application logs at each node
In the AWS Console
- 1
Lambda → [function] → Configuration → Monitoring and operations tools → Active tracing
Enable Active Tracing on a Lambda function.
This alone, with no SDK code changes, gets basic per-invocation tracing for a Lambda function.
- 2
X-Ray → Service map, then Traces → [select a trace]
View the service map and drill into a specific slow trace.
Sort traces by 'Response time' descending to jump straight to your slowest recent requests.
- 3
AWS Health Dashboard (top-right Support menu → AWS Health) → Your account health
Check for any account-specific operational issues or scheduled changes.
🎤 Interview questions
What problem does X-Ray solve that CloudWatch metrics alone can't? (Listen for: CloudWatch tells you THAT a service is slow (e.g., high p99 latency) but not WHERE in a multi-service request chain the time is being spent; X-Ray traces a single request end-to-end across every service it touches, showing exactly which downstream call — a specific DynamoDB query, a specific downstream microservice — accounts for the latency)
What is a trace, and what is a segment? (Listen for: a trace represents one end-to-end request's journey through your application; a segment represents the work done by one service/component for that request, with subsegments representing finer-grained work within a segment like an individual SQL query or HTTP call — segments from every service a request touches are stitched together into one trace using a shared trace ID)
How does a request's trace ID get propagated across service boundaries? (Listen for: via an HTTP header (X-Amzn-Trace-Id) that the X-Ray SDK automatically injects into outgoing calls and reads from incoming calls, so every service in the chain contributes its segment to the same trace as long as it's instrumented with the X-Ray SDK/daemon)
What's the difference between the AWS Health Dashboard and the AWS Service Health Dashboard? (Listen for: the Service Health Dashboard is public and shows AWS-wide service status/outages visible to everyone; the AWS Health Dashboard (Personal Health Dashboard) is account-specific and shows issues and scheduled changes that specifically affect YOUR resources — e.g., an EC2 instance scheduled for retirement, or an API throttling issue tied to your account — which the public dashboard would never mention)
How would X-Ray help debug a microservices request that's slow, when all individual services report healthy CPU/memory in CloudWatch? (Listen for: the bottleneck might not be compute-bound at all — X-Ray's trace would show, for example, that one specific downstream call (an external API, a slow DB query, an underprovisioned Lambda) is consuming most of the end-to-end latency even while every service's own resource metrics look fine)