AWS Systems Manager
Operational tooling for managing fleets of EC2 instances and on-premises servers — patching, secure shell-free access, and centralized configuration storage.
Want a visual for this topic?
Generate a diagram tailored to AWS Systems Manager — the AI picks whichever visual (architecture, flowchart, ER diagram, etc.) best fits this specific AWS concept.
Sign in to generate a visual →🎓 Learning objectives
- •Explain what Session Manager solves and why it's preferred over SSH/RDP
- •Explain Parameter Store's role for centralized configuration and secrets
- •Explain what Patch Manager automates
- •Explain the role of the SSM Agent in enabling these capabilities
What is it?
AWS Systems Manager is a collection of operational management capabilities for EC2 instances (and on-premises/other-cloud servers registered as 'managed instances') — including Session Manager (secure shell access without SSH keys or open ports), Parameter Store (centralized, optionally encrypted configuration and secrets storage), Patch Manager (automated OS/software patching), and Run Command (executing commands across many instances without individually logging into each one).
Why it exists
Managing a fleet of servers traditionally requires SSH keys distributed and rotated carefully, open inbound ports for SSH/RDP access (a real attack surface), manual or custom-scripted patching processes, and configuration values scattered across config files or hardcoded in application code. Systems Manager exists to replace these ad-hoc, individually-risky practices with centralized, auditable, AWS-managed tooling.
Problem it solves
Session Manager solves the SSH-key-and-open-port problem (secure shell access with no inbound port needing to be open at all, and every session logged for audit). Parameter Store solves the scattered-configuration problem (one centralized, access-controlled, optionally-encrypted place for configuration values and secrets, referenced by ID rather than hardcoded). Patch Manager solves the manual-patching problem (scheduled, automated OS patching across a fleet, with compliance reporting). Run Command solves the individual-server-access problem (execute a command across many instances simultaneously without SSHing into each one).
Intuition
Session Manager is like replacing individually-cut physical keys to every building (SSH key pairs, open ports) with a single, centrally-managed keycard system that logs every entry and can be revoked instantly for any specific person, with no physical door needing to stay unlocked for the system to work. Parameter Store is like a company's central, access-controlled filing cabinet holding shared reference information instead of every employee keeping their own scattered sticky notes.
Analogy
Session Manager is like a modern building's badge-access system instead of physical keys under the doormat (SSH keys) and propped-open doors (open SSH ports) — access is centrally granted, logged, and revocable instantly, with no door that has to stay physically accessible to work. Parameter Store is a shared, secured company reference binder instead of everyone keeping their own individual copies of important numbers scattered across sticky notes and personal notebooks (hardcoded config values).
Technical explanation
Session Manager establishes its connection using the SSM Agent's outbound connection to the Systems Manager service, meaning no inbound Security Group rule for SSH/RDP is required at all — a meaningfully smaller attack surface than traditional SSH access, and it works even for instances in fully private subnets with no direct network path from the user's location. IAM policies control exactly who can start a session and on which instances, and session activity (including a full transcript of commands and output) can be logged to S3 or CloudWatch Logs for later audit. Parameter Store supports three parameter types: String and StringList for plain configuration values, and SecureString for values encrypted via KMS (like database passwords or API keys) — applications retrieve parameters via the SDK/CLI/instance metadata at runtime rather than having secrets embedded in code or unencrypted config files. Patch Manager uses Patch Baselines (defining which patches are approved, and how quickly critical vs non-critical patches should be auto-approved) applied to instances via Maintenance Windows (scheduled time windows for patching operations to run, avoiding disruption during business hours).
Architecture
A production environment removes SSH/RDP access entirely from all Security Groups (no inbound rule for those ports at all, on any instance), relying exclusively on Session Manager for any needed shell access, with every session logged to a centralized S3 bucket for security audit. Application configuration and secrets (database connection strings, API keys) are stored in Parameter Store as SecureString values, retrieved by the application at startup via IAM role permissions rather than being present in any deployed code or config file. A weekly Maintenance Window runs Patch Manager against all instances, auto-approving critical security patches after a 3-day delay (allowing time to catch any widely-reported patch issues) while non-critical patches follow a slower, more conservative approval cycle.
Workflow
- Confirm the SSM Agent is installed and running on target instances (pre-installed on most standard AWS AMIs) and that the instance's IAM role includes the necessary Systems Manager permissions. 2) Remove SSH/RDP inbound Security Group rules where Session Manager access is sufficient, reducing attack surface. 3) Migrate hardcoded or scattered configuration values and secrets into Parameter Store, using SecureString for anything sensitive. 4) Define Patch Baselines matching your organization's patching policy (how quickly to apply critical vs non-critical patches) and schedule them via Maintenance Windows. 5) Use Run Command for one-off or scheduled operational tasks across a fleet instead of manually accessing each instance individually.
Example
A financial services company's compliance requirements prohibit any direct SSH access to production servers. They rely entirely on Session Manager for operational access, with every session's full command transcript logged to a dedicated, access-restricted S3 bucket that the compliance team can audit. Their application retrieves its database credentials from Parameter Store as a SecureString value at startup (decrypted transparently via the instance's IAM role and KMS permissions), meaning no plaintext credential ever exists in their deployed code or configuration files, and rotating the credential requires updating only the Parameter Store value, not redeploying application code.
Real-world usage
Session Manager has become the standard AWS-recommended alternative to traditional SSH/RDP access, extensively documented in AWS's own security best practices specifically for eliminating the SSH-key-management and open-port attack surface; Parameter Store (and its more feature-rich sibling, AWS Secrets Manager, used when automatic secret rotation is specifically needed) is a common default for centralizing application configuration and secrets across AWS-hosted applications.
Trade-offs
Session Manager's elimination of open SSH/RDP ports is a clear security improvement, but requires the SSM Agent and appropriate network connectivity (direct internet access, NAT, or a VPC endpoint) to actually function — a fully air-gapped instance with no path to the Systems Manager service can't use it. Parameter Store's Standard tier is free and sufficient for most configuration needs, but AWS Secrets Manager (a related, separate service) costs more while adding automatic secret rotation — worth the extra cost specifically when a secret genuinely needs to rotate on a schedule (like a database password), not for general configuration values.
Visual explanation
Picture an EC2 instance running the SSM Agent (pre-installed on most AWS-provided AMIs), which maintains an outbound-only connection to the Systems Manager service — no inbound port needs to be open at all. A user wanting shell access goes through Session Manager, authenticated and authorized via IAM, with the entire session optionally logged to S3/CloudWatch Logs for audit. Separately, Parameter Store holds configuration values (some as SecureString, encrypted via KMS) that applications and instances reference by name at runtime, rather than values being hardcoded or scattered across config files.
Advantages
- —
Session Manager eliminates the need for SSH key management and open inbound ports entirely, meaningfully reducing attack surface
- —
Every Session Manager session can be fully logged for audit, unlike traditional SSH access which typically isn't recorded at the command level by default
- —
Parameter Store centralizes configuration and secrets with fine-grained IAM access control and optional KMS encryption, removing hardcoded values from code
- —
Patch Manager automates a genuinely tedious, error-prone manual process (fleet-wide patching) with compliance reporting built in
Disadvantages
- —
Requires the SSM Agent to be installed, running, and able to reach the Systems Manager service endpoint (directly or via a VPC endpoint for fully private subnets) — a misconfigured or missing agent breaks all these capabilities
- —
The IAM permission model for Systems Manager capabilities requires careful configuration to avoid being either too permissive (broad access to manage many instances) or too restrictive (blocking legitimate operational needs)
- —
Parameter Store's free tier has practical limits on parameter count and retrieval throughput that can require moving to AWS Secrets Manager or Parameter Store's paid Advanced tier at larger scale
- —
Teams accustomed to traditional SSH workflows have a real transition and tooling adjustment when moving to Session Manager-only access
Common mistakes
- —
Leaving SSH/RDP Security Group rules open 'just in case' after adopting Session Manager, failing to actually realize the attack-surface reduction that removing them would provide
- —
Storing sensitive secrets as a plain String parameter instead of SecureString, leaving them unencrypted at rest and readable by anyone with basic parameter-read IAM permissions rather than requiring KMS decrypt permission too
- —
Not verifying the SSM Agent is actually running and up to date on instances, leading to instances that unexpectedly can't be managed via Session Manager or Patch Manager when needed
- —
Applying overly broad IAM permissions for Systems Manager actions (e.g. allowing Session Manager access to every instance in the account) instead of scoping access to specific instances or tag-based conditions matching actual operational need
- —
Not testing Patch Manager's patch baseline and maintenance window configuration in a non-production environment before applying it broadly, risking an unexpected patch-induced issue across the whole fleet simultaneously
In the AWS Console
- 1
AWS Console → Systems Manager → Fleet Manager
Confirm your instances appear as 'Managed' — this requires the SSM Agent running and the instance's IAM role including the AmazonSSMManagedInstanceCore policy.
An instance not appearing here means Session Manager, Run Command, and Patch Manager won't work on it until the agent/IAM/networking prerequisites are resolved.
- 2
Systems Manager → Session Manager → Start session
Select a managed instance and start a session directly in the browser — no SSH key or open port needed.
Configure session logging (to S3 and/or CloudWatch Logs) under Session Manager's preferences for an audit trail of session activity.
- 3
Systems Manager → Parameter Store → Create parameter
Create a parameter, choosing 'SecureString' type for any sensitive value, and select the KMS key to encrypt it with.
Applications retrieving a SecureString parameter need both Parameter Store read permission AND kms:Decrypt permission on the specific KMS key used.
- 4
Systems Manager → Patch Manager → Patch baselines / Maintenance Windows
Define or select a patch baseline specifying auto-approval rules for critical vs non-critical patches, and schedule a Maintenance Window for when patching operations should run.
Test a new patch baseline against a small, non-production instance group first before applying it broadly across production.
🎤 Interview questions
Why is Session Manager considered more secure than traditional SSH access? (Listen for: no inbound port needs to be open at all — uses the SSM Agent's outbound connection; access controlled and logged via IAM, works even for fully private-subnet instances.)
What's the purpose of Parameter Store, and when would you use SecureString versus String? (Listen for: centralized, access-controlled configuration/secrets storage; SecureString for anything sensitive (KMS-encrypted), String for non-sensitive plain configuration values.)
How does Patch Manager help with fleet-wide OS patching? (Listen for: automates patch approval (via baselines) and scheduled application (via maintenance windows) across many instances, with compliance reporting, replacing manual or custom-scripted patching.)
What has to be true for an EC2 instance to be manageable via Systems Manager? (Listen for: the SSM Agent must be installed and running, the instance's IAM role must include the necessary Systems Manager permissions, and it needs network connectivity to the Systems Manager service endpoint.)
How would you eliminate SSH access entirely from a production environment while still allowing operational shell access when needed? (Listen for: remove SSH/RDP inbound Security Group rules entirely, rely on Session Manager instead, which requires no open inbound port.)