Storage Gateway, DataSync & Snow Family
Moving data between on-premises and AWS — online for ongoing sync, or offline via physical devices when the network itself is the bottleneck.
Want a visual for this topic?
Generate a diagram tailored to Storage Gateway, DataSync & Snow Family — the AI picks whichever visual (architecture, flowchart, ER diagram, etc.) best fits this specific AWS concept.
Sign in to generate a visual →🎓 Learning objectives
- •Explain when to use DataSync versus Storage Gateway versus the Snow Family
- •Explain the three Storage Gateway types and what each bridges
- •Calculate roughly when a network transfer becomes slower than physically shipping a Snow device
What is it?
This trio of services moves data between on-premises environments and AWS. AWS DataSync automates and accelerates online data transfer and synchronization. AWS Storage Gateway bridges on-premises applications to AWS storage as if it were local, with several gateway types for different protocols. The Snow Family (Snowcone, Snowball, Snowmobile) provides physical devices for offline, massive-scale data transfer when network transfer would be impractically slow.
Why it exists
Moving large amounts of data into AWS over the internet or even Direct Connect has a hard physical limit — bandwidth. A petabyte of data over even a fast dedicated connection can take weeks; for genuinely enormous datasets, that math simply doesn't work regardless of connection quality. These services exist to cover the full spectrum: DataSync for efficient, automated ongoing online transfer; Storage Gateway for making AWS storage feel local to existing on-premises applications; and Snow Family devices for when physically shipping hardware is faster than any network transfer could be.
Problem it solves
DataSync solves the manual-scripting problem for online transfers (handles retries, integrity verification, and incremental sync automatically, much faster than naive copy tools). Storage Gateway solves the application-compatibility problem (legacy on-premises applications keep using familiar file/tape/block protocols while data actually lives durably in AWS). The Snow Family solves the bandwidth-physics problem (some transfers are only practical by physically shipping storage hardware).
Intuition
DataSync is like a smart moving company that efficiently packs and transports your boxes over the highway, verifying nothing is lost or damaged along the way, and can keep making repeat trips to keep both locations in sync. Storage Gateway is like a magic doorway installed in your office that looks and acts like a local storage closet but secretly connects through to a much larger, off-site warehouse. Snowball is literally what it sounds like when the highway itself is too slow: you ship a shipping container instead.
Analogy
If you needed to send 100TB of data to a friend, DataSync is like a very good internet connection with a program that manages the transfer reliably. Storage Gateway is like giving your friend a special drive that looks local but is actually a live window into your storage. And if the internet connection would take three months, you'd just put a hard drive in a box and mail it — that's the Snow Family, at a scale from a small tabletop device (Snowcone) up to a shipping container on a truck (Snowmobile).
Technical explanation
AWS DataSync deploys a lightweight agent (a VM) in your on-premises environment that reads from a source (NFS/SMB file share, on-premises object storage) and writes to a destination (S3, EFS, FSx), handling parallelized transfer, automatic retry, and data integrity verification, and can run on a schedule for ongoing incremental synchronization. AWS Storage Gateway offers three types: File Gateway presents an NFS/SMB file share backed by S3, Volume Gateway presents iSCSI block storage backed by EBS snapshots (in Cached mode, only recently accessed data stays local; in Stored mode, all data stays local with async backup to AWS), and Tape Gateway presents a virtual tape library interface for backup software expecting physical tape, backed by S3/Glacier. Snowcone is a small, portable device (a few TB) suited for edge locations with limited space or power; Snowball Edge (tens of TB, with optional onboard compute) suits larger one-time migrations or edge computing; Snowmobile is a literal shipping-container-sized truck for exabyte-scale migrations.
Architecture
A media company migrating a 2 petabyte on-premises archive to S3 would order multiple Snowball Edge devices, copy data onto them on-premises, ship them to AWS for direct ingestion into S3 (far faster than any feasible network link), then switch to DataSync for ongoing incremental sync of new files being created after the initial bulk migration completes. A separate legacy backup system that only knows how to write to physical tape would instead use Tape Gateway to redirect those tape-writing backup jobs into S3/Glacier-backed virtual tapes with zero changes to the backup software itself.
Workflow
- Estimate data volume and available network bandwidth to decide whether online transfer (DataSync) or offline shipping (Snow Family) is faster. 2) For ongoing, automated online transfer or sync, deploy a DataSync agent and configure source/destination and schedule. 3) For legacy on-premises applications needing to keep using file, block, or tape protocols while data lives in AWS, choose the matching Storage Gateway type. 4) For one-time massive migrations where bandwidth math doesn't work, order the appropriately-sized Snow Family device.
Example
A hospital system migrating decades of medical imaging archives (1.5 petabytes) from on-premises storage orders several Snowball Edge devices, since even a 1 Gbps dedicated connection would take over 100 days to transfer that volume online, while shipping and re-ingesting the devices takes about 2 weeks total. Their ongoing daily imaging output after the migration (a few hundred GB per day) is handled by a scheduled DataSync task instead, since that volume comfortably fits their available bandwidth.
Real-world usage
AWS's own migration case studies repeatedly cite the Snow Family for petabyte-and-above one-time data center exits and media/entertainment archive migrations; Storage Gateway's Tape Gateway is commonly used by enterprises with existing tape-based backup software (Veeam, NetBackup, and similar) to redirect backups into S3/Glacier without replacing that software; DataSync is the standard recommendation for ongoing file share synchronization to EFS or S3 in hybrid architectures.
Trade-offs
The core tradeoff across all three is network dependency versus logistics overhead: DataSync and Storage Gateway depend entirely on available bandwidth but require no physical logistics; Snow Family devices remove the bandwidth constraint entirely but introduce real shipping time and physical handling overhead. The right choice is a straightforward function of data volume divided by available bandwidth compared against Snow device shipping turnaround time — there's a genuine crossover point, not a matter of preference.
Visual explanation
Picture a bandwidth-versus-data-size chart: for small to moderate data volumes, DataSync over your existing network connection is fastest and simplest. As data volume grows into the tens or hundreds of terabytes (or petabytes), a crossover point is reached where physically shipping a Snow device and letting AWS ingest it directly from the device becomes faster than any network transfer — that crossover point is exactly what determines the Snow Family's use case.
Advantages
- —
DataSync automates integrity verification and retry logic that a naive rsync/copy script wouldn't handle as reliably, often at significantly higher effective transfer speed
- —
Storage Gateway lets legacy applications keep their existing protocol expectations while data durability and scale genuinely live in AWS
- —
Snow Family devices make otherwise physically-impractical data volumes transferable within a reasonable timeframe
- —
All three integrate with standard AWS security (encryption in transit and at rest, IAM-based access control)
Disadvantages
- —
DataSync still fundamentally depends on available network bandwidth — it doesn't help if the bandwidth itself is the hard constraint
- —
Storage Gateway's Cached Volume mode means local read latency for non-cached (cold) data depends on network conditions back to AWS, unlike a fully local disk
- —
Snow Family devices have real logistics overhead (ordering, physical security during transit, shipping time) unsuited to anything needing faster-than-days turnaround
- —
Choosing the wrong tool (e.g. attempting a petabyte-scale transfer purely over DataSync on a modest connection) can make a migration timeline unrealistic
Common mistakes
- —
Attempting a very large one-time migration purely over the network without calculating whether available bandwidth actually supports completing it in a reasonable timeframe
- —
Choosing Storage Gateway's Cached mode for a workload that actually needs consistently low latency on cold (non-cached) reads, not realizing cache misses depend on network round-trip to AWS
- —
Not accounting for Snow Family device ordering and shipping lead time in a migration project timeline, causing schedule surprises
- —
Using a full DataSync/network migration approach when a hybrid strategy (Snow device for the bulk historical data, DataSync for ongoing new data) would have been faster overall
In the AWS Console
- 1
AWS Console → DataSync → Agents → Create agent
Download and deploy the DataSync agent VM image in your on-premises hypervisor, then activate it by entering the activation key shown after the VM boots.
The agent needs outbound network access to AWS DataSync service endpoints — check firewall rules before deployment if activation fails.
- 2
DataSync → Tasks → Create task
Select your configured source location (e.g. an NFS share via the agent) and destination location (e.g. an S3 bucket), then configure a schedule for ongoing sync or run it once for a one-time transfer.
DataSync only transfers changed/new data on subsequent runs by default, making it efficient for ongoing incremental synchronization, not just one-time copies.
- 3
AWS Console → Storage Gateway → Create gateway
Choose the gateway type (File, Volume, or Tape) matching your on-premises application's protocol expectation, deploy the gateway VM on-premises, and activate it.
Each gateway type has a genuinely different use case — File Gateway for NFS/SMB shares, Volume Gateway for iSCSI block storage, Tape Gateway specifically for backup software expecting a virtual tape library.
- 4
AWS Console → AWS Snow Family → Create job
Select the device type and size based on your data volume, specify the shipping address, and once the device arrives, connect it on-premises to load data before shipping it back to AWS.
Snow devices encrypt data at rest using keys you control, and AWS validates the device tamper-evident seal on return — check this if handling sensitive data.
🎤 Interview questions
How would you decide between DataSync and a Snow Family device for a large data migration? (Listen for: calculate data volume divided by available network bandwidth, compare against Snow device shipping turnaround time — the faster option wins, it's a math problem, not a preference.)
What's the difference between Storage Gateway's Cached and Stored volume modes? (Listen for: Cached keeps only recently-accessed data locally with the rest in S3-backed EBS snapshots; Stored keeps all data locally with asynchronous backup to AWS.)
Why would a company with existing tape backup software use Tape Gateway instead of rewriting their backup process? (Listen for: Tape Gateway presents a virtual tape library interface, letting existing backup software keep working unchanged while data actually lands in S3/Glacier.)
What's the actual bottleneck that makes Snowball devices sometimes faster than internet transfer? (Listen for: available network bandwidth is a hard physical limit; for large enough data volumes, no realistic connection speed beats physically shipping the data.)
Does DataSync only support one-time transfers? (Listen for: no — it supports scheduled, incremental synchronization, transferring only changed/new data on subsequent runs, well-suited to ongoing hybrid sync, not just migrations.)