AWS Knowledge
Understanding AWS DataSync Pricing and Costs
Stuart Lundberg
Dec 6, 2024
Transferring large datasets quickly, efficiently, and securely remains one of the most significant pain points among data administrators and the broader AWS ecosystem. Thankfully, such a transfer is made significantly more straightforward, quicker, and cheaper with DataSync, slashing manual AWS data transfer costs by up to 70%. However, knowing the prices is critical to regulating expenses and tweaking your operations.
In this blog, we’ll break down the ins and outs of pricing for AWS DataSync, provide tips on saving money, and explain how to optimize it for the best results. Whether it is data migration to Amazon S3, file synchronization on on-premises storage devices, or AWS data transfer scaling - this article has you covered with everything AWS DataSync related you require while simultaneously being cost-effective!!
What is AWS DataSync?
AWS DataSync is a completely run server and a service that automatically decentralizes data transfer from and to on-premises devices and the Amazon S3 cloud altogether. Migrating large amounts of Data is always complex and time-consuming, but transferring services helps streamline and speed up this process.
Think of it like a helpful wizard moving precious items from an unseen vault to a tucked-away cloud storage. Transfers are fast, secure, and robust. Each transfer is effortless, regardless of how big or small it is. Manually moving files is a thing of the past, according to AWS DataSync.
Key Features of AWS DataSync:
Automated Data Transfer: Eliminate the need for manual processes by automating data movement tasks.
Accelerated Migration: Transfers data up to 10 times faster than traditional methods using optimized network protocols.
Flexible Task Scheduling: Schedule transfers periodically or prioritize them with task configurations.
Secure Transfers: Employ TLS encryption during transit and validate data integrity.
Seamless Integration: Supports direct transfers to major AWS storage services, such as Amazon S3, S3 Glacier, and Amazon EC2.
Common Use Cases:
Moving large volumes of datasets into the cloud storage.
Backing up on-premises data to AWS for disaster recovery.
Data Synchronization between hybrid cloud setups
Replicating datasets for global scaling and analytics.
How Does AWS DataSync Work?
(Image Source: AWS DataSync)
AWS DataSync can easily transfer data between on-premises storage, AWS storage services, or other cloud providers. Here’s how it works:
Source Location: This is where your data is located at this point. It can consist of an on-premises storage facility, AWS services such as Amazon S3, EFS, and many more, or, yet again, a different provider on the cloud.
Destination Location: This is where you would like to transfer your data. The most common include the S3 bucket and the Amazon EFS file systems.
DataSync Agent: This is a virtual machine hosted in on-premises. Its primary role is to link on-premises storage systems to AWS storage and ensure that the data transfer is fast, safe and efficient.
Moving data from a source to a destination is done by creating and managing missions through the AWS Management Console. Suppose you need insight, for example, about the transfers or even want to keep an eye on the performance of these transfers. In that case, you can easily use the Amazon CloudWatch metrics to provide the system with the data needed.
Deep Dive Into AWS DataSync Pricing
The AWS DataSync pricing model is based on a pay-as-you-go system. This means that is you only get charged for services that you use. Prices vary depending on factors such as:
The volume of data transferred.
The number of times a task is attempted.
The AWS storage service that a person utilizes.
Pricing Breakdown:
Per-Gigabyte Fee: AWS service pricing starts at $0.0125 per/GB for basic mode and $0.015 per/GB for enhanced mode for the first 10TB of data transferred monthly in US East (N. Virginia). Once additional amounts are transferred, the price is likely to shift, so expect an odd price per GB.
Task Execution Fee: There is a fee of $0.55 per task execution in Enhanced modes to boost performance on parallel data processing.
Storage Service Costs: Various costs are incurred when using the standard rates for other services from AWS, such as Amazon S3, EFS, and S3 Glacier.
Example Calculation (on-premises to Amazon S3 Standard storage):
Transferring 10 TB of data in Basic mode:
DataTransfer = 10 TB × 1024 × $0.0125 = $128
TaskExecution = 10 executions × $0.55 = $5.50
Total = $128 + $5.50 = $133.50
Enhanced vs. Basic Mode:
Enhanced Mode: is suited for more complex tasks which process higher charges while ensuring faster completion. For example, suppose you are running a data-intensive task that requires real-time analysis of a large dataset. In that case, the enhanced mode will ensure the task is completed within the specified time period.
Basic mode: is utilized when relatively simple tasks are performed that do not require a faster turnaround as they are low cost. For example, if one transfers and verifies files one by one to send archived files to a backup server, then the basic mode can be employed in this scenario as it would save costs.
Additional Charges:
CloudWatch Metrics: The charge for using Amazon CloudWatch to monitor DataSync is based on CloudWatch metrics. These metrics charge for their retrieval and storage.
PrivateLink Interface Endpoints: Additional charges for using PrivateLink to connect securely to DataSync via Amazon Virtual Private cloud endpoints are incurred.
Data Transfers Between Regions: The transfer of data while using DataSync services, such as copying or archiving, would incur some costs, especially when the data has to move from the original source region at an extra cost.
Cost Optimization Benefits
Understanding AWS DataSync pricing can unlock numerous benefits:
Predictable Budgeting: Pay-as-you-go pricing ensures no surprises in your monthly cloud costs.
Incremental Discounts: Users in the higher tier and above the 10 TB mark have the advantage of benefiting from a lower price per GB
Efficient AWS Integration: With the integration of DataSync and AWS Cost Explorer, there is better and easier collaboration for different cost tracking in a busy and convoluted environment.
Case Study: How London Stock Exchange Group Migrated 30 PB of Data and Cut Storage Costs by 80%
To better store & access data, the London Stock Exchange Group (LSEG) moved 30 PB of market data and 60 TB of additional data daily from Wasabi to Amazon Simple Storage Service. In partnership with AWS and DataArt, a semi-automated approach of using AWS DataSync for transferring and validating data was executed alongside Step Functions and Dynamo DB, all for data efficiency. New data was transferred to S3 Intelligent-Tiering, while older data was stored in Glacier Deep Archive.
Results:
30 PB migrated in under 3 months.
80% storage cost reduction.
Improved customer access via AWS Data Exchange.
Data availability increased and freed the personnel to perform other important activities.
Tools and Tips for Cutting AWS DataSync Costs
If you’re looking to rein in expenses, try these practical strategies:
Transfer Optimization Tips:
Batch Transfers: Group smaller data files together to minimize task executions.
Optimize Task Scheduling: Carry tasks in Scheduled times or periods when the network is less used.
Utilize Cost Management Tools:
AWS Pricing Calculator: Easily estimate the cost of upcoming DataSync tasks.
AWS Cost Explorer: Look into previous costs to discover avenues for enhancement.
DataSync Monitoring: Use Amazon CloudWatch to track tasks in real-time.
Configure Transfers Wisely:
Use Basic Mode for less frequent transfers or transfers with smaller amounts of data
Opt for compression techniques to reduce data volumes before transfer.
Use local AWS Regions to store data and avoid inter-regional data transfer costs.
Conclusion
With AWS DataSync in place, the movement of massive amounts of data becomes simple, cheap and secure, taking the guesswork out of scaling cloud resources. Learning about its pricing and cost optimization often allows for secure data management without breaking the bank.
Take a closer look at AWS DataSync to understand how you would transfer your data and get onto your free transfer, guaranteeing effortless, productive data movement.
Log onto the AWS DataSync console or access the AWS Pricing Calculator to begin the process.
Join Pump for Free
If you found this post interesting, consider checking out Pump, which can save you up to 60% off AWS for early-stage startups, and it’s completely free (yes, that's right!). Pump has tailor-made solutions to take you in control of your AWS and GCP spend in an effective way. So, are you ready to take charge of cloud expenses and maximize the most from your investment in AWS? Learn more here.
