Amazon Simple Storage Service (referred to as AWS S3) is AWS’s cloud-based file storage platform. Amazon S3 is one of Amazon’s most widely used services, but it’s also one of the easiest to overspend on. If you’re not familiar with AWS S3’s pricing model or taking advantage of lifecycle policies, you’re likely paying more for storage than you need.
In this AWS S3 Pricing and Cost Optimization Guide, you’ll learn the following:
- Amazon S3 Pricing and Cost Factors
- Storage
- S3 Requests (PUT, COPY, POST, LIST, GET, SELECT, Lifecycle Transition, and Data Retrievals)
- S3 Data Transfer
- S3 Management and Analytics
- S3 Replication
- S3 Object Lambda
- S3 Storage Pricing and Classes
- S3 Standard
- S3 Intelligent-Tiering
- S3 Standard-IA
- S3 One Zone-IA
- S3 Glacier Instant Retrieval
- S3 Glacier Flexible Retrieval
- S3 Glacier Deep Archive
- S3 Outposts
- Understanding Your S3 Access Patterns
- Creating an S3 Lifecycle Rule
- AWS S3 Cost Optimization Tips
Looking For Other AWS Cost Guides?
- Data Transfer Costs and Optimization Guide
- CloudFront Pricing and Cost Optimization Guide
- DynamoDB Pricing and Cost Optimization Guide
- ECS Pricing and Container Optimization Guide
- EMR Cost Optimization Guide
- NAT Gateway Pricing and Cost Reduction Guide
- RDS Pricing and Cost Optimization Guide
- Tagging Best Practices and Strategies
Amazon S3 Pricing and Cost
Before you start optimizing your S3 buckets, you need to understand Amazon S3’s pricing model. At a high level, you are charged based on six factors:
Storage:
- The total size of all files in AWS S3.
S3 Requests (PUT, COPY, POST, LIST, GET, SELECT, Lifecycle Transition, and Data Retrievals)
- Every API call to save or retrieve a file from S3 incurs a cost.
- PUT, COPY, or POST requests are most expensive for the S3 Glacier Category (up to $0.05 per 1000 requests) and cheapest for the Standard Category ($0.005 per 1000 requests).
- S3 Infrequent Access category ($0.001 per 1000 requests) Glacier Instant ($0.01 per 1000 requests)is the most expensive option for data retrieval.
S3 Data Transfer
- Data transfer IN to the AWS S3 service is free.
- Data transfer OUT of the AWS S3 servers to the Internet is charged at $0.09 per GB (first 10TB per month) and $0.05 per GB after 150TB of transfer.
- Transfer to other AWS Regions and Multi-Region Access Points is typically charged $0.01-$0.02 per GB
S3 Management and Analytics
- Storing data records: $0.0025 per million objects listed
- Monitoring records: $0.10 per million objects monitored per month; Tags: $0.01 per 10,000 tags per month
- Advanced Metrics: $0.20 per million objects monitored per month
- Batch Operations: $0.25 per job
S3 Replication
- In addition to the storage and transfer fees for replication, you may also need to pay for S3 Replication Time Control.
- Incurs pricing for S3 Storage and data management: $0.015 per GB for time-control data transfer.
S3 Object Lambda
- Customizing data access with AWS Lambda functions incurs additional charges: $0.005 per GB of data returned.
Amazon S3 Storage Pricing and Classes
The amount you will pay for each of these factors depends on your S3 storage class. Using the correct S3 storage class is the easiest way to help control AWS S3 cost, so it’s important to start here. You can set the S3 storage class at the S3 bucket or individual object level.
Amazon S3 Standard Storage
S3 Standard Storage is the default and general-purpose storage class designed for a wide range of cloud storage use cases. Rapid retrieval, low latency access, and high throughput features make the Standard class suitable for the storage of frequently accessed data. S3 Standard data items can be configured with objects from other S3 storage tiers such as intelligent-Tiering and Infrequency-Access, allowing for additional flexibility on your storage use cases.
Key Features:
- Low latency and high throughput.
- High data durability of 11 9’s (99.999999999%).
- Designed Availability: 99.99%; True SLA Availability: 99.9%.
- SSL and data encryption.
- Automated migration capability between other S3 classes.
Monthly Price: Starting at $0.021-$0.023 per GB, US Regions.
Use Case: Data lakes, log storage, configuration files, cloud-native applications, website, and content distribution.
Consideration: Suitable for frequent data access use cases that require an optimal tradeoff between cost and performance.
Amazon S3 Intelligent-Tiering
The S3 Intelligent-Tiering class introduces a cost-saving mechanism by moving data workloads between alternative S3 Storage tiers based on parameters such as data access frequency and usage patterns. Moving infrequently accessed data assets to the low-cost Infrequent Access tier optimizes the overall storage cost while maintaining high throughput and low latency as offered in the S3 Standard class. An optional Archive Access tier is also available for rarely accessed data workloads.
Key Features:
- Automatic tiering between multiple storage options: Frequent, Infrequent, and Archive Instant Access.
- Opt-in activation of automated archiving for asynchronously accessible data.
- Higher Cost Savings: Up to 40% for Infrequent Access; 68% for Archive Instant Access.
- Milliseconds in latency, slightly higher than the Standard S3 class.
- Similar Security and Durability as S3 Standard class.
- High data durability of 11 9’s (99.999999999%).
- SSL and eata encryption
- Designed Availability: 99.9%; True SLA Availability: 99%.
Monthly Price: Starting at $0.021-$0.023 per GB for Frequent Access tier, $0.0125 per GB for Infrequent Access tier and $0.004 per GB for Archive Instant Access tier, US Regions.
Use Case: Disaster recovery and backup data workloads are suitable applications for the Infrequent usage tier, while other data assets with unknown or changing access patterns can take advantage of the automated tiering mechanism.
Considerations: Suitable for diverse data use cases access policies. Access and retrieval incurs additional charges. Accessing data from Archive Access takes several minutes or hours to access the first byte.
Amazon S3 Infrequent Access Storage
The S3 Standard – Infrequent Access Storage tier is designed specifically for data workloads that are rarely accessed but require fast retrieval when necessary. The S3 storage class offers a cost tradeoff suitable for long-term workloads such as disaster recovery data: the cost of storage is low but data access cost higher than comparable storage classes. Users can apply S3 Lifecycle policies to automatically configure objects in this class in the same bucket of objects from Intelligent Tiering and Standard classes.
Key Features:
- High data durability of 11 9’s (99.999999999%).
- SSL and eata encryption.
- S3 Lifecycle management for automated migration between S3 classes.
- Designed Availability: 99.9%; True SLA Availability: 99%.
Monthly Price: $0.0125 per GB, US region. Minimum 30 days charge applied, requires objects of 128KB or more.
Use Cases: Disaster recovery and backup of non-mission-critical business information.
Consideration: The storage tier is quite similar to the S3 class in terms of performance, except for millisecond retrieval delays.
Amazon S3 One Zone – Infrequent Access Storage
The S3 Storage One Zone Infrequent Access Storage is a low-cost service offering primarily focused on storing infrequently accessed data. While the standard S3 Infrequent Access stores data in three different regions to maximize data durability and availability, the S3 One Zone service tier stores all data into a single Availability Zone. The zone can be chosen for rapid access – lowest latency and throughput – but susceptible to service outages facing the chosen AWS Region.
Key Features:
- High data durability of 11 9’s (99.999999999%).
- Single Availability Zone.
- SSL and eata encryption
- S3 Lifecycle management for automated migration between S3 classes.
- Designed Availability: 99.5%; True SLA Availability: 99%.
Monthly Price: $0.01 per GB.
Use Cases: Disaster recovery and backup of data that may not be mission-critical but substantial in volume.
Consideration: The stored data is lost permanently in the event of a destructive event facing the selected One Zone region. Such an event is described as a disaster that destroys multiple AWS data centers in the region, causing irreversible damages to the stored data that’s not replicated in alternative AWS regions.
Amazon S3 Glacier Flexible Retrieval (Formerly S3 Glacier)
The S3 Glacier Flexible Retrieval class is a low-cost alternative to the standard Glacier Instant Retrieval tier: 10% cheaper while introducing three retrieval speed options: expedited (1-5 mins), standard (3-5 hours), and bulk (12 hours). The service tier makes large data assets accessible at no additional cost. The longer delays are typically applicable to accessing data in bulk, while smaller volumes remain accessible quickly.
Key Features:
- Latency and throughput performance similar to the Standard class.
- Three data retrieval options: expedited (minutes), standard (hours) and bulk (up to 12 hours).
- High data durability of 11 9’s (99.999999999%).
- SSL and eata encryption.
- S3 Lifecycle management for automated migration between S3 classes.
- Designed Availability: 99.99%; True SLA Availability: 99%.
Monthly Price: $0.0036 per GB
Use Cases: Record retention for compliance use cases, where the retrieval frequency may vary.
Consideration: Multiple retrieval speeds make this tier particularly suitable for archived long-term data that doesn’t require rapid retrieval and would incur significant bulk access fees as applicable to other tiers.
Amazon S3 Glacier Deep Archive
The S3 Glacier Deep Archive is the most economic solution for long-term storage of data that may be restored a few times per year. This class is designed to meet the needs of tightly regulated industry verticals such as financial, healthcare and public services sectors that must retain data items for the longer term (7-10 years, or more).
Key Features:
- Latency and throughput performance similar to the Standard class, introducing minute – 12 hours in data retrieval delay.
- High data durability of 11 9’s (99.999999999%).
- SSL and eata encryption.
- S3 Lifecycle management for automated migration between S3 classes.
- Designed Availability: 99.99%; True SLA Availability: 99.9%.
Monthly Price: $0.00099 per GB, US Regions.
Use Cases: Long-term data archives and record retention for compliance purposes.
Consideration: Consider the service tier as a cloud alternative to magnetic tapes and hard drives.
Amazon S3 on Outposts
S3 on Outposts is the managed storage service offering for on-premise storage use cases. The service tier runs native S3 Storage capabilities data stored on your local on-premise servers. S3 capabilities including security, control and data management are delivered to the AWS Outposts as an API connection. The service is designed to meet stringent compliance requirements of local data storage while maintaining the security, performance and dependability of the cloud-based S3 Storage service.
Key Features:
- S3 Storage Class compatible storage service and data management.
- SSE encryption.
- AWS IAM and S3 Access Point security controls.
- S3 Lifecycle management for automated migration between S3 classes.
Monthly Price: [variable]
Use Cases: Tightly regulated data assets that must be stored locally.
Consideration: Significant investments into data redundancy and disaster recovery required to match data durability offered in the cloud-based service tiers of AWS S3.
S3 Pricing by Region
Another way to influence pricing in Amazon S3 is to select the cheapest region that makes sense for your application. Some AWS regions are as much as 50 percent cheaper, but you have to be careful if you pick a region purely based on price.
First, latency between your S3 storage and other AWS resources could be a problem. If most of your infrastructure is hosted in South America, but your S3 files are in Virginia, it could add precious milliseconds to each API call.
Similarly, putting your S3 region far from your users might degrade performance if file access is a critical part of your application. That said, if you use S3 for longer-term storage and cache most of your files in a CDN like CloudFront, users might not notice the difference between S3 regions much.
Finally, if you’re a very light user of Amazon S3 or you’re just trying it out, it’s worth noting that AWS offers up to 5 GB of storage for twelve months as part of their Free tier. This might be helpful for early-stage startups or side projects looking for a free way to get started with S3 file storage.
Understanding Your S3 Access Patterns
It’s helpful to know about the different storage classes in S3, but in practice, most organizations start by putting everything in the Standard tier. Depending on your usage patterns, you’ll be able to optimize your S3 storage classes after AWS has a few weeks of data to analyze.
In this section, I’ll show you how to use the Amazon S3 storage class analysis to determine which Standard tier files can be moved to a less frequent access tier. This will help you decide whether it’s worth investing in a S3 lifecycle policy and how much you might save by using one. It could also be used to tag objects in S3 and move them to the right tier accordingly.
Running AWS S3 Storage Class Analysis
You can run storage class analysis on an entire bucket or a subset of your S3 files using prefixes or tags. To configure the analysis, open your S3 bucket and click the Metrics tab.
Scroll down and click Create analytics configuration. From here you can decide whether to analyze the whole bucket or just a subset of files. You can also decide whether to export the results or not.
Understanding the Results
AWS will start the storage class analysis, and depending on the size of your bucket, you might be able to see results fairly quickly. Here’s an example of what the results might look like:
The chart and summary boxes in the image show you the total size of the objects in Amazon S3 that you are storing versus the amount that’s been retrieved. These results show you that even many of the older files in this bucket are being retrieved regularly. While this bucket only contains 13.92 GB of files over a year old, 40.84 GB have been retrieved in the past seven days.
On the other hand, the analytics in the following image show a very different usage pattern:
In this bucket, none of the files over thirty days old have been accessed at all in the past seven days. This might indicate that you can move some of these older files to an Infrequent Access tier or even Glacier. You’ll have to be careful, though, as moving files in and out of these longer-term storage patterns incurs a cost.
Once the storage class analysis is complete, AWS will offer some recommendations as well. Here’s an example from another bucket:
This shows you that the rate of retrieval (the solid orange line) drops to almost zero after a file is thirty days old, and that there is a lot of storage being consumed by older, infrequently accessed files in this bucket.
Now that you know you can use a better Amazon S3 storage tier, what do you do with this information? That’s where lifecycle rules come in.
Creating an S3 Lifecycle Rule
By creating a lifecycle rule, you are instructing Amazon S3 to move files to another storage tier or delete files automatically based on their age. These rules can be applied to all files in your bucket or you can use tags or file prefixes to filter files first.
To get started, go to the Management tab in your S3 bucket and click Create lifecycle rule.
For the second example above where no files older than thirty days are being accessed, you could create a lifecycle rule that:
- Moves files to the Standard-IA tier after they’re forty-five days old (this gives you a little bit of margin for error, but you could probably use thirty days if you like)
- Moves files to Glacier after they’re 365 days old
- Expires files after they’re ten years old
In this case, your access rule setup would look like this in the Amazon S3 GUI:
These rules can also be created in XML and applied using the S3 API if you would like to automate this process to apply policies to many buckets at once. Be careful when setting up these policies, though. Moving files in and out of Glacier frequently will get very expensive, so a small mistake in your lifecycle policy could add thousands to your AWS bill. One way to avoid this is to sign up for CloudForecast’s daily AWS spending alerts to help monitor cost. In addition, CloudForecast’s ZeroWaste Health Check report helps you quickly identity all your S3 buckets that do not have a lifecycle policy attached.
Once you’ve set up the appropriate policies, your S3 files will be automatically moved to the desired storage tier as they age. This is one of the best ways to lower your Amazon S3 bill, but it’s not the only strategy you can take.
AWS S3 Cost Optimization Tips
Depending on the size of your organization and the way you’ve stored data in Amazon S3, access patterns and lifecycle policies might only get you so far. If you’re taking a deep look at optimizing your S3 cost, here are a few more things you can explore.
Prioritize the Biggest Wins First
If your organization manages a lot of data in Amazon S3, going through the storage class analysis, checking with the appropriate teams, and implementing lifecycle policies for every set of data can be overwhelming.
Start with the areas where you can get the biggest wins first and invest time in them accordingly. If you’re only storing a few gigabytes in a bucket, it might not be worth setting up a lifecycle policy at all.
S3 Bucket Organization
Because lifecycle policies can use file prefixes or tags as a filter, it’s vital that you implement an organization system across your S3 buckets. Common tagging strategies might include:
- Prefixing file names with an expiration date
- Tagging objects with the responsible team’s name
- Adding user ID’s to the front of file names so you can archive old user data
- Tagging data with each customer’s company name
Having a consistent system for organizing objects in Amazon S3 will ensure that you can write effective lifecycle policies, and it will prevent you from deleting important files.
S3 Usage Reports
If you want a granular look at how each file in your S3 bucket has been created, modified, or accessed, you can download an AWS usage report.
S3 Bulk Retrieval and Storage
If you decide to move data into or out of Glacier, you might want to consider zipping your files first. This will help you save money on your data transfer costs and make retrieving files that are likely to be needed at the same time easier. The time to compress your data is negligible compared to the five-minute access time Glacier imposes.
Partial File Uploads
If you use Amazon S3 for multipart file uploads, be sure to remove unfinished parts from your buckets. Amazon provides a lifecycle policy for this in their documentation, and it’s an easy way to make sure you’re not paying for files you don’t need.
Don’t Forget Your Logs
If you’re storing CloudWatch logs in Amazon S3, one of the easiest ways to decrease your S3 cost is to expire (ie, delete) old log files automatically. You can do this manually or using a lifecycle policy (as mentioned above), but be careful. Many regulations require organizations to retain logs for months or years.
Conclusion
Amazon S3 is one of the most widely used AWS products, and because it has such a wide array of use cases, it can be especially hard to understand its pricing. After you get a handle on the different storage tiers available, you can start to dive into your S3 buckets.
Run the storage class analysis tool to understand your usage, and create a lifecycle policy that properly transfers your S3 files to the appropriate storage class. Keep prioritization and organization in mind, though, because S3 pricing optimization is an ongoing process.
If you’d like help understanding and keeping up with your AWS spending, CloudForecast can help. Reach out to our CTO, [emailprotected] to help reduce your cost or schedule a time with us.