The cloud services market is becoming more and more widespread, with 78% of organizations either using or planning to use the cloud. The market is projected to reach $206.2 billion in 2019. What is not apparent is that with this growth, unanticipated costs arise due to wasteful habits, poor business strategies, and hidden expenditures. Cutting cloud waste became an important matter.
Most organizations not only fail to take full advantage of cloud capabilities, but are unknowingly pouring company assets down the drain. Wasted cloud spend is astronomical—forecasted to exceed $14.1 billion in 2019 and to reach $21 billion a year by 2021. Clearly, it is dangerously easy to waste money in the cloud. Cloud pricing can be complex and deceptive, and expenses can add up in insidious ways. The bottom line is that resources that could go towards company growth are instead silently evaporating into the cloud. What is at the root of all this?
This blog post explores the most prevalent causes of funds wasted in the cloud, and delves into solutions that can immediately reduce monthly bills. Businesses that become aware of this will be able to both significantly improve their cloud-management practices and avoid superfluous expenses.
A Lack of Awareness
Wasting money in the cloud is first and foremost a conceptual problem. The truth of the matter is that many teams fail to realize just how much is wasted. In fact, according to a recent survey, 62% of cloud users report costs to be higher than expected. It’s easy to upload data to the cloud and assume that’s the last you’ll ever need to think about it. However, storing data in the cloud is not like throwing clothes in a closet—it calls for a vigilant, organized, and flexible mindset. Adopting such a mindset can be extremely valuable in more ways than one, primarily because thinking cost efficiently will often also lead to superior data practices.
Using the cloud intelligently and effectively boils down to one simple principle: pay for what you need, when you need it, the way you need it—and nothing more. Putting this into practice, however, requires a carefully thought-out strategy.
Let’s break this down into the most recurring problems.
So What are the Common Pitfalls to Watch out For?
Over provisioned Resources
Many companies are paying for more than they actually need. Over 50% of cloud spend is on instances, and 40% of these instances are actually one to two sizes bigger than required for workload. This is like filling an entire swimming pool just to take a bath. Whether due to passively selecting default settings or workloads that have changed over time, this is a significant source of waste. It is estimated that $5.3 billion are wasted annually just on oversized resources.
Another form of over-provisioning is through the selection of expensive instance types. Different instance types are optimized for different functions, such as compute, memory, or storage. The cost of these varies accordingly. When instance types are selected carelessly, projects end up using more expensive instances than they actually need.
Idle Resources
These are resources that are only needed at certain times and yet are paid for round the clock. This occurs mostly in non-production environments such as development, testing, staging, and QA. It can also occur in production, albeit more rarely (when provisioning multiple resources for HA or DR, for example). This form of waste is estimated at about $8.8 billion a year (according to the ParkMyCloud article previously cited).
Unused Resources
Another significant source of waste is resources that are continuing to run, even though they are no longer needed. Some examples of these are:
- Orphaned resources: Resources that are attached to a virtual machine that has been terminated, and continue to run on their own.
- Old snapshots: Outdated snapshots are often not cleaned up and are consequently retained for longer than necessary.
- Unattached volumes: When an instance is terminated forever, volumes attached to it are not always deleted, incurring additional costs.
Taking Too Many Snapshots
Backing up data to Amazon S3 by taking EBS snapshots is the recommended thing to do, but taking too many of these without setting proper Amazon S3 lifecycle rules will lead to snapshot sprawl, exponentially driving up storage costs.
Access, Compute, and Transfer Costs
If businesses are not careful about where they store their data, they might end up paying through the nose on access and compute costs. Seemingly cheap storage can result in overwhelming access costs, accumulated over tens of thousands of GET requests. In addition to this, transfer costs stealthily add up, as data is transferred within the AWS environment. If companies aren’t mindful about their infrastructure, data will probably not flow through the most cost-effective routes. It is worth noting here that the choice of regions, which vary in costs, can significantly affect transfer costs in this way.
Misuse of On-Demand Resources
Another form of waste occurs when companies do not plan ahead regarding how they will use their instance resources. Using on-demand instances is tempting and easy to deploy. However, stopping and starting these resources generates unexpected costs, and is much more expensive than using reserved or Spot instances.
So Now What?
The ancient sages knew it long ago; the key to wisdom is to know thyself. The clearer the picture you have of your current workflow (how it breaks down; what data is used; when, how, and for what; and how things are projected to change over time), the better you can adapt the cloud to fit your specific needs—and the lower your cloud bills will be. Below are a few concrete steps to cutting your cloud waste.
Cleaning up your environment: Delete any resources you no longer need, such as EC2 instances, orphaned snapshots, EBS volumes, etc. This will immediately reduce your cloud bills.
Efficient lifecycle management: Use automatic policies that ensure efficient lifecycle management of your resources. For instance, make sure to use a snapshot retention policy.
Rightsize: Analyze usage statistics and capacity demands to make sure all your instances are right-sized. AWS offers useful tools for this, such as AWS CloudWatch and AWS CLI. By fine-tuning your resources to meet your actual performance needs, you can considerably lower your storage costs.
Adopt an RI strategy: If you can predict your future workloads, then you can pay upfront for reserved instances, which can save up to 75% in comparison to the standard cost of on-demand instances. You should check your monthly bills to ensure that your consumption matches your usage credits. Using Spot instances can also offer a great way to reduce costs.
Customize instance types: Another way to fine-tune your resources and reduce your storage costs is by selecting instance types carefully. Use expensive and higher-performance instance types, such as Amazon EFS and Amazon EBS, only when strictly necessary. Otherwise, use the cost-efficient Amazon S3 as a larger cold-storage tier.
Scheduling: Use scheduling tools to shut down instances when they are not in use, thus eliminating idle resources. One thing to keep in mind though is the cost of stopping and starting instances manually.
Auto scaling: Dynamically scaling the size of resources based on certain target values, or by using scheduling tools, is another useful cost-optimization practice. Another advantage of using Amazon S3 is that it scales automatically, preventing overprovisioning.
Tagging: Enforce global tagging policies to keep track of resources and how they are allocated. To put it simply, if you can give things a name, and do so consistently, you will gain a deeper understanding of what is actually going on. By categorizing instances, you create an extra layer of information for reporting, which can ultimately lead to cost optimization. Tagging can be useful when utilizing AWS billing tools, such as AWS Cost Explorer.
Avoid the cost of downtime: Beyond cost efficiency, it is essential to have a solid backup and recovery plan in place to prevent the cost of downtime, which can be hugely damaging to your business. Third-party data management tools, such as N2WS, are designed expressly for this purpose.
Cutting cloud waste in AWS with N2WS Backup & Recovery
In addition to AWS own set of native solutions, third-party tools such as N2WS Backup & Recovery can play a central role in improving cloud management strategies and in cutting costs.
N2WS Backup & Recovery allows businesses to:
- Optimize data storage with cost-efficient storage methods:
- Control where data is and have the ability to move it around flexibly in the AWS environment.
- Transfer EBS snapshots to a lower cost Amazon S3/Glacier repository, saving up to 98%.
- Customize retention periods based on data lifecycle requirements.
- Save space and scale efficiently with block-level incremental snapshots.
- Reduce compute costs and the cost of idle resources: Start, stop, and hibernate groups of Amazon EC2 or Amazon RDS instances, thus saving on compute costs (with N2WS LightSwitch).
- Scheduling: Tailor backup schedules and policies with the new “Archive Snapshots to Amazon S3” feature, reducing long-term archival costs.
- Avoiding the hazardous cost of downtime: N2WS’s backup and recovery process allows for efficient backup and recovery to any region or account.
Summary
Billions of dollars are wasted every year due to poor cloud-management practices. This wasteful behavior can be uprooted by cultivating organizational awareness, and by forming a thought out cloud management strategy. Doing so can have an immediate impact on your next AWS bill.
N2WS Backup & Recovery serves as an excellent tool for cost-effective cloud management. This service aids in eliminating idle resources with automated scheduling policies, moving instances to the cost-efficient Amazon S3 repository, simplifying backup and recovery, and gaining better overall control of the way you use the cloud. In this way, N2WS can help you put your days of wasteful cloud practices behind you.
