When running workloads in the AWS cloud, often you will find yourself in need of a shared storage capacity. And while the Amazon cloud does offer such a solution with S3 object storage, the performance it provides is simply not good enough, as it does not have high enough IOPS or low enough latency that many use cases may require. The EBS volumes (the AWS offering for block storage) do provide great performance, but they don’t have the capabilities to be shared between instances—each volume can only be attached to a single instance at a time.
The obvious need for a high-performance shared storage solution in the cloud led to Amazon releasing the Elastic File System (EFS) back in June 2016, and in this article we will take a closer look at this service, as well as look into some details that you might find very useful when deciding what is best for your company’s needs and requirements.
What Is Amazon EFS?
Amazon EFS is a service offering a shared, scalable, and elastic file system designed for various workloads running in the Amazon cloud, supporting Network File System (NFS) 4.0 and 4.1 protocols. EFS is designed to be used with Linux instances (if you need a shares storage for Windows you can look into Amazon FSx—a fully managed native Microsoft Windows file system), and is a highly available and durable service which is built to scale automatically—it will grow (but also shrink) on-demand as you add more files to it—all the while supporting low latency shared access to a large number of instances at the same time.
With Amazon EFS, access is secured using POSIX permissions, and there is no need to make any changes to your existing applications (something which is very desired). Amazon has also just recently released a new storage class for EFS, allowing you to choose between the two—Standard (originally the only class available) for you regular workloads, and a cheaper Infrequent Access class for files which are rarely accessed (priced $0.045 per GB stored compared to $0.30).
6 Things You Didn’t Know About Amazon EFS
At first glance, EFS looks like a very simple service, but when you dig a little deeper you will find that there is a lot more to it. It is a very powerful tool, and one you can use to your great benefit. Let’s take a look at some things you probably didn’t know about EFS, and why you should consider it for your cloud (or even hybrid cloud) environments.
EFS Is Fully Managed and Very Simple to Use
Amazon EFS is a fully managed service, meaning there is no overhead for deploying or maintaining the underlying storage at all. You have a simple web interface which allows you to create an EFS file system (or you can do it via the command line interface, making it easy to automate), which mounts to your Linux instances quickly and easily—this process requires a bare minimum of familiarity with both AWS and Linux. This ease of use will be very beneficial to any company without a DevOps team available, especially smaller startups, as EFS can be handled by almost anyone.
Also, if you are running a hybrid cloud environment, and you have already relied on the shared storage on-premises, moving to EFS will be seamless.
EFS Provides Shared Storage, Accessible from Both AWS, and On-Premises
EFS can be accessed simultaneously by literally thousands of instances at the same time, and supports not only those running in the AWS cloud, but also your on-premises instances using a VPN connection or AWS Direct Connect. This makes it a great tool for various hybrid cloud solutions where you might have the application requirement for shared storage from both an on-premises data center and the cloud at the same time (for example, it can help greatly when doing a lift and shift of your application to the cloud). EFS also allows access across various Availability Zones and Regions (something that EBS can’t provide you with), so if you have workloads in multiple geographical areas that need to access the same data simultaneously, this is the tool to look into.
EFS Has Scalable Performance, Along with Dynamic Elasticity
EFS is designed for low latency scalable performance, providing IOPS and throughput that scale as your usage of the file system grows. The more files you add, the larger your storage size becomes, which in turn increases the overall performance. At its peak, EFS can support over 10 GB/sec and up to 500,000 IOPS. But on the other hand, also keep in mind that when used minimally, EFS might not provide you with what your application needs—you could be using too little storage to achieve the desired performance you require.
With EFS you also have to be careful when making backups, as this can have a negative impact —decreasing the performance of the file system, since the throughput used by backup counts towards your total file system throughput. So make sure to plan well and schedule your backups to avoid affecting your production environment.
EFS Is Cheaper Than EBS When Utilized Properly
Even though on paper EFS costs you more than an EBS volume would (with $0.30 per GB compared to $0.10 per GB), in practice you will have multiple instances connected to your EFS shared storage at most times. And since you are paying per EFS share, and not per instance using it, EFS will cost you the same whether utilized by one or more instances (though data transfer costs have to be kept in mind). This means that if you have dozens of EC2 instances connected to your EFS volume, you will pay a lot less for the storage than if you relied on EBS volumes for each of those instances.
On top of that, EBS volumes are only replicated within their Availability Zone (AZ), while EFS is automatically replicated within the region where it is residing (meaning across multiple AZs)— which makes it a much more durable storage solution for your data.
EFS Is Secured and Meets Many Compliances
EFS allows you to have multiple layers of security. It relies on your existing cloud security infrastructure, and you can use Identity and Access Management (IAM) roles to secure access to the service itself. For network access VPC security groups are used, giving you the ability to decide which instances will have access to the shared storage. And in terms of access to the files themselves, a standard POSIX is used to define various permissions.
EFS also meets various eligibility and compliance requirements, so whatever your use case might be, you can be sure that you are covered—some of the compliance programs worth mentioning are PCI DSS, HIPAA, SOC, etc.
EFS Can Be Used in a Multitude of Different Ways
EFS is a very versatile tool, and as a shared storage it can be used for many different use cases. We have already mentioned things like doing a lift and shift, but EFS is also great for other hybrid cloud workloads like big data analytics, thanks to its ability to scale performance and capacity as needed. It is also useful as a shared code repository in dev environments, so you can, for example, run Artifactory on EFS. Another way to utilize EFS would be in media processing—not only does it have capabilities which make it a great use case for audio and video editing or rendering, it also allows you to centralize the storage of all the files.
EFS can also be used for container storage, database backups, serving web content, and many other things.
Summary
EFS was introduced as a requirement as shared cloud storage grew, but what we got was much more than a service fulfilling a simple need. EFS offers familiar, but very powerful storage, that can scale performance to support almost any use case you may have. Also the ability to access its file system from any location, within or outside of AWS cloud, makes it very versatile when looking at designing more complex cloud environments. And yet, EFS still keeps things simple by being a fully managed service, providing you with all you need, while removing all the complexity that maintaining a storage requires. So if your company is looking at a shared storage solution for your workloads, make sure to give EFS a test run.
