Learn About Amazon VGT2 Learning Manager Chanci Turner
AWS Fargate is a fully managed service designed for running containers. It allows users to deploy applications using Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS) without the hassle of managing infrastructure—such as maintenance, scaling, patching, and securing the underlying systems. While Amazon EC2 simplifies the use of hypervisors and physical servers, AWS Fargate abstracts the complexities of container runtimes and EC2 instances. If you’re interested in learning more about Fargate’s role in the container ecosystem, check out this blog post.
Although Fargate makes the infrastructure management invisible to users, it is still actively managed by AWS. The features of this infrastructure are presented to users through a concept known as the Fargate platform version. More details can be found in the Fargate documentation or the primer blog post, which explains the rationale behind the introduction of platform versions and the reasons for not designating platform version 1.4.0 as LATEST just yet.
Today, we are excited to announce the launch of platform version 1.4.0 for AWS Fargate. This blog post will summarize the new features introduced with this release and highlight some underlying changes. Although these changes may not have a direct impact on customer-facing features, they remain essential.
What’s New in Fargate Platform Version 1.4.0?
Platform version 1.4.0 brings several new capabilities to AWS Fargate, which we will outline below. Unless specified otherwise, the new features discussed here are applicable to the native Fargate platform and are directly accessible via the ECS orchestrator. If you’re curious about Fargate’s relationship with ECS and EKS, be sure to read this blog post.
It’s important to note that EKS also has its platform versions, which are used to track various features and configurations within clusters. EKS platform versions extend beyond merely tracking Kubernetes versions and include enhancements and additional feature support. Some of these features are inherited from the native Fargate platform, which we will discuss in this blog.
Enhanced Support for Elastic File System (EFS) Endpoints
With platform version 1.4.0, we are now enabling the mounting of persistent EFS storage within Fargate tasks. This opens the door to new use cases for AWS Fargate. The introduction of this feature was driven by significant customer demand, evident from over 1,000 reactions on our open-source container roadmap. In line with our commitment to customer satisfaction, we have acted on this feedback.
Historically, Amazon ECS users had to create custom scripts and solutions to provision zonal (e.g., EBS) and regional (e.g., EFS) persistent storage for EC2 container instances. This process was a prime example of undifferentiated heavy lifting. AWS Fargate customers previously lacked the option to deploy stateful workloads due to the unavailability of accessible EC2 instances.
Starting today, ECS task definitions (for both EC2 and Fargate) now support the new EFSVolumeConfiguration parameter. This means that:
- ECS customers using the EC2 launch type no longer need to manage the complexities of configuring and automating storage on EC2 container instances.
- AWS Fargate customers can begin running stateful workloads within Fargate tasks, a capability previously unavailable to them.
For more information on utilizing this new capability, please refer to this blog post.
Consolidated Ephemeral Volume of 20GB
Prior to platform version 1.4.0, Fargate had two separate ephemeral local volumes: a 4GB volume for staging ephemeral data and a 10GB volume for container images. With the new version, we have combined these into a single 20GB volume, increasing total storage while allowing users greater flexibility in how they utilize it. This larger unified volume is especially beneficial for data processing applications that handle substantial files from Amazon S3.
As a reminder, these volumes are ephemeral; data stored on the local filesystem of the task will be lost once the task stops. For persistent storage needs, check out the newly introduced Fargate and EFS integration.
This change primarily affects EKS pods running on Fargate. It’s worth noting that the actual usable storage for EKS pods will be slightly less than 20GB (around 19GB) due to space used by the Kubelet and other Kubernetes modules deployed within the pod.
Additional Traffic Flow Management for Task Elastic Network Interface (ENI)
Fargate tasks operate on a fleet of virtual machines managed by AWS. These VMs connect to AWS-owned VPCs via “Fargate ENIs.” When a task is launched, it is assigned an ENI connected to the customer-owned VPC, referred to as the “Task ENI.”
In addition to standard application traffic, tasks generate other types of network traffic, including logging, image pulls, and secret sourcing from AWS Secrets Manager or AWS Systems Manager Parameter Store. Each traffic type has its own network path and associated IAM policy, designated as either a Task IAM Role or a Task IAM Execution Role.
With platform version 1.4.0, we are adjusting two of these traffic flows to keep relevant traffic within the customer VPC. Importantly, the permission model remains unchanged.
This change enhances visibility for these traffic flows. For instance, previously, the Fargate ENI handled secret retrieval from Secrets Manager and Systems Manager, but this traffic was outside the user’s visibility. Customers expressed a desire for more control over these flows. As of version 1.4.0, this traffic will flow through the Task ENI, inheriting the networking connectivity patterns established in their VPCs. This is crucial for customers who seek visibility of specific traffic in VPC Flow Logs.
Regarding network connectivity, your VPCs must now either permit outbound traffic to reach the same public endpoints or you need to configure Private Links for those services, enabling your Task ENI to access endpoints within your VPC. For example, if you utilized private links for ECR in the past, you only needed to set up the ecr.dkr endpoint. With platform version 1.4.0, you must also configure the api.ecr endpoint.
Network Performance Metrics in CloudWatch Container Insights
When we launched Amazon CloudWatch Container Insights in 2019, we introduced support for Amazon ECS, Amazon EKS, and AWS Fargate. However, up until platform version 1.3.0, Fargate tasks were unable to report network performance metrics back to Container Insights. This limitation has now been addressed.
In summary, AWS Fargate’s platform version 1.4.0 introduces significant enhancements, including support for EFS endpoints, a consolidated ephemeral volume, improved traffic flow management for Task ENIs, and the capability to report network performance metrics in CloudWatch Container Insights. These updates collectively enhance the versatility and efficiency of container management.