You're ready to deploy containers to production on AWS and now you’re wondering the best way to go about it. Teams today have more options than ever before when it comes to containerization tools.
You’re probably trying to make sense of the relevant AWS products like the Elastic Beans, Elastic Container Service (ECS), Elastic Container Service for Kubernetes (EKS), or Fargate, and other solutions such as Kubernetes, DCOS, and Docker Datacenter. You may also be scratching your head over how to roll your own infrastructure. This post covers the various options for deploying containers on AWS, the trade-offs to consider, and recommendations for common scenarios.
Meet the Players
AWS offers three containerization tools: ECS, EKS, and Fargate. EKS and Fargate were launched at Re:Invent 2017, while ECS predates the field — EKS, Fargate, and even Kubenertes.
ECS is first and foremost a stable container orchestration platform, built on EC2, Elastic Compute Cloud. Based on user-defined tasks, ECS coordinates the containers across a cluster of EC2 instances. ECS integrates into other AWS services such as CodeDeploy, automated app deployments across EC2 or on-premises instances or serverless Lambda functions, and CloudWatch, cloud monitoring service.
EKS is AWS' Kubernetes-as-a-Service offering. EKS runs upstream Kubernetes with AWS-specific features like Identity and Access Management (IAM) for role-based access control (RBAC) and networking between containers and Virtual Private Cloud (VPC).
Fargate is a hands-off platform that runs on ECS, with EKS support coming in 2018. Fargate's goal is to completely remove EC2 instances from the deployment equation. It's roughly similar to Functions-as-a-Service, where users provide the functions and the platform takes care of the rest.
Although the trio of AWS services covers many use cases, there's always the option of running your own orchestrator such as Kubernetes, Mesos, or Docker Swarm on AWS. This approach requires more knowledge, but provides more flexibility and control. It’s better to use a tool that automates installation and common tasks. Kops, Kubespray & kubeadm are among the better options for self-managed Kubernetes clusters on AWS.
On the other hand, while many container early adopters resorted to home grown orchestration, for lack of fully formed alternatives, the landscape is now very different. There are more mature, safer, and scalable options, giving you far fewer and far less compelling reasons to veer away from more standard platform options.
Of course, it's wise to reserve judgement until after a complete and tailored assessment can be made. With that in mind, let's dive into some of the relevant details.
1. ECS – Elastic Container Service
ECS is a container orchestration service that works well for straightforward web applications, microservice architectures, and data processing pipelines. Users deploy applications as task definitions and ECS takes care of scaling and auto-healing the application.
ECS requires running EC2 instances in an ECS cluster. AWS can create the EC2 instances and cluster for you, or you can opt to build and manage EC2 instances yourself. The self-management approach may work well with teams already building and managing an EC2 fleet, while AWS-managed EC2 instances may be more suitable for teams needing something more hands-offs.
ECS also integrates with X-Ray for request tracing—useful for distributed systems—and CloudWatch/CloudTail for logging. All in, ECS provides a familiar AWS experience for deploying containerized applications without having to learn too many new tools.
ECS will not work for you if you’d like to use distributed workloads across other cloud providers. ECS is AWS only. You’ll need something else in this case.
2. EKS – Amazon Elastic Container Service for Kubernetes
Simply put, EKS is managed Kubernetes. It promises to own the heavy lifting of cluster provisioning, master management, HA, patching, and upgrades, with an AWS-specific twist. EKS uses a custom Kubernetes Container Network Interface (CNI) to work with AWS VPCs. It uses IAM for RBAC between containers in the cluster.
There's still much to learn about the technical specifications of EKS, as it's still in preview. Regardless of how the details shake out, however, the big win is that EKS is just Kubernetes. Your application can move seamlessly among Kubernetes clusters inside AWS, on-prem, hybrid, or anywhere else.
If you're committed to Kubernetes, then you may want request preview access to EKS and see what's in store. However, it's too early to commit to EKS for production workloads because it's still not clear when it will launch, what regions it will support, and a host of other questions.
Fargate is AWS' completely managed container deployment platform, built on top of ECS or EKS, with EKS support coming in 2018. Fargate isn't a new orchestrator; it complements ECS and EKS by handling all aspects of the container deployment lifecycle. Declare container memory and CPU and Fargate handles scaling up and down.
Fargate also handles instance type selection, which should lead to higher resource utilization and lower cost compared to using ECS or EKS directly.
While ECS or EKS are billed per hour, Fargate's billing is per second, which makes it a strong contender for short-lived workloads or extremely dynamic environments.
An interesting combination of Kubernetes and cloud provider container as a service is offered in Virtual Kubelet. It is an open source Kubernetes kubelet implementation that masquerades as a kubelet for the purposes of connecting Kubernetes to other APIs. This allows the Kubernetes nodes to be backed by other services like AWS Fargate, Azure ACI, etc.
4. Self-Managed Kubernetes with Kops
Sometimes teams need more control over their infrastructure, want to avoid vendor lock-in, or need to overcome limitations in AWS offerings. Kops automates the provisioning, upgrading, scaling, and decommissioning of Kubernetes clusters on AWS according to community best practices.
Using Kops does not absolve users from understanding Kubernetes thoroughly or from taking responsibility for their container deployments. You can use Kops to configure the internal components such as the networking provider and AWS specifics like VPCs, subnets, or even Amazon Machine Image (AMI). Once the cluster is provisioned, however, the bulk of responsibility falls on you.
The same points discussed above for Kops also apply to Mesos or Docker Swarm, which have their own tools to automate cluster creation and management.
Containerization Tools: Mapping Capabilities to Requirements
It's nice to see AWS driving container adoption by introducing Fargate and EKS. Now all three major cloud providers — AWS, Google, and Microsoft — offer managed Kubernetes containerization tools. The competition is heating up and will undoubtedly drive better options for deploying containers.
Unfortunately, AWS users must wait for the EKS launch, leaving three options if you need to deploy containers to production on AWS today: ECS, Fargate, and self-managed. Deciding your best fit is largely based on:
- Your existing technical architecture and unique use cases
- Your preference for either AWS-specific technologies, or becoming cloud-agnostic
- Your operations staff, capabilities, and practices
You may have already chosen Kubernetes to deploy, scale and manage your containerized applications — a choice that provides a strong ecosystem and future flexibility. However, that choice also means that you'll need to self-manage your container deployments on AWS, which is no small task and certainly not a job for a single person.
Operations engineers will need to be trained and equipped to handle their container management responsibilities. When EKS launches it will be possible to shift some operational responsibility to AWS, but the team must still have a good grasp of how Kubernetes works. Migrating from your AWS self-managed cluster to EKS might go through Ark, collection of tools to backup and restore your Kubernetes cluster resources all the way done to your persistent volumes.
If you haven't already decided on a platform, then AWS offer two strong, yet simple enough options in ECS and Fargate running on ECS. Automating and managing ECS clusters may be the best choice along with ECS command line interface (CLI) support for docker-compose. It will jump start your development teams and enable you to take advantage of ECS' integration with other AWS services.
If your team has no experience or no capacity to handle infrastructure, then Fargate is likely the better choice. Because Fargate uses the same definitions as ECS, you can move to Fargate without too much legwork.
Choose the right tool to get the job done. In fact, hybrid solutions provide a good balance of trade-offs. When EKS launches there will be even more hybrid options. Core teams that need Kubernetes can use EKS, while other members of the organization can use Fargate on top of that EKS cluster. It's also possible to test out ECS or EKS via Fargate before deciding.
The best fit balances trade-offs among existing technical capabilities, commitment to AWS, and operations staff. Remember though that just because it's hosted doesn't mean you're completely off the hook.
Choose containerization tools that deliver the most value to your team so that they can focus on what they do best: adding value to the business, without worrying about container provisioning and management.