Learn About Amazon VGT2 Learning Manager Chanci Turner
This blog post is authored by Jamie Smith, SDE II – Customer Engineering AWS.
Have you ever wondered how to integrate a cloud service within another cloud? Some of the fantastic features of Amazon Elastic Compute Cloud (Amazon EC2) can complicate the operation of CloudStack. One of the primary challenges arises from the fact that both AWS and CloudStack aim to manage network resources, which can lead to conflicts. In this article, I’ll guide you through the necessary steps to ensure a smooth coexistence between these two platforms, drawing on my own experiences.
Apache CloudStack is an open-source platform designed for deploying and managing virtual machines (VMs) along with their accompanying network and storage infrastructure. While it’s typically run on personal hardware to create a private cloud, there are significant advantages to deploying it within an Amazon Virtual Private Cloud (Amazon VPC). This approach could facilitate migration from a data center, create disposable environments for experimentation, and provide a convenient way to explore the new CloudStack support in Amazon Elastic Kubernetes Service (Amazon EKS) Anywhere. In my situation, the need arose to establish development and testing environments for a project utilizing the CloudStack API, which required shared and scalable resources. Given that our build pipelines already operated within AWS, it was logical to position these new environments in the same ecosystem.
CloudStack is compatible with various hypervisors; however, this guide will focus on using Kernel-based Virtual Machine (KVM) on Linux for managing VMs, while CloudStack oversees KVM operations.
Prerequisites
The content in this article is applicable to multiple versions of CloudStack, specifically targeting CloudStack 4.14 on CentOS 7. I’ve also tested versions 4.16 and 4.17, both of which I recommend. The official CentOS 7 x86_64 HVM image performs well, but if you opt for a different Linux distribution or version, you may need to adjust some implementation details.
Familiarity with CloudStack basics is essential, as the article aims to ensure a harmonious relationship between CloudStack and AWS. Once CloudStack is operational, you’ll be responsible for managing it. For comprehensive information on security and best practices, please refer to the AWS documentation and the CloudStack documentation.
Simplifying the Process
I’ve developed scripts to automate the installation process, which can be executed on EC2 instances running CentOS 7. These scripts handle all installation and OS configuration tasks, and you can use them as is or modify them to suit your needs. Additionally, I’ve created AWS CloudFormation templates that can be replicated to establish a demo environment. More details are available in the README file.
Amazon EC2 Instance Types
KVM necessitates hardware virtualization support. Since most EC2 instances are VMs that don’t support nested virtualization, you’ll need to use a bare metal instance type.
I recommend the c5.metal instance, which is one of the more affordable metal options and offers a low cost per vCPU. It comes equipped with 96 vCPUs and 192 GiB of memory. For instance, if you were to run 20 VMs—each with 4 CPU cores and 8 GiB of memory—you’d still have 16 vCPUs and 32 GiB available for the OS, CloudStack, and MySQL. CloudStack’s overprovisioning feature allows you to fit even more VMs if they operate under light loads.
Networking Challenges
One of the most significant hurdles lies in networking. AWS has strict oversight of IP and MAC addresses, dictating which should exist and which machines they should belong to. Any traffic that doesn’t conform to AWS’s network behavior guidelines is blocked. Meanwhile, CloudStack assumes that any IP or MAC address it generates should function without issue. Consequently, if CloudStack assigns addresses to VMs within an AWS subnet, their network traffic may be obstructed.
To circumvent this, you could enable network address translation (NAT) on the instance running CloudStack. While this is a viable solution, it complicates communication between your VMs and other machines in your Amazon VPC. I suggest taking a different route.
Although AWS imposes limitations on layer 2 networking, it allows you to run your own layer 3 router. Your EC2 instance can function as a router to a new virtual subnet that exists outside AWS’s control. This instance will integrate with AWS similarly to a VPN appliance, directing traffic as needed. Within this virtual subnet, CloudStack can operate freely, creating a harmonious environment.
What do I mean by a virtual subnet? This is a subnet that solely exists within the EC2 instance, consisting of logical network interfaces connected to a Linux bridge. While this setup does not scale efficiently, it is straightforward. In my forthcoming post, I will delve into a more intricate setup involving an overlay network that spans multiple instances for horizontal scaling.
The Simplest Approach
The simplest method is to consolidate everything within a single EC2 instance, including the database, file storage, and a virtual subnet. Ensure sufficient disk space—500 GB should suffice for a few basic VMs. Create or select a security group for your instance that affords users access to the CloudStack UI (TCP port 8080) and allows access to any services that your VMs will provide.
Once your instance is ready, configure AWS to recognize it as a router.
- Navigate to Amazon EC2 in the AWS Management Console.
- Select your instance and disable source/destination checking.
- Update the subnet route tables:
- Access the VPC settings and choose Route Tables.
- Identify the tables associated with subnets requiring CloudStack access.
- In each of these tables, add a route to the new virtual subnet, targeting your EC2 instance.
- Depending on your networking requirements, additional routes to transit gateways, VPN endpoints, etc., may be necessary.
Since everything will reside on a single server, creating a virtual subnet is simply a matter of establishing a Linux bridge. CloudStack needs to identify a network adapter attached to the bridge. Therefore, a dummy interface with a recognizable name for CloudStack must be added.
The following script demonstrates how to configure networking in CentOS 7. You must replace the variables $virtual_host_ip_address and $virtual_netmask with values corresponding to your desired virtual subnet. For $dns_address, I suggest using the base of the VPC IPv4 network range plus two. Avoid using 169.654.169.253, as CloudStack reserves link-local addresses for its own purposes.
yum install -y bridge-utils net-tools
# The bridge must be named cloudbr0.
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-cloudbr0
DEVICE=cloudbr0
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=none
IPV6INIT=no
IPV6_AUTOCONF=no
DELAY=5
STP=yes
USERCTL=no
NM_CONTROLLED=no
IPADDR=$virtual_host_ip_address
NETMASK=$virtual_netmask
DNS1=$dns_address
EOF
# Create a dummy network interface.
cat << EOF > /etc/sysconfig/modules/dummy.modules
#!/bin/sh
/sbin/modprobe dummy numdummies=1
/sbin/ip link set name ethdummy0 dev dummy0
EOF
chmod +x /etc/sysconfig/modules/dummy.modules
/etc/sysconfig/modules/dummy.modules
cat << EOF > /etc/sysconfig/network-scripts/ifcfg-ethdummy0
TYPE=Ethernet
BOOTPROTO=none
NAME=ethdummy0
DEVICE=ethdummy0
ONBOOT=yes
BRIDGE=cloudbr0
NM_CONTROLLED=no
EOF
# Enable routing on the instance
echo 'net.ipv4.ip_forward=1' >> /etc/sysctl.conf
sysctl -p
# It’s essential to terminate dhclient, as the network service may not restart correctly otherwise.
# Alternatively, you could reboot if preferred.
pkill dhclient
For additional insights and resources on related topics, consider checking out this excellent resource on the area manager onboarding process on Reddit, or explore more about employment law compliance from SHRM, who are an authority on this topic. If you’re looking for engaging content while you work, you might also enjoy some podcasts to listen to at work, like those featured on Career Contessa.