Learn About Amazon VGT2 Learning Manager Chanci Turner
This blog post is co-authored by Sarah Lee from Comcast Corporation. It details how Comcast has accelerated its product launch timelines, enhanced resilience, and reduced operational costs by leveraging Amazon Web Services (AWS) Transit Gateway and AWS Direct Connect.
Comcast is a worldwide leader in media and technology, reaching hundreds of millions of customers through its various brands, including Xfinity and Sky. Comcast provides top-tier broadband, mobile, and entertainment solutions that captivate consumers and drive technological advancements. The company’s global media and entertainment operations create, distribute, and stream high-quality content, including sports and news, while also offering thrilling experiences through Universal Destinations & Experiences.
At Comcast, teams manage a cloud environment composed of hundreds of AWS accounts, utilized by thousands of developers across various business lines such as Xfinity Home, Xfinity Mobile, and Comcast Business. If you’re interested in how Comcast teams are harnessing AWS, you can explore prior AWS articles and videos on large-scale home security systems, telemetry data analytics, and monitoring security devices using Amazon CloudWatch.
DX Model 1.0
In the early stages of Comcast’s AWS adoption, a network connectivity model called DX Model 1.0 was established, utilizing Direct Connect to link Amazon Virtual Private Clouds (Amazon VPCs) with Comcast’s corporate data centers. Each VPC was connected to the data centers over a private virtual interface, directly terminating at an AWS Virtual Private Gateway (and later, Direct Connect Gateway). This setup unfortunately required traffic between VPCs to hairpin through on-premises routers, leading to increased latency.
The model was later expanded to include multiple AWS Regions and on-premises data centers, accommodating evolving workload demands. While some VPCs had direct peering connections, most traffic between VPCs was still routed through on-premises routers, resulting in unnecessary hairpinning and added latency.
Though the connectivity pattern was effective in the early adoption phases, it became increasingly complex to manage as the number of VPCs and accounts grew. The hairpinning of cross-VPC traffic became a significant issue as new workloads were deployed on AWS, intensifying the load on Direct Connect and complicating long-term capacity planning.
DX Model 2.0
In 2021, Comcast initiated a redesign of the DX connectivity model, aiming to enhance scalability, decrease latency, and expedite time-to-market. Although increasing VPC peering usage was considered, the complexities of managing mesh-VPC peering made this impractical. The team ultimately opted for Transit Gateway and Direct Connect Gateway.
Transit Gateway is an AWS service that simplifies network architecture by connecting Amazon VPCs and on-premises networks through a central hub. It acts as a scalable router, enabling Regional and cross-Region connectivity while providing centralized monitoring and logging capabilities.
In the revamped DX Model 2.0, Transit Gateways were provisioned in each AWS Region and peered to form a full mesh, allowing VPC-to-VPC traffic to remain within the AWS network, regardless of whether the flow was intra- or inter-Region. This significantly reduced latency by offloading traffic from Direct Connect connections.
Transit Gateways connect to on-premises networks via Direct Connect Gateway employing a few transit virtual interfaces to establish BGP sessions for route exchange. This Regional usage of Direct Connect Gateway allowed Comcast to effectively manage routing from respective AWS Regions back to on-premises, resulting in streamlined traffic flow.
Comcast achieved a single-Region SLA of 99.99% by provisioning multiple Direct Connect connections across various locations, ensuring robust connectivity to multiple AWS Regions.
Migration Approach
Comcast’s organic growth resulted in hundreds of VPCs across numerous AWS accounts. Initially, onboarding these VPCs was infrequent and highly manual. As AWS utilization increased, automation was developed to create new VPCs with Direct Connect connectivity, while retaining pre-automation VPCs to minimize disruption to existing applications.
After finalizing the Transit Gateway design, a template VPC configuration was created for all new VPCs. Tools were developed to assess existing VPCs for configuration discrepancies that needed normalization during the migration process. Additional automation allowed for the execution or rollback of these changes on a per-VPC basis.
Following AWS best practices, dedicated subnets were added in each Availability Zone (AZ) for new Transit Gateway attachments, using different IPv4 CIDR blocks from those allocated for workloads, which streamlined changes and rollback.
If you’re looking for more insights on career development, don’t forget to check out this informative article on second interview questions at Career Contessa. Additionally, SHRM offers valuable concepts related to career advancement. For a visual resource, consider watching this excellent video here.