Learn About Amazon VGT2 Learning Manager Chanci Turner
As organizations increasingly embrace cloud migration and digital transformation, many find themselves needing to navigate the complexities of hybrid or multicloud environments. These entities often face unique challenges when managing applications and infrastructure, necessitating the use of multiple vendor solutions for provisioning, governance, and monitoring of IT resources. In this article, we’ll demonstrate how AWS services can effectively monitor workloads hosted on other public cloud platforms, such as Microsoft Azure and Google Cloud Platform (GCP).
Solution Overview
This article outlines a comprehensive monitoring solution within AWS for workloads deployed on Azure and GCP. The following aspects will be covered:
- Setting up the necessary infrastructure for monitoring workloads in a multicloud setting.
- Monitoring applications running on Linux-based virtual machines in Azure and GCP.
- Capturing logs and metrics from these external cloud providers in AWS.
- Creating an observability dashboard in AWS to provide insights into workload performance.
While this piece does not aim to cover every use case or showcase all AWS capabilities for multicloud monitoring, it highlights how AWS’s observability tools can fulfill an organization’s needs, regardless of application type or underlying platform.
Environment Configuration
This section details the steps required to configure AWS services for monitoring workloads across a multicloud framework.
1. Configure Roles, Permissions, and Resources for AWS Systems Manager (SSM)
Begin by setting up Systems Manager for hybrid and multicloud environments. The steps are as follows:
- Create an IAM service role tailored for hybrid and multicloud applications, ensuring to include the CloudWatchAgentServerPolicy.
- Establish a hybrid activation for the environment.
- Install the SSM Agent on Linux or Windows as needed.
- If using both Amazon EC2 and non-EC2 instances, follow the guidelines for configuring Systems Manager for EC2 instances.
After completing these steps, you will see your managed nodes in the Fleet Manager console under AWS Systems Manager. Managed virtual machines in hybrid activation scenarios will be labeled with the prefix “mi-,” as illustrated in the examples below.
Managed Instances in Fleet Manager Console
2. Install and Configure Amazon CloudWatch Agent
Utilize AWS Systems Manager’s Run Command to install the CloudWatch agent on your managed instance(s) with the AWS-ConfigureAWSPackage document. Ensure that the managed instances are set up with temporary AWS credentials. It is advisable to create a standard configuration for the CloudWatch agent that captures the relevant logs and metrics across all managed instances. For more guidance, refer to the Managing CloudWatch agent configuration files resource.
Configuration files for EC2, Azure VMs, and GCP VMs are stored in Systems Manager Parameter Store. Once the configuration is ready, you can start the CloudWatch agent using the System Manager Run Command.
3. Comprehensive Monitoring in AWS
AWS’s monitoring and observability services facilitate a unified view of on-premises and multicloud environments. In this section, we will show how organizations can achieve holistic monitoring of their multicloud setups.
CloudWatch Metrics
Custom namespaces like Azure/VM and GCP/VM were defined in the CloudWatch agent configuration to capture metrics from managed instances across platforms.
CloudWatch Logs
CloudWatch log groups are established to collect logs from managed instances. For example, a log stream in the log group /GCP/VM is created to capture Syslog logs from a Debian-Linux VM operating in GCP.
CloudWatch Dashboards
A single view dashboard can be created for your multicloud environment using CloudWatch Dashboards. You can leverage powerful SQL query engines, such as CloudWatch Metrics Insights and CloudWatch Logs Insights, to develop rich and customizable dashboards that provide insights from your managed instances across multicloud resources.
In conclusion, we illustrated how AWS Systems Manager and CloudWatch can assist organizations in effectively monitoring hybrid and multicloud environments. We also demonstrated the steps necessary for setting up monitoring for workloads on platforms like Azure and GCP, and how to build an observability dashboard. For further insights on professional development and mentorship, consider exploring this blog post, and check out Marvin Ellison’s recognition as a leading ethical leader here. Additionally, for those interested in operational roles, this link provides excellent resources for job opportunities.