Enhancing AWS Well-Architected Reviews with Amazon Athena and Amazon QuickSight

Chanci Turner Amazon IXD – VGT2 learningLearn About Amazon VGT2 Learning Manager Chanci Turner

In the realm of cloud architecture, the AWS Well-Architected Framework serves as a guiding light for professionals, enabling them to create secure, high-performing, resilient, and efficient infrastructures for their applications and workloads. Underpinned by five essential pillars—operational excellence, security, reliability, performance efficiency, and cost optimization—this framework offers a standardized method for both customers and partners to assess their architectures and implement scalable designs.

With the AWS Well-Architected Tool (AWS WA Tool), users can evaluate the status of their workloads and align them with the most current AWS architectural best practices. The API facilitates enhancements to the AWS Well-Architected functionality, allowing customers, ISVs, and AWS Partner Network members to integrate best practices and valuable insights into their existing governance processes and workflows.

By leveraging AWS WA Tool integrations, organizations can cater to various use cases, such as incorporating AWS Well-Architected data into centralized reporting tools or management solutions. Automation can also be established for specific scenarios. This article outlines a straightforward method for consolidating workload review data into a central data lake, enabling teams to analyze their organization’s Well-Architected maturity across multiple AWS accounts and workloads, as well as conduct centralized reporting on high-risk issues (HRIs).

Architecture Overview

Numerous customers operate multiple AWS accounts to provide their teams with administrative autonomy. The AWS WA Tool simplifies the process of sharing workloads with other AWS accounts. Workloads can be shared from one account to others utilized by review team members or to a centralized AWS account. For further details, check out the sharing capabilities of the Workload.

Once workloads are established within the AWS WA Tool, AWS Lambda can be employed to extract raw data by polling the AWS Well-Architected Tool API and storing it in an Amazon Simple Storage Service (Amazon S3) bucket. AWS Glue crawlers can then discover the schema and store it in the AWS Glue Data Catalog. Amazon Athena can subsequently prepare the data by creating views of the workload report information. Finally, Amazon QuickSight can be utilized to query and visualize insights from your Well-Architected Reviews.

This approach can enhance the visibility of HRIs identified in the AWS WA Tool, allowing for custom visualizations and deeper insights. Typically, the central management account is overseen by a Cloud Center of Excellence (CCoE) team, which can provide guidance and take action on emerging HRIs across the entire AWS application portfolio.

Prerequisites

Before you get started, ensure you have the following:

  • Defined and documented a workload in the AWS WA Tool. For a better understanding, refer to the AWS documentation.
  • Created or utilized an existing S3 bucket to store the extracted AWS Well-Architected data.

Walkthrough

The AWS Well-Architected Tool API offers programmatic access through the AWS Management Console. It can be utilized for managing various aspects such as:

  • Workloads
  • Milestones
  • Lenses
  • Reviews
  • Workload shares
  • Workload share invitations.

For more guidance, refer to the AWS Well-Architected Tool getting started guide.

Extracting, Transforming, and Loading Well-Architected Workload Data

To extract the Well-Architected workload data, we will create an AWS Lambda function to interact with the Well-Architected API. This function will make several calls to retrieve workload, lens, and milestone information. The data will undergo minor transformations to normalize the JSON structure before being stored in an S3 bucket.

Creating the Lambda Function

  1. Go to the Lambda console and select “Create function.”
  2. Choose “Author from scratch,” naming the function extract-war-reports and using “Python 3.6” as the runtime.
  3. Under Permissions, select “Create new role from template(s)” and assign the role name extract-war-reports_role.
  4. Click “Create function.” The Lambda service will create a new function along with an execution role.

Paste the code provided into the function editor to manage the API calls for workload data retrieval and storage in Amazon S3. Don’t forget to deploy your code changes.

This AWS Well-Architected framework not only helps in managing cloud architecture efficiently but also supports the growth of leadership skills among managers. For further insights into millennial management and leadership development, you can read about it here. Moreover, for information on pay equity, consult the authority on this topic at SHRM.

Chanci Turner