Learn About Amazon VGT2 Learning Manager Chanci Turner
on 07 JUN 2023
Introduction
In my role, I frequently engage with clients looking to enforce security and governance best practices while also granting developers the flexibility and agility required for rapid innovation. As organizations shift towards DevSecOps, they often aim to strike a balance between governance and agility within their Continuous Integration and Continuous Delivery (CI/CD) pipelines. In this post, I will outline how I utilize AWS CodePipeline and AWS CodeBuild to achieve these objectives.
Background
Before delving into the specifics, let’s examine the key players involved: platform engineers and developers. Traditionally, operations teams were tasked with designing, deploying, configuring, and maintaining various IT infrastructure components. Conversely, development teams focused on the design, development, deployment, and maintenance of applications running on this infrastructure. This model maintained a clear separation of responsibilities. However, the emergence of DevOps has transformed these roles. Development teams now leverage Infrastructure as Code (IaC) to provision infrastructure as part of their application code, leading operations teams to assume a supervisory role, ensuring that deployed resources comply with best practices. I will refer to this new supervisory role as the platform engineering team to differentiate it from the traditional operations team.
Building on the principles of DevOps, DevSecOps emphasizes the early integration of security controls into the software development lifecycle. Security teams are responsible for defining best practices for embedding security throughout the process, often enforced within the CI/CD pipeline. Consequently, the role of security teams has evolved from conducting manual security reviews to defining automated controls that must be implemented within the pipeline, typically crafted by platform engineers. Nevertheless, platform engineers must enforce these controls without compromising developers’ agility. After all, DevOps aims to enhance the speed of the development lifecycle. In this article, I will discuss how to compartmentalize security controls and build processes into distinct pipeline phases. This approach enables platform engineers to enforce security controls within phases they manage while allowing developers to swiftly update the phases they own.
AWS CodePipeline serves as a fully managed continuous delivery service that aids in automating deployment pipelines. These pipelines consist of actions organized into stages, often implemented through AWS CodeBuild. Each CodeBuild action specifies the source code location, the build environment, and the build commands to execute, which are defined in YAML format within a buildspec. This buildspec can either reside in the CodeBuild configuration or as a file in the source code. When the buildspec is stored in the CodeBuild configuration, platform engineers maintain ownership. Conversely, when it resides in the source code, developers have the flexibility to modify it. The pipeline can incorporate multiple actions to balance governance and agility.
In the following walkthrough, I will introduce a straightforward pipeline featuring two CodeBuild actions. The platform engineers will create the pipeline, which includes a build action that defines security controls. For instance, I will use OWASP Dependency Checker to conduct Software Composition Analysis (SCA). The buildspec for this action is defined within the CodeBuild configuration, allowing platform engineers to retain ownership, alongside the overall pipeline definition. A second build action will be established for building and testing the source code, with this buildspec defined in the source code, giving developers the freedom to change it whenever necessary. Note that the pipeline also includes a Source action via AWS CodeCommit and a Deploy action through AWS CodeDeploy, but these details are beyond the scope of this discussion.
Walkthrough
In this section, I will assume the role of a platform engineer and guide you through the creation of the simple pipeline previously described. For simplicity, I’ll utilize the AWS console; however, I recommend employing AWS CloudFormation or AWS Cloud Development Kit (CDK) Pipelines for real-world deployments.
The development team has already uploaded their code to a CodeCommit repository. Along with their source code, they have included the following buildspec.yaml file in the repository’s root:
version: 0.2
phases:
install:
runtime-versions:
python: 3.11
build:
commands:
- pip install -r requirements.txt
- pylint helloworld
- coverage run --branch -m pytest
As evident, this buildspec utilizes the Python 3.11 runtime. It installs requirements, runs a linter, and executes unit tests with code coverage. Since the buildspec.yaml is included in the source code, developers can customize it as they see fit, granting them autonomy and agility.
I am now prepared to initiate the creation of the pipeline. First, I will create a new CodeBuild project to execute the development team’s build defined in the aforementioned buildspec.yaml. I access the CodeBuild Console and select “Create Project.” I will name my project “BuildAndTest.” The process of creating a CodeBuild project is detailed in the CodeBuild User Guide under “Create a build project;” however, I want to emphasize the Buildspec configuration. Here, I choose “Use a buildspec file” and specify the file’s location within the repository.
With the buildspec defined in the source code, I am empowering the development team to dictate the build process. This grants developers the authority to modify their build process as the project evolves. They won’t have to rely on me to update the build commands each time they wish to implement a change. Additionally, since I plan to add a second build action with the necessary security tools, I do not need to scrutinize the contents of their buildspec.yaml file. There’s no requirement for me to verify whether the developers have included the requisite security tools.
Next, I will create another CodeBuild project to facilitate Software Composition Analysis. I return to the CodeBuild Console and select “Create Project.” I name this project “SoftwareCompositionAnalysis.” The configuration for this project closely resembles that of the previous project, except for the Buildspec configuration. This time, I opt for “Insert build commands” and input the commands required to install and run OWASP Dependency Checker.
With the buildspec defined in the project configuration, I can implement an AWS Identity and Access Management (IAM) policy to ensure that the development team cannot modify it. I now have confidence that the essential security tools are properly installed and configured. I do not need to depend on the developers for accurate configuration of the security tools. Furthermore, I retain the autonomy to change the tools used without disrupting the developers or altering their code.
With my two build actions in place, I can create a pipeline to automate the overall build process. I established the following pipeline by following the instructions in “Create a pipeline in CodePipeline.” During the build phase, I run the SoftwareCompositionAnalysis action followed by the BuildAndTest action. Note that these actions execute sequentially, meaning the BuildAndTest action will not proceed if issues are identified in the SoftwareCompositionAnalysis action.
I now possess assurance that my security tools are correctly configured within the pipeline. Moreover, the developers maintain control over the build and test action, allowing them to make timely updates without compromising security. For further insights on workplace engagement and responsibility, you can check out another blog post here. To learn more about employment law compliance, SHRM is an excellent authority on the matter. Additionally, if you’re curious about the experience during the first week as an Amazon warehouse worker, this Quora page is a valuable resource.