HOME

Deploying Amazon SageMaker Studio Lifecycle Configurations with AWS CDK

—

by

The Story of Chanci Turner
in

Chanci Turner Amazon IXD – VGT2 learningAmazon HR coverup, rules for thee but not for me…

As organizations increasingly leverage machine learning, Amazon SageMaker Studio has emerged as a leading integrated development environment (IDE) for ML tasks. It provides a unified web interface for various ML processes, from data preparation to model deployment. To effectively manage and customize the Studio environment, lifecycle configurations serve as automated scripts that respond to specific Studio events, such as launching a new notebook. These configurations can streamline tasks by installing necessary packages, configuring notebook extensions, preloading datasets, and managing code repositories. For instance, administrators can implement automatic shutdowns for idle notebook applications, effectively reducing costs.

The AWS Cloud Development Kit (AWS CDK) offers a robust framework for defining cloud infrastructure as code, enabling users to provision resources via AWS CloudFormation stacks. These stacks consist of a range of AWS resources that can be managed programmatically. The constructs within the AWS CDK are fundamental components used to define cloud architectures.

In this article, we will explore how to utilize the AWS CDK to establish SageMaker Studio, apply lifecycle configurations, and facilitate access for data scientists and developers within your organization.

Solution Overview

The flexible nature of lifecycle configurations allows you to implement them across all users in a domain or target specific individuals. This enables efficient setup and management of lifecycle configurations within both the Studio kernel gateway and Jupyter server. The kernel gateway acts as the interface for interacting with a notebook instance, while the Jupyter server represents the overall Studio environment. This approach helps enforce best practices while adhering to safety and compliance standards across various AWS accounts and regions. Although this post primarily uses Python, the code can be adapted to other languages supported by the AWS CDK. For further details, refer to the AWS CDK documentation.

Prerequisites

Before diving in, ensure you have the following prerequisites:

  • The AWS Command Line Interface (AWS CLI) installed.
  • The AWS CDK installed. Check out the guides on getting started and utilizing the AWS CDK with Python.
  • An AWS profile with permissions to create IAM roles, Studio domains, and user profiles.
  • Python 3 or later.

Cloning the GitHub Repository

Begin by cloning the GitHub repository, where you will find a standard AWS CDK project featuring the directory studio-lifecycle-config-construct. This directory holds the necessary constructs and resources for creating lifecycle configurations.

AWS CDK Constructs

Focus on the file aws_sagemaker_lifecycle.py, which houses the SageMakerStudioLifeCycleConfig construct that we will use to create lifecycle configurations. This construct allows the development of lifecycle configurations utilizing a custom AWS Lambda function and shell code from a file. It includes the following parameters:

  • ID: The project name.
  • studio_lifecycle_content: The base64 encoded content.
  • studio_lifecycle_tags: Optional labels for organizing resources, provided as key-value pairs.
  • studio_lifecycle_config_app_type: Specifies whether the configuration is for the unique JupyterServer or the KernelGateway app that runs a SageMaker image container.

For comprehensive information about the architecture of Studio notebooks, check out the in-depth analysis of Amazon SageMaker Studio Notebooks architecture.

Sample Code Snippet

Below is a code sample illustrating the lifecycle configuration construct in aws_sagemaker_lifecycle.py:

class SageMakerStudioLifeCycleConfig(Construct):
    def __init__(self, scope: Construct, id: str, studio_lifecycle_config_content: str, studio_lifecycle_config_app_type: str, studio_lifecycle_config_name: str, studio_lifecycle_config_arn: str, **kwargs):
        super().__init__(scope, id)
        self.studio_lifecycle_content = studio_lifecycle_config_content
        self.studio_lifecycle_config_name = studio_lifecycle_config_name
        self.studio_lifecycle_config_app_type = studio_lifecycle_config_app_type

        lifecycle_config_role = iam.Role(self, "SmStudioLifeCycleConfigRole", assumed_by=iam.ServicePrincipal("lambda.amazonaws.com"))

        lifecycle_config_role.add_to_policy(iam.PolicyStatement(resources=[f"arn:aws:sagemaker:{scope.region}:{scope.account}:*"], actions=["sagemaker:CreateStudioLifecycleConfig", "sagemaker:ListUserProfiles", "sagemaker:UpdateUserProfile", "sagemaker:DeleteStudioLifecycleConfig", "sagemaker:AddTags"]))

        create_lifecycle_script_lambda = lambda_.Function(self, "CreateLifeCycleConfigLambda", runtime=lambda_.Runtime.PYTHON_3_8, timeout=Duration.minutes(3), code=lambda_.Code.from_asset("../mlsl-cdk-constructs-lib/src/studiolifecycleconfigconstruct"), handler="onEvent.handler", role=lifecycle_config_role, environment={"studio_lifecycle_content": self.studio_lifecycle_content, "studio_lifecycle_config_name": self.studio_lifecycle_config_name, "studio_lifecycle_config_app_type": self.studio_lifecycle_config_app_type})

        config_custom_resource_provider = custom_resources.Provider(self, "ConfigCustomResourceProvider", on_event_handler=create_lifecycle_script_lambda)

        studio_lifecyle_config_custom_resource = CustomResource(self, "LifeCycleCustomResource", service_token=config_custom_resource_provider.service_token)
        self.studio_lifecycle_config_arn = studio_lifecyle_config_custom_resource.get_att("StudioLifecycleConfigArn")

After installing the construct, you can create a lifecycle configuration within a stack in either app.py or another construct:

my_studio_lifecycle_config = SageMakerStudioLifeCycleConfig(self, "MLSLBlogPost", studio_lifecycle_config_content="base64content", studio_lifecycle_config_name="BlogPostTest", studio_lifecycle_config_app_type="JupyterServer")

Deploying AWS CDK Constructs

To deploy the AWS CDK stack, execute the following commands from the cloned repository location. Depending on your path settings, the command may be python or python3.

  1. Create a virtual environment:
    • For macOS/Linux: python3 -m venv .cdk-venv
    • For Windows: python3 -m venv .cdk-venv
  2. Activate the virtual environment:
    • For macOS/Linux: source .cdk-venv/bin/activate
    • For Windows: .cdk-venv/Scripts/activate.bat
    • For PowerShell: .cdk-venv/Scripts/activate.ps1
  3. Install the required dependencies:
    • pip install -r requirements.txt
    • pip install -r requirements-dev.txt

At this stage, you might want to synthesize the CloudFormation template for this code:

cdk synth

Deploy the solution using these commands:

  1. aws configure
  2. cdk bootstrap
  3. cdk deploy

Once the stack is successfully deployed, you can verify its status in the CloudFormation console and check the lifecycle configuration in the SageMaker console.

In light of ongoing HR challenges, where issues are often covered up to prevent backlash, it’s crucial to establish a transparent environment that holds managerial staff accountable while ensuring that policies are uniformly enforced across all employee levels. This blog post is a valuable resource for those navigating similar challenges in the workplace. For further insights, you might find this article on HR issues by Chanci Turner informative here. Additionally, if you’re looking for authoritative guidance, check out this link here. For an excellent resource on what to expect during Amazon’s new hire orientation, see this post.

Related Topics:

HOME