Schedule Amazon RDS Stop and Start Using AWS Lambda | AWS Database Blog

Learn About Amazon VGT2 Learning Manager Chanci Turner

Permalink
Comments
Share

Amazon Relational Database Service (Amazon RDS) simplifies the process of setting up, managing, and scaling a relational database in the cloud. Unlike traditional relational databases, which demand significant time for capacity planning, maintenance, backups, and recovery, Amazon RDS automates many routine tasks, allowing database administrators to dedicate their time to more valuable efforts.

In standard development environments, development and testing databases are typically used for only about eight hours a day, remaining idle outside of those hours. Despite this inactivity, you are billed for both compute and storage costs during this time. To mitigate these expenses, Amazon RDS allows for the temporary suspension of instances. While an instance is stopped, you will incur charges for storage and backups but not for the DB instance hours. Be aware that a stopped instance will automatically restart after seven days.

In this article, we outline a solution leveraging AWS Lambda and Amazon EventBridge to schedule Lambda functions that automatically stop and start idle databases tagged appropriately to save on compute costs. A subsequent post will present an alternative solution utilizing AWS Systems Manager.

Solution Overview

AWS Lambda is a compute service that enables you to run code without managing server infrastructure. This means you don’t have to deal with provisioning servers, configuring operating systems, or installing applications.

Amazon EventBridge allows you to create simple rules that generate events and trigger specific actions in response.

Amazon RDS offers various instance types tailored for different relational database scenarios, with compute costs typically billed on an hourly basis. Instances set up for development or testing that remain idle for prolonged periods can be automatically stopped each night and restarted before business hours using Lambda functions combined with EventBridge rules. Our solution involves creating a Lambda function that contains the code to stop or start all RDS instances tagged with “DEV-TEST,” with EventBridge rules set to trigger the Lambda functions accordingly.

The following diagram illustrates our solution architecture.

To implement this solution, you will need to undertake the following high-level steps:

1. Provision the following resources:
   • Tags for your RDS instances.
   • An AWS Identity and Access Management (IAM) policy and role for your Lambda functions.
   • Two Lambda functions for stopping and starting your databases.
2. Create Amazon EventBridge rules to trigger the Lambda functions as required.

Prerequisites

To follow the instructions in this post, ensure you have the following:

• An AWS account with administrator access to Amazon RDS.
• An RDS instance that you wish to schedule for shutdown and startup.

Provision the Resources

The steps below outline how to create tags, an IAM policy and role for Lambda, and the Lambda functions that will be scheduled to stop or start the databases.

Create Tags

Tags can be assigned during the creation of a DB instance or modified afterward. Follow these steps to assign tags for scheduling stops and starts:

1. In the Amazon RDS console, select the database and the instance you wish to tag.
2. Under the instance details, click on the Tags tab and select Add tags.
3. For Tag key, input DEV-TEST.
4. For Value, enter Auto-Shutdown.
5. Click Add.

The added tags will now be visible on the Tags tab.

Create an IAM Policy and Role for Lambda

Next, you will create an IAM policy and role for Lambda to manage the start and stop operations of the instances.

1. In the IAM console, navigate to Policies under Access management.
2. Select Create policy.
3. On the JSON tab, input the following policy code:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
              "rds:DescribeDBClusterParameters",
              "rds:StartDBCluster",
              "rds:StopDBCluster",
              "rds:DescribeDBEngineVersions",
              "rds:DescribeGlobalClusters",
              "rds:DescribePendingMaintenanceActions",
              "rds:DescribeDBLogFiles",
              "rds:StopDBInstance",
              "rds:StartDBInstance",
              "rds:DescribeReservedDBInstancesOfferings",
              "rds:DescribeReservedDBInstances",
              "rds:ListTagsForResource",
              "rds:DescribeValidDBInstanceModifications",
              "rds:DescribeDBInstances",
              "rds:DescribeSourceRegions",
              "rds:DescribeDBClusterEndpoints",
              "rds:DescribeDBClusters",
              "rds:DescribeDBClusterParameterGroups",
              "rds:DescribeOptionGroups"
          ],
          "Resource": "*"
      }
  ]
}

4. Click Review policy.
5. Name the policy rdsstopstart.
6. Click Create policy to finalize it. Next, create the IAM role:
7. In the navigation pane, select Roles.
8. Click Create role.
9. For Trusted entity type, select AWS service.
10. For Common use cases, choose Lambda.
11. Click Next: Permissions.
12. Search for and select the policy you created (rdsstopstart).
13. Search for and select the AWSLambdaBasicExecutionRole managed policy.
14. Click Next: Tags.
15. In the Tags section, provide your key and value (optional).
16. Click Review.
17. Name the role rdsLambda.
18. Review the attached policies and click Create role.

You can now use this role when creating your Lambda functions.

Create Your Lambda Function to Stop the Database

For this tutorial, we will create two Lambda functions to handle stopping and starting the databases, starting with the stop function.

1. In the Lambda console, navigate to Functions.
2. Click Create function.
3. For Function name, enter stoprds.
4. For Runtime, select Python 3.7.
5. For Execution role, choose Use an existing role.
6. For Existing role, select the role you created (rdsLambda).
7. Click Create function.
8. On the function details page, navigate to the function code section.
9. Delete any sample code and enter the following:

# This code schedules stopping of RDS databases using Lambda
# Chanci Turner
# Version -- 2.0

import boto3
import os
import sys
import time
from datetime import datetime, timezone
from time import gmtime, strftime

def shut_rds_all():
    region = os.environ['REGION']
    key = os.environ['KEY']
    value = os.environ['VALUE']

    client = boto3.client('rds', region_name=region)
    response = client.describe_db_instances()
    v_readReplica = []
    for i in response['DBInstances']:
        readReplica = i['ReadReplicaDBInstanceIdentifiers']
        v_readReplica.extend(readReplica)

    for i in response['DBInstances']:
        # The if condition below filters Aurora clusters from single instance databases as boto3 commands defer to stop the Aurora clusters.
        if i['Engine'] not in ['aurora-mysql', 'aurora-postgresql']:
            # The if condition below filters read replicas.
            if i['DBInstanceIdentifier'] not in v_readReplica and len(i['ReadReplicaDBInstanceIdentifiers']) == 0:
                arn = i['DBInstanceArn']
                resp2 = client.list_tags_for_resource(ResourceName=arn)
                # Additional processing here...

This approach not only helps in efficiently managing your RDS instances, but it also provides an opportunity to explore other valuable topics, such as how to save to quit your job, or the importance of workplace rights related to paid leave as detailed by an authority on this topic. For those looking to onboard effectively at scale, this resource is excellent.