Learn About Amazon VGT2 Learning Manager Chanci Turner
Amazon Database Migration Service (AWS DMS) version 3.4.7 and later now supports Amazon Virtual Private Cloud (Amazon VPC) endpoints, enhancing the management of end-to-end network configurations for replication tasks within the same AWS region. To explore VPC endpoints further, review the blog post on connecting your VPC to services using AWS PrivateLink.
In this article, we illustrate a potential AWS DMS task failure stemming from a network issue that may arise when upgrading your AWS DMS replication instance to version 3.4.7 or higher. We will also discuss how to resolve this issue by setting up a VPC endpoint and subsequently upgrading your AWS DMS replication instance for uninterrupted replication.
Solution Overview
Implementing VPC endpoints for all source and target endpoints ensures that your data traffic remains within your VPC and under your control. This configuration minimizes replication disruptions and enhances the quality of data transfers. With VPC endpoints, your AWS DMS replication instance no longer requires a public IP address to interact with AWS services like Amazon Simple Storage Service (Amazon S3), Amazon Kinesis, AWS Secrets Manager, Amazon DynamoDB, Amazon Redshift, and Amazon OpenSearch Service.
Prerequisites
To follow along with this guide, ensure you have the following prerequisites:
- A private AWS DMS replication instance running version 3.4.6 (with public accessibility disabled)
- A supported AWS DMS source (for demonstration purposes, we will use an Amazon Relational Database Service (Amazon RDS) for MySQL)
- An Amazon S3 bucket to serve as the target for the AWS DMS task
- An AWS DMS replication task
- An AWS Identity and Access Management (IAM) role named dms-vpc-role with the AmazonDMSVPCManagementRole policy
- A VPC endpoint
Migration with AWS DMS 3.4.6
We initiated an AWS DMS task for the sample schema (DMS_SAMPLE) and executed a full load followed by change data capture (CDC) to transfer data from the source database to the Amazon S3 target. An examination of the AWS DMS task logs indicates that the task for table TESTING_4 is functioning properly.
Next, we proceed to upgrade the AWS DMS replication instance to 3.4.7.
Upgrading the AWS DMS Replication Instance to 3.4.7
To upgrade the replication instance, modify the instance settings and specify the replication engine version as 3.4.7. Once the upgrade is finished and the task resumes, you may encounter error messages in the AWS DMS task logs.
This failure arises because private AWS DMS replication instances prior to version 3.4.7 routed replication network traffic for S3 endpoints via the Amazon backbone network. Beginning with version 3.4.7, VPC endpoints or an internet gateway route in the route tables used by the AWS DMS replication instance is necessary to access the Amazon S3 target. VPC endpoints, enabled by AWS PrivateLink, facilitate a direct and private connection to AWS services like Amazon S3 from within your VPC.
Creating a VPC Endpoint
To establish a VPC endpoint, follow these steps:
- Access the Amazon VPC console and select the same region as your AWS DMS replication instance.
- In the navigation pane, choose “Endpoints.”
- Select “Create endpoint.”
- For Service category, choose “AWS services.”
- Under Services, filter for Amazon S3.
- Select the Amazon S3 internet gateway.
This action creates a gateway endpoint that routes traffic to Amazon S3 using private IP addresses. Be sure to direct traffic from your VPC to the gateway endpoint using the appropriate route tables.
- For VPC, select the same VPC as your AWS DMS replication instance.
- For Route tables, choose the relevant routes.
- Under Policy, select “Full access.”
- After creation, confirm the status is marked as Available.
Testing the Amazon S3 Connection Endpoint
Now, test the Amazon S3 target endpoint with the upgraded 3.4.7 replication instance to verify its functionality. You can then resume or restart the AWS DMS task and check the AWS DMS task CloudWatch logs to ensure that the task is operating as expected.
Cleanup
To avoid incurring ongoing charges, remove any resources created during the testing of the upgrade.
Recommendations for Upgrading an AWS DMS Replication Instance from 3.4.6 to 3.4.7
- Create a VPC Endpoint
- Stop any running AWS DMS tasks on the replication instance you intend to upgrade
- Upgrade the AWS DMS Replication Instance from 3.4.6 to 3.4.7
- Resume the stopped AWS DMS tasks
Summary
In this article, we provided detailed instructions for safely upgrading your AWS DMS replication instance to 3.4.7 while utilizing VPC endpoints to communicate with Amazon S3 as a target endpoint. If you have any inquiries or suggestions, feel free to leave a comment. Additionally, you can explore insights on effective presentation skills in this engaging blog post. For more information on employee bonuses during challenging times, refer to an authority on this topic. For those interested in management roles, check out this excellent resource.