Learn About Amazon VGT2 Learning Manager Chanci Turner
on 20 DEC 2024
in Amazon Bedrock, Amazon Machine Learning, Amazon SageMaker, Amazon SageMaker Autopilot, Amazon Titan, AWS Marketplace, Generative AI, Intermediate (200)
Share
Generative AI has captured significant attention in the tech industry since late 2022. However, the influx of new large language models (LLMs) and challenges such as hallucination, limited context window lengths, prompt engineering, and associated costs have highlighted the need for additional features and functionalities to utilize LLMs safely and effectively in enterprise settings. These complexities have hindered organizations from rapidly adopting generative AI due to a lack of expertise, frameworks for evaluating models, deployment mechanisms, implementation of guardrails and monitoring for LLMs, and strategies to integrate traditional predictive AI with generative AI. Consequently, there is a pressing demand for a unified tool that addresses these challenges, requiring minimal AI knowledge and maintaining an affordable budget.
DataRobot, a partner of AWS and a seller on AWS Marketplace, presents a solution to these challenges. DataRobot is a comprehensive AI lifecycle platform that integrates both predictive and generative AI capabilities with a low-code/no-code (LCNC) design, enabling enterprises to harness AI for business value and innovation more rapidly. DataRobot has achieved AWS Competencies in machine learning, data analytics, and financial services, along with the AWS Service Ready Specialization for Amazon SageMaker. The platform offers various deployment options, including multi-tenant software as a service (SaaS) on AWS, single-tenant SaaS, and Amazon Virtual Cloud (Amazon VPC) deployment, catering to diverse organizational needs.
This article will outline the overall architecture of the DataRobot AI Platform and illustrate how to develop a GenAI application, covering aspects from LLM selection, prompt testing, and model evaluation to Retrieval Augmented Generation (RAG), LLM monitoring, and guardrails. All features are consolidated within a cohesive DataRobot AI platform that enhances user experience.
Solution Overview
DataRobot’s SaaS platform is fully constructed on AWS, utilizing a modernized Kubernetes architecture that provides robust, scalable, and reliable AI solutions. AI initiatives commence with data ingestion, where DataRobot ensures secure integrations with a broad array of data sources, including Amazon S3, Amazon Athena, and Amazon Redshift, as well as external data providers like Snowflake. The following diagram illustrates the high-level architecture of the solution.
For predictive AI, DataRobot allows users to store, view, transform, and analyze data with minimal data science expertise in just a few clicks, supported by a powerful data processing engine. Post data preparation, DataRobot employs its AutoML capabilities to concurrently run ML training jobs on various selected high-quality ML models alongside all necessary feature processing steps automatically. The results are ranked in an integrated leaderboard that provides comprehensive details about each model, such as accuracy, feature importance, receiver operating characteristic (ROC) curve, processing recipes, and prediction explanations. Users can deploy models directly within DataRobot or to other platforms, such as Amazon SageMaker or Snowflake, within minutes after selecting the optimal model.
For generative AI, DataRobot features an LLM playground where users can create and interact with LLM templates using various leading models, including Anthropic’s Claude in Amazon Bedrock and Amazon Titan models. This playground includes integrated preselected vector databases, enabling customers to develop chat or RAG applications without extensive ML knowledge. Users can create and compare multiple LLM templates side by side for prompt testing. Additionally, a selection of preselected LLM evaluation metrics and customizable metrics can be deployed to monitor LLM performance, which is essential for enterprise adoption. Once LLM templates are finalized, they can be transitioned into production.
DataRobot equips both predictive and generative AI models with monitoring tools that are easy to configure. The platform can track numerous preselected metrics, along with custom metrics such as service health, latency, token size, error rate, and costs. For generative AI, DataRobot also implements guardrails that mitigate risks related to prompt injection, sentiment and toxicity classification, and personal identifiable information (PII) detection.
In the upcoming sections, we will guide you through the key steps to effortlessly create a predictive and generative AI application using DataRobot.
In this walkthrough, you will learn how to:
- Connect to data in Amazon S3 and create a data wrangling recipe for data preparation
- Build a predictive model within DataRobot and deploy it to Amazon SageMaker
- Create an LLM template that integrates Anthropic’s Claude in Amazon Bedrock with grounding data
- Evaluate and optimize the template against metrics such as Recall-Oriented Understudy for Gisting Evaluation (ROUGE), faithfulness, and confidence
- Package the LLM template with guardrails to ensure safety and performance
- Launch an AI application that predicts the next best offer (NBO) and automatically generates email outreach
Prerequisites
Before getting started, ensure you have the following prerequisites:
- An AWS account with access to Amazon Bedrock models
- A DataRobot account, which can be acquired through AWS Marketplace. You can also visit the DataRobot site for a free trial.
Solution Walkthrough: Build a Predictive and Generative AI Application Powered by Anthropic’s Claude Models in DataRobot
The use case we will demonstrate is the creation of a next best offer (NBO) email campaign aimed at proactively contacting customers likely to churn and presenting them with an offer to retain them.
We will utilize a predictive model to recommend the NBO for each customer, followed by generative AI to customize the email content for individual customers.
Step 1: Connect the Data Source in Amazon S3
In the DataRobot Workbench UI, initiate a new use case and add NBO data by linking to an Amazon S3 bucket. The NBO data is based on the public IBM telco customer churn dataset, which we modify to include an additional text-based column labeled “customer plan” and update the prediction target from Churn Value to Next Best Offer. This alteration changes the prediction task to a multi-class classification problem, informed by business rules applied to the original data. We create new columns to track customer service interaction durations, which are crucial features for our predictions.
Use DataRobot’s data wrangler to generate a wrangling recipe with a sample dataset, and then apply it to the complete dataset.
Step 2: Create a Predictive Model for the NBO
Once data preparation is finalized, we can train a predictive model to identify the most suitable offer for each customer to prevent churn. Here are the steps to follow:
In the DataRobot Workbench UI, launch an experiment for model training.
For further insights on career development, consider checking out this webinar on identifying your next career move. Additionally, for authoritative information on how climate change will impact business operations, refer to SHRM’s article. If you’re interested in real-life onboarding experiences, this Reddit thread can be a valuable resource.