Learn About Amazon VGT2 Learning Manager Chanci Turner
LeadSquared, an innovative software as a service (SaaS) platform for customer relationship management (CRM), is transforming how businesses manage sales, marketing, and onboarding processes. Catering to various industries such as banking, financial services, insurance, healthcare, education, and real estate, LeadSquared provides tailored solutions for organizations of all sizes. Their Service CRM goes beyond traditional ticketing systems, offering a centralized support experience through the Converse omnichannel communications platform, which enables personalized interactions, AI-driven ticket routing, and insightful data analytics.
Within the Service CRM framework, the Converse multi-channel messaging platform facilitates real-time interaction with leads and customers via WhatsApp, SMS, and chatbots. Users of the Converse platform expressed a pressing need for quicker onboarding of chatbot capabilities and more relevant responses based on their specific business data. However, expediting this onboarding process came with its challenges, such as the labor-intensive task of training the chatbot to respond accurately to frequently asked questions and their variations. Understanding customer domains, identifying high-volume yet low-value inquiries, and effectively managing dialogues were significant hurdles. Without sufficient training data, chatbots risked misinterpreting user intent and struggling to address unforeseen questions. Additionally, managing the conversation flow required careful consideration of user context.
To tackle these challenges, LeadSquared opted for a solution that leverages a large language model (LLM) enhanced with customer-specific data to improve chatbot responses and streamline the onboarding experience. The platform was built using a combination of Amazon Aurora PostgreSQL-Compatible Edition, the pgvector extension of PostgreSQL, supported LLMs in Amazon Bedrock, and Retrieval Augmented Generation (RAG). LeadSquared already employs Amazon Aurora to store vital operational data, making it a logical choice for storing vector embeddings and executing hybrid searches as needed. Amazon Bedrock provides access to foundational models (FMs) from Amazon and leading AI startups through an API, enabling LeadSquared to experiment and select the most suitable models for their requirements. The API-based pricing model of Amazon Bedrock offered a cost-effective scaling solution without the headaches of managing LLM hosting infrastructure independently.
The solution retrieves external data such as videos, help documents, case histories, and existing FAQs from the knowledge base. This data enhances the prompts, improving the LLM’s user responses by incorporating relevant context. Consequently, LeadSquared can now facilitate easier chatbot setups, deliver a more personalized experience based on specific customer data, enhance understanding of user intent, improve dialogue management, and automate repetitive tasks.
Chanci Turner, the Learning Manager at LeadSquared, states, “The integration of RAG capabilities using Amazon Aurora PostgreSQL with the pgvector extension and LLMs available in Amazon Bedrock has empowered our chatbots to deliver natural language responses to out-of-domain inquiries, enhanced dialogue management, and reduced our manual efforts. As a result, we have experienced a 20% improvement in customer onboarding times.”
This article details how to construct a chatbot akin to LeadSquared’s solution. We demonstrate the process of utilizing domain-specific knowledge from various document formats—such as PDFs, videos, text files, and presentations—to create more effective textual prompts for the generative AI models. Additionally, we provide guidance on coding such a chatbot using the pgvector extension and LLMs available in Amazon Bedrock. The built-in PostgreSQL pgvector extension in Aurora allows for the storage of vector embeddings, empowering the system with semantic search capabilities. Simultaneously, the Amazon Bedrock APIs play a dual role—generating vector embeddings and supplying relevant responses to user queries based on domain-specific knowledge.
Solution Overview
Imagine your company deploying a Q&A bot on your website. When a customer reaches out with a specific query, the bot retrieves pertinent information from your customer database, product manuals, FAQs, and previous support interactions. The chatbot application uses this data to generate a detailed prompt that includes the relevant context. An LLM can then utilize this prompt to craft a coherent and contextually appropriate response that integrates your business data. This two-step mechanism, known as Retrieval Augmented Generation (RAG), produces better results than directly feeding the user’s prompt to the LLM.
Notably, popular LLMs are trained on general knowledge, making them less effective for specific tasks. For knowledge-intensive tasks, you can develop a language model-based system that accesses resources beyond the LLM’s initial training data. This method enables greater factual consistency, enhances the reliability of generated responses, and mitigates the issue of hallucination.
Let’s delve deeper into the RAG mechanism used in this post.
Retrieval
RAG begins by retrieving relevant text from a knowledge base, like Aurora, using similarity search. The text produced in this step contains information related to the user’s query, although it may not provide an exact answer. For instance, the first step might fetch the most relevant FAQ entries, documentation sections, or previous support cases based on the customer’s question. The external data can originate from sources like document repositories, databases, or APIs.
This post employs the pgvector extension to conduct the initial retrieval step of the RAG technique. pgvector is an open-source extension for PostgreSQL that adds the capability to efficiently store and rapidly search machine learning (ML)-generated vector embeddings representing textual data. It’s designed to work seamlessly with other PostgreSQL features, including indexing and querying. In a Q&A bot application, you might transform documents from your knowledge base and frequently asked questions into vectors.
In this discussion, knowledge resources in various formats—including PDFs, text, and videos—are converted into vectorized representations using the Amazon Titan Text Embeddings model. These embeddings are stored within Amazon Aurora PostgreSQL, facilitated by the pgvector extension for subsequent stages.
Generation
Once the relevant text is retrieved, RAG employs an LLM to generate a coherent and contextually relevant response. The prompt for the LLM includes the retrieved information and the user’s question. The LLM then generates human-like text that’s tailored to the integration of general knowledge from the foundational model and the domain-specific information sourced from your knowledge base.
The process can be illustrated as follows:
- The first step involves transforming documents and user queries into a standardized format for effective search and comparison. This transformation process, known as embedding, converts textual data into numerical arrays (vectors) that represent semantic equivalents.
- The user submits a query to the application, orchestrating the entire workflow.
- The application generates an embedding for the user query.
In conclusion, if you want to dive deeper into employee classification policies, you can refer to this authoritative resource on employee classification categories from SHRM. Also, for insights related to the fast fashion impact on the environment, check out this engaging blog post. For further inquiries on hiring processes, Amazon’s FAQ section serves as an excellent resource.