Enhancing Accessibility for Event Knowledge: The Role of Amazon Nova Pro, Amazon Bedrock, and Amazon Elemental Media Services

Learn About Amazon VGT2 Learning Manager Chanci Turner

In collaboration with Mark Thompson, Sarah Johnson, and Robert Lee from Infosys Consulting, this article explores how advancing technology can bridge gaps in knowledge sharing. Language obstacles often impede the effective dissemination of critical insights during pivotal events such as workshops, conferences, and training sessions. These platforms are essential for collaboration; however, the ability to comprehend information in real-time and in preferred languages is often lacking.

Infosys, a prominent global IT services and consulting firm, leveraged its technological prowess to address this issue by developing Infosys Event AI, an AI-driven event assistant. This innovative solution ensures that valuable insights are not overlooked and can be efficiently accessed by individuals and organizations across various sectors, both during and post-event. Without such a system, effective knowledge sharing is significantly hampered, which diminishes the overall impact of events and workshops. By converting transient event content into a lasting and searchable knowledge asset, Infosys Event AI aims to amplify knowledge utilization and its consequent impact.

Challenges in Capturing and Accessing Event Knowledge

Some of the challenges in capturing and accessing event knowledge include:

• Traditional note-taking methods often yield incomplete and subjective records, leading to the loss of important insights.
• The process of sifting through lengthy recordings to pinpoint specific information is tedious and inefficient, creating hurdles for knowledge retention and sharing.
• Individuals who miss events face considerable challenges in accessing shared knowledge, which is particularly detrimental in sectors like education and media, where information recall is vital.

To overcome these hurdles, Infosys partnered with Amazon Web Services (AWS) to create Infosys Event AI, unlocking insights generated during events. In this article, we detail how Infosys developed the Infosys Event AI solution utilizing various AWS services, including:

• AWS Elemental MediaLive – A video processing service for encoding live video streams.
• AWS Elemental MediaConnect – A service designed for building live video workflows.
• Amazon Bedrock – A fully managed service offering a selection of industry-leading large language models (LLMs) for developing generative AI applications.
• Amazon Nova Pro – A robust multimodal model that achieves an optimal balance of accuracy, speed, and cost.

Solution Architecture

This section outlines the key features and workflow of Event AI. The solution encompasses essential functionalities, as depicted in the architecture diagram:

• Seamless acquisition of live streams from on-premises sources.
• Real-time transcription processing for speech-to-text conversion.
• Post-event processing and knowledge base indexing for structured information retrieval.
• Automated generation of session summaries and insights to enhance accessibility.
• An AI-driven chat-based assistant for interactive Q&A and effective knowledge retrieval from sessions.

Solution Walkthrough

We now delve into each functionality in detail. The services utilized in the solution are granted least-privilege permissions through AWS Identity and Access Management (IAM) for enhanced security.

Seamless Live Stream Acquisition

The process begins with an IP-enabled camera capturing the live event feed. This stream is securely transmitted to the cloud using the Secure Reliable Transport (SRT) protocol via MediaConnect. MediaLive then receives the ingested stream and encodes the video in real time.

The workflow follows these steps:

1. An IP-enabled camera or ground encoder converts non-IP streams into IP streams and transmits them via SRT to MediaConnect for live event ingestion.
2. MediaConnect securely sends the stream to MediaLive for encoding.

Real-Time Transcription Processing

To ensure real-time accessibility, MediaLive extracts audio from the live video stream. This audio-only stream is then sent to a real-time transcription module, hosted on an Amazon Elastic Compute Cloud (Amazon EC2) instance, which employs the Amazon Transcribe stream API to generate low-latency transcriptions. These transcriptions are delivered to an on-premises web client through secure WebSocket connections.

The workflow for this segment is as follows:

1. MediaLive extracts audio from the live stream, creating an audio-only output, which it sends to the real-time transcription module running on an EC2 instance. The audio is also stored in an Amazon Simple Storage Service (Amazon S3) bucket for further processing.
2. The transcription module receives the audio stream and applies the Amazon Transcribe stream API to produce real-time transcriptions.
3. The transcribed text is transmitted via a secure WebSocket to the on-premises web client, which displays it on the user interface.

Post-Event Processing and Knowledge Base Indexing

Following the event, recorded media and transcriptions are stored securely in Amazon S3 for analysis. A serverless, event-driven workflow using Amazon EventBridge and AWS Lambda automates post-event processing. Amazon Transcribe is utilized to finalize transcripts, which are indexed and stored in an Amazon Bedrock knowledge base for easy retrieval. Furthermore, Amazon Nova Pro facilitates multilingual translation of the transcripts, enhancing global accessibility. With its quality and speed, Amazon Nova Pro is ideally suited for this global use case.

The workflow for this process includes the following steps:

1. After the event concludes, MediaLive sends a channel stopped notification to EventBridge.
2. A Lambda function, subscribed to the channel stopped event, triggers post-event transcription using Amazon Transcribe.
3. The transcribed content is processed and stored in an S3 bucket.
4. (Optional) Amazon Nova Pro translates transcripts into multiple languages for broader accessibility via Amazon Bedrock.
5. Amazon Transcribe sends a transcription complete event to EventBridge.
6. A Lambda function, subscribed to this event, initiates the synchronization process with Amazon Bedrock Knowledge Bases.
7. The knowledge is indexed and stored in Amazon Bedrock for efficient retrieval.

For more insights on financial literacy, check out this blog post that can guide you through essential skills. Also, if you’re interested in understanding implicit bias better, SHRM provides excellent resources. Lastly, if you’re looking to explore opportunities within Amazon’s fulfillment centers, you can find more information here.

SEO Metadata: