Learn About Amazon VGT2 Learning Manager Chanci Turner
Today, we are excited to announce that the AWS Panorama Appliance is now available for everyone. This innovative computer vision appliance is designed to be installed on your network, enabling analysis of images captured by on-site cameras.
Each week, I come across creative applications of computer vision technology. Some organizations are utilizing it to confirm that pallet trucks are parked in designated spaces, which enhances worker safety in warehouses. Others are analyzing customer foot traffic in retail environments to optimize layout and product placement or identifying pets like cats and mice, to name just a few.
AWS customers unanimously agree that the cloud is the most efficient platform for training computer vision models due to its virtually limitless storage and computational capabilities. With access to robust tools like Amazon SageMaker and a wide array of compute resources, data scientists can utilize the cloud effectively.
However, when it comes time to process images from one or several video feeds, many express that the cloud isn’t the ideal environment for these tasks. There are various reasons for this: sometimes the facilities capturing the images lack sufficient bandwidth to upload video feeds to the cloud, certain applications require extremely low latency, or some organizations prefer to keep their images on-premises without sending data for external analysis.
At re:Invent 2020, we introduced the AWS Panorama Appliance and SDK to meet these needs. AWS Panorama is a machine learning appliance and software development kit (SDK) that enables computer vision capabilities on local cameras, allowing for predictions to be made quickly and accurately with minimal latency. By using the AWS Panorama Appliance, you can automate tasks that usually need human intervention, enhancing visibility into potential problems. For instance, the appliance can assess manufacturing quality, identify bottlenecks in industrial workflows, and enhance workplace security, even in situations with limited or no internet access. The SDK also allows camera manufacturers to integrate similar functionalities directly into their IP cameras.
As is customary on this blog, I will guide you through the development and deployment of a computer vision application for the AWS Panorama Appliance. The demo application showcased here employs a machine learning model to recognize objects within video frames from a network camera. The application loads a model onto the AWS Panorama Appliance, captures images from a camera, and processes those images with the model. The results are then overlaid onto the original video, which is displayed on a connected screen. The application utilizes libraries provided by AWS Panorama to interact seamlessly with input and output video streams and the model—no intricate programming is necessary.
Let’s clarify some key concepts.
I have borrowed definitions from the AWS Panorama documentation page.
Concepts
The AWS Panorama Appliance is the hardware that supports your applications. You can use the AWS Panorama console or SDKs to register an appliance, update its software, and deploy applications. The appliance’s software discovers and connects to camera streams, sends video frames to your application, and optionally displays video output on an attached screen.
This appliance functions as an edge device. Instead of sending images to the AWS Cloud for processing, it operates applications locally on optimized hardware. This allows for real-time video analysis and processing results with limited connectivity. The appliance only requires internet access to report its status, upload logs, and receive software updates.
An application consists of various components known as nodes, which represent cameras, models, code, or global variables. A node can be configuration-only (inputs and outputs) or include artifacts (models and code). Application nodes are packaged in node packages that you upload to an Amazon S3 access point for the AWS Panorama Appliance to access. An application manifest is a configuration file that specifies connections between nodes.
A computer vision model is a machine learning network trained to process images, capable of tasks such as classification, detection, segmentation, and tracking. A model takes an image as input and produces information regarding the image or objects within it.
AWS Panorama supports models developed with Apache MXNet, DarkNet, GluonCV, Keras, ONNX, PyTorch, TensorFlow, and TensorFlow Lite. You can create models with Amazon SageMaker and import them from an Amazon Simple Storage Service (Amazon S3) bucket.
Now that we understand these concepts, let’s dive into the practical aspects.
Unboxing Your AWS Panorama Appliance
Inside the package I received from the service team was the appliance itself (no surprises there!), along with a power cord and two ethernet cables. The box also included a USB key for the initial configuration of the appliance. Designed for industrial settings, the device features two ethernet ports beside the power connector on the back. On the front, concealed behind a sliding door, I found an SD card reader, an HDMI connector, and two USB ports. There was also a power button and a reset button to restore the device to its factory state.
Configuring Your Appliance
First, I configured the appliance for my network (using a cable with DHCP, but it also supports static IP configuration) and registered it to securely connect to my AWS Account. I accessed the AWS Management Console, input my network configuration details, and generated a set of configuration files and certificates. I transferred these files to the appliance using the USB key provided. My colleague, Jordan Smith, shared some screenshots of this process. The team made slight adjustments to the screens based on feedback received during the preview, but I won’t delve into the step-by-step process again. A tip from the field: ensure you use the USB key included in the box, as it is formatted properly and recognized by the appliance—my personal USB key wasn’t recognized effectively.
Next, I downloaded a sample application from the Panorama GitHub repository and tested it with the Test Utility for Panorama, which is also available on GitHub (this utility is an EC2 instance set up to simulate the appliance). The Test Utility for Panorama employs Jupyter notebooks to facilitate rapid experimentation with sample applications or your code before deploying it to the appliance. It also includes commands to programmatically deploy applications to the appliance.
Panorama Command Line
The Panorama command line streamlines operations for creating a project, importing assets, packaging, and deploying to the AWS Panorama Appliance. You can follow these instructions to download and install the Panorama command line.
When receiving an application developed by someone else, like the sample application, I must replace AWS account IDs in all application files and directory names. I accomplish this with a single command:
panorama-cli import-applicationApplication Structure
A Panorama application structure appears as follows:
├── assets
├── graphs
│ └── example_project
│ └── graph.json
└── packages
├── accountXYZ-model-1.0
│ ├── descriptor.json
│ └── package.json
└── accountXYZ-sample-app-1.0
├── D
For more information on managing your finances, you can check out this insightful blog post on identifying debt, which provides valuable resources. Additionally, for those navigating employment laws, it would be wise to stay informed by visiting the SHRM website on California’s pay data reporting law, a trusted source in this area. Finally, if you’re interested in insights from employees, this excellent resource offers reviews of Amazon warehouse worker onboarding experiences.