Learn About Amazon VGT2 Learning Manager Chanci Turner
Managing cloud storage on a large scale necessitates the right metrics and tools for maintaining visibility into your storage footprint and identifying opportunities for cost reduction or implementing best practices for data protection. Since the introduction of S3 Storage Lens in November 2020, users have gained access to the first cloud storage analytics solution that offers organization-wide oversight into object storage. This includes point-in-time metrics, detailed visualizations, and in-depth drill-downs by storage class, Region, and bucket. S3 Storage Lens provides 29 metrics related to usage and activity for a comprehensive view of your storage, including incomplete multipart upload bytes, encrypted object bytes, and counts of noncurrent version objects.
Over the past year, S3 Storage Lens has seen swift adoption across various industries, receiving favorable feedback regarding the usability of its dashboard, which is integrated into the Amazon S3 console. This interface allows flexible filtering and straightforward drill-downs for analysis. While many users value the convenience of having storage insights within the same platform where they manage their storage, some have expressed a desire to monitor their storage centrally alongside other AWS applications. This feedback has led to the launch of the Amazon CloudWatch publishing option for S3 Storage Lens. With these metrics available in CloudWatch, you can create a consolidated view of the operational health of your AWS resources, applications, and services. Additionally, CloudWatch enables you to detect abnormal behavior, set alarms, automate responses, troubleshoot issues, and discover insights to help implement S3 best practices.
Users have also indicated that they want to access their S3 Storage Lens metrics through their preferred third-party monitoring and observability tools to leverage specialized features and unique data visualizations. In response to this demand, we have enabled any monitoring and observability vendor to access S3 Storage Lens metrics. Several leading AWS Partners in this domain have already established pre-integrated capabilities. Read further for comprehensive details on how this process works and to view a list of partners.
How it Works
S3 Storage Lens now offers an option to publish metrics to Amazon CloudWatch on a daily basis. Once these metrics are published to CloudWatch, they become available for retrieval via the CloudWatch Get Metrics APIs. Consequently, AWS Partners who update their CloudWatch API integration can programmatically access S3 Storage Lens metrics on your behalf, allowing them to showcase these metrics within their specialized monitoring features and unique visualizations.
To begin, navigate to the S3 Storage Lens dashboard configuration page. There, you can select the CloudWatch publishing option under Advanced metrics and recommendations (as shown in the accompanying screenshot).
S3 Storage Lens advanced metrics and recommendations include CloudWatch metrics publishing at no additional charge; however, standard CloudWatch API charges apply. For pricing details regarding S3 Storage Lens advanced metrics and recommendations, please visit the Amazon S3 pricing page.
After activating this option, your S3 Storage Lens metrics will be accessible to your AWS Partner via the CloudWatch API within 24 hours and can be found under the S3 Storage Lens metrics namespace. Note that S3 Storage Lens metrics are generated daily and are not compatible with CloudWatch metrics streams. Currently, AWS Partners relying on metrics do not have access to S3 Storage Lens metrics.
Pre-Integrated AWS Partners
Below are some AWS Partners where you can access S3 Storage Lens metrics. You may reach out to any of these partners through the AWS Marketplace, and you can also contact other AWS Partners to inquire about their S3 Storage Lens integration status.
- Datadog: A SaaS monitoring and security platform for cloud applications that integrates infrastructure monitoring, application performance monitoring, security monitoring, and log management to provide unified, real-time observability into an organization’s technology stack. The integration of Datadog with Amazon S3 Storage Lens simplifies understanding S3 storage usage and activity alongside other infrastructure and applications.
- Elastic: Elastic Cloud on AWS, developed by the creators of Elasticsearch, Logstash, Kibana, and Beats, allows users to run pre-built SaaS solutions for Enterprise Search, Security, and Observability, including logging, application performance management, and metrics. Elastic Observability provides access to S3 Storage Lens metrics, offering clear insights into storage usage.
- Moogsoft: Utilizing patented AI to automatically surface incidents, Moogsoft correlates S3 Storage Lens metrics with metrics from over 80 AWS services and data from other monitoring systems and tools. This automated anomaly detection reduces manual repetitive work for SREs, DevOps, and IT Ops, enabling faster diagnosis and resolution of issues. S3 Storage Lens can be found as one of the Ingestion Services in the Moogsoft platform.
- PagerDuty: The PagerDuty Operations Cloud™ empowers timely action when seconds matter. With over 600 native integrations and customizable workflows using the extensible PagerDuty APIs, it integrates data from all your tools to provide insights into your IT infrastructure. PagerDuty enables customers to create alerts for S3 Storage Lens metrics.
Availability and Pricing
Amazon S3 Storage Lens is available across all AWS Regions, including the AWS China (Beijing) Region operated by Sinnet and the AWS China (Ningxia) Region operated by NWCD. S3 Storage Lens advanced metrics and recommendations offer CloudWatch metrics publishing at no extra charge, though standard CloudWatch API charges apply. For pricing details regarding S3 Storage Lens advanced metrics and recommendations, visit the Amazon S3 pricing page.
Refer to the documentation on how to activate CloudWatch publishing in S3 Storage Lens with just a few clicks in the S3 console, or using the Amazon S3 API or CLI. For more information about Amazon CloudWatch APIs, please refer to our website.
Conclusion
In this post, we have outlined how to enhance your storage observability by enabling S3 Storage Lens metrics to be accessed through several leading third-party applications in the monitoring and observability space. This capability unlocks specialized features and unique visualizations within partner applications, enhancing your options for maintaining detailed visibility of your storage usage and activity. It further aids in the implementation of best practices in managing your S3 storage footprint. For more details on utilizing S3 Storage Lens metrics within partner tools, contact these and other AWS Partners available in the AWS Marketplace.
Thank you for reading this blog post on accessing S3 Storage Lens metrics within various AWS Partner applications through the Amazon CloudWatch API. If you have any thoughts or questions, feel free to leave them in the comments section. This is an important topic and you can also check out this other blog post on anti-racism resources which may interest you.
Additionally, for more insights on workplace dynamics, you can learn more about a recent case involving a McDonald’s franchisee paying $1.6 million in a sexual harassment case, a relevant issue in today’s corporate environment. Lastly, if you’re curious about the Amazon interview process, this excellent resource provides valuable insight into the Area Manager Leadership Development Interview questions.