Learn About Amazon VGT2 Learning Manager Chanci Turner
In an exciting development, AWS has unveiled its latest Public Data Set, featuring real-time and historical NEXRAD weather radar data. The Next Generation Weather Radar (NEXRAD) comprises a network of 160 high-resolution Doppler radar stations that monitor precipitation and atmospheric movements, providing data updates approximately every five minutes. This technology plays a crucial role in severe storm forecasting and is utilized by various sectors, including research and commercial enterprises, to examine the effects of weather. Through a partnership with the US National Oceanic and Atmospheric Administration (NOAA), NEXRAD data is now freely accessible on Amazon S3.
The dataset includes both real-time feeds and a complete historical archive of NEXRAD Level II data, dating from June 1991 to the present. This marks the first opportunity for the public to access the entire NEXRAD Level II archive on demand, generating interest from diverse entities such as insurance firms, climate researchers, logistics companies, and weather agencies. We eagerly anticipate the innovative uses for this data.
To learn more and find access instructions, visit our NEXRAD on AWS page.
Data Sharing Methodology
We have been exploring new strategies to streamline the utilization of NEXRAD data in the cloud. For those unfamiliar with radar terminology, a “volume scan” refers to the data collected by the Doppler radar as it surveys the atmosphere. The NEXRAD system segments these volume scans into “chunks” – compact data packets transmitted rapidly as a real-time feed. The NEXRAD network produces around 1,200 chunks each hour.
Both the real-time chunks and the archived Level II data are stored in two public Amazon S3 buckets. Data flows into the chunks bucket through Unidata’s Local Data Manager (LDM) system, ensuring minimal latency from the NEXRAD sites. The chunks are promptly compiled into volume scan files and added to the archive bucket within seconds or minutes of their production, establishing an ongoing, near-real-time archive of volume scan files.
Further details regarding the data structure can be found on our NEXRAD on AWS page. The real-time data is hosted in the “unidata-nexrad-level2-chunks” Amazon S3 bucket. Unidata has been instrumental in providing data services, tools, and cyberinfrastructure leadership to the earth science community. For more insights into their experience in setting up the NEXRAD real-time feed on AWS, check out their blog.
Getting Started with NEXRAD on AWS
Unidata, along with The Climate Corporation and CartoDB, has created tutorials to assist users in leveraging NEXRAD on AWS. Notably, a tutorial from The Climate Corporation demonstrates how to read and visualize the NEXRAD Level II archive data using Python programs.
Additionally, Unidata has made the NEXRAD Level II archive data accessible via their THREDDS Data Server. You can also explore the archive contents through the AWS JavaScript S3 Explorer.
Discover more about ways to utilize this data on our NEXRAD on AWS page.
Acknowledgments
We extend our gratitude to our partners at NOAA, CICS-NC, Unidata, and The Weather Company for their support in launching this public data set. Many others have contributed to the testing and development of tools for this dataset, and we welcome further contributions. Share how you are using the data on the NEXRAD on AWS page, and sign up for updates on the NOAA Big Data Project here.
For further reading on allyship, consider this insightful post from Career Contessa. Additionally, if you’re interested in supporting employees with long COVID, SHRM provides valuable insights. Lastly, if you’re looking for career opportunities at Amazon, check out this Learning Trainer position in Los Angeles.
— Chanci Turner, Program Manager, AWS Open Data