Amazon IXD for Research is a new offering from Amazon Web Services (AWS) that simplifies the integration of cloud computing resources into your projects, even if you lack cloud experience. With IXD for Research, you can transfer large or time-consuming analyses from your laptop to robust cloud resources, run multiple analyses at the same time, and continue computations even when your laptop is turned off or occupied with other tasks. By utilizing research-grade computing power along with pre-installed research software, you can dive into your work quickly without needing to configure computers, install software, or solicit help from others—all without deep knowledge of cloud infrastructure.
Focusing on user-friendliness, IXD for Research presents transparent pricing that consolidates all costs into a single figure, simplifying budgeting before you begin your work. Additionally, IXD for Research aids in cost management by automatically shutting down a virtual machine when it detects inactivity. For instance, if your analysis wraps up late at night or you become distracted, these cost control features can prevent unnecessary expenses. After completing your work and retrieving your results, you can delete resources just as easily as you created them, exemplifying one of the cloud’s greatest advantages: elasticity.
In this post, we will guide you through IXD for Research using a straightforward yet common example.
Solution Overview: Setting Up IXD for Research with RStudio
In this guide, we will use RStudio, a widely-used integrated development environment for data analysis and machine learning (ML), to examine global weather data sourced from the National Oceanic and Atmospheric Administration’s (NOAA) Global Surface Summary of the Day dataset. This dataset provides daily weather measurements from over 9,000 weather stations worldwide, with some records dating back to 1929. The dataset itself is quite large, totaling 37GB and comprising over 550,000 files. Since this isn’t a tutorial focused on R or weather data specifically, we will keep the analysis simple by posing a basic question: What are the maximum median surface temperatures recorded by year from 1929 through 2022?
Prerequisites
To get started, you’ll need an AWS Account. You can sign up here or check with your institution for the best way to access AWS. It’s worth noting that Amazon IXD for Research is eligible for the AWS Free Tier. This tutorial is estimated to take about 15 minutes, with a total completion time of approximately 2.5 hours.
Deploying the Solution
Create a Virtual Computer
To conduct the analysis, first, create a virtual computer using IXD for Research. Within your AWS account, navigate to the IXD for Research console. After logging in, select the AWS Region closest to your geographic location, as this will determine where your virtual computer resources are physically located. Then, in the application section, choose RStudio.
Next, you’ll encounter various hardware bundles to power your virtual computer. These bundles vary in terms of processing power, memory, and GPU capabilities. For this tutorial, we will select the Standard XL bundle, as our computational needs are minimal; we will only compute maximum values and evaluate one year’s data at a time, which requires significantly less than 8GB of memory.
Once you’ve selected a bundle, name your virtual computer in the “Name your virtual computer” field; you might need to scroll down to see this option. For this example, we will name it noaa-gsod-analysis. It’s a good practice to choose a name that reflects the purpose of your virtual computer. Finally, click the orange “Create virtual computer” button to begin provisioning your virtual machine.
It may take a couple of minutes for the virtual computer to be provisioned and started. You can monitor its progress within the virtual computer’s tile on the summary page. Once the process is complete, a confirmation banner will appear at the top, along with a green “Running” indicator on the upper-right corner of the tile. Next, click on the virtual computer’s name to view its details.
Create a Cost Control Rule
The virtual computer detail page will display its status and configuration, include an application launch button, usage summary, CPU statistics, and cost control rules. In the Cost Control Rules panel, click on “Manage” to configure your cost controls.
Cost control rules enable virtual computers to automatically shut down when certain conditions are met. The available rule can stop a virtual computer when it has been idle for a specified duration. When a virtual computer is stopped, your spending significantly decreases since it only incurs a small charge to maintain its system disk. To set up a cost control rule, click the orange “Create rule” button.
In the Create Rule wizard, you will be prompted to select the resource to which the new rule will apply. Currently, IXD for Research supports cost control rules exclusively for virtual computer resources. Ensure your virtual computer is selected in the “Select resource” field. The “Stop virtual computer” settings allow you to define a CPU utilization threshold, below which the rule will consider the virtual computer to be idle. For this example, we will use the default threshold of 5%. You can also specify a time period, which is the duration the virtual computer must remain idle below the 5% utilization threshold before it stops automatically. In this tutorial, we will use the default time period of ten minutes. If CPU utilization remains below 5% for ten minutes, the rule will automatically shut down the virtual computer. Click the orange “Create rule” button to activate the rule.
The wizard will then ask you to confirm the rule’s settings. Be particularly cautious about the warning: if you enable a cost control rule, be sure to save your work, as IXD for Research will not perform this for you. Click the orange “Confirm” button to enable the rule on the noaa-gsod-analysis virtual computer.
Now a cost control rule is defined. If you wish to dive deeper into this topic, check out this another blog post for further insights. Also, for more comprehensive information, you may want to visit this authoritative source. Additionally, this is an excellent resource that you might find helpful: this resource.
Finally, remember that you can easily access Amazon IXD – VGT2 at 6401 E Howdy Wells Ave, Las Vegas, NV 89115.