The aim of this page is to describe how to configure an AWS cluster and run EFDC+ on it.
Setting up a cluster on AWS
Steps to setup AWS parallel cluster command line interface. The goal here is to install the necessary tools to setup a cluster from the command line and access the cluster from the command line.
All of this is assumed to be done with Ubuntu 18.04 and Python 3.6+, and Pip installed. It also requires having and AWS account.
Followed the guide to install AWS command line tools:
https://docs.aws.amazon.com/cli/latest/userguide/install-virtualenv.html
I installed in a virtual environment, which proved to be a good move for managing dependencies.
Additionally, install the aws-parallelcluster command line tools. Instructions found at:
https://docs.aws.amazon.com/parallelcluster/latest/ug/install.html
If you want a reference for the parallel cluster interface:
https://docs.aws.amazon.com/parallelcluster/latest/ug/aws-parallelcluster-ug.pdf
Prior to running it is helpful to have the AWS console open and logged in to your account, which can be accessed at:
https://console.aws.amazon.com/
You will need some information to connect your command line interface to your AWS account. To find this info click on your username in the top right hand side of the page and select “My Security Credentials”
Then click the “Access keys (access key ID and secret access key” in the center of a page and a drop down will appear. The click the blue “Create New Access Key” button. You will need this info to configure the aws parallel cluster, which we will look at next.
The first step is to connect your AWS account information, to do so run:
$ aws configure
Enter the info you just generated from the new access key
AWS access key ID:
Secret Access Key (unique value, can only have a max of 2 with a single aws user)
default region: use us-east-1
default output format: none
Next, to setup the parallelcluster configuration (this is the config file that determines how the cluster will look)
$ pcluster configure
I forgot all of the options that come up at this prompt. But they can all be changed later anyway. all that matters is running this script creates the parallel_config file. I have attached a sample one that should be used. I think the same one has 3 nodes, we might want to set this system up with 4.
Setting Up Intel MPI
https://docs.aws.amazon.com/parallelcluster/latest/ug/intelmpi.html