/
Running EFDC+ with MPI

Running EFDC+ with MPI

Owned by Zander Mausolff (Deactivated)
22 Apr, 2020
2 min read

This section provides some example scripts on how to run EFDC+ with MPI

Modifying the EFDC+ Run Script

Lunching the EFDC+ executable with MPI requires several changes to the run_efdc.bat script.

The main components that need to be modified in these run scripts are as follows:

  • The mpiexec.exe executable needs to precede the EFDC+ executable.

  • the '- n # processes', this specifies the total number of MPI processes. It should match up to the total number of subdomains specified in the domain.inp file.

Additionally, depending on the if OpenMP threading is utilized an additional run time environment variable must be set. If running on a cluster system a host_file must be specified as well. To differentiate between the run options, each method is described in the sections below.

Running with MPI Only on a Single Node

To run EFDC+ with only MPI only requires the addition of the mpiexec.exe and '-n # of processes' as shown in the sample script below.

SET KMP_AFFINITY=granularity=fine,compact,1,0 TITLE Title of the model CD "C:\Location\of\Working\Directory" EE\EFDC\mpi_distribution\mpiexec -n 4 "\path\to\EFDCPlus_101_MPI.exe" -NT1 -NOP PAUSE

Running with MPI and OpenMP Only on a Single Node
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

To run with MPI and OpenMP an additional enviornment variable should be specified in the execution line, i.e., -genv I_MPI_PIN_DOMAIN=omp. An example script is given below.

SET KMP_AFFINITY=granularity=fine,compact,1,0 TITLE Title of the model CD "C:\Location\of\Working\Directory" \EE\EFDC\mpi_distribution\mpiexec -n 4 -genv I_MPI_PIN_DOMAIN=omp "\path\to\EFDCPlus_101_MPI.exe" -NT2 -NOP PAUSE

Running on MPI a Cluster (Multi Node) [Advanced Feature]

To configure the host file you need to get the IP address of the node of the DNS name. This can be accomplished by querying the DNS server.

  • Launch Windows Command Prompt by navigating to Start > Command Prompt or via Run > CMD.

  • Type NSLOOKUP and hit Enter.  The default Server is set to your local DNS, the Address will be your local IP.

To run on a cluster with multiple nodes requires the specification of a host_file. The host_file specifies a list of hosts on which to run the executable. These hosts correspond to address of each computer.

https://software.intel.com/en-us/articles/micro-cluster-setup-with-intel-mpi-a-stepwise-guide-to-setting-up-the-smallest-windows

SET KMP_AFFINITY=granularity=fine,compact,1,0 TITLE Title of the model CD "C:\Location\of\Working\Directory" EE\EFDC\mpi_distribution\mpiexec -f host_file -n 4 "\path\to\EFDCPlus_101_MPI.exe" -NT1 -NOP PAUSE

 

Related content