The main principle to consider when running with MPI is that EE will divide the model into several subdomains. Each subdomain will be simulated on the number of cores set by the user. How many subdomains are used to divide the model, and how to arrange those subdomains is also important and the method is explained below, however, some experimentation may be needed to determine the optimal arrangement. Guiding principles for domain decomposition for running MPI are also provided here.
It is recommended that restart files be written as infrequently as reasonably possible to avoid reduced performance as EFDC+ writes out the output files.
...
Hydraulic Structures with a downstream receiving cell. Both the upstream and the downstream cells must be in the same subdomain
Cell Connectors. Both end ends of the connectors must be in the same subdomain
Withdrawal/Return boundaries. Both the withdrawal and the return must be in the same subdomain
Flow Boundary with group redistribution of flows. All cells in the boundary group must be in the same subdomain
Model grids that do not have two columns of extra cells on the North, South, East, and West sides. It is a requirement with the MPI model that there be a buffer zone around the model cells of at least two rows and columns. Without this, the model may run but the results will be incorrect. Check this with the View Cell Map option.
To modify any of the boundaries so that they follow the restrictions mentioned above it is easiest to utilize View Cell Map for viewing and editing. This can can be used to move the cell blocks either manually or automatically.
...
EE can automatically divide the model into the subdomains entered using the Run Automatic Domain Decomposition button. The number of cells chosen in each subdomain is done such that each subdomain has roughly the same number of active cells. Selecting this will use an algorithm to divide the whole domain into the number of subdomains as defined above, i.e.. e… six subdomains from 1 x 6 cuts. Initially, EE always assigns # Subdomains in I direction to one. The user can modify this number from 1 x 6 to 6 x 1 or some other combination by selecting the up and down arrow next to the text box.
...
The number of cells contained in each subdomain is based on the number of rows and grid columns in the model domain. Figure 3 shows an example of a model domain divided into four subdomains, this model domain has 46 rows and 89 columns. For this example, EE requires the total grid rows to add to 46 and total grid colums columns to add to 89 in this case. So, the user can divide the 46 rows into 24 + 22, and the 89 columns into 55 + 34.
...