Four main factors will determine how you configure a model to optimize the efficiency of a simulation. To run a model most efficiently requires balancing these factors:
Number of grid cells
Number of constituents to be simulated
Number of physical cores available for processing
The amount of data exchanged between sub-domains in each time step
1. Number of grid cells
A general rule of thumb is that the number of cells assigned to a given core should be in the order of some thousands. A minimum of approximately 1,000 cells is needed to make an MPI computation worthwhile. An individual core would be expected to be assigned from 500 to 10,000 cells. 20,000 cells should be the maximum number of cells on any one core. For example, if we have a hydrodynamic model with 60,000 cells and 6 cores, then model could initially divided into six subdomains, each with 10,000 cells. The user should then adjust the run configuration by increasing or decreasing subdomains.
2. Number of constituents
Beside hydrodynamics, each transport process added to the simulation such as salinity, temperature, dye, sediment, toxic or water quality parameters, requires more equations to be solved. Although the processing of the constituents in each sub-domain can be parallelized using OMP, if more constituents need to be considered, then it is recommended that more domains be used.
3. Number of physical cores
The goal of the modeler should be to ensure that equal computational load is place on each processor. This means that if some models have different numbers of layers in different parts of the grid, thought should be given to how to divide the domain. For example, a Sigma-Zed model could fewer layers in some parts of the domain. The domain should then be divided in such a way that more horizontal cells are in that one sub-domain so that the total number of vertical and horizontal cells are roughly equal for each sub-domain.
4. The amount of data exchanged between sub-domains in each time step
The information at the interfaces of the subdomain is exchanged between the subdomains at each time step. If more constituents are simulated, especially for water quality problems, the amount of data exchanged between the subdomain could be very significant. This should be minimized by optimization of the number of cell connections between the subdomains at the interfaces.