KMP Offset Theory

Owned by Bui Bich Thuy (Deactivated)
Last updated: 07 Nov, 2023 by Kester Scandrett
2 min read

When using OMP, it is possible to use the KMP Offset option, which is the core offset for the current run. This option is not applicable for MPI runs. The KMP Offset relates to the number of cores already being used by the system and is, therefore, usually set to zero. The KMP Offset should only be adjusted if the user wishes to run more than one model on the same machine. In order to run the second model, the KMP Offset should be set to the number of cores being used in the already running model. For example, if you are using a machine with 16 cores, 32 threads, then

  • Run 1:   # Threads = 8, KMP Offset = 0
  • Run 2:   # Threads = 8, KMP Offset = 8

This will lock each thread to a unique core.  Run 1 will be using 8 cores and Run 2 the other 8 cores.  The total CPU usage will only show 50% because those calculations are based on the total number of threads, i.e. 32.

A typical Intel I7 computer has from 4 to 6 cores. The way these cores are handled in the EFDC code is with the KMP_AFFINITY variable, defined as:

KMP_AFFINITY=[<modifier>,...]<type>[,<permute>][,<offset>]

For example, to list a machine topology map, specify "KMP_AFFINITY=verbose, none" to use a modifier of verbose and a type of none.

"KMP_AFFINITY = granular = fine, compact, 1,0", will assign individual threads to different cores. When it assigns the threads it also spreads them out. If the modifier is set to verbose, then a map of threads will also be produced.

Note that the file "0_RunEFDC" is created in the model folder each time the EFDC+ model is run. This file is also used by EE to determine whether the model ran to completion or not, as it is deleted on successful model completion. If the user opens that file then they can see the KMP offset-related parameters

Figure 1 illustrates the binding of the OpenMP thread to hardware thread contexts when specifying "KMP_AFFINITY=granularity=fine, compact".

Figure 1.  Open MP thread IDs (graphic courtesy of Intel).


To learn more about the KMP_AFFINITY environment and the specific syntax that must be used, please visit the following website:
http://software.intel.com/sites/products/documentation/hpc/composerxe/en-us/cpp/mac/optaps/common/optaps_openmp_thread_affinity.htm