KMP Offset Theory _OLD

Owned by Duy Pham
Last updated: 05 Sept, 2022
2 min read

The user may also set the KMP Offset, which is the core offset for the current run. The KMP Offset relates to the number of cores already being used by the system and is therefore usually set to zero. The KMP Offset should only be adjusted if the user wishes to run more than one model on the same machine. In order to run the second model, the KMP Offset should be set to the number of cores being used in the already running model. For example, if you are using a machine with 16 cores, 32 threads, then

Run 1:   # Threads = 8, KMP Offset = 0

Run 2:   # Threads = 8, KMP Offset = 8

This will lock each thread to a unique core.  Run 1 will be using 8 cores and Run 2 the other 8 cores.  The total CPU usage will only show 50% because those calculations are based on the total number of threads, i.e. 32.

A typical Intel I7 computer has from has 4 to 6 cores. The way these cores are handled in the EFDC code is with the KMP_AFFINITY variable, defined as:

KMP_AFFINITY=[<modifier>,...]<type>[,<permute>][,<offset>]


For example, to list a machine topology map, specify "KMP_AFFINITY=verbose, none" to use a modifier of verbose and a type of none.

"KMP_AFFINITY = granular = fine, compact, 1,0", will assign individual threads to different cores. When it assigns the threads it also spreads them out. If the modifier is set to verbose, then a map of threads will also be produced.

Note that the file "0_RunEFDC" is created in the model folder each time the EFDC+ model is run. This file is also used by EE to determine whether the model ran to completion or not, as it is deleted on successful model completion. If the user opens that file then they can see this KMP offset related parameters


  Figure 1 illustrates the binding of OpenMP thread to hardware thread contexts when specifying "KMP_AFFINITY=granularity=fine, compact".

Figure 1  Open MP thread IDs (graphic courtesy of Intel).


To learn more about the KMP_AFFINITY environment and the specific syntax that must be used please visit the following website:
http://software.intel.com/sites/products/documentation/hpc/composerxe/en-us/cpp/mac/optaps/common/optaps_openmp_thread_affinity.htm