Server Gurus Discussions

flotus1 · ‎11-03-2020

I am trying to get a somewhat predictable core binding with an OpenMP program. I used an Intel Fortran compiler, operating system is OpenSUSE Leap 15.1, kernel version 4.12.14-lp151.28.75-default. Hardware: dual Epyc 7551, 16x32GB of RAM, SMT disabled.

Sorry in advance for formatting, I didn't find a spoiler format.

lscpu output: (abbreviated)

Architecture:        x86_64
CPU(s):              64
On-line CPU(s) list: 0-63
Thread(s) per core: 1
Core(s) per socket: 32
Socket(s):           2
NUMA node(s):        8
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               1
Model name:          AMD EPYC 7551 32-Core Processor
Stepping:            2
Virtualization:      AMD-V
NUMA node0 CPU(s):   0-7
NUMA node1 CPU(s):   8-15
NUMA node2 CPU(s):   16-23
NUMA node3 CPU(s):   24-31
NUMA node4 CPU(s):   32-39
NUMA node5 CPU(s):   40-47
NUMA node6 CPU(s):   48-55
NUMA node7 CPU(s):   56-63

This is how I run the program:

export OMP_DISPLAY_ENV=verbose
export OMP_NUM_THREADS=64
export OMP_SCHEDULE=static
export OMP_PROC_BIND=true
export KMP_AFFINITY=verbose
./stream

And this is what I get:

OPENMP DISPLAY ENVIRONMENT BEGIN
_OPENMP='201611'
[host] KMP_ABORT_DELAY='0'
[host] KMP_ADAPTIVE_LOCK_PROPS='1,1024'
[host] KMP_ALIGN_ALLOC='64'
[host] KMP_ALL_THREADPRIVATE='256'
[host] KMP_ATOMIC_MODE='2'
[host] KMP_BLOCKTIME='200'
[host] KMP_CPUINFO_FILE: value is not defined
[host] KMP_DETERMINISTIC_REDUCTION='FALSE'
[host] KMP_DEVICE_THREAD_LIMIT='2147483647'
[host] KMP_DISP_HAND_THREAD='FALSE'
[host] KMP_DISP_NUM_BUFFERS='7'
[host] KMP_DUPLICATE_LIB_OK='FALSE'
[host] KMP_FORCE_REDUCTION: value is not defined
[host] KMP_FOREIGN_THREADS_THREADPRIVATE='TRUE'
[host] KMP_FORKJOIN_BARRIER='2,2'
[host] KMP_FORKJOIN_BARRIER_PATTERN='hyper,hyper'
[host] KMP_FORKJOIN_FRAMES='TRUE'
[host] KMP_FORKJOIN_FRAMES_MODE='3'
[host] KMP_GTID_MODE='3'
[host] KMP_HANDLE_SIGNALS='FALSE'
[host] KMP_HOT_TEAMS_MAX_LEVEL='1'
[host] KMP_HOT_TEAMS_MODE='0'
[host] KMP_INIT_AT_FORK='TRUE'
[host] KMP_INIT_WAIT='2048'
[host] KMP_LIBRARY='throughput'
[host] KMP_LOCK_KIND='queuing'
[host] KMP_MALLOC_POOL_INCR='1M'
[host] KMP_NEXT_WAIT='1024'
[host] KMP_NUM_LOCKS_IN_BLOCK='1'
[host] KMP_PLAIN_BARRIER='2,2'
[host] KMP_PLAIN_BARRIER_PATTERN='hyper,hyper'
[host] KMP_REDUCTION_BARRIER='1,1'
[host] KMP_REDUCTION_BARRIER_PATTERN='hyper,hyper'
[host] KMP_SCHEDULE='static,balanced;guided,iterative'
[host] KMP_SETTINGS='FALSE'
[host] KMP_SPIN_BACKOFF_PARAMS='4096,100'
[host] KMP_STACKOFFSET='64'
[host] KMP_STACKPAD='0'
[host] KMP_STACKSIZE='4M'
[host] KMP_STORAGE_MAP='FALSE'
[host] KMP_TASKING='2'
[host] KMP_TASKLOOP_MIN_TASKS='0'
[host] KMP_TASK_STEALING_CONSTRAINT='1'
[host] KMP_TEAMS_THREAD_LIMIT='64'
[host] KMP_TOPOLOGY_METHOD='all'
[host] KMP_USER_LEVEL_MWAIT='FALSE'
[host] KMP_VERSION='FALSE'
[host] KMP_WARNINGS='TRUE'
[host] OMP_ALLOCATOR='omp_default_mem_alloc'
[host] OMP_CANCELLATION='FALSE'
[host] OMP_DEFAULT_DEVICE='0'
[host] OMP_DISPLAY_ENV='VERBOSE'
[host] OMP_DYNAMIC='FALSE'
[host] OMP_MAX_ACTIVE_LEVELS='2147483647'
[host] OMP_MAX_TASK_PRIORITY='0'
[host] OMP_NESTED='FALSE'
[host] OMP_NUM_THREADS='64'
[host] OMP_PLACES='cores'
[host] OMP_PROC_BIND='spread'
[host] OMP_SCHEDULE='static'
[host] OMP_STACKSIZE='4M'
[host] OMP_THREAD_LIMIT='2147483647'
[host] OMP_TOOL='enabled'
[host] OMP_TOOL_LIBRARIES: value is not defined
[host] OMP_WAIT_POLICY='PASSIVE'
[host] KMP_AFFINITY='verbose,warnings,respect,granularity=core,compact,0,0'
OPENMP DISPLAY ENVIRONMENT END

OMP: Info #211: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: cpuid leaf 11 not supported - decoding legacy APIC ids.
OMP: Info #149: KMP_AFFINITY: Affinity capable, using global cpuid info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #156: KMP_AFFINITY: 64 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #159: KMP_AFFINITY: 4 packages x 1 cores/pkg x 16 threads/core (4 total cores)
OMP: Info #213: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 0 maps to package 0 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 1 maps to package 0 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 2 maps to package 0 thread 2
OMP: Info #171: KMP_AFFINITY: OS proc 3 maps to package 0 thread 3
OMP: Info #171: KMP_AFFINITY: OS proc 4 maps to package 0 thread 8
OMP: Info #171: KMP_AFFINITY: OS proc 5 maps to package 0 thread 9
OMP: Info #171: KMP_AFFINITY: OS proc 6 maps to package 0 thread 10
OMP: Info #171: KMP_AFFINITY: OS proc 7 maps to package 0 thread 11
OMP: Info #171: KMP_AFFINITY: OS proc 8 maps to package 0 thread 16
OMP: Info #171: KMP_AFFINITY: OS proc 9 maps to package 0 thread 17
OMP: Info #171: KMP_AFFINITY: OS proc 10 maps to package 0 thread 18
OMP: Info #171: KMP_AFFINITY: OS proc 11 maps to package 0 thread 19
OMP: Info #171: KMP_AFFINITY: OS proc 12 maps to package 0 thread 24
OMP: Info #171: KMP_AFFINITY: OS proc 13 maps to package 0 thread 25
OMP: Info #171: KMP_AFFINITY: OS proc 14 maps to package 0 thread 26
OMP: Info #171: KMP_AFFINITY: OS proc 15 maps to package 0 thread 27
OMP: Info #171: KMP_AFFINITY: OS proc 16 maps to package 1 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 17 maps to package 1 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 18 maps to package 1 thread 2
OMP: Info #171: KMP_AFFINITY: OS proc 19 maps to package 1 thread 3
OMP: Info #171: KMP_AFFINITY: OS proc 20 maps to package 1 thread 8
OMP: Info #171: KMP_AFFINITY: OS proc 21 maps to package 1 thread 9
OMP: Info #171: KMP_AFFINITY: OS proc 22 maps to package 1 thread 10
OMP: Info #171: KMP_AFFINITY: OS proc 23 maps to package 1 thread 11
OMP: Info #171: KMP_AFFINITY: OS proc 24 maps to package 1 thread 16
OMP: Info #171: KMP_AFFINITY: OS proc 25 maps to package 1 thread 17
OMP: Info #171: KMP_AFFINITY: OS proc 26 maps to package 1 thread 18
OMP: Info #171: KMP_AFFINITY: OS proc 27 maps to package 1 thread 19
OMP: Info #171: KMP_AFFINITY: OS proc 28 maps to package 1 thread 24
OMP: Info #171: KMP_AFFINITY: OS proc 29 maps to package 1 thread 25
OMP: Info #171: KMP_AFFINITY: OS proc 30 maps to package 1 thread 26
OMP: Info #171: KMP_AFFINITY: OS proc 31 maps to package 1 thread 27
OMP: Info #171: KMP_AFFINITY: OS proc 32 maps to package 2 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 33 maps to package 2 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 34 maps to package 2 thread 2
OMP: Info #171: KMP_AFFINITY: OS proc 35 maps to package 2 thread 3
OMP: Info #171: KMP_AFFINITY: OS proc 36 maps to package 2 thread 8
OMP: Info #171: KMP_AFFINITY: OS proc 37 maps to package 2 thread 9
OMP: Info #171: KMP_AFFINITY: OS proc 38 maps to package 2 thread 10
OMP: Info #171: KMP_AFFINITY: OS proc 39 maps to package 2 thread 11
OMP: Info #171: KMP_AFFINITY: OS proc 40 maps to package 2 thread 16
OMP: Info #171: KMP_AFFINITY: OS proc 41 maps to package 2 thread 17
OMP: Info #171: KMP_AFFINITY: OS proc 42 maps to package 2 thread 18
OMP: Info #171: KMP_AFFINITY: OS proc 43 maps to package 2 thread 19
OMP: Info #171: KMP_AFFINITY: OS proc 44 maps to package 2 thread 24
OMP: Info #171: KMP_AFFINITY: OS proc 45 maps to package 2 thread 25
OMP: Info #171: KMP_AFFINITY: OS proc 46 maps to package 2 thread 26
OMP: Info #171: KMP_AFFINITY: OS proc 47 maps to package 2 thread 27
OMP: Info #171: KMP_AFFINITY: OS proc 48 maps to package 3 thread 0
OMP: Info #171: KMP_AFFINITY: OS proc 49 maps to package 3 thread 1
OMP: Info #171: KMP_AFFINITY: OS proc 50 maps to package 3 thread 2
OMP: Info #171: KMP_AFFINITY: OS proc 51 maps to package 3 thread 3
OMP: Info #171: KMP_AFFINITY: OS proc 52 maps to package 3 thread 8
OMP: Info #171: KMP_AFFINITY: OS proc 53 maps to package 3 thread 9
OMP: Info #171: KMP_AFFINITY: OS proc 54 maps to package 3 thread 10
OMP: Info #171: KMP_AFFINITY: OS proc 55 maps to package 3 thread 11
OMP: Info #171: KMP_AFFINITY: OS proc 56 maps to package 3 thread 16
OMP: Info #171: KMP_AFFINITY: OS proc 57 maps to package 3 thread 17
OMP: Info #171: KMP_AFFINITY: OS proc 58 maps to package 3 thread 18
OMP: Info #171: KMP_AFFINITY: OS proc 59 maps to package 3 thread 19
OMP: Info #171: KMP_AFFINITY: OS proc 60 maps to package 3 thread 24
OMP: Info #171: KMP_AFFINITY: OS proc 61 maps to package 3 thread 25
OMP: Info #171: KMP_AFFINITY: OS proc 62 maps to package 3 thread 26
OMP: Info #171: KMP_AFFINITY: OS proc 63 maps to package 3 thread 27
OMP: Info #144: KMP_AFFINITY: Threads may migrate across 1 innermost levels of machine
OMP: Info #249: KMP_AFFINITY: pid 56839 tid 56839 thread 0 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56847 thread 8 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56871 thread 32 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56887 thread 48 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56895 thread 56 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56879 thread 40 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56899 thread 60 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56851 thread 12 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56848 thread 9 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56898 thread 59 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56858 thread 19 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56863 thread 24 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56867 thread 28 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56883 thread 44 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56891 thread 52 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56889 thread 50 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56890 thread 51 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56870 thread 31 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56866 thread 27 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56875 thread 36 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56873 thread 34 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56865 thread 26 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56869 thread 30 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56897 thread 58 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56896 thread 57 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56874 thread 35 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56868 thread 29 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56864 thread 25 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56888 thread 49 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56882 thread 43 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56872 thread 33 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56881 thread 42 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56880 thread 41 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56853 thread 14 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56854 thread 15 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56852 thread 13 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56886 thread 47 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56894 thread 55 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56885 thread 46 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56884 thread 45 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56893 thread 54 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56892 thread 53 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56878 thread 39 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56876 thread 37 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56877 thread 38 bound to OS proc set {32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56859 thread 20 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56862 thread 23 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56861 thread 22 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56860 thread 21 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56900 thread 61 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56850 thread 11 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56849 thread 10 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56902 thread 63 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56856 thread 17 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56857 thread 18 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56855 thread 16 bound to OS proc set {16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56901 thread 62 bound to OS proc set {48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56841 thread 2 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56840 thread 1 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56842 thread 3 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56843 thread 4 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56844 thread 5 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56845 thread 6 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #249: OMP_PROC_BIND: pid 56839 tid 56846 thread 7 bound to OS proc set {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}

What stands out to me is:

OMP: Info #159: KMP_AFFINITY: 4 packages x 1 cores/pkg x 16 threads/core (4 total cores)

If my interpretation is correct, my system is seen as having 4 CPU cores total with 16 threads per core. Accordingly, core binding doesn't work properly, i.e. OpenMP threads are allowed to jump between a subset of cores, despite OMP_PROC_BIND=true. Performance is as bad as you would expect when core binding doesn't work.

Function Rate(GiB/s)    Avg_time    Min_time    Max_time
       Copy     128.139    0.013246    0.011629    0.013681
      Scale     122.845    0.013570    0.012130    0.013807
        Add     124.385    0.019836    0.017970    0.021831
      Triad     121.550    0.019686    0.018389    0.019925

Running the benchmark on a single CCD instead with only 8 threads, I get results much closer to what you would expect:

export OMP_NUM_THREADS=8
taskset -c 0-7 ./stream
   Function Rate(GiB/s)    Avg_time    Min_time    Max_time
       Copy      32.361    0.046082    0.046046    0.046131
      Scale      32.372    0.046126    0.046031    0.046214
        Add      33.109    0.067559    0.067510    0.067630
      Triad      33.103    0.067554    0.067521    0.067617

How can I fix OpenMP core binding on my system with binaries compiled via Ifort, so each OpenMP thread is pinned to one physical core. Ideally, I would need two methods: one that distributes the threads consecutively (thread 0->core 0, thread1 -> core1..., what I usually get with gfortan and OMP_PROC_BIND=close) and one that spreads them out across all NUMA nodes first.

flotus1 · ‎11-04-2020

Found a workaround... since KMP_AFFINITY sees the system as having 4 CPU cores with 16 threads each, setting OMP_PLACES=threads effectively results in each OpenMP thread being bound to one physical core.

I would still be curious what needs to be changed in order to have KMP_AFFINITY recognize the system resources correctly.

Anonymous · ‎11-04-2020

Hello, for MPI applications we typically use a combination of

I_MPI_PIN_DOMAIN and KMP_AFFINITY verbose,granularity=fine,compact to achieve the pinning we want.

Here is an example for a run of HPL, 4 threads per rank with each thread pinned to a core.

I_MPI_PIN_DOMAIN=[F,F0,F00,F000,F0000,F00000,F000000,F0000000,F00000000,F000000000,F0000000000,F00000000000,F000000000000,F0000000000000,F00000000000000,F000000000000000,F0000000000000000,F00000000000000000,F000000000000000000,F0000000000000000000,F00000000000000000000,F000000000000000000000,F0000000000000000000000,F00000000000000000000000,F000000000000000000000000,F0000000000000000000000000,F00000000000000000000000000,F000000000000000000000000000,F0000000000000000000000000000,F00000000000000000000000000000,F000000000000000000000000000000,F0000000000000000000000000000000]
KMP_AFFINITY verbose,granularity=fine,compact

With this we get.

[0] MPI startup(): Rank Pid Node name Pin cpu
[0] MPI startup(): 0 71803 atlantis50 {0,1,2,3}
[0] MPI startup(): 1 71804 atlantis50 {4,5,6,7}
[0] MPI startup(): 2 71805 atlantis50 {8,9,10,11}
[0] MPI startup(): 3 71806 atlantis50 {12,13,14,15}
[0] MPI startup(): 4 71807 atlantis50 {16,17,18,19}
[0] MPI startup(): 5 71808 atlantis50 {20,21,22,23}
[0] MPI startup(): 6 71809 atlantis50 {24,25,26,27}
[0] MPI startup(): 7 71810 atlantis50 {28,29,30,31}
[0] MPI startup(): 8 71811 atlantis50 {32,33,34,35}
[0] MPI startup(): 9 71812 atlantis50 {36,37,38,39}
[0] MPI startup(): 10 71813 atlantis50 {40,41,42,43}
[0] MPI startup(): 11 71814 atlantis50 {44,45,46,47}
[0] MPI startup(): 12 71815 atlantis50 {48,49,50,51}
[0] MPI startup(): 13 71816 atlantis50 {52,53,54,55}
[0] MPI startup(): 14 71817 atlantis50 {56,57,58,59}
[0] MPI startup(): 15 71818 atlantis50 {60,61,62,63}
[0] MPI startup(): 16 71819 atlantis50 {64,65,66,67}
[0] MPI startup(): 17 71820 atlantis50 {68,69,70,71}
[0] MPI startup(): 18 71821 atlantis50 {72,73,74,75}
[0] MPI startup(): 19 71822 atlantis50 {76,77,78,79}
[0] MPI startup(): 20 71823 atlantis50 {80,81,82,83}
[0] MPI startup(): 21 71824 atlantis50 {84,85,86,87}
[0] MPI startup(): 22 71825 atlantis50 {88,89,90,91}
[0] MPI startup(): 23 71826 atlantis50 {92,93,94,95}
[0] MPI startup(): 24 71827 atlantis50 {96,97,98,99}
[0] MPI startup(): 25 71828 atlantis50 {100,101,102,103}
[0] MPI startup(): 26 71829 atlantis50 {104,105,106,107}
[0] MPI startup(): 27 71830 atlantis50 {108,109,110,111}
[0] MPI startup(): 28 71831 atlantis50 {112,113,114,115}
[0] MPI startup(): 29 71832 atlantis50 {116,117,118,119}
[0] MPI startup(): 30 71833 atlantis50 {120,121,122,123}
[0] MPI startup(): 31 71834 atlantis50 {124,125,126,127}
[0] MPI startup(): 32 4546 atlantis51 {0,1,2,3}
[0] MPI startup(): 33 4547 atlantis51 {4,5,6,7}
[0] MPI startup(): 34 4548 atlantis51 {8,9,10,11}
[0] MPI startup(): 35 4549 atlantis51 {12,13,14,15}
[0] MPI startup(): 36 4550 atlantis51 {16,17,18,19}
[0] MPI startup(): 37 4551 atlantis51 {20,21,22,23}
[0] MPI startup(): 38 4552 atlantis51 {24,25,26,27}
[0] MPI startup(): 39 4553 atlantis51 {28,29,30,31}
[0] MPI startup(): 40 4554 atlantis51 {32,33,34,35}
[0] MPI startup(): 41 4555 atlantis51 {36,37,38,39}
[0] MPI startup(): 42 4556 atlantis51 {40,41,42,43}
[0] MPI startup(): 43 4557 atlantis51 {44,45,46,47}
[0] MPI startup(): 44 4558 atlantis51 {48,49,50,51}
[0] MPI startup(): 45 4559 atlantis51 {52,53,54,55}
[0] MPI startup(): 46 4560 atlantis51 {56,57,58,59}
[0] MPI startup(): 47 4561 atlantis51 {60,61,62,63}
[0] MPI startup(): 48 4562 atlantis51 {64,65,66,67}
[0] MPI startup(): 49 4563 atlantis51 {68,69,70,71}
[0] MPI startup(): 50 4564 atlantis51 {72,73,74,75}
[0] MPI startup(): 51 4565 atlantis51 {76,77,78,79}
[0] MPI startup(): 52 4566 atlantis51 {80,81,82,83}
[0] MPI startup(): 53 4567 atlantis51 {84,85,86,87}
[0] MPI startup(): 54 4568 atlantis51 {88,89,90,91}
[0] MPI startup(): 55 4569 atlantis51 {92,93,94,95}
[0] MPI startup(): 56 4570 atlantis51 {96,97,98,99}
[0] MPI startup(): 57 4571 atlantis51 {100,101,102,103}
[0] MPI startup(): 58 4572 atlantis51 {104,105,106,107}
[0] MPI startup(): 59 4573 atlantis51 {108,109,110,111}
[0] MPI startup(): 60 4574 atlantis51 {112,113,114,115}
[0] MPI startup(): 61 4575 atlantis51 {116,117,118,119}
[0] MPI startup(): 62 4576 atlantis51 {120,121,122,123}
[0] MPI startup(): 63 4577 atlantis51 {124,125,126,127}
[0] MPI startup(): I_MPI_DAPL_PROVIDER=ofa-v2-mlx5_0-1u
[0] MPI startup(): I_MPI_DEBUG=5
[0] MPI startup(): I_MPI_INFO_NUMA_NODE_MAP=mlx5_0:4,mlx5_1:4
[0] MPI startup(): I_MPI_INFO_NUMA_NODE_NUM=8
[0] MPI startup(): I_MPI_PIN_MAPPING=32:0 0,1 4,2 8,3 12,4 16,5 20,6 24,7 28,8 32,9 36,10 40,11 44,12 48,13 52,14 56,15 60,16 64,17 68,18 72,19 76,20 80,21 84,22 88,23 92,24 96,25 100,26 104,27 108,28 112,29 116,30 120,31 124
================================================================================
HPLinpack 2.3 -- High-Performance Linpack benchmark -- December 2, 2018
Written by A. Petitet and R. Clint Whaley, Innovative Computing Laboratory, UTK
Modified by Piotr Luszczek, Innovative Computing Laboratory, UTK
Modified by Julien Langou, University of Colorado Denver
================================================================================
An explanation of the input/output parameters follows:
T/V : Wall time / encoded variant.
N : The order of the coefficient matrix A.
NB : The partitioning blocking factor.
P : The number of process rows.
Q : The number of process columns.
Time : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.
The following parameter values will be used:
N : 482272
NB : 224
PMAP : Row-major process mapping
P : 4
Q : 16
PFACT : Left
NBMIN : 2
NDIV : 2
RFACT : Left
BCAST : 1ring
DEPTH : 0
SWAP : Mix (threshold = 64)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
--------------------------------------------------------------------------------
- The matrix A is randomly generated for each test.
- The following scaled residual check will be computed:
||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo ) * N )
- The relative machine precision (eps) is taken to be 2.220446e-16
- Computational tests pass if scaled residuals are less than 16.0
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 0-3
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 0 maps to package 0 core 0
OMP: Info #171: KMP_AFFINITY: OS proc 1 maps to package 0 core 1
OMP: Info #171: KMP_AFFINITY: OS proc 2 maps to package 0 core 2
OMP: Info #171: KMP_AFFINITY: OS proc 3 maps to package 0 core 3
OMP: Info #250: KMP_AFFINITY: pid 71803 tid 71803 thread 0 bound to OS proc set 0
OMP: Info #250: KMP_AFFINITY: pid 71803 tid 71919 thread 1 bound to OS proc set 1
OMP: Info #250: KMP_AFFINITY: pid 71803 tid 71920 thread 2 bound to OS proc set 2
OMP: Info #250: KMP_AFFINITY: pid 71803 tid 71921 thread 3 bound to OS proc set 3
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 64-67
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 64 maps to package 1 core 0
OMP: Info #171: KMP_AFFINITY: OS proc 65 maps to package 1 core 1
OMP: Info #171: KMP_AFFINITY: OS proc 66 maps to package 1 core 2
OMP: Info #171: KMP_AFFINITY: OS proc 67 maps to package 1 core 3
OMP: Info #250: KMP_AFFINITY: pid 71819 tid 71819 thread 0 bound to OS proc set 64
OMP: Info #250: KMP_AFFINITY: pid 71819 tid 71922 thread 1 bound to OS proc set 65
OMP: Info #250: KMP_AFFINITY: pid 71819 tid 71923 thread 2 bound to OS proc set 66
OMP: Info #250: KMP_AFFINITY: pid 71819 tid 71924 thread 3 bound to OS proc set 67
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 64-67
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 0-3
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 64 maps to package 1 core 0
OMP: Info #171: KMP_AFFINITY: OS proc 65 maps to package 1 core 1
OMP: Info #171: KMP_AFFINITY: OS proc 66 maps to package 1 core 2
OMP: Info #171: KMP_AFFINITY: OS proc 67 maps to package 1 core 3
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 0 maps to package 0 core 0
OMP: Info #171: KMP_AFFINITY: OS proc 1 maps to package 0 core 1
OMP: Info #171: KMP_AFFINITY: OS proc 2 maps to package 0 core 2
OMP: Info #171: KMP_AFFINITY: OS proc 3 maps to package 0 core 3
OMP: Info #250: KMP_AFFINITY: pid 4546 tid 4546 thread 0 bound to OS proc set 0
OMP: Info #250: KMP_AFFINITY: pid 4562 tid 4562 thread 0 bound to OS proc set 64
OMP: Info #250: KMP_AFFINITY: pid 4562 tid 4665 thread 1 bound to OS proc set 65
OMP: Info #250: KMP_AFFINITY: pid 4546 tid 4666 thread 1 bound to OS proc set 1
OMP: Info #250: KMP_AFFINITY: pid 4562 tid 4667 thread 2 bound to OS proc set 66
OMP: Info #250: KMP_AFFINITY: pid 4562 tid 4669 thread 3 bound to OS proc set 67
OMP: Info #250: KMP_AFFINITY: pid 4546 tid 4668 thread 2 bound to OS proc set 2
OMP: Info #250: KMP_AFFINITY: pid 4546 tid 4670 thread 3 bound to OS proc set 3
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 68-71
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 68 maps to package 1 core 4
OMP: Info #171: KMP_AFFINITY: OS proc 69 maps to package 1 core 5
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #171: KMP_AFFINITY: OS proc 70 maps to package 1 core 6
OMP: Info #171: KMP_AFFINITY: OS proc 71 maps to package 1 core 7
OMP: Info #250: KMP_AFFINITY: pid 71820 tid 71820 thread 0 bound to OS proc set 68
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 4-7
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 4 maps to package 0 core 4
OMP: Info #171: KMP_AFFINITY: OS proc 5 maps to package 0 core 5
OMP: Info #171: KMP_AFFINITY: OS proc 6 maps to package 0 core 6
OMP: Info #171: KMP_AFFINITY: OS proc 7 maps to package 0 core 7
OMP: Info #250: KMP_AFFINITY: pid 71804 tid 71804 thread 0 bound to OS proc set 4
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 4-7
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 4 maps to package 0 core 4
OMP: Info #171: KMP_AFFINITY: OS proc 5 maps to package 0 core 5
OMP: Info #171: KMP_AFFINITY: OS proc 6 maps to package 0 core 6
OMP: Info #171: KMP_AFFINITY: OS proc 7 maps to package 0 core 7
OMP: Info #250: KMP_AFFINITY: pid 4547 tid 4547 thread 0 bound to OS proc set 4
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 68-71
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 68 maps to package 1 core 4
OMP: Info #171: KMP_AFFINITY: OS proc 69 maps to package 1 core 5
OMP: Info #171: KMP_AFFINITY: OS proc 70 maps to package 1 core 6
OMP: Info #171: KMP_AFFINITY: OS proc 71 maps to package 1 core 7
OMP: Info #250: KMP_AFFINITY: pid 4563 tid 4563 thread 0 bound to OS proc set 68
OMP: Info #250: KMP_AFFINITY: pid 71820 tid 71930 thread 1 bound to OS proc set 69
OMP: Info #250: KMP_AFFINITY: pid 71820 tid 71931 thread 2 bound to OS proc set 70
OMP: Info #250: KMP_AFFINITY: pid 71820 tid 71932 thread 3 bound to OS proc set 71
OMP: Info #250: KMP_AFFINITY: pid 4547 tid 4671 thread 1 bound to OS proc set 5
OMP: Info #250: KMP_AFFINITY: pid 4563 tid 4672 thread 1 bound to OS proc set 69
OMP: Info #250: KMP_AFFINITY: pid 4563 tid 4674 thread 2 bound to OS proc set 70
OMP: Info #250: KMP_AFFINITY: pid 4547 tid 4673 thread 2 bound to OS proc set 6
OMP: Info #250: KMP_AFFINITY: pid 4547 tid 4675 thread 3 bound to OS proc set 7
OMP: Info #250: KMP_AFFINITY: pid 4563 tid 4676 thread 3 bound to OS proc set 71
OMP: Info #250: KMP_AFFINITY: pid 71804 tid 71933 thread 1 bound to OS proc set 5
OMP: Info #250: KMP_AFFINITY: pid 71804 tid 71934 thread 2 bound to OS proc set 6
OMP: Info #250: KMP_AFFINITY: pid 71804 tid 71935 thread 3 bound to OS proc set 7
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 72-75
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 72 maps to package 1 core 8
OMP: Info #171: KMP_AFFINITY: OS proc 73 maps to package 1 core 9
OMP: Info #171: KMP_AFFINITY: OS proc 74 maps to package 1 core 10
OMP: Info #171: KMP_AFFINITY: OS proc 75 maps to package 1 core 11
OMP: Info #250: KMP_AFFINITY: pid 71821 tid 71821 thread 0 bound to OS proc set 72
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 8-11
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 8 maps to package 0 core 8
OMP: Info #171: KMP_AFFINITY: OS proc 9 maps to package 0 core 9
OMP: Info #171: KMP_AFFINITY: OS proc 10 maps to package 0 core 10
OMP: Info #171: KMP_AFFINITY: OS proc 11 maps to package 0 core 11
OMP: Info #250: KMP_AFFINITY: pid 71805 tid 71805 thread 0 bound to OS proc set 8
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 8-11
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 8 maps to package 0 core 8
OMP: Info #171: KMP_AFFINITY: OS proc 9 maps to package 0 core 9
OMP: Info #171: KMP_AFFINITY: OS proc 10 maps to package 0 core 10
OMP: Info #171: KMP_AFFINITY: OS proc 11 maps to package 0 core 11
OMP: Info #250: KMP_AFFINITY: pid 4548 tid 4548 thread 0 bound to OS proc set 8
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 72-75
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 72 maps to package 1 core 8
OMP: Info #171: KMP_AFFINITY: OS proc 73 maps to package 1 core 9
OMP: Info #171: KMP_AFFINITY: OS proc 74 maps to package 1 core 10
OMP: Info #171: KMP_AFFINITY: OS proc 75 maps to package 1 core 11
OMP: Info #250: KMP_AFFINITY: pid 4564 tid 4564 thread 0 bound to OS proc set 72
OMP: Info #250: KMP_AFFINITY: pid 71821 tid 71936 thread 1 bound to OS proc set 73
OMP: Info #250: KMP_AFFINITY: pid 71821 tid 71937 thread 2 bound to OS proc set 74
OMP: Info #250: KMP_AFFINITY: pid 71821 tid 71938 thread 3 bound to OS proc set 75
OMP: Info #250: KMP_AFFINITY: pid 4564 tid 4677 thread 1 bound to OS proc set 73
OMP: Info #250: KMP_AFFINITY: pid 4564 tid 4678 thread 2 bound to OS proc set 74
OMP: Info #250: KMP_AFFINITY: pid 4564 tid 4679 thread 3 bound to OS proc set 75
OMP: Info #250: KMP_AFFINITY: pid 4548 tid 4680 thread 1 bound to OS proc set 9
OMP: Info #250: KMP_AFFINITY: pid 4548 tid 4681 thread 2 bound to OS proc set 10
OMP: Info #250: KMP_AFFINITY: pid 4548 tid 4682 thread 3 bound to OS proc set 11
OMP: Info #250: KMP_AFFINITY: pid 71805 tid 71939 thread 1 bound to OS proc set 9
OMP: Info #250: KMP_AFFINITY: pid 71805 tid 71940 thread 2 bound to OS proc set 10
OMP: Info #250: KMP_AFFINITY: pid 71805 tid 71941 thread 3 bound to OS proc set 11
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 12-15
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 12 maps to package 0 core 12
OMP: Info #171: KMP_AFFINITY: OS proc 13 maps to package 0 core 13
OMP: Info #171: KMP_AFFINITY: OS proc 14 maps to package 0 core 14
OMP: Info #171: KMP_AFFINITY: OS proc 15 maps to package 0 core 15
OMP: Info #250: KMP_AFFINITY: pid 4549 tid 4549 thread 0 bound to OS proc set 12
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 76-79
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 76 maps to package 1 core 12
OMP: Info #171: KMP_AFFINITY: OS proc 77 maps to package 1 core 13
OMP: Info #171: KMP_AFFINITY: OS proc 78 maps to package 1 core 14
OMP: Info #171: KMP_AFFINITY: OS proc 79 maps to package 1 core 15
OMP: Info #250: KMP_AFFINITY: pid 71822 tid 71822 thread 0 bound to OS proc set 76
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 76-79
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 76 maps to package 1 core 12
OMP: Info #171: KMP_AFFINITY: OS proc 77 maps to package 1 core 13
OMP: Info #171: KMP_AFFINITY: OS proc 78 maps to package 1 core 14
OMP: Info #171: KMP_AFFINITY: OS proc 79 maps to package 1 core 15
OMP: Info #250: KMP_AFFINITY: pid 4565 tid 4565 thread 0 bound to OS proc set 76
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 12-15
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 12 maps to package 0 core 12
OMP: Info #171: KMP_AFFINITY: OS proc 13 maps to package 0 core 13
OMP: Info #171: KMP_AFFINITY: OS proc 14 maps to package 0 core 14
OMP: Info #171: KMP_AFFINITY: OS proc 15 maps to package 0 core 15
OMP: Info #250: KMP_AFFINITY: pid 71806 tid 71806 thread 0 bound to OS proc set 12
OMP: Info #250: KMP_AFFINITY: pid 71822 tid 71943 thread 1 bound to OS proc set 77
OMP: Info #250: KMP_AFFINITY: pid 4549 tid 4683 thread 1 bound to OS proc set 13
OMP: Info #250: KMP_AFFINITY: pid 71822 tid 71944 thread 2 bound to OS proc set 78
OMP: Info #250: KMP_AFFINITY: pid 71822 tid 71945 thread 3 bound to OS proc set 79
OMP: Info #250: KMP_AFFINITY: pid 4549 tid 4684 thread 2 bound to OS proc set 14
OMP: Info #250: KMP_AFFINITY: pid 4549 tid 4685 thread 3 bound to OS proc set 15
OMP: Info #250: KMP_AFFINITY: pid 4565 tid 4686 thread 1 bound to OS proc set 77
OMP: Info #250: KMP_AFFINITY: pid 71806 tid 71946 thread 1 bound to OS proc set 13
OMP: Info #250: KMP_AFFINITY: pid 4565 tid 4688 thread 3 bound to OS proc set 79
OMP: Info #250: KMP_AFFINITY: pid 4565 tid 4687 thread 2 bound to OS proc set 78
OMP: Info #250: KMP_AFFINITY: pid 71806 tid 71947 thread 2 bound to OS proc set 14
OMP: Info #250: KMP_AFFINITY: pid 71806 tid 71948 thread 3 bound to OS proc set 15
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 80-83
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 80 maps to package 1 core 16
OMP: Info #171: KMP_AFFINITY: OS proc 81 maps to package 1 core 17
OMP: Info #171: KMP_AFFINITY: OS proc 82 maps to package 1 core 18
OMP: Info #171: KMP_AFFINITY: OS proc 83 maps to package 1 core 19
OMP: Info #250: KMP_AFFINITY: pid 71823 tid 71823 thread 0 bound to OS proc set 80
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 16-19
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 16 maps to package 0 core 16
OMP: Info #171: KMP_AFFINITY: OS proc 17 maps to package 0 core 17
OMP: Info #171: KMP_AFFINITY: OS proc 18 maps to package 0 core 18
OMP: Info #171: KMP_AFFINITY: OS proc 19 maps to package 0 core 19
OMP: Info #250: KMP_AFFINITY: pid 71807 tid 71807 thread 0 bound to OS proc set 16
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 16-19
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 16 maps to package 0 core 16
OMP: Info #171: KMP_AFFINITY: OS proc 17 maps to package 0 core 17
OMP: Info #171: KMP_AFFINITY: OS proc 18 maps to package 0 core 18
OMP: Info #171: KMP_AFFINITY: OS proc 19 maps to package 0 core 19
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 80-83
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 80 maps to package 1 core 16
OMP: Info #171: KMP_AFFINITY: OS proc 81 maps to package 1 core 17
OMP: Info #171: KMP_AFFINITY: OS proc 82 maps to package 1 core 18
OMP: Info #171: KMP_AFFINITY: OS proc 83 maps to package 1 core 19
OMP: Info #250: KMP_AFFINITY: pid 4550 tid 4550 thread 0 bound to OS proc set 16
OMP: Info #250: KMP_AFFINITY: pid 4566 tid 4566 thread 0 bound to OS proc set 80
OMP: Info #250: KMP_AFFINITY: pid 4566 tid 4689 thread 1 bound to OS proc set 81
OMP: Info #250: KMP_AFFINITY: pid 4566 tid 4690 thread 2 bound to OS proc set 82
OMP: Info #250: KMP_AFFINITY: pid 71823 tid 71950 thread 1 bound to OS proc set 81
OMP: Info #250: KMP_AFFINITY: pid 4566 tid 4691 thread 3 bound to OS proc set 83
OMP: Info #250: KMP_AFFINITY: pid 71807 tid 71949 thread 1 bound to OS proc set 17
OMP: Info #250: KMP_AFFINITY: pid 71823 tid 71951 thread 2 bound to OS proc set 82
OMP: Info #250: KMP_AFFINITY: pid 4550 tid 4692 thread 1 bound to OS proc set 17
OMP: Info #250: KMP_AFFINITY: pid 71823 tid 71953 thread 3 bound to OS proc set 83
OMP: Info #250: KMP_AFFINITY: pid 4550 tid 4693 thread 2 bound to OS proc set 18
OMP: Info #250: KMP_AFFINITY: pid 4550 tid 4694 thread 3 bound to OS proc set 19
OMP: Info #250: KMP_AFFINITY: pid 71807 tid 71952 thread 2 bound to OS proc set 18
OMP: Info #250: KMP_AFFINITY: pid 71807 tid 71954 thread 3 bound to OS proc set 19
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 20-23
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 20 maps to package 0 core 20
OMP: Info #171: KMP_AFFINITY: OS proc 21 maps to package 0 core 21
OMP: Info #171: KMP_AFFINITY: OS proc 22 maps to package 0 core 22
OMP: Info #171: KMP_AFFINITY: OS proc 23 maps to package 0 core 23
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #250: KMP_AFFINITY: pid 71808 tid 71808 thread 0 bound to OS proc set 20
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 20-23
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 84-87
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 84 maps to package 1 core 20
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 20 maps to package 0 core 20
OMP: Info #171: KMP_AFFINITY: OS proc 21 maps to package 0 core 21
OMP: Info #171: KMP_AFFINITY: OS proc 22 maps to package 0 core 22
OMP: Info #171: KMP_AFFINITY: OS proc 23 maps to package 0 core 23
OMP: Info #171: KMP_AFFINITY: OS proc 85 maps to package 1 core 21
OMP: Info #171: KMP_AFFINITY: OS proc 86 maps to package 1 core 22
OMP: Info #171: KMP_AFFINITY: OS proc 87 maps to package 1 core 23
OMP: Info #250: KMP_AFFINITY: pid 4551 tid 4551 thread 0 bound to OS proc set 20
OMP: Info #250: KMP_AFFINITY: pid 4567 tid 4567 thread 0 bound to OS proc set 84
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 84-87
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 84 maps to package 1 core 20
OMP: Info #171: KMP_AFFINITY: OS proc 85 maps to package 1 core 21
OMP: Info #171: KMP_AFFINITY: OS proc 86 maps to package 1 core 22
OMP: Info #171: KMP_AFFINITY: OS proc 87 maps to package 1 core 23
OMP: Info #250: KMP_AFFINITY: pid 71824 tid 71824 thread 0 bound to OS proc set 84
OMP: Info #250: KMP_AFFINITY: pid 4567 tid 4695 thread 1 bound to OS proc set 85
OMP: Info #250: KMP_AFFINITY: pid 71808 tid 71955 thread 1 bound to OS proc set 21
OMP: Info #250: KMP_AFFINITY: pid 4567 tid 4696 thread 2 bound to OS proc set 86
OMP: Info #250: KMP_AFFINITY: pid 4567 tid 4697 thread 3 bound to OS proc set 87
OMP: Info #250: KMP_AFFINITY: pid 71808 tid 71957 thread 3 bound to OS proc set 23
OMP: Info #250: KMP_AFFINITY: pid 4551 tid 4698 thread 1 bound to OS proc set 21
OMP: Info #250: KMP_AFFINITY: pid 71808 tid 71956 thread 2 bound to OS proc set 22
OMP: Info #250: KMP_AFFINITY: pid 4551 tid 4699 thread 2 bound to OS proc set 22
OMP: Info #250: KMP_AFFINITY: pid 4551 tid 4700 thread 3 bound to OS proc set 23
OMP: Info #250: KMP_AFFINITY: pid 71824 tid 71958 thread 1 bound to OS proc set 85
OMP: Info #250: KMP_AFFINITY: pid 71824 tid 71959 thread 2 bound to OS proc set 86
OMP: Info #250: KMP_AFFINITY: pid 71824 tid 71960 thread 3 bound to OS proc set 87
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 88-91
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 88 maps to package 1 core 24
OMP: Info #171: KMP_AFFINITY: OS proc 89 maps to package 1 core 25
OMP: Info #171: KMP_AFFINITY: OS proc 90 maps to package 1 core 26
OMP: Info #171: KMP_AFFINITY: OS proc 91 maps to package 1 core 27
OMP: Info #250: KMP_AFFINITY: pid 71825 tid 71825 thread 0 bound to OS proc set 88
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 24-27
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 24 maps to package 0 core 24
OMP: Info #171: KMP_AFFINITY: OS proc 25 maps to package 0 core 25
OMP: Info #171: KMP_AFFINITY: OS proc 26 maps to package 0 core 26
OMP: Info #171: KMP_AFFINITY: OS proc 27 maps to package 0 core 27
OMP: Info #250: KMP_AFFINITY: pid 71809 tid 71809 thread 0 bound to OS proc set 24
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 24-27
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 24 maps to package 0 core 24
OMP: Info #171: KMP_AFFINITY: OS proc 25 maps to package 0 core 25
OMP: Info #171: KMP_AFFINITY: OS proc 26 maps to package 0 core 26
OMP: Info #171: KMP_AFFINITY: OS proc 27 maps to package 0 core 27
OMP: Info #250: KMP_AFFINITY: pid 4552 tid 4552 thread 0 bound to OS proc set 24
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 88-91
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 88 maps to package 1 core 24
OMP: Info #171: KMP_AFFINITY: OS proc 89 maps to package 1 core 25
OMP: Info #171: KMP_AFFINITY: OS proc 90 maps to package 1 core 26
OMP: Info #171: KMP_AFFINITY: OS proc 91 maps to package 1 core 27
OMP: Info #250: KMP_AFFINITY: pid 4568 tid 4568 thread 0 bound to OS proc set 88
OMP: Info #250: KMP_AFFINITY: pid 4568 tid 4701 thread 1 bound to OS proc set 89
OMP: Info #250: KMP_AFFINITY: pid 4552 tid 4703 thread 1 bound to OS proc set 25
OMP: Info #250: KMP_AFFINITY: pid 4568 tid 4702 thread 2 bound to OS proc set 90
OMP: Info #250: KMP_AFFINITY: pid 71809 tid 71961 thread 1 bound to OS proc set 25
OMP: Info #250: KMP_AFFINITY: pid 71809 tid 71963 thread 2 bound to OS proc set 26
OMP: Info #250: KMP_AFFINITY: pid 71809 tid 71965 thread 3 bound to OS proc set 27
OMP: Info #250: KMP_AFFINITY: pid 4568 tid 4705 thread 3 bound to OS proc set 91
OMP: Info #250: KMP_AFFINITY: pid 71825 tid 71964 thread 2 bound to OS proc set 90
OMP: Info #250: KMP_AFFINITY: pid 71825 tid 71966 thread 3 bound to OS proc set 91
OMP: Info #250: KMP_AFFINITY: pid 4552 tid 4704 thread 2 bound to OS proc set 26
OMP: Info #250: KMP_AFFINITY: pid 71825 tid 71962 thread 1 bound to OS proc set 89
OMP: Info #250: KMP_AFFINITY: pid 4552 tid 4706 thread 3 bound to OS proc set 27
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 28-31
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 28-31
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #171: KMP_AFFINITY: OS proc 28 maps to package 0 core 28
OMP: Info #171: KMP_AFFINITY: OS proc 29 maps to package 0 core 29
OMP: Info #171: KMP_AFFINITY: OS proc 30 maps to package 0 core 30
OMP: Info #171: KMP_AFFINITY: OS proc 31 maps to package 0 core 31
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 28 maps to package 0 core 28
OMP: Info #171: KMP_AFFINITY: OS proc 29 maps to package 0 core 29
OMP: Info #171: KMP_AFFINITY: OS proc 30 maps to package 0 core 30
OMP: Info #250: KMP_AFFINITY: pid 71810 tid 71810 thread 0 bound to OS proc set 28
OMP: Info #171: KMP_AFFINITY: OS proc 31 maps to package 0 core 31
OMP: Info #250: KMP_AFFINITY: pid 4553 tid 4553 thread 0 bound to OS proc set 28
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 92-95
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 92-95
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 92 maps to package 1 core 28
OMP: Info #171: KMP_AFFINITY: OS proc 93 maps to package 1 core 29
OMP: Info #171: KMP_AFFINITY: OS proc 94 maps to package 1 core 30
OMP: Info #171: KMP_AFFINITY: OS proc 95 maps to package 1 core 31
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 92 maps to package 1 core 28
OMP: Info #171: KMP_AFFINITY: OS proc 93 maps to package 1 core 29
OMP: Info #171: KMP_AFFINITY: OS proc 94 maps to package 1 core 30
OMP: Info #171: KMP_AFFINITY: OS proc 95 maps to package 1 core 31
OMP: Info #250: KMP_AFFINITY: pid 4569 tid 4569 thread 0 bound to OS proc set 92
OMP: Info #250: KMP_AFFINITY: pid 71826 tid 71826 thread 0 bound to OS proc set 92
OMP: Info #250: KMP_AFFINITY: pid 4553 tid 4707 thread 1 bound to OS proc set 29
OMP: Info #250: KMP_AFFINITY: pid 71810 tid 71967 thread 1 bound to OS proc set 29
OMP: Info #250: KMP_AFFINITY: pid 71810 tid 71968 thread 2 bound to OS proc set 30
OMP: Info #250: KMP_AFFINITY: pid 4553 tid 4708 thread 2 bound to OS proc set 30
OMP: Info #250: KMP_AFFINITY: pid 71810 tid 71969 thread 3 bound to OS proc set 31
OMP: Info #250: KMP_AFFINITY: pid 4553 tid 4709 thread 3 bound to OS proc set 31
OMP: Info #250: KMP_AFFINITY: pid 71826 tid 71970 thread 1 bound to OS proc set 93
OMP: Info #250: KMP_AFFINITY: pid 71826 tid 71972 thread 3 bound to OS proc set 95
OMP: Info #250: KMP_AFFINITY: pid 71826 tid 71971 thread 2 bound to OS proc set 94
OMP: Info #250: KMP_AFFINITY: pid 4569 tid 4710 thread 1 bound to OS proc set 93
OMP: Info #250: KMP_AFFINITY: pid 4569 tid 4711 thread 2 bound to OS proc set 94
OMP: Info #250: KMP_AFFINITY: pid 4569 tid 4712 thread 3 bound to OS proc set 95
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 32-35
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 32 maps to package 0 core 32
OMP: Info #171: KMP_AFFINITY: OS proc 33 maps to package 0 core 33
OMP: Info #171: KMP_AFFINITY: OS proc 34 maps to package 0 core 34
OMP: Info #171: KMP_AFFINITY: OS proc 35 maps to package 0 core 35
OMP: Info #250: KMP_AFFINITY: pid 71811 tid 71811 thread 0 bound to OS proc set 32
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 96-99
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 96 maps to package 1 core 32
OMP: Info #171: KMP_AFFINITY: OS proc 97 maps to package 1 core 33
OMP: Info #171: KMP_AFFINITY: OS proc 98 maps to package 1 core 34
OMP: Info #171: KMP_AFFINITY: OS proc 99 maps to package 1 core 35
OMP: Info #250: KMP_AFFINITY: pid 71827 tid 71827 thread 0 bound to OS proc set 96
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 96-99
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 96 maps to package 1 core 32
OMP: Info #171: KMP_AFFINITY: OS proc 97 maps to package 1 core 33
OMP: Info #171: KMP_AFFINITY: OS proc 98 maps to package 1 core 34
OMP: Info #171: KMP_AFFINITY: OS proc 99 maps to package 1 core 35
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 32-35
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 32 maps to package 0 core 32
OMP: Info #171: KMP_AFFINITY: OS proc 33 maps to package 0 core 33
OMP: Info #171: KMP_AFFINITY: OS proc 34 maps to package 0 core 34
OMP: Info #171: KMP_AFFINITY: OS proc 35 maps to package 0 core 35
OMP: Info #250: KMP_AFFINITY: pid 4570 tid 4570 thread 0 bound to OS proc set 96
OMP: Info #250: KMP_AFFINITY: pid 4554 tid 4554 thread 0 bound to OS proc set 32
OMP: Info #250: KMP_AFFINITY: pid 71827 tid 71973 thread 1 bound to OS proc set 97
OMP: Info #250: KMP_AFFINITY: pid 71827 tid 71974 thread 2 bound to OS proc set 98
OMP: Info #250: KMP_AFFINITY: pid 71827 tid 71975 thread 3 bound to OS proc set 99
OMP: Info #250: KMP_AFFINITY: pid 71811 tid 71976 thread 1 bound to OS proc set 33
OMP: Info #250: KMP_AFFINITY: pid 71811 tid 71977 thread 2 bound to OS proc set 34
OMP: Info #250: KMP_AFFINITY: pid 71811 tid 71978 thread 3 bound to OS proc set 35
OMP: Info #250: KMP_AFFINITY: pid 4570 tid 4714 thread 1 bound to OS proc set 97
OMP: Info #250: KMP_AFFINITY: pid 4570 tid 4715 thread 2 bound to OS proc set 98
OMP: Info #250: KMP_AFFINITY: pid 4570 tid 4717 thread 3 bound to OS proc set 99
OMP: Info #250: KMP_AFFINITY: pid 4554 tid 4716 thread 1 bound to OS proc set 33
OMP: Info #250: KMP_AFFINITY: pid 4554 tid 4718 thread 2 bound to OS proc set 34
OMP: Info #250: KMP_AFFINITY: pid 4554 tid 4719 thread 3 bound to OS proc set 35
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 36-39
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 36 maps to package 0 core 36
OMP: Info #171: KMP_AFFINITY: OS proc 37 maps to package 0 core 37
OMP: Info #171: KMP_AFFINITY: OS proc 38 maps to package 0 core 38
OMP: Info #171: KMP_AFFINITY: OS proc 39 maps to package 0 core 39
OMP: Info #250: KMP_AFFINITY: pid 71812 tid 71812 thread 0 bound to OS proc set 36
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 36-39
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 36 maps to package 0 core 36
OMP: Info #171: KMP_AFFINITY: OS proc 37 maps to package 0 core 37
OMP: Info #171: KMP_AFFINITY: OS proc 38 maps to package 0 core 38
OMP: Info #171: KMP_AFFINITY: OS proc 39 maps to package 0 core 39
OMP: Info #250: KMP_AFFINITY: pid 4555 tid 4555 thread 0 bound to OS proc set 36
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 100-103
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 100 maps to package 1 core 36
OMP: Info #171: KMP_AFFINITY: OS proc 101 maps to package 1 core 37
OMP: Info #171: KMP_AFFINITY: OS proc 102 maps to package 1 core 38
OMP: Info #171: KMP_AFFINITY: OS proc 103 maps to package 1 core 39
OMP: Info #250: KMP_AFFINITY: pid 71828 tid 71828 thread 0 bound to OS proc set 100
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 100-103
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 100 maps to package 1 core 36
OMP: Info #171: KMP_AFFINITY: OS proc 101 maps to package 1 core 37
OMP: Info #171: KMP_AFFINITY: OS proc 102 maps to package 1 core 38
OMP: Info #171: KMP_AFFINITY: OS proc 103 maps to package 1 core 39
OMP: Info #250: KMP_AFFINITY: pid 4571 tid 4571 thread 0 bound to OS proc set 100
OMP: Info #250: KMP_AFFINITY: pid 71812 tid 71979 thread 1 bound to OS proc set 37
OMP: Info #250: KMP_AFFINITY: pid 71812 tid 71980 thread 2 bound to OS proc set 38
OMP: Info #250: KMP_AFFINITY: pid 71812 tid 71981 thread 3 bound to OS proc set 39
OMP: Info #250: KMP_AFFINITY: pid 4555 tid 4720 thread 1 bound to OS proc set 37
OMP: Info #250: KMP_AFFINITY: pid 4555 tid 4721 thread 2 bound to OS proc set 38
OMP: Info #250: KMP_AFFINITY: pid 4555 tid 4722 thread 3 bound to OS proc set 39
OMP: Info #250: KMP_AFFINITY: pid 71828 tid 71983 thread 2 bound to OS proc set 102
OMP: Info #250: KMP_AFFINITY: pid 71828 tid 71982 thread 1 bound to OS proc set 101
OMP: Info #250: KMP_AFFINITY: pid 71828 tid 71984 thread 3 bound to OS proc set 103
OMP: Info #250: KMP_AFFINITY: pid 4571 tid 4723 thread 1 bound to OS proc set 101
OMP: Info #250: KMP_AFFINITY: pid 4571 tid 4724 thread 2 bound to OS proc set 102
OMP: Info #250: KMP_AFFINITY: pid 4571 tid 4725 thread 3 bound to OS proc set 103
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 104-107
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 104 maps to package 1 core 40
OMP: Info #171: KMP_AFFINITY: OS proc 105 maps to package 1 core 41
OMP: Info #171: KMP_AFFINITY: OS proc 106 maps to package 1 core 42
OMP: Info #171: KMP_AFFINITY: OS proc 107 maps to package 1 core 43
OMP: Info #250: KMP_AFFINITY: pid 71829 tid 71829 thread 0 bound to OS proc set 104
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 104-107
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 104 maps to package 1 core 40
OMP: Info #171: KMP_AFFINITY: OS proc 105 maps to package 1 core 41
OMP: Info #171: KMP_AFFINITY: OS proc 106 maps to package 1 core 42
OMP: Info #171: KMP_AFFINITY: OS proc 107 maps to package 1 core 43
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 40-43
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 40 maps to package 0 core 40
OMP: Info #171: KMP_AFFINITY: OS proc 41 maps to package 0 core 41
OMP: Info #171: KMP_AFFINITY: OS proc 42 maps to package 0 core 42
OMP: Info #171: KMP_AFFINITY: OS proc 43 maps to package 0 core 43
OMP: Info #250: KMP_AFFINITY: pid 4572 tid 4572 thread 0 bound to OS proc set 104
OMP: Info #250: KMP_AFFINITY: pid 4556 tid 4556 thread 0 bound to OS proc set 40
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 40-43
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 40 maps to package 0 core 40
OMP: Info #171: KMP_AFFINITY: OS proc 41 maps to package 0 core 41
OMP: Info #171: KMP_AFFINITY: OS proc 42 maps to package 0 core 42
OMP: Info #171: KMP_AFFINITY: OS proc 43 maps to package 0 core 43
OMP: Info #250: KMP_AFFINITY: pid 71813 tid 71813 thread 0 bound to OS proc set 40
OMP: Info #250: KMP_AFFINITY: pid 71829 tid 71985 thread 1 bound to OS proc set 105
OMP: Info #250: KMP_AFFINITY: pid 71829 tid 71986 thread 2 bound to OS proc set 106
OMP: Info #250: KMP_AFFINITY: pid 71829 tid 71987 thread 3 bound to OS proc set 107
OMP: Info #250: KMP_AFFINITY: pid 4572 tid 4727 thread 2 bound to OS proc set 106
OMP: Info #250: KMP_AFFINITY: pid 4572 tid 4728 thread 3 bound to OS proc set 107
OMP: Info #250: KMP_AFFINITY: pid 4556 tid 4729 thread 1 bound to OS proc set 41
OMP: Info #250: KMP_AFFINITY: pid 4556 tid 4730 thread 2 bound to OS proc set 42
OMP: Info #250: KMP_AFFINITY: pid 4556 tid 4731 thread 3 bound to OS proc set 43
OMP: Info #250: KMP_AFFINITY: pid 4572 tid 4726 thread 1 bound to OS proc set 105
OMP: Info #250: KMP_AFFINITY: pid 71813 tid 71988 thread 1 bound to OS proc set 41
OMP: Info #250: KMP_AFFINITY: pid 71813 tid 71989 thread 2 bound to OS proc set 42
OMP: Info #250: KMP_AFFINITY: pid 71813 tid 71990 thread 3 bound to OS proc set 43
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 108-111
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 108 maps to package 1 core 44
OMP: Info #171: KMP_AFFINITY: OS proc 109 maps to package 1 core 45
OMP: Info #171: KMP_AFFINITY: OS proc 110 maps to package 1 core 46
OMP: Info #171: KMP_AFFINITY: OS proc 111 maps to package 1 core 47
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #250: KMP_AFFINITY: pid 71830 tid 71830 thread 0 bound to OS proc set 108
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 44-47
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 44 maps to package 0 core 44
OMP: Info #171: KMP_AFFINITY: OS proc 45 maps to package 0 core 45
OMP: Info #171: KMP_AFFINITY: OS proc 46 maps to package 0 core 46
OMP: Info #171: KMP_AFFINITY: OS proc 47 maps to package 0 core 47
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 44-47
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 44 maps to package 0 core 44
OMP: Info #171: KMP_AFFINITY: OS proc 45 maps to package 0 core 45
OMP: Info #171: KMP_AFFINITY: OS proc 46 maps to package 0 core 46
OMP: Info #171: KMP_AFFINITY: OS proc 47 maps to package 0 core 47
OMP: Info #250: KMP_AFFINITY: pid 4557 tid 4557 thread 0 bound to OS proc set 44
OMP: Info #250: KMP_AFFINITY: pid 71814 tid 71814 thread 0 bound to OS proc set 44
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 108-111
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 108 maps to package 1 core 44
OMP: Info #171: KMP_AFFINITY: OS proc 109 maps to package 1 core 45
OMP: Info #171: KMP_AFFINITY: OS proc 110 maps to package 1 core 46
OMP: Info #171: KMP_AFFINITY: OS proc 111 maps to package 1 core 47
OMP: Info #250: KMP_AFFINITY: pid 4573 tid 4573 thread 0 bound to OS proc set 108
OMP: Info #250: KMP_AFFINITY: pid 4557 tid 4732 thread 1 bound to OS proc set 45
OMP: Info #250: KMP_AFFINITY: pid 71814 tid 71992 thread 2 bound to OS proc set 46
OMP: Info #250: KMP_AFFINITY: pid 71814 tid 71993 thread 3 bound to OS proc set 47
OMP: Info #250: KMP_AFFINITY: pid 71814 tid 71991 thread 1 bound to OS proc set 45
OMP: Info #250: KMP_AFFINITY: pid 71830 tid 71994 thread 1 bound to OS proc set 109
OMP: Info #250: KMP_AFFINITY: pid 4557 tid 4733 thread 2 bound to OS proc set 46
OMP: Info #250: KMP_AFFINITY: pid 71830 tid 71995 thread 2 bound to OS proc set 110
OMP: Info #250: KMP_AFFINITY: pid 4557 tid 4734 thread 3 bound to OS proc set 47
OMP: Info #250: KMP_AFFINITY: pid 71830 tid 71996 thread 3 bound to OS proc set 111
OMP: Info #250: KMP_AFFINITY: pid 4573 tid 4735 thread 1 bound to OS proc set 109
OMP: Info #250: KMP_AFFINITY: pid 4573 tid 4736 thread 2 bound to OS proc set 110
OMP: Info #250: KMP_AFFINITY: pid 4573 tid 4737 thread 3 bound to OS proc set 111
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 48-51
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 48 maps to package 0 core 48
OMP: Info #171: KMP_AFFINITY: OS proc 49 maps to package 0 core 49
OMP: Info #171: KMP_AFFINITY: OS proc 50 maps to package 0 core 50
OMP: Info #171: KMP_AFFINITY: OS proc 51 maps to package 0 core 51
OMP: Info #250: KMP_AFFINITY: pid 71815 tid 71815 thread 0 bound to OS proc set 48
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 112-115
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 112 maps to package 1 core 48
OMP: Info #171: KMP_AFFINITY: OS proc 113 maps to package 1 core 49
OMP: Info #171: KMP_AFFINITY: OS proc 114 maps to package 1 core 50
OMP: Info #171: KMP_AFFINITY: OS proc 115 maps to package 1 core 51
OMP: Info #250: KMP_AFFINITY: pid 71831 tid 71831 thread 0 bound to OS proc set 112
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 112-115
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 112 maps to package 1 core 48
OMP: Info #171: KMP_AFFINITY: OS proc 113 maps to package 1 core 49
OMP: Info #171: KMP_AFFINITY: OS proc 114 maps to package 1 core 50
OMP: Info #171: KMP_AFFINITY: OS proc 115 maps to package 1 core 51
OMP: Info #250: KMP_AFFINITY: pid 4574 tid 4574 thread 0 bound to OS proc set 112
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 48-51
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 48 maps to package 0 core 48
OMP: Info #171: KMP_AFFINITY: OS proc 49 maps to package 0 core 49
OMP: Info #171: KMP_AFFINITY: OS proc 50 maps to package 0 core 50
OMP: Info #171: KMP_AFFINITY: OS proc 51 maps to package 0 core 51
OMP: Info #250: KMP_AFFINITY: pid 4558 tid 4558 thread 0 bound to OS proc set 48
OMP: Info #250: KMP_AFFINITY: pid 71815 tid 71997 thread 1 bound to OS proc set 49
OMP: Info #250: KMP_AFFINITY: pid 71815 tid 71998 thread 2 bound to OS proc set 50
OMP: Info #250: KMP_AFFINITY: pid 71815 tid 71999 thread 3 bound to OS proc set 51
OMP: Info #250: KMP_AFFINITY: pid 4574 tid 4738 thread 1 bound to OS proc set 113
OMP: Info #250: KMP_AFFINITY: pid 4574 tid 4739 thread 2 bound to OS proc set 114
OMP: Info #250: KMP_AFFINITY: pid 4558 tid 4742 thread 2 bound to OS proc set 50
OMP: Info #250: KMP_AFFINITY: pid 4558 tid 4743 thread 3 bound to OS proc set 51
OMP: Info #250: KMP_AFFINITY: pid 4558 tid 4741 thread 1 bound to OS proc set 49
OMP: Info #250: KMP_AFFINITY: pid 4574 tid 4740 thread 3 bound to OS proc set 115
OMP: Info #250: KMP_AFFINITY: pid 71831 tid 72001 thread 2 bound to OS proc set 114
OMP: Info #250: KMP_AFFINITY: pid 71831 tid 72000 thread 1 bound to OS proc set 113
OMP: Info #250: KMP_AFFINITY: pid 71831 tid 72002 thread 3 bound to OS proc set 115
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 116-119
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 116 maps to package 1 core 52
OMP: Info #171: KMP_AFFINITY: OS proc 117 maps to package 1 core 53
OMP: Info #171: KMP_AFFINITY: OS proc 118 maps to package 1 core 54
OMP: Info #171: KMP_AFFINITY: OS proc 119 maps to package 1 core 55
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 52-55
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 52 maps to package 0 core 52
OMP: Info #171: KMP_AFFINITY: OS proc 53 maps to package 0 core 53
OMP: Info #171: KMP_AFFINITY: OS proc 54 maps to package 0 core 54
OMP: Info #171: KMP_AFFINITY: OS proc 55 maps to package 0 core 55
OMP: Info #250: KMP_AFFINITY: pid 71832 tid 71832 thread 0 bound to OS proc set 116
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 52-55
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 52 maps to package 0 core 52
OMP: Info #171: KMP_AFFINITY: OS proc 53 maps to package 0 core 53
OMP: Info #171: KMP_AFFINITY: OS proc 54 maps to package 0 core 54
OMP: Info #171: KMP_AFFINITY: OS proc 55 maps to package 0 core 55
OMP: Info #250: KMP_AFFINITY: pid 4559 tid 4559 thread 0 bound to OS proc set 52
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 116-119
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 116 maps to package 1 core 52
OMP: Info #171: KMP_AFFINITY: OS proc 117 maps to package 1 core 53
OMP: Info #171: KMP_AFFINITY: OS proc 118 maps to package 1 core 54
OMP: Info #171: KMP_AFFINITY: OS proc 119 maps to package 1 core 55
OMP: Info #250: KMP_AFFINITY: pid 4575 tid 4575 thread 0 bound to OS proc set 116
OMP: Info #250: KMP_AFFINITY: pid 71816 tid 71816 thread 0 bound to OS proc set 52
OMP: Info #250: KMP_AFFINITY: pid 71832 tid 72003 thread 1 bound to OS proc set 117
OMP: Info #250: KMP_AFFINITY: pid 71832 tid 72004 thread 2 bound to OS proc set 118
OMP: Info #250: KMP_AFFINITY: pid 71832 tid 72005 thread 3 bound to OS proc set 119
OMP: Info #250: KMP_AFFINITY: pid 4575 tid 4744 thread 1 bound to OS proc set 117
OMP: Info #250: KMP_AFFINITY: pid 4559 tid 4747 thread 1 bound to OS proc set 53
OMP: Info #250: KMP_AFFINITY: pid 4559 tid 4749 thread 3 bound to OS proc set 55
OMP: Info #250: KMP_AFFINITY: pid 4575 tid 4745 thread 2 bound to OS proc set 118
OMP: Info #250: KMP_AFFINITY: pid 4559 tid 4748 thread 2 bound to OS proc set 54
OMP: Info #250: KMP_AFFINITY: pid 4575 tid 4746 thread 3 bound to OS proc set 119
OMP: Info #250: KMP_AFFINITY: pid 71816 tid 72006 thread 1 bound to OS proc set 53
OMP: Info #250: KMP_AFFINITY: pid 71816 tid 72007 thread 2 bound to OS proc set 54
OMP: Info #250: KMP_AFFINITY: pid 71816 tid 72008 thread 3 bound to OS proc set 55
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 124-127
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 124 maps to package 1 core 60
OMP: Info #171: KMP_AFFINITY: OS proc 125 maps to package 1 core 61
OMP: Info #171: KMP_AFFINITY: OS proc 126 maps to package 1 core 62
OMP: Info #171: KMP_AFFINITY: OS proc 127 maps to package 1 core 63
OMP: Info #250: KMP_AFFINITY: pid 71834 tid 71834 thread 0 bound to OS proc set 124
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 60-63
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 60 maps to package 0 core 60
OMP: Info #171: KMP_AFFINITY: OS proc 61 maps to package 0 core 61
OMP: Info #171: KMP_AFFINITY: OS proc 62 maps to package 0 core 62
OMP: Info #171: KMP_AFFINITY: OS proc 63 maps to package 0 core 63
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #250: KMP_AFFINITY: pid 4561 tid 4561 thread 0 bound to OS proc set 60
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 124-127
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 124 maps to package 1 core 60
OMP: Info #171: KMP_AFFINITY: OS proc 125 maps to package 1 core 61
OMP: Info #171: KMP_AFFINITY: OS proc 126 maps to package 1 core 62
OMP: Info #171: KMP_AFFINITY: OS proc 127 maps to package 1 core 63
OMP: Info #250: KMP_AFFINITY: pid 4577 tid 4577 thread 0 bound to OS proc set 124
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 60-63
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 56-59
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 56 maps to package 0 core 56
OMP: Info #171: KMP_AFFINITY: OS proc 57 maps to package 0 core 57
OMP: Info #171: KMP_AFFINITY: OS proc 58 maps to package 0 core 58
OMP: Info #171: KMP_AFFINITY: OS proc 59 maps to package 0 core 59
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 120-123
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #250: KMP_AFFINITY: pid 4560 tid 4560 thread 0 bound to OS proc set 56
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 120 maps to package 1 core 56
OMP: Info #171: KMP_AFFINITY: OS proc 121 maps to package 1 core 57
OMP: Info #171: KMP_AFFINITY: OS proc 122 maps to package 1 core 58
OMP: Info #171: KMP_AFFINITY: OS proc 123 maps to package 1 core 59
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 56-59
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #250: KMP_AFFINITY: pid 4576 tid 4576 thread 0 bound to OS proc set 120
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 56 maps to package 0 core 56
OMP: Info #171: KMP_AFFINITY: OS proc 57 maps to package 0 core 57
OMP: Info #171: KMP_AFFINITY: OS proc 58 maps to package 0 core 58
OMP: Info #171: KMP_AFFINITY: OS proc 59 maps to package 0 core 59
OMP: Info #250: KMP_AFFINITY: pid 71817 tid 71817 thread 0 bound to OS proc set 56
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 120-123
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 120 maps to package 1 core 56
OMP: Info #171: KMP_AFFINITY: OS proc 121 maps to package 1 core 57
OMP: Info #171: KMP_AFFINITY: OS proc 122 maps to package 1 core 58
OMP: Info #171: KMP_AFFINITY: OS proc 123 maps to package 1 core 59
OMP: Info #250: KMP_AFFINITY: pid 71833 tid 71833 thread 0 bound to OS proc set 120
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 4 cores/pkg x 1 threads/core (4 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 60 maps to package 0 core 60
OMP: Info #171: KMP_AFFINITY: OS proc 61 maps to package 0 core 61
OMP: Info #171: KMP_AFFINITY: OS proc 62 maps to package 0 core 62
OMP: Info #171: KMP_AFFINITY: OS proc 63 maps to package 0 core 63
OMP: Info #250: KMP_AFFINITY: pid 71818 tid 71818 thread 0 bound to OS proc set 60
OMP: Info #250: KMP_AFFINITY: pid 71834 tid 72009 thread 1 bound to OS proc set 125
OMP: Info #250: KMP_AFFINITY: pid 4577 tid 4750 thread 1 bound to OS proc set 125
OMP: Info #250: KMP_AFFINITY: pid 4577 tid 4752 thread 3 bound to OS proc set 127
OMP: Info #250: KMP_AFFINITY: pid 71834 tid 72010 thread 2 bound to OS proc set 126
OMP: Info #250: KMP_AFFINITY: pid 71834 tid 72011 thread 3 bound to OS proc set 127
OMP: Info #250: KMP_AFFINITY: pid 4577 tid 4751 thread 2 bound to OS proc set 126
OMP: Info #250: KMP_AFFINITY: pid 4561 tid 4753 thread 1 bound to OS proc set 61
OMP: Info #250: KMP_AFFINITY: pid 4561 tid 4754 thread 2 bound to OS proc set 62
OMP: Info #250: KMP_AFFINITY: pid 4561 tid 4755 thread 3 bound to OS proc set 63
OMP: Info #250: KMP_AFFINITY: pid 4560 tid 4757 thread 1 bound to OS proc set 57
OMP: Info #250: KMP_AFFINITY: pid 4576 tid 4756 thread 1 bound to OS proc set 121
OMP: Info #250: KMP_AFFINITY: pid 4576 tid 4758 thread 2 bound to OS proc set 122
OMP: Info #250: KMP_AFFINITY: pid 4576 tid 4759 thread 3 bound to OS proc set 123
OMP: Info #250: KMP_AFFINITY: pid 4560 tid 4760 thread 2 bound to OS proc set 58
OMP: Info #250: KMP_AFFINITY: pid 4560 tid 4761 thread 3 bound to OS proc set 59
OMP: Info #250: KMP_AFFINITY: pid 71818 tid 72014 thread 1 bound to OS proc set 61
OMP: Info #250: KMP_AFFINITY: pid 71833 tid 72012 thread 1 bound to OS proc set 121
OMP: Info #250: KMP_AFFINITY: pid 71833 tid 72013 thread 2 bound to OS proc set 122
OMP: Info #250: KMP_AFFINITY: pid 71818 tid 72015 thread 2 bound to OS proc set 62
OMP: Info #250: KMP_AFFINITY: pid 71833 tid 72016 thread 3 bound to OS proc set 123
OMP: Info #250: KMP_AFFINITY: pid 71817 tid 72018 thread 1 bound to OS proc set 57
OMP: Info #250: KMP_AFFINITY: pid 71817 tid 72019 thread 2 bound to OS proc set 58
OMP: Info #250: KMP_AFFINITY: pid 71817 tid 72020 thread 3 bound to OS proc set 59
OMP: Info #250: KMP_AFFINITY: pid 71818 tid 72017 thread 3 bound to OS proc set 63

flotus1 · ‎11-13-2020

KMP_AFFINITY=verbose,granularity=fine,compact works fine too for binding each thread to a unique physical core. But I am still curious why kmp affinity gets my systems topology so wrong.

Server Gurus Discussions

What's wrong with OpenMP binding on my Epyc 7551 system