Repository navigation
New buffer access in xxx (and ffv) functions - remove ipagV from C++ api - #313
Merged
Merged
Conversation
…anges
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.940136e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.113848e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.734095 sec
393,467,151 cycles:u # 0.408 GHz
709,346,008 instructions:u # 1.80 insn per cycle
1.027992796 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.303097e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.221896 sec
19,053,500,298 cycles:u # 2.633 GHz
48,696,237,229 instructions:u # 2.56 insn per cycle
7.240189441 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 636) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.908897e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.635227 sec
8,968,023,902 cycles:u # 2.457 GHz
16,446,697,673 instructions:u # 1.83 insn per cycle
3.652880396 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2704) (512y: 52) (512z: 0)
=========================================================================
…892107 None of these is still used anywhere - performance remains the same
…u - nan in cuda, ok in c++
…1/eemumu - all ok, good performances
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.927727e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.112874e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.934527 sec
759,358,027 cycles:u # 0.664 GHz
1,460,203,275 instructions:u # 1.92 insn per cycle
1.234834519 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.312128e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.242481 sec
19,031,081,712 cycles:u # 2.622 GHz
48,696,253,110 instructions:u # 2.56 insn per cycle
7.266153195 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 636) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.916675e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.672978 sec
9,027,104,402 cycles:u # 2.445 GHz
16,446,713,690 instructions:u # 1.82 insn per cycle
3.695645301 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2704) (512y: 52) (512z: 0)
=========================================================================
…n pointers
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.969003e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.111533e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.959843 sec
758,028,751 cycles:u # 0.657 GHz
1,460,327,449 instructions:u # 1.93 insn per cycle
1.273118826 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.311472e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.291478 sec
19,185,553,275 cycles:u # 2.625 GHz
48,696,253,284 instructions:u # 2.54 insn per cycle
7.314469200 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 636) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.911322e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.675334 sec
9,028,582,206 cycles:u # 2.444 GHz
16,446,713,774 instructions:u # 1.82 insn per cycle
3.697490131 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2704) (512y: 52) (512z: 0)
=========================================================================
valassi
marked this pull request as draft
December 11, 2021 10:25
…k to fptype_sv in xxx functions
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.967032e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.114019e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 1.096448 sec
773,971,897 cycles:u # 0.663 GHz
1,458,037,221 instructions:u # 1.88 insn per cycle
1.398319897 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.310183e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.262135 sec
19,070,073,943 cycles:u # 2.619 GHz
48,696,258,394 instructions:u # 2.55 insn per cycle
7.286654464 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 636) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.916607e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.691983 sec
9,051,495,450 cycles:u # 2.438 GHz
16,446,719,020 instructions:u # 1.82 insn per cycle
3.716089213 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2704) (512y: 52) (512z: 0)
=========================================================================
…- add implicit conversion p4type_sv to fptype_sv*
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.953641e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.114671e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.935976 sec
768,960,916 cycles:u # 0.661 GHz
1,467,936,740 instructions:u # 1.91 insn per cycle
1.248062395 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.311101e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.254034 sec
19,066,177,157 cycles:u # 2.622 GHz
48,696,258,582 instructions:u # 2.55 insn per cycle
7.278082466 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 636) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.905104e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.687230 sec
9,047,700,943 cycles:u # 2.440 GHz
16,446,718,506 instructions:u # 1.82 insn per cycle
3.711097818 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2704) (512y: 52) (512z: 0)
=========================================================================
…nel access and addressing function
…sMomenta - used for cuda, not yet fully in c++
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.958293e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.114393e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.766816 sec
414,707,867 cycles:u # 0.412 GHz
722,267,590 instructions:u # 1.74 insn per cycle
1.066649196 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.305823e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.283923 sec
19,045,988,578 cycles:u # 2.609 GHz
48,696,259,263 instructions:u # 2.56 insn per cycle
7.308598173 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 636) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.925262e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.674796 sec
9,022,915,294 cycles:u # 2.442 GHz
16,446,719,005 instructions:u # 1.82 insn per cycle
3.698812056 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2704) (512y: 52) (512z: 0)
=========================================================================
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 6.057387e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.141808e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 1.010526 sec
765,676,850 cycles:u # 0.666 GHz
1,459,667,841 instructions:u # 1.91 insn per cycle
1.311940329 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.311844e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.251677 sec
19,048,456,703 cycles:u # 2.621 GHz
48,696,258,480 instructions:u # 2.56 insn per cycle
7.275597617 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 636) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.904536e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.687137 sec
9,051,969,019 cycles:u # 2.442 GHz
16,446,719,045 instructions:u # 1.82 insn per cycle
3.710988915 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2704) (512y: 52) (512z: 0)
=========================================================================
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.911154e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.105379e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.765036 sec
418,055,605 cycles:u # 0.416 GHz
716,983,446 instructions:u # 1.72 insn per cycle
1.065857170 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.312343e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.235976 sec
19,044,928,715 cycles:u # 2.624 GHz
48,696,258,019 instructions:u # 2.56 insn per cycle
7.260246122 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 636) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.929951e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.709707 sec
9,109,382,472 cycles:u # 2.442 GHz
16,446,719,092 instructions:u # 1.81 insn per cycle
3.733559283 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2704) (512y: 52) (512z: 0)
=========================================================================
…ok (~2% slower in c++?)
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.924484e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.113887e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.762733 sec
413,030,276 cycles:u # 0.412 GHz
721,969,023 instructions:u # 1.75 insn per cycle
1.062606040 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.300178e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.289449 sec
19,175,817,992 cycles:u # 2.624 GHz
48,343,933,614 instructions:u # 2.52 insn per cycle
7.313170955 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 574) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.820191e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.741431 sec
9,175,766,443 cycles:u # 2.441 GHz
16,248,536,622 instructions:u # 1.77 insn per cycle
3.765174905 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2754) (512y: 52) (512z: 0)
=========================================================================
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.963277e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.114171e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.965056 sec
764,711,277 cycles:u # 0.664 GHz
1,461,264,678 instructions:u # 1.91 insn per cycle
1.265600027 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.292914e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.312846 sec
19,242,259,949 cycles:u # 2.625 GHz
48,343,933,778 instructions:u # 2.51 insn per cycle
7.336570575 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 574) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.844702e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.719313 sec
9,132,329,454 cycles:u # 2.442 GHz
16,248,536,673 instructions:u # 1.78 insn per cycle
3.742908010 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2754) (512y: 52) (512z: 0)
=========================================================================
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.975366e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.115482e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.941425 sec
762,940,296 cycles:u # 0.663 GHz
1,468,151,082 instructions:u # 1.92 insn per cycle
1.242135101 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.300589e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.280185 sec
19,148,003,713 cycles:u # 2.623 GHz
48,318,767,984 instructions:u # 2.52 insn per cycle
7.304646286 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 574) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.832714e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.704604 sec
9,094,062,506 cycles:u # 2.441 GHz
16,248,536,525 instructions:u # 1.79 insn per cycle
3.728801864 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2754) (512y: 52) (512z: 0)
=========================================================================
…UNCTIONS...
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.996332e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.120328e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.975801 sec
761,377,731 cycles:u # 0.656 GHz
1,466,316,940 instructions:u # 1.93 insn per cycle
1.279572623 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.297201e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.301089 sec
19,198,479,392 cycles:u # 2.623 GHz
48,318,768,371 instructions:u # 2.52 insn per cycle
7.325079977 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 574) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.826184e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.710541 sec
9,097,646,008 cycles:u # 2.439 GHz
16,248,537,127 instructions:u # 1.79 insn per cycle
3.733582744 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2754) (512y: 52) (512z: 0)
=========================================================================
…ower
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.954583e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.113358e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.761552 sec
413,546,907 cycles:u # 0.414 GHz
727,127,575 instructions:u # 1.76 insn per cycle
1.061515791 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.293610e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.326218 sec
19,281,456,331 cycles:u # 2.625 GHz
48,318,767,477 instructions:u # 2.51 insn per cycle
7.349502477 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.695919e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.748591 sec
9,197,162,277 cycles:u # 2.441 GHz
16,688,941,210 instructions:u # 1.81 insn per cycle
3.772120405 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2752) (512y: 52) (512z: 0)
=========================================================================
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.918101e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.111580e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.769473 sec
411,091,298 cycles:u # 0.410 GHz
717,754,322 instructions:u # 1.75 insn per cycle
1.069596435 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.301696e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.285865 sec
19,178,043,588 cycles:u # 2.625 GHz
48,318,767,095 instructions:u # 2.52 insn per cycle
7.309316569 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.512116e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.718906 sec
9,080,082,207 cycles:u # 2.429 GHz
16,386,951,251 instructions:u # 1.80 insn per cycle
3.743047818 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2794) (512y: 52) (512z: 0)
=========================================================================
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.969161e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.113501e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.773621 sec
415,155,217 cycles:u # 0.411 GHz
717,379,589 instructions:u # 1.73 insn per cycle
1.074332216 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.301150e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.281312 sec
19,151,429,304 cycles:u # 2.623 GHz
48,318,768,412 instructions:u # 2.52 insn per cycle
7.304786269 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.818740e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.703873 sec
9,089,320,140 cycles:u # 2.441 GHz
16,248,536,398 instructions:u # 1.79 insn per cycle
3.727230595 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2754) (512y: 52) (512z: 0)
=========================================================================
…lty if anything (now 4.86?!)
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.936738e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.113886e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.765109 sec
413,197,548 cycles:u # 0.412 GHz
723,218,856 instructions:u # 1.75 insn per cycle
1.064332586 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.302016e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.307847 sec
19,231,018,339 cycles:u # 2.625 GHz
48,318,767,089 instructions:u # 2.51 insn per cycle
7.331490647 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.855278e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.715297 sec
9,125,943,724 cycles:u # 2.443 GHz
16,273,702,614 instructions:u # 1.78 insn per cycle
3.738512895 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2762) (512y: 52) (512z: 0)
=========================================================================
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.930947e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.110873e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.758637 sec
411,453,190 cycles:u # 0.413 GHz
717,627,965 instructions:u # 1.74 insn per cycle
1.058214819 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.301208e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.281200 sec
19,166,963,587 cycles:u # 2.625 GHz
48,318,767,492 instructions:u # 2.52 insn per cycle
7.304740930 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
WARNING! arbitrary array
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.299171e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.870550 sec
9,473,318,297 cycles:u # 2.434 GHz
17,506,839,400 instructions:u # 1.85 insn per cycle
3.895081499 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2783) (512y: 52) (512z: 0)
=========================================================================
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.935073e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.113731e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.761975 sec
411,482,675 cycles:u # 0.410 GHz
718,980,829 instructions:u # 1.75 insn per cycle
1.064540223 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.301737e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.274357 sec
19,150,038,986 cycles:u # 2.625 GHz
48,318,767,059 instructions:u # 2.52 insn per cycle
7.298094984 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
WARNING! arbitrary array
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.491809e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.718218 sec
9,068,572,731 cycles:u # 2.429 GHz
16,500,197,400 instructions:u # 1.82 insn per cycle
3.741645759 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2784) (512y: 52) (512z: 0)
=========================================================================
…ent check every time)
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.958651e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.111818e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.768364 sec
414,840,251 cycles:u # 0.413 GHz
716,621,300 instructions:u # 1.73 insn per cycle
1.068316030 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.292061e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.321030 sec
19,252,455,161 cycles:u # 2.624 GHz
48,318,767,801 instructions:u # 2.51 insn per cycle
7.344887374 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.843689e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.702359 sec
9,079,231,621 cycles:u # 2.440 GHz
16,273,702,715 instructions:u # 1.79 insn per cycle
3.726554931 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2762) (512y: 52) (512z: 0)
=========================================================================
…rate MemoryAccess.h - all ok
This will need a much more throught cleanup, once we agree better what is needed in the Fortran bridge
But for the moment this is a reasonable separation of data access and computations
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.525339e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.162881e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 1.123886 sec
785,576,398 cycles:u # 0.880 GHz
1,490,945,195 instructions:u # 1.90 insn per cycle
1.427680604 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.301289e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.290284 sec
19,162,611,591 cycles:u # 2.621 GHz
48,318,763,032 instructions:u # 2.52 insn per cycle
7.316508951 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.851138e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.716292 sec
9,109,552,318 cycles:u # 2.436 GHz
16,273,698,120 instructions:u # 1.79 insn per cycle
3.743027573 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2762) (512y: 52) (512z: 0)
=========================================================================
…sAmplitude (trivial, for the moment) - all ok
(No functional change for the moment - kernelAccessAmplitude is still the same for cuda and c++)
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.578246e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.176616e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 1.033650 sec
780,589,876 cycles:u # 0.661 GHz
1,496,096,610 instructions:u # 1.92 insn per cycle
1.336930631 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.302509e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.292469 sec
19,173,073,325 cycles:u # 2.621 GHz
48,318,763,508 instructions:u # 2.52 insn per cycle
7.319408337 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.847707e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.713602 sec
9,102,914,462 cycles:u # 2.437 GHz
16,273,698,804 instructions:u # 1.79 insn per cycle
3.739403482 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2762) (512y: 52) (512z: 0)
=========================================================================
…RD_PATH in epoch1 as done in epochX (issue madgraph5#194) On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]: ========================================================================= Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0] FP precision = DOUBLE (NaN/abnormal=0, zero=0) EvtsPerSec[MatrixElems] (3) = ( 5.430542e+08 ) sec^-1 EvtsPerSec[MECalcOnly] (3a) = ( 1.112191e+09 ) sec^-1 MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0 TOTAL : 0.790035 sec 432,265,595 cycles:u # 0.419 GHz 752,034,458 instructions:u # 1.74 insn per cycle 1.093489337 seconds time elapsed ==PROF== Profiling "sigmaKin": launch__registers_per_thread 130 ==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100% ========================================================================= Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0] FP precision = DOUBLE (NaN/abnormal=0, zero=0) Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD) OMP threads / `nproc --all` = 1 / 4 EvtsPerSec[MECalcOnly] (3a) = ( 1.303264e+06 ) sec^-1 MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0 TOTAL : 7.297318 sec 19,169,051,850 cycles:u # 2.618 GHz 48,318,765,409 instructions:u # 2.52 insn per cycle 7.324402289 seconds time elapsed =Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0) ------------------------------------------------------------------------- Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0] FP precision = DOUBLE (NaN/abnormal=0, zero=0) Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES] OMP threads / `nproc --all` = 1 / 4 EvtsPerSec[MECalcOnly] (3a) = ( 4.805005e+06 ) sec^-1 MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0 TOTAL : 3.741404 sec 9,120,132,021 cycles:u # 2.422 GHz 16,273,702,231 instructions:u # 1.78 insn per cycle 3.769439753 seconds time elapsed =Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2762) (512y: 52) (512z: 0) =========================================================================
…ce diffs to epochX)
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.431251e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.111897e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 1.132435 sec
1,061,482,814 cycles:u # 0.809 GHz
2,042,247,419 instructions:u # 1.92 insn per cycle
1.449279763 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.303611e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.302474 sec
19,182,786,382 cycles:u # 2.618 GHz
48,318,767,121 instructions:u # 2.52 insn per cycle
7.331050665 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.839633e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.732373 sec
9,131,519,732 cycles:u # 2.432 GHz
16,273,702,812 instructions:u # 1.78 insn per cycle
3.760368811 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2762) (512y: 52) (512z: 0)
=========================================================================
…r the ifdef MGONGPU_HARDCODE_CIPC)
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.450328e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.113992e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.791362 sec
435,349,317 cycles:u # 0.420 GHz
752,035,433 instructions:u # 1.73 insn per cycle
1.098707209 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.302442e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.304491 sec
19,181,226,930 cycles:u # 2.618 GHz
48,318,766,149 instructions:u # 2.52 insn per cycle
7.331533054 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.855146e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.729151 sec
9,132,014,260 cycles:u # 2.434 GHz
16,273,703,332 instructions:u # 1.78 insn per cycle
3.756437859 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2762) (512y: 52) (512z: 0)
=========================================================================
… LOOP" COMMENTS IN THE XXX FUNCTIONS...
… ipagV, use the kernelAccess functions) - all ok
…ier on in HelAmps_sm.h and add Amplitude function
…ions earlier on in HelAmps_sm.h and add Amplitude
On itscrd70.cern.ch [CPU: Intel(R) Xeon(R) Silver 4216 CPU] [GPU: 1x Tesla V100S-PCIE-32GB]:
=========================================================================
Process = EPOCH1_EEMUMU_CUDA [nvcc 11.1.105 (gcc 10.2.0)] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
EvtsPerSec[MatrixElems] (3) = ( 5.441266e+08 ) sec^-1
EvtsPerSec[MECalcOnly] (3a) = ( 1.120748e+09 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 0.987913 sec
792,480,290 cycles:u # 0.670 GHz
1,496,770,892 instructions:u # 1.89 insn per cycle
1.292641075 seconds time elapsed
==PROF== Profiling "sigmaKin": launch__registers_per_thread 130
==PROF== Profiling "sigmaKin": sm__sass_average_branch_targets_threads_uniform.pct 100%
=========================================================================
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = SCALAR ('none': ~vector[1], no SIMD)
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 1.300619e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 7.324798 sec
19,215,567,029 cycles:u # 2.616 GHz
48,318,766,461 instructions:u # 2.51 insn per cycle
7.353073181 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 576) (avx2: 0) (512y: 0) (512z: 0)
-------------------------------------------------------------------------
Process = EPOCH1_EEMUMU_CPP [gcc 10.2.0] [inlineHel=0] [hardcodeCIPC=0]
FP precision = DOUBLE (NaN/abnormal=0, zero=0)
Internal loops fptype_sv = VECTOR[4] ('512y': AVX512, 256bit) [cxtype_ref=YES]
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MECalcOnly] (3a) = ( 4.850439e+06 ) sec^-1
MeanMatrixElemValue = ( 1.371706e-02 +- 3.270315e-06 ) GeV^0
TOTAL : 3.726553 sec
9,120,667,109 cycles:u # 2.432 GHz
16,273,702,551 instructions:u # 1.78 insn per cycle
3.754039059 seconds time elapsed
=Symbols in CPPProcess.o= (~sse4: 0) (avx2: 2762) (512y: 52) (512z: 0)
=========================================================================
…mu epochX - most likely it will not be needed
…al to codegen and eemumu auto
…lignment check every time)
…ge in performance
…est manual - all ok, no change in performance
valassi
marked this pull request as ready for review
December 13, 2021 13:09
Member
Author
|
This is now complete thus far and can be merged. Main changes
|
Member
Author
|
Self merging |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a WIP PR for the ideas discussed in the first bullet of #175 (comment)
The idea is to start separating CUDA and C++, so that CUDA does a data access in a large buffer inside the XX functions (and eventually in the FFV functions to split into smaller kernels), while C++ takes a single event or at most a SIMD vector of events.
In practice this means removing ipagV from many interfaces.
Note that ipar is still there for CUDA - this cannot be changed unless/until a new memory structure is introduced, as discussed in the SOAOSOA #309
Note also that ip4 is still in the interface of the bufferAccessMomenta. I think this is unavoidable for CUDA: we want to have an AOSOA for events and 4-components, with a striding not fixed a priori. This also does not harm for c++: it could be avoided as one could just get an fptype_sv[4] and index it in a very intuitive way, but it does not harm to have it as a function.
The same idea could be used eventually for splitting the kernels (#310): it would be enough to add for instance bufferAccessWavefunction( fptype* buffer, int iw6 ) wherethe iw6 index is passed in the same way as the ip4 now.
This is WIP, a lot to do, but the basic idea is there now.