heFFTe

 Installation of heFFTe with MKL on SPACK;

% spack install heffte%intel@2021.4.0+mkl ^intel-oneapi-mpi@2021.4.0 ^intel-mkl threads=openmp

To use installed Intel OneAPI and CMAKE on SPACK

In order to avoid multiple installations of Intel-OneAPI and CMAKE, we can use installed ones.
Please install Install OneAPI and CMAKE via APT and put the packages.yaml under ~/.spack/ whose contents are as follows;

 $ cat ~/.spack/packages.yaml

packages:

intel-oneapi-mkl:

buildable: false

externals:

- spec: intel-oneapi-mkl@2021.4.0

prefix: /opt/intel/oneapi/

intel-oneapi-mpi:

buildable: false

externals:

- spec: intel-oneapi-mpi@2021.4.0

prefix: /opt/intel/oneapi/

cmake:

buildable: false

externals:

- spec: cmake@3.16.3

prefix: /usr

buildable: false prevents us from installing packages via SPACK.

MPI/OpenMP hybrid with Intel MPI

  mpirun -np 8 -genv OMP_NUM_THREADS=16 -hosts node2-ib,node3-ib,node4-ib,node5-ib -ppn 2 ./sol

To run an OpenMP/MPI hybrid code with IntelMPI, the above command is needed.

The meanings are;

8 MPI processes (-np 8),

16 OpenMP threads (-genv OMP_NUM_THREADS=16) ,

2 MPI processes per node (-ppn 2).