You are here

MKL

This numerical library contains BLAS (including a C interface), LAPACK, BLACS, ScaLAPACK, a sparse system solver, random number generators and vector versions of common mathematical functions. It also includes an FFT library.

On  the Westgrid systems that use Intel processors, MKL is the fastest option for BLAS.  It comes bundled with Intel compiler suite, so in many cases, linking C, Fortran and C++ code against MKL libraries is as simple as giving the corresponding Intel compiler the "-mkl" option.

MKL provides both sequential and threaded versions of these libraries. Note that "-mkl" would link against the threaded version by default. To control the resource usage of the threaded versions in the production calculations,  MKL_NUM_THREADS environment variable should be set in agreement to the number of the CPU cores requested for the batch job from Torque.

Parallel BLACS and ScaLAPACK parts of MKL depend on the MPI libraries. OpenMPI and IntelMPI distributions are supported by MKL. It is important to link against the same flavour of MPI for BLACS as for the rest of your application is using.

If there is a need for finer control over linking options such as selecting particular MPI dependency for ScaLAPACK or size of the array indeces (which can be 64 bit or 32 but wide), or if you want to use compilers other than Intels, MKL allows for a variety explicit linking options. To help the user with selecting the right options, Intel provides a Web-based "Interactive Intel MKL link options adviser" (see the link below).

Running MKL on Grex

On Grex, the environment module for Intel compilers version 12.1.5 that includes MKL version 10.3.11 is loaded by default. The simplest way to use MKL with Intel C, C++ or Fortran compilers is to use "-mkl"  options for the compiler/linker. Note that by default it will link against the threaded version of MKL. The Intel compilers module can be loaded explicitly as

module load intel/12.1.5.339

It is also possible to load MKL only, for use with compilers other than Intel's, and not having the later in the PATH. There is the special module for it, called gcc/mkl (as GNU GCC is the most widely used non-Intel compilers suite).

module load gcc/mkl/10.3.11

Either of Intel or GCC MKL modules would set the MKLROOT environment variable, and add necessary directories to LD_LIBRARY_PATH. The MKLROOT is handy when using explicit linking against libraries. It can be useful if you want to select a particular compiler (Intel or GCC), pointer width (the corresponding libraries have suffix _lp64 for 32bit pointers and_ilp64 for 64bit ones; the later is needed for, for example, Fortran codes with INTEGER*8 array indexes, explicit or set by -i8 compiler option ) and kind of MPI library to be used in BLACS (OpenMPI or IntelMPI which both are available on Grex). An example of the linker options to link against sequential, 64bit pointed vesion of BLAS, LAPACK for an Intel Fortran code:

ifort -O2 -i8 main.f -L$MKLROOT/lib/intel64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lpthread -lm

MKL also has FFTW bindings. They have to be enabled separately from the general Intel compilers installation; and therefore details of the usage might be different between particular Westgrid systems. On Grex, these libraries are pesent in two versions: 32 bit pointers (libfftw3xf_intel_lp64) and 64 bit pointers (fftw3xf_intel_ilp64). To link against these FFT libraries, the following include and library options to the compilers can be used (for the _lp64 case):

-I$MKLROOT/include/fftw -I$MKLROOT/interfaces/fftw3xf \
-L$MKLROOT/interfaces/fftw3xf -lfftw3xf_intel_lp64

The above to lines are, admittedly, rather elaborate but give the benefit of compiling and building all of the code with MKL, without the need for maintaining a separtate library such as FFTW3.