Application Development Environment¶
The application development environment on Tesseract is primarily controlled through the modules environment. By loading and switching modules you control the compilers, libraries and software available.
This means that for compiling on Tesseract you typically set the compiler you wish to use using the appropriate modules, then load all the required library modules (e.g. numerical libraries, IO format libraries).
Additionally, if you are compiling parallel applications using MPI (or SHMEM, etc.) then you will need to load the MPI environment and use the appropriate compiler wrapper scripts.
By default, all users on Tesseract start with no modules loaded.
Basic usage of the
module command on Tesseract is covered below. For
full documentation please see:
Using the modules environment¶
Information on the available modules¶
Finding out which modules (and hence which compilers, libraries and
software) are available on the system is performed using the
module avail command:
[user@tesseract-login1 ~]$ module avail ...
This will list all the names and versions of the modules available on the service. Not all of them may work in your account though due to, for example, licencing restrictions. You will notice that for many modules we have more than one version, each of which is identified by a version number. One of these versions is the default. As the service develops the default version will change.
You can list all the modules of a particular type by providing an
argument to the
module avail command. For example, to list all
available versions of the Intel libraries, compilers and tools:
[user@tesseract-login1 ~]$ module avail intel ------------------------------ /tessfs1/sw/modulefiles ------------------------------- intel-cc-18/18.1.163 intel-fc-18/18.1.163 intel-tools-18 intel-cmkl-18/18.1.163 intel-mpi-18/18.1.163 intel-vtune-18/18.1.163
If you want more info on any of the modules, you can use the
module help command:
[user@tesseract-login1 ~]$ module help intel-cmkl-18/18.1.163 ----------- Module Specific Help for 'intel-cmkl-18/18.1.163' --------------------------- Sets up the paths for Intel Cluster Math Kernal Library 18.1.163
module list command will give the names of the modules
and their versions you have presently loaded in your envionment:
[user@tesseract-login1 ~]$ module list Currently Loaded Modulefiles: 1) intel-cc-18/18.1.163 4) intel-mpi-18/18.1.163 2) intel-fc-18/18.1.163 5) intel-vtune-18/18.1.163 3) intel-cmkl-18/18.1.163 6) intel-tools-18
Loading, unloading and swapping modules¶
To load a module to use
module add or
module load. For example,
to load the Intel Fortran compilers into the development environment:
module load intel-fc-18
This will load the default version of the intel fortran compilers. If you need a specfic version of the module, you can add more information:
module load intel-fc-18/18.1.163
will load version 18.1.163 for you, regardless of the default. If you
want to clean up,
module remove will remove a loaded module:
module remove intel-fc-18
module rm intel-fc-18 or
module unload intel-fc-18) will unload what ever version of
intel-fc-17 (even if it is not the default) you might have
loaded. There are many situations in which you might want to change the
presently loaded version to a different one, such as trying the latest
version which is not yet the default or using a legacy version to keep
compatibility with old data. This can be achieved most easily by using
module swap oldmodule newmodule.
Available Compiler Suites¶
As Tesseract uses dynamic linking by default you will generally also need to load any modules you used to compile your code in your job submission script when you run your code.
Intel Compiler Suite¶
The Intel compiler suite is accessed by loading the
intel-tools-* module. For example:
module load intel-tools-18
Once you have loaded the module, the compilers are available as:
GCC Compiler Suite¶
The GCC 4.8.5 compiler suite is available by default without loading any modules.
The compilers are available as:
Compiling MPI codes¶
Tesseract currently supports the Intel MPI library.
You should also consult the chapter on running jobs through the batch system for examples of how to run jobs compiled against MPI.
By default, all compilers produce dynamic executables on Tesseract. This means that you must load the same modules at runtime (usually in your job submission script) as you have loaded at compile time.
Using Intel MPI¶
To compile MPI code with Intel MPI, using any compiler, you must first load the “intel-mpi-18” module:
module load intel-mpi-18
(If you loaded the
intel-tools-18 module then this automatically loads the Intel
MPI module for you.)
This makes the compiler wrapper scripts available to you. The name of the wrapper script depends on the compiler suite you are using. In summary:
Further details on using the different compiler suites with Intel MPI are available in the following sections.
Using Intel Compilers and Intel MPI¶
You should make the Intel compilers and MPI environment available by loading the
module load intel-tools-18
MPI compilers are then available as
mpiifort- Fortran with MPI
mpiicc- C with MPI
mpiicpc- C++ with MPI
Intel compilers with Intel MPI use non-standard compiler wrapper script names. If you use the standard names you will end up using the GCC compilers.
Using GCC Compilers and Intel MPI¶
Once you have loaded the
intel-tools-18 module, MPI compilers are then available as
mpif90- Fortran with MPI
mpicc- C with MPI
mpicxx- C++ with MPI
Compiler Information and Options¶
Help is available for the different compiler suites
gfortran --help, C/C++
man ifort, C/C++
Useful compiler options¶
For best performance on Tesseract we currently advise that you should use the Intel compilers wherever possible as the version of GCC available on the system is very old. We aim to install a more up to date version of GCC soon.
Whilst difference codes will benefit from compiler optimisations in different ways, for reasonable performance on Tesseract, at least initially, we suggest the following compiler options:
-O2 -ftree-vectorize -funroll-loops -ffast-math
When you have a application that you are happy is working correctly and has reasonable performance you may wish to investigate some more aggressive compiler optimisations. Below is a list of some further optimisations that you can try on your application (Note: these optimisations may result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions):
Vectorisation, which is one of the important compiler optimisations for Tesseract, is enabled by default as follows:
-O3and above or when using
To promote integer and real variables from four to eight byte precision for Fortran codes the following compiler flags can be used:
-real-size 64 -integer-size 64 -xAVX(Sometimes the Intel compiler incorrectly generates AVX2 instructions if the
-r8options are set. Using the
-xAVXoption prevents this.)