{{indexmenu_n>1}}
====== Software environment ======
On Hábrók a lot of software has been made available. In order to allow multiple versions of software to be available the software is arranged in so-called modules.
===== module system =====
By default, only the main modules for getting access to the software stack are loaded: these are named ''StdEnv'' and ''2023.01'' (which denotes the current version of the software stack).
Other than that, no modules for actual applications are loaded by default, which means that the software will not yet be available on the command line prompt. Throughout this page we will use R as an example. Normally the command ''R'' should fire up a command line session for R:
[username@habrok ~]$ R
-bash: R: command not found
[username@habrok ~]$
As you can see R cannot be found. In order to find the options that are available and add the software to our environment we have to make use of the module system. The command ''module'' allows for finding and loading software into the environment. For this the command has to be used together with a subcommand to specify the action we want to perform.
The subcommands are:
* ''avail'' for showing a list of available software
* ''load'' or ''add'' for loading software into the environment
* ''list'' for showing the currently loaded modules
* ''unload'' or ''del'' for unloading a module from the environment
* ''purge'' for unloading all loaded modules from the environment
* ''save'' for saving a set of loaded modules into a configuration for restoring at a later time
* ''restore'' for restoring a saved set of modules
==== module avail ====
One way to find information about software is to run the command ''module'' with the subcommand ''avail''. If we run it without extra arguments we get something like:
[username@habrok ~]$ module avail
---- /cvmfs/hpc.rug.nl/versions/2023.01/rocky8/x86_64/amd/zen3/modules/ai ----
PyTorch/1.12.1-foss-2022a-CUDA-11.7.0
---- /cvmfs/hpc.rug.nl/versions/2023.01/rocky8/x86_64/amd/zen3/modules/astro ----
astropy/5.1.1-foss-2022a
---- /cvmfs/hpc.rug.nl/versions/2023.01/rocky8/x86_64/amd/zen3/modules/bio ----
angsd/0.940-GCC-11.2.0
BA3-SNPS-autotune/2.1.2-GCC-11.3.0
BamTools/2.5.2-GCC-11.3.0
BayesAss3-SNPs/1.1-GCC-11.3.0
BCFtools/1.15.1-GCC-11.3.0
BEDTools/2.30.0-GCC-11.3.0
BioPerl/1.7.8-GCCcore-11.3.0
Biopython/1.79-foss-2022a
BLAST+/2.13.0-gompi-2022a
BLAST/2.11.0-Linux_x86_64
... et cetera...
We can use the spacebar to scroll through the extensive list. As you can see the list contains software packages. For each package one or more versions is available.
In order to bring down the list to certain packages only we can specify an argument to ''module avail''. If we specify 'R', or 'r' as an argument it will show us all packages with an 'r' in its name. This will not help much, so we have to be more precise. A more useful search string would be 'r/3':
[username@habrok ~]$ module avail r/4
-------------------------------------- /cvmfs/hpc.rug.nl/versions/2023.01/rocky8/x86_64/amd/zen3/modules/lang --------------------------------------
R/4.2.1-foss-2022a
-------------------------------------- /cvmfs/hpc.rug.nl/versions/2023.01/rocky8/x86_64/amd/zen3/modules/lib ---------------------------------------
NSPR/4.34-GCCcore-11.3.0
-------------------------------------- /cvmfs/hpc.rug.nl/versions/2023.01/rocky8/x86_64/amd/zen3/modules/math --------------------------------------
MPFR/4.1.0-GCCcore-11.3.0
Now we can see the actual version(s) of R that is/are available. If multiple versions of the same application are available, only one of them will have '(D)' behind the name, denoting that this is the default versions.
==== module load/add ====
In order to load R, we can use the ''load'' or equivalent ''add'' subcommand for module. Note that we have to be precise with respect to upper- and lowercase now.
[username@habrok ~]$ module load R
[username@habrok ~]$ R --version
R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.
[username@habrok ~]$
As can be seen from the last output R version 4.2.1 is now in our environment. If no version is specified the default version of the package is loaded.
**CAVEAT** Although packages have default versions, these defaults are subject to change without notice. It is therefore better to be explicit in the version that is to be loaded, especially in jobscripts. Otherwise you may find out that jobscripts break when the default suddenly changes.
For loading a specific version the full version has to be specified like:
[username@habrok ~]$ module load R/4.2.1-foss-2022a
==== module spider ====
Another way to find out more information about available packages is through module spider. With this subcommand you also have to make sure to use an uppercase R:
[username@habrok ~]$ module spider R
------------------------------------------------------------------------------------------------------------------------------------------------
R: R/4.2.1-foss-2022a
------------------------------------------------------------------------------------------------------------------------------------------------
Description:
R is a free software environment for statistical computing and graphics.
Other possible modules matches:
Amber, AmberTools, Arrow, BioPerl, Brotli, CapnProto, DendroPy, FastTree, FreeSurfer, FriBidi, GCCcore, GDRCopy, GObject-Introspection, ...
You will need to load all module(s) on any one of the lines below before the "R/4.2.1-foss-2022a" module is available to load.
2023.01
This module provides the following extensions:
[removed the long output showing all included R libraries>
This command shows some extra information on the package.
==== module list ====
In order to find out which modules have been loaded the subcommand ''list'' can be used:
[username@habrok ~]$ module list
Currently Loaded Modules:
1) 2023.01 (S) 35) X11/20220504-GCCcore-11.3.0 69) LAME/3.100-GCCcore-11.3.0
2) StdEnv (S) 36) gzip/1.12-GCCcore-11.3.0 70) libsndfile/1.1.0-GCCcore-11.3.0
3) GCCcore/11.3.0 37) lz4/1.9.3-GCCcore-11.3.0 71) ICU/71.1-GCCcore-11.3.0
4) zlib/1.2.12-GCCcore-11.3.0 38) zstd/1.5.2-GCCcore-11.3.0 72) Szip/2.1.1-GCCcore-11.3.0
5) binutils/2.38-GCCcore-11.3.0 39) libdrm/2.4.110-GCCcore-11.3.0 73) HDF5/1.12.2-gompi-2022a
6) GCC/11.3.0 40) libglvnd/1.4.0-GCCcore-11.3.0 74) UDUNITS/2.2.28-GCCcore-11.3.0
7) numactl/2.0.14-GCCcore-11.3.0 41) libunwind/1.6.2-GCCcore-11.3.0 75) GSL/2.7-GCC-11.3.0
8) XZ/5.2.5-GCCcore-11.3.0 42) LLVM/14.0.3-GCCcore-11.3.0 76) ATK/2.38.0-GCCcore-11.3.0
9) libxml2/2.9.13-GCCcore-11.3.0 43) Mesa/22.0.3-GCCcore-11.3.0 77) Gdk-Pixbuf/2.42.8-GCCcore-11.3.0
10) libpciaccess/0.16-GCCcore-11.3.0 44) libGLU/9.0.2-GCCcore-11.3.0 78) HarfBuzz/4.2.1-GCCcore-11.3.0
11) hwloc/2.7.1-GCCcore-11.3.0 45) pixman/0.40.0-GCCcore-11.3.0 79) FriBidi/1.0.12-GCCcore-11.3.0
12) OpenSSL/1.1 46) libffi/3.4.2-GCCcore-11.3.0 80) Pango/1.50.7-GCCcore-11.3.0
13) libevent/2.1.12-GCCcore-11.3.0 47) gettext/0.21-GCCcore-11.3.0 81) GTK2/2.24.33-GCCcore-11.3.0
14) UCX/1.12.1-GCCcore-11.3.0 48) PCRE/8.45-GCCcore-11.3.0 82) Ghostscript/9.56.1-GCCcore-11.3.0
15) libfabric/1.15.1-GCCcore-11.3.0 49) GLib/2.72.1-GCCcore-11.3.0 83) JasPer/2.0.33-GCCcore-11.3.0
16) PMIx/4.1.2-GCCcore-11.3.0 50) cairo/1.17.4-GCCcore-11.3.0 84) LittleCMS/2.13.1-GCCcore-11.3.0
17) UCC/1.0.0-GCCcore-11.3.0 51) libreadline/8.1.2-GCCcore-11.3.0 85) ImageMagick/7.1.0-37-GCCcore-11.3.0
18) OpenMPI/4.1.4-GCC-11.3.0 52) Tcl/8.6.12-GCCcore-11.3.0 86) GLPK/5.0-GCCcore-11.3.0
19) OpenBLAS/0.3.20-GCC-11.3.0 53) SQLite/3.38.3-GCCcore-11.3.0 87) nodejs/16.15.1-GCCcore-11.3.0
20) FlexiBLAS/3.2.0-GCC-11.3.0 54) PCRE2/10.40-GCCcore-11.3.0 88) Python/3.10.4-GCCcore-11.3.0
21) FFTW/3.3.10-GCC-11.3.0 55) NASM/2.15.05-GCCcore-11.3.0 89) netCDF/4.9.0-gompi-2022a
22) gompi/2022a 56) libjpeg-turbo/2.1.3-GCCcore-11.3.0 90) GEOS/3.10.3-GCC-11.3.0
23) FFTW.MPI/3.3.10-gompi-2022a 57) jbigkit/2.1-GCCcore-11.3.0 91) nlohmann_json/3.10.5-GCCcore-11.3.0
24) ScaLAPACK/2.2.0-gompi-2022a-fb 58) libdeflate/1.10-GCCcore-11.3.0 92) PROJ/9.0.0-GCCcore-11.3.0
25) foss/2022a 59) LibTIFF/4.3.0-GCCcore-11.3.0 93) libgeotiff/1.7.1-GCCcore-11.3.0
26) bzip2/1.0.8-GCCcore-11.3.0 60) Java/11.0.16 94) pybind11/2.9.2-GCCcore-11.3.0
27) expat/2.4.8-GCCcore-11.3.0 61) Tk/8.6.12-GCCcore-11.3.0 95) SciPy-bundle/2022.05-foss-2022a
28) libpng/1.6.37-GCCcore-11.3.0 62) cURL/7.83.0-GCCcore-11.3.0 96) libtirpc/1.3.2-GCCcore-11.3.0
29) Brotli/1.0.9-GCCcore-11.3.0 63) GMP/6.2.1-GCCcore-11.3.0 97) HDF/4.2.15-GCCcore-11.3.0
30) freetype/2.12.1-GCCcore-11.3.0 64) NLopt/2.7.1-GCCcore-11.3.0 98) GDAL/3.5.0-foss-2022a
31) ncurses/6.3-GCCcore-11.3.0 65) libogg/1.3.5-GCCcore-11.3.0 99) MPFR/4.1.0-GCCcore-11.3.0
32) util-linux/2.38-GCCcore-11.3.0 66) FLAC/1.3.4-GCCcore-11.3.0 100) libgit2/1.4.3-GCCcore-11.3.0
33) fontconfig/2.14.0-GCCcore-11.3.0 67) libvorbis/1.3.7-GCCcore-11.3.0 101) R/4.2.1-foss-2022a
34) xorg-macros/1.19.3-GCCcore-11.3.0 68) libopus/1.3.1-GCCcore-11.3.0
As you can see not only R itself has been loaded, but also a lot of other modules. These modules are all the tools and libraries this version of R depends on.
One of the issues that you will probably encounter is that if you want to use different modules, and load these, that you run into conflicts with respect to the underlying dependencies. More on this later.
==== module del/unload ====
In order to unload a certain module from the environment the ''unload'' or equivalent ''del'' subcommand can be used:
[username@habrok ~]$ module unload R
[username@habrok ~]$ module list
Currently Loaded Modules:
1) 2023.01 (S) 35) X11/20220504-GCCcore-11.3.0 69) LAME/3.100-GCCcore-11.3.0
2) StdEnv (S) 36) gzip/1.12-GCCcore-11.3.0 70) libsndfile/1.1.0-GCCcore-11.3.0
3) GCCcore/11.3.0 37) lz4/1.9.3-GCCcore-11.3.0 71) ICU/71.1-GCCcore-11.3.0
4) zlib/1.2.12-GCCcore-11.3.0 38) zstd/1.5.2-GCCcore-11.3.0 72) Szip/2.1.1-GCCcore-11.3.0
5) binutils/2.38-GCCcore-11.3.0 39) libdrm/2.4.110-GCCcore-11.3.0 73) HDF5/1.12.2-gompi-2022a
6) GCC/11.3.0 40) libglvnd/1.4.0-GCCcore-11.3.0 74) UDUNITS/2.2.28-GCCcore-11.3.0
7) numactl/2.0.14-GCCcore-11.3.0 41) libunwind/1.6.2-GCCcore-11.3.0 75) GSL/2.7-GCC-11.3.0
8) XZ/5.2.5-GCCcore-11.3.0 42) LLVM/14.0.3-GCCcore-11.3.0 76) ATK/2.38.0-GCCcore-11.3.0
9) libxml2/2.9.13-GCCcore-11.3.0 43) Mesa/22.0.3-GCCcore-11.3.0 77) Gdk-Pixbuf/2.42.8-GCCcore-11.3.0
10) libpciaccess/0.16-GCCcore-11.3.0 44) libGLU/9.0.2-GCCcore-11.3.0 78) HarfBuzz/4.2.1-GCCcore-11.3.0
11) hwloc/2.7.1-GCCcore-11.3.0 45) pixman/0.40.0-GCCcore-11.3.0 79) FriBidi/1.0.12-GCCcore-11.3.0
12) OpenSSL/1.1 46) libffi/3.4.2-GCCcore-11.3.0 80) Pango/1.50.7-GCCcore-11.3.0
13) libevent/2.1.12-GCCcore-11.3.0 47) gettext/0.21-GCCcore-11.3.0 81) GTK2/2.24.33-GCCcore-11.3.0
14) UCX/1.12.1-GCCcore-11.3.0 48) PCRE/8.45-GCCcore-11.3.0 82) Ghostscript/9.56.1-GCCcore-11.3.0
15) libfabric/1.15.1-GCCcore-11.3.0 49) GLib/2.72.1-GCCcore-11.3.0 83) JasPer/2.0.33-GCCcore-11.3.0
16) PMIx/4.1.2-GCCcore-11.3.0 50) cairo/1.17.4-GCCcore-11.3.0 84) LittleCMS/2.13.1-GCCcore-11.3.0
17) UCC/1.0.0-GCCcore-11.3.0 51) libreadline/8.1.2-GCCcore-11.3.0 85) ImageMagick/7.1.0-37-GCCcore-11.3.0
18) OpenMPI/4.1.4-GCC-11.3.0 52) Tcl/8.6.12-GCCcore-11.3.0 86) GLPK/5.0-GCCcore-11.3.0
19) OpenBLAS/0.3.20-GCC-11.3.0 53) SQLite/3.38.3-GCCcore-11.3.0 87) nodejs/16.15.1-GCCcore-11.3.0
20) FlexiBLAS/3.2.0-GCC-11.3.0 54) PCRE2/10.40-GCCcore-11.3.0 88) Python/3.10.4-GCCcore-11.3.0
21) FFTW/3.3.10-GCC-11.3.0 55) NASM/2.15.05-GCCcore-11.3.0 89) netCDF/4.9.0-gompi-2022a
22) gompi/2022a 56) libjpeg-turbo/2.1.3-GCCcore-11.3.0 90) GEOS/3.10.3-GCC-11.3.0
23) FFTW.MPI/3.3.10-gompi-2022a 57) jbigkit/2.1-GCCcore-11.3.0 91) nlohmann_json/3.10.5-GCCcore-11.3.0
24) ScaLAPACK/2.2.0-gompi-2022a-fb 58) libdeflate/1.10-GCCcore-11.3.0 92) PROJ/9.0.0-GCCcore-11.3.0
25) foss/2022a 59) LibTIFF/4.3.0-GCCcore-11.3.0 93) libgeotiff/1.7.1-GCCcore-11.3.0
26) bzip2/1.0.8-GCCcore-11.3.0 60) Java/11.0.16 94) pybind11/2.9.2-GCCcore-11.3.0
27) expat/2.4.8-GCCcore-11.3.0 61) Tk/8.6.12-GCCcore-11.3.0 95) SciPy-bundle/2022.05-foss-2022a
28) libpng/1.6.37-GCCcore-11.3.0 62) cURL/7.83.0-GCCcore-11.3.0 96) libtirpc/1.3.2-GCCcore-11.3.0
29) Brotli/1.0.9-GCCcore-11.3.0 63) GMP/6.2.1-GCCcore-11.3.0 97) HDF/4.2.15-GCCcore-11.3.0
30) freetype/2.12.1-GCCcore-11.3.0 64) NLopt/2.7.1-GCCcore-11.3.0 98) GDAL/3.5.0-foss-2022a
31) ncurses/6.3-GCCcore-11.3.0 65) libogg/1.3.5-GCCcore-11.3.0 99) MPFR/4.1.0-GCCcore-11.3.0
32) util-linux/2.38-GCCcore-11.3.0 66) FLAC/1.3.4-GCCcore-11.3.0 100) libgit2/1.4.3-GCCcore-11.3.0
33) fontconfig/2.14.0-GCCcore-11.3.0 67) libvorbis/1.3.7-GCCcore-11.3.0
34) xorg-macros/1.19.3-GCCcore-11.3.0 68) libopus/1.3.1-GCCcore-11.3.0
As you can see R has been removed from the environment. This is not true, however, for the dependencies that were loaded as well. In most cases the ''unload'' or ''del'' subcommand is therefore not very useful.
==== module purge ====
In order to completely clear the list of loaded modules the subcommand ''purge'' must be used. This is an effective way of making sure all modules are gone, before loading a new package.
[username@habrok ~]$ module purge
The following modules were not unloaded:
(Use "module --force purge" to unload all):
1) 2023.01 2) StdEnv
[username@habrok ~]$ module list
Currently Loaded Modules:
1) 2023.01 (S) 2) StdEnv (S)
Where:
S: Module is Sticky, requires --force to unload or purge
[username@habrok ~]$
Note that, even with ''purge'', it will not unload the two default modules ''StdEnv'' and ''2023.01'', as you will lose access to all modules in that case. If you do really want to unload those ones as well, you can use ''module --force purge''. In order to regain access to the modules of the ''2023.01'' software stack later on, you can use ''module load 2023.01''.
==== module save ====
If you have a set of modules that you need to use more often, you can save this set into a configuration which is stored for reuse later. This can be done using the subcommand ''restore''.
[username@habrok ~]$ module load BLAST+/2.13.0-gompi-2022a
[username@habrok ~]$ module load Stacks/2.62-foss-2022a
[username@habrok ~]$ module save my_favorite_modules
Saved current collection of modules to: "my_favorite_modules"
[username@habrok ~]$
==== module restore ====
In order to restore a set of saved modules into the environment the subcommand ''restore'' has to be used:
[username@habrok ~]$ module purge
The following modules were not unloaded:
(Use "module --force purge" to unload all):
1) 2023.01 2) StdEnv
[username@habrok ~]$ module list
Currently Loaded Modules:
1) 2023.01 (S) 2) StdEnv (S)
Where:
S: Module is Sticky, requires --force to unload or purge
[username@habrok ~]$ module restore my_favorite_modules
Restoring modules from user's my_favorite_modules
[username@habrok ~]$ module list
Currently Loaded Modules:
1) 2023.01 (S) 13) libevent/2.1.12-GCCcore-11.3.0 25) libpng/1.6.37-GCCcore-11.3.0
2) StdEnv (S) 14) UCX/1.12.1-GCCcore-11.3.0 26) NASM/2.15.05-GCCcore-11.3.0
3) GCCcore/11.3.0 15) libfabric/1.15.1-GCCcore-11.3.0 27) libjpeg-turbo/2.1.3-GCCcore-11.3.0
4) zlib/1.2.12-GCCcore-11.3.0 16) PMIx/4.1.2-GCCcore-11.3.0 28) LMDB/0.9.29-GCCcore-11.3.0
5) binutils/2.38-GCCcore-11.3.0 17) UCC/1.0.0-GCCcore-11.3.0 29) BLAST+/2.13.0-gompi-2022a
6) GCC/11.3.0 18) OpenMPI/4.1.4-GCC-11.3.0 30) OpenBLAS/0.3.20-GCC-11.3.0
7) numactl/2.0.14-GCCcore-11.3.0 19) gompi/2022a 31) FlexiBLAS/3.2.0-GCC-11.3.0
8) XZ/5.2.5-GCCcore-11.3.0 20) bzip2/1.0.8-GCCcore-11.3.0 32) FFTW/3.3.10-GCC-11.3.0
9) libxml2/2.9.13-GCCcore-11.3.0 21) PCRE/8.45-GCCcore-11.3.0 33) FFTW.MPI/3.3.10-gompi-2022a
10) libpciaccess/0.16-GCCcore-11.3.0 22) ICU/71.1-GCCcore-11.3.0 34) ScaLAPACK/2.2.0-gompi-2022a-fb
11) hwloc/2.7.1-GCCcore-11.3.0 23) Boost/1.79.0-GCC-11.3.0 35) foss/2022a
12) OpenSSL/1.1 24) GMP/6.2.1-GCCcore-11.3.0 36) Stacks/2.62-foss-2022a
[username@habrok ~]$
----
**Next section: [[..:data_management:transferring_data]]**