Software and Modules
HPC platforms service large numbers of users, over a broad variety of scientific and computational discipilines. Consequently, the domain of software operated on most HPCs is astronomical. Decades of experience managing shared HPC have taught administrators two imporant lessons:
- There is no one, perfect (or even functional) configuration.
- It is impractical for a sys admin – or even a team of admins, to attempt to dictate the best research software solutions.
Consequently, HPC administration has adopted tools and practices to provide central repositories of core packages, while permitting users to compile, install, mix, and match the software they need to do research. The general principles and tools employed include:
- Users will not have
root
access and cannot install software into the (typically) default, system locations (eg.,/usr/local
). Note this means that package managers such asapt, apt-get, yum, and brew
will not be available to ordinary users. - Users can still install software to various locations, including
$HOME
,$GROUP_HOME
,$SCRATCH
,$GROUP_SCRATCH
, or any other space where the user has write and execute privilegse. - NOTE: The specific paths available for (2) above will vary from one HPC platform to another. Also, installing to these paths may require setting some environment variables (eg
$PATH
,$LD_LIBRARY_PATH
) to make the software available. - Software is available as “modules”, using GNU
modules
,LMOD
, or some other modules platform. These systems make software available, enabling users to configure their software environment, primarily by setting environment and path variables. - Modules are not just for admins! Users can develop module scripts for their own custom insalled software.
Building and Installing Software
Build packages, such as *.rpm
and *.deb
files will typically not work properly without sudo
access. Users will not be granted sudo
access on the HPC, so plese don’t ask.
Nonetheless, software can almost always be installed and/or compiled to a user defined location. This is typically accompilished via something like:
./configure --prefix=$HOME/my_sw
cmake -DCMAKE_INSTALL_PREFIX:PATH=$HOME/my_sw
pip install --user my_sw_package
conda install my_sw_package
It may then be necessary to amend some path variables. Assuming that we install binaries (executables) and libraries to $HOME/my_sw/bin
and $HOME/my_sw/lib
, respectively,
in the command line (for temorary scope), in your ~/.bashrc
, or even better in a module file:
export PATH=$HOME/my_sw/bin:$PATH
export LD_LIBRARY_PATH=$HOME/my_sw/lib:$LD_LIBRARY_PATH
Modules
Sherlock uses the popular LMOD
module system. Module scripts are written for individual packages, or sometimes to make a group of package
available together.
To see available modules,
module avail
To look for a module, for example a module “like” NetCDF, use module spider
, for example:
module spider netcdf
Sherlock may include some hierarchical module stacks, and sets of modules that take advantage of LMOD’s built in package, version interpretation.
Typically, LMOD/lua modules are stored and interepred like {package_name}/{version}.lua
, and LMOD is smart enoug to interpret some incomplete entries. For example,
if my_sw
is available in versions 2.1.
, 2.0.0
, 1.9
, module load my_sw/
will load v2.1
, unless a different default is defined; module load my_sw/2.0
and
module load my_sw/1.
will load versions 2.0.0
and 1.9
, respectively.
In addition to the standard SW modules available on Sherlock, some SDSS specific software, including CMG
, dune
, seissol
, and Schlumberger applications have been compiled (or installed) and can be loaded as modules. These modules can be activated by using the module use
command:
module use /home/groups/sh_s-dss/share/sdss/modules/modulefiles
General purpose, comprehensive SW stacks may also be available, but they are currently very much in beta
state. If standard Sherlock software modules are not sufficient to compile and maintain your software, please request support.
Note that software and modules may have somewhat rigorous compatibility requirements for various dependencies and supporting packages. Of most particular interest, some packages will have dependencies on a specific version of a compiler or MPI.