Skip to content

Software and Modules

HPC platforms service large numbers of users, over a broad variety of scientific and computational discipilines. Consequently, the domain of software operated on most HPCs is astronomical. Decades of experience managing shared HPC have taught administrators two imporant lessons:

  1. There is no one, perfect (or even functional) configuration.
  2. It is impractical for a sys admin – or even a team of admins, to attempt to dictate the best research software solutions.

Consequently, HPC administration has adopted tools and practices to provide central repositories of core packages, while permitting users to compile, install, mix, and match the software they need to do research. The general principles and tools employed include:

  1. Users will not have root access and cannot install software into the (typically) default, system locations (eg., /usr/local). Note this means that package managers such as apt, apt-get, yum, and brew will not be available to ordinary users.
  2. Users can still install software to various locations, including $HOME, $GROUP_HOME, $SCRATCH, $GROUP_SCRATCH, or any other space where the user has write and execute privilegse.
  3. NOTE: The specific paths available for (2) above will vary from one HPC platform to another. Also, installing to these paths may require setting some environment variables (eg $PATH, $LD_LIBRARY_PATH) to make the software available.
  4. Software is available as “modules”, using GNU modules, LMOD, or some other modules platform. These systems make software available, enabling users to configure their software environment, primarily by setting environment and path variables.
  5. Modules are not just for admins! Users can develop module scripts for their own custom insalled software.

Building and Installing Software

Build packages, such as *.rpm and *.deb files will typically not work properly without sudo access. Users will not be granted sudo access on the HPC, so plese don’t ask. Nonetheless, software can almost always be installed and/or compiled to a user defined location. This is typically accompilished via something like:

  • ./configure --prefix=$HOME/my_sw
  • cmake -DCMAKE_INSTALL_PREFIX:PATH=$HOME/my_sw
  • pip install --user my_sw_package
  • conda install my_sw_package

It may then be necessary to amend some path variables. Assuming that we install binaries (executables) and libraries to $HOME/my_sw/bin and $HOME/my_sw/lib, respectively, in the command line (for temorary scope), in your ~/.bashrc, or even better in a module file:

  • export PATH=$HOME/my_sw/bin:$PATH
  • export LD_LIBRARY_PATH=$HOME/my_sw/lib:$LD_LIBRARY_PATH

Modules

Sherlock uses the popular LMOD module system. Module scripts are written for individual packages, or sometimes to make a group of package available together.

To see available modules,

module avail

To look for a module, for example a module “like” NetCDF, use module spider, for example:

module spider netcdf

Sherlock may include some hierarchical module stacks, and sets of modules that take advantage of LMOD’s built in package, version interpretation. Typically, LMOD/lua modules are stored and interepred like {package_name}/{version}.lua, and LMOD is smart enoug to interpret some incomplete entries. For example, if my_sw is available in versions 2.1., 2.0.0, 1.9, module load my_sw/ will load v2.1, unless a different default is defined; module load my_sw/2.0 and module load my_sw/1. will load versions 2.0.0 and 1.9, respectively.

In addition to the standard SW modules available on Sherlock, some SDSS specific software, including CMG, dune, seissol, and Schlumberger applications have been compiled (or installed) and can be loaded as modules. These modules can be activated by using the module use command:

module use /home/groups/sh_s-dss/share/sdss/modules/modulefiles

General purpose, comprehensive SW stacks may also be available, but they are currently very much in beta state. If standard Sherlock software modules are not sufficient to compile and maintain your software, please request support.

Note that software and modules may have somewhat rigorous compatibility requirements for various dependencies and supporting packages. Of most particular interest, some packages will have dependencies on a specific version of a compiler or MPI.