Available Software
At this time the cluster really only has Anaconda and Cuda installed for work on systems like Tensorflow and Pytorch. We use environment modules which you can find out more about here. The preferred method of installing software is to work off of Anaconda environments we will have updated documentation on this shortly. If you need additional software installed please email rc@zi.columbia.edu to request it.
Environment Modules
Axon uses a system of environmental modules, specifically using a piece of software called lmod, which allows people to specify the version of the program they want to run enabling the cluster to seamlessly have multiple versions of a single program. Below is an example of how it works.
[aa3301@axon ~]$ python --version Python 2.7.5 [aa3301@axon ~]$ which python /usr/bin/python [aa3301@axon ~]$ ml anaconda3-2019.03 [aa3301@axon ~]$ python --version Python 3.7.3 [aa3301@axon ~]$ which python /share/apps/anaconda3-2019.03/bin/python
In the example above when we login and run python we get the default system python, but if we run ml (which can also be run as module load) and load anaconda we get a much newer version with a bunch of modules already installed.
[aa3301@axon ~]$ ml Currently Loaded Modules: 1) anaconda3-2019.03 [aa3301@axon ~]$ ml av ------------------------------------------------------------------------------------------------------- /share/modulefiles/manual ---------------------------------------------------------------------------------------------------- anaconda3-2019.03 (L) cuda@10.0.130 cuda@10.1.168 cuda@9.2.88 cudnn@7.3.0 cudnn@7.5.1-10.0 cudnn@7.6.1.34-10.0 cudnn@7.6.1.34-10.1 Where: L: Module is loaded Use "module spider" to find all possible modules. Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
Running the ml (or module list) will show you the currently loaded modules, and ml av (or module avail) will show you the available modules to load.
[aa3301@axon ~]$ ml add cuda@10.0.130 cudnn@7.3.0 cudnn@7.6.1.34-10.1 keyword purge save spider unuse whatis anaconda3-2019.03 cuda@10.1.168 cudnn@7.5.1-10.0 delete list restore show swap update avail cuda@9.2.88 cudnn@7.6.1.34-10.0 help load rm sl unload use [aa3301@axon ~]$ module load cud cuda@10.0.130 cuda@10.1.168 cuda@9.2.88 cudnn@7.3.0 cudnn@7.5.1-10.0 cudnn@7.6.1.34-10.0 cudnn@7.6.1.34-10.1
The ml alias is convenient shortcut to most commands. If you type ml and press tab it will show you all the possible completion the same is true of the module load command.