Page Comparison

In order for the scripts in these examples to work, you will need to replace <ACCOUNT> with your group's account name.

...

This program will print out "Hello World!" when run on a gpu server or print "Hello Hello" when no gpu module is found. 

Singularity 

Singularity is a software tool that brings Docker-like containers and reproducibility to scientific computing and HPC. Singularity has Docker container support and enables users to easily  run different flavors of Linux with different software stacks. These containers provide a single universal on-ramp from the laptop, to HPC, to cloud.

Users can run Singularity containers just as they run any other program on our HPC clusters. Example usage of Singularity is listed below. For additional details on how to use Singularity, please contact us or refer to the Singularity User Guide.

Downloading Pre-Built Containers

Singularity makes it easy to quickly deploy and use software stacks or new versions of software. Since Singularity has Docker support, users can simply pull existing Docker images from Docker Hub or download docker images directly from software repositories that increasingly support the Docker format. Singularity Container Library also provides a number of additional containers.

You can use the pull command to download pre-built images from an external resource into your current working directory. The docker:// uri reference can be used to pull Docker images. Pulled Docker images will be automatically converted to the Singularity container format. 

...

Here's an example of pulling the latest stable release of the Tensorflow Docker image and running it with Singularity. (Note: these pre-built versions may not be optimized for use with our CPUs.)

...

Singularity - Interactive Shell 

The shell command allows you to spawn a new shell within your container and interact with it as though it were a small virtual machine:

...

Singularity tensorflow.simg:~> python
>>> import tensorflow as tf
>>> print(tf.__version__)
1.13.1
>>> exit()

When done, you may exit the Singularity interactive shell with the "exit" command.

Singularity tensorflow.simg:~> exit

Singularity: Executing Commands

The exec command allows you to execute a custom command within a container by specifying the image file. This is the way to invoke commands in your job submission script.

...

Singularity: Running a Batch Job

Below is an example of job submission script named submit.sh that runs Singularity. Note that you may need to specify the full path to the Singularity image you wish to run.

#!/bin/bash
# Singularity example submit script for Slurm.
#
# Replace <ACCOUNT> with your account name before submitting.
#
#SBATCH -A <ACCOUNT>           # Set Account name
#SBATCH --job-name=tensorflow  # The job name
#SBATCH -c 1                   # Number of cores
#SBATCH -t 0-0:30              # Runtime in D-HH:MM
#SBATCH --mem-per-cpu=4gb      # Memory per cpu core

module load singularity
singularity exec tensorflow.simg python -c 'import tensorflow as tf; print(tf.__version__)'

Then submit the job to the scheduler. This example prints out the tensorflow version.

$ sbatch submit.sh

For additional details on how to use Singularity, please contact us or refer to the Singularity User Guide.

Example of R run

For this example, the R code below is used to generate a graph ''Rplot.pdf'' of a discrete Delta-hedging of a call. It hedges along a path and repeats over many paths. There are two R files required:

...

Tensorflow

The following describes is how you can import to run tensorflow on a Terremoto GPU node.

First, run an interactive job requesting one GPU on a GPU node

$ srun --pty -t 0-02:00:00 --gres=gpu:1 -A <group_name> /bin/bash

Load modules:Then load the singularity environment module and run the tensorflow container, which was built from the Tensorflow docker image. You can start an interactive singularity shell and specify the --nv flag which instructs singularity to use the Nvidia GPU driver.

$ module load cuda92/toolkit cuda92/blas cudnn/5.1

Start python and test tensorflow:

$ load singularity


$ singularity shell --nv /moto/opt/singularity/tensorflow-1.13-gpu-py3-moto.simg


Singularity tensorflow-1.13-gpu-py3-moto.simg:~> python
Python 23.75.132 |Anaconda custom (64-bit)| (default, DecNov 2012 20162018, 2313:0943:1514) 
[GCC 45.4.7 20120313 (Red Hat 4.4.7-1)0 20160609] on linux2
linux
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
..
>>> print(sess.run(hello))
Hello, TensorFlow!

The following describes is how you can import tensorflow on GPU P100 node.

Load modules:

$ module load cuda92/toolkit 
$ module load cuda92/blas 
$ module load cudnn


****
Default anaconda module comes with python 3.5.2
$ module load anaconda


If you never installed tensorflow-gpu package please use this command once:
$ pip install upgrade tensorflow-gpu --user

Start python and test tensorflow:

$ python 

Python 3.5.2 |Anaconda 4.2.0 (64-bit)| (default, Jul 2 2016, 17:53:06) 

[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux 

Type "help", "copyright", "credits" or "license" for more information. 



>>> import tensorflow as tf 

>>> print(tf.__version__) 

1.8.0 

>>> hello = tf.constant('Hello, TensorFlow!') 


>>> sess = tf.Session() 

totalMemory: 15.89GiB freeMemory: 14.61GiB 

name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285 




>>> print(sess.run(hello).decode()) 

Hello, TensorFlow! 

exit()

You may type "exit" to exit when you're done with the Singularity shell.

Singularity tensorflow-1.13-gpu-py3-moto.simg:~> exit

Below is an example of job submission script named submit.sh that runs Tensorflow with GPU support using Singularity. 

#!/bin/bash
# Tensorflow with GPU support example submit script for Slurm.
#
# Replace <ACCOUNT> with your account name before submitting.
#
#SBATCH -A <ACCOUNT>           # Set Account name
#SBATCH --job-name=tensorflow  # The job name
#SBATCH -c 1                   # Number of cores
#SBATCH -t 0-0:30              # Runtime in D-HH:MM
#SBATCH --gres=gpu:1           # Request a gpu module

module load singularity
singularity exec --nv /moto/opt/singularity/tensorflow-1.13-gpu-py3-moto.simg python -c 'import tensorflow as tf; print(tf.__version__)'

Then submit the job to the scheduler. This example prints out the tensorflow version.

$ sbatch submit.sh

For additional details on how to use Singularity, please contact us, see our Singularity documentation, or refer to the Singularity User Guide.

Jupyter Notebooks

This is one way to set up and run a jupyter notebook on Terremoto. As your notebook will listen on a port that will be accessible to anyone logged in on a submit node you should first create a password.

...