Page Comparison

Ginsburg is currently running Red Hat Enterprise Linux release 8.2.

The table below shows software already installed on the cluster system-wide.

...

Running LBPM in a Singularity container from Nvidia's NGC Catalog

LBPM (Lattice Boltzmann Methods for Porous Media) is an open source software framework designed to model flow processes based on digital rock physics, and is freely available through the Open Porous Media project. Digital rock physics refers to a growing class of methods that leverage microscopic data sources to obtain insight into the physical behavior of fluids in rock and other porous materials. LBPM simulation protocols are based on two-fluid lattice Boltzmann methods, focusing in particular on wetting phenomena. The Department of Earth and Environmental Engineering and the Kelly Lab are the main users

...

The first time you launch Mathematica you will need to provide the host details of the MathLM (license) server. Using the Activating Mathematica guide, click 'Other ways to activate' then choose 'Connect to a Network License Server' and enter the IP address, 128.59.30.140

OpenMPI Settings 

Insomnia has a few MPI options loadable as modules:

• openmpi5/5.0.2

• mpi/mpich-x86_64/4.1.1

• mpi/openmpi-x86_64/4.1.1

• Intel oneAPI/hpctoolkit/mpi/2021.11

If you find that a mpirun hangs or does not complete, try adding the following options:

 -mca coll ^hcollThe latest version of OpenMPI on Insomnia is  4.1.5a1, which is provided by Nvidia Mellanox and optimized for the MOFED stack. You will receive the following warnings when using mpirun/mpiexec:

WARNING: There was an error initializing an OpenFabrics device.

  Local host:   g###
  Local device: mlx5_0

Default device parameters will be used, which may result in lower performance. You can edit any of the files specified by the
btl_openib_device_param_files MCA parameter to set values for your device.

NOTE: You can turn off this warning by setting the MCA parameter btl_open ib_warn_no_device_params_found to 0.

You can pass the following option, which will use ucx which is default as of version 3.x:

--mca pml ucx --mca btl '^openib' 

To help with the following warning:

Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

You can also add:

--mca orte_base_help_aggregate 0

If you choose to use the openmpi/gcc/64/4.1.1_cuda_11.0.3_aware module, this version expects a GPU and will throw the following warning on non-GPU nodes:

The library attempted to open the following supporting CUDA libraries, but each of them failed. CUDA-aware support is disabled.
libcuda.so.1: cannot open shared object file: No such file or directory
libcuda.dylib: cannot open shared object file: No such file or directory
/usr/lib64/libcuda.so.1: cannot open shared object file: No such file or directory
/usr/lib64/libcuda.dylib: cannot open shared object file: No such file or directory

If you are not interested in CUDA-aware support, then run with --mca opal_warn_on_missing_libcuda 0 to suppress this message. If you are interested in CUDA-aware support, then try setting LD_LIBRARY_PATH to the location of libcuda.so.1 to get passed this issue.

You can pass this option:

--mca opal_warn_on_missing_libcuda 0

RStudio in a Singularity container

...

singularity pull --name rstudio.simg docker://rocker/rstudio:4.3.1

In order for RStudio to start in a browser via an interactive session you will need the IP address of the compute node. Note that the IP below will likely be different for you:

IP_ADDR=`hostname -i`
echo $IP_ADDR
10.197.16.39

From RStudio 4.2 and later, some added security features require binding of a locally created database file to the database in the container. Don't forget to change the password.

mkdir -p run var-lib-rstudio-server

printf 'provider=sqlite\ndirectory=/var/lib/rstudio-server\n' > database.conf

PASSWORD='CHANGEME' singularity exec \
   --bind run:/run,var-lib-rstudio-server:/var/lib/rstudio-server,database.conf:/etc/rstudio/database.conf \
   rstudio.simg \
   /usr/lib/rstudio-server/bin/rserver --auth-none=0 --auth-pam-helper-path=pam-helper --server-user=$USER

This will run the Rstudio server in a Singularity container.

Now open another Terminal and start the RStudio rserver session using Port Forwarding to connect a local port on your computer to a remote one on Ginsburg.

...