Overview
To better server serve our research community who require high data usage and demand in our environmentHPC environment to run ML/AI kind of jobs, DSBIT is re-vamping the storage to ultra fast NVMe based Weka storage.
...
Storage Tiers | C2B2 Cluster | New Cluster | ||
|---|---|---|---|---|
home | home/PI_gp/memberUNI | users/memberUNI | ||
data | data/PI_gp/memberUNI | groups/PI_gp/memberUNI | scratch | scratch/PI_gp/memberUNI |
groups/PI_gp/scratch/memberUNI | archive | archive/PI_gp/memberUNI | archive/PI_gp/memberUNI |
...
Shared space of 1TB storage allocated by default to each PI_group.
General space for group software, common data, etc.
POSIX/NFS shared
Backed up weekly
Scratch
Shared space under Groups.
Temporary storage space that can be used while running jobs.
NFS mounted on HPC cluster only
Never, but you can archive your files to archive
...
Archive
Shared space for each group.
PI can purchase spinning disk storage to store their data for long term.
SMB mounted only, can be accessed via Globus
Backed up monthly to another S3 block storage device (in the same building but different floor)
NOTE: Groups, Scratch
Scratch storage tier is not offered in Weka, although PI can create a directory called Scratch to temporarily copy data required to run their jobs under groups/PI_gp storage tier.
Groups and Archive tier storage are available in capacity increments of 1TB.
Pricing details for each of the tiers will be shared once the New Cluster launchis launched.
For any questions/comments, please send an email to dsbit_help@cumc.columbia.edu
...