Overview
To keep up with the high data usage and demand in our environment, DSBIT is re-vamping the storage to ultra fast NVMe based Weka storage.
WekaFS is a fully distributed parallel filesystem leveraging NVMe Flash for file services. Integrated tiering seamlessly expands the namespace to and from HDD object storage, simplifying data management. WekaFS stands out with its unique architecture, overcoming legacy systems’ scaling and file-sharing limitations. Supporting POSIX, NFS, SMB, S3, and GPUDirect Storage.
What’s NEW in storage?
Storage Tiers | C2B2 Cluster | New Cluster |
|---|---|---|
home | home/PI_gp/memberUNI | users/memberUNI |
data | data/PI_gp/memberUNI | groups/PI_gp/memberUNI |
scratch | scratch/PI_gp/memberUNI | groups/PI_gp/scratch/memberUNI |
archive | archive/PI_gp/memberUNI | archive/PI_gp/memberUNI |
Users (Previously home)
Exclusive storage space of 20GB assigned to each C2B2 Cluster user.
Used for small input files, source code files and executables, software builds etc.
POSIX/NFS shared
Backed up nightly.
Groups (Previously data)
Shared space of 1TB storage allocated by default to each PI_group.
General space for group software, common data, etc.
POSIX/NFS shared
Backed up weekly
Scratch
Shared space under Groups.
Temporary storage space that can be used while running jobs.
NFS mounted on HPC cluster only
Never, but you can archive your files to archive
NOTE: Files will be periodically deleted as per our autodelete policy.
Archive
Shared space for each group.
PI can purchase spinning disk storage to store their data for long term.
SMB mounted only, can be accessed via Globus
Backed up monthly to another S3 block storage device (in the same building but different floor)
NOTE: Groups, Scratch and Archive tier storage are available in capacity increments of 1TB.