Zuckerman Storage Service

The Research Computing team has launched a dedicated, robust, large capacity storage system called Engram to support research and collaboration for Zuckerman researchers.

The system has an initial capacity of roughly two (2) petabytes (2,000 terabytes) and is designed to accommodate growth of tens of thousands of terabytes (TB) over the next years if required.

We can help you find the best way to use Engram for your particular research.

Contact us at support@zi.columbia.edu.

About Engram

Engram is a large-scale scientific data storage system. It has 24 storage nodes and each node serves data. It is designed as the primary storage for the Institute.

You are encouraged to centralize your data and workflows around it. You can do any or all of this on Engram:

• Connect all your computers and servers to it
  
• Collect data directly to it
• Compute and analyze data on it
  

Your data on Engram is backed up to tape, and a copy is stored off-site in a safe and secure location, for disaster recovery cases.

If you have questions or would like to sit down together to determine how to best take advantage of Engram, please contact us at support@zi.columbia.edu.

How to Connect to Engram

Connect to Engram from Mac
Connect to Engram from Windows
Connect to Engram from Linux
Connect to Engram from Synology

Types of Storage and Rates

Engram has three storage levels.

Each lab gets either:

• 10 TB of free locker level disk space, or
•   7 TB of free labshare level disk space, or
•   5 TB of free staging level disk space

Additional storage can be purchased in 1 TB increments.

Billing occurs every 6 months via your chart string.

Rates (in the table below) are per Terabyte (TB) per year.

We can help you decide which storage level to start with. Please contact us at support@zi.columbia.edu.

Data Life Cycle

You control the data stored on Engram. Your data will never be deleted. Only you can delete your data.

Accessing Your Data

Engram is network-attached storage (NAS), also known as network drive in Windows or linux drive on Mac/Linux.

It is just like a USB drive you plug into your computer - but instead of plugging the USB drive into your computer with a USB cable, Engram is on the network.

In Windows, Engram shows up as a network drive with a letter (for example P:). On Mac, you will see it as a folder in Finder. In Linux, it shows up just like a USB drive, mounted under its own directory.

For all practical purposes, Engram network drive has a functionality of a local drive or folder. It is directly attached to your computer or server. You can access data on it just like on a USB drive or a local directory.

And since it is a network drive, it can be accessed from multiple computers and servers. In other words, multiple computers can access the same data, at the same time.

Engram can be accessed:

• From anywhere on Columbia University downtown campus or Manhattanville
  • By connecting your computer to the network with a physical network cable
  • By using "Columbia U Secure" Wi-Fi
  • Note: "Columbia University" Wi-Fi is open and insecure and cannot be used to reach Engram
• From outside the University by using Virtual Private Network (VPN) ( VPN Setup )
• From CUMC: under special circumstances we will work with CUMC IT to open their firewall to allow access to Engram from CUMC locations. Please contact us at support@zi.columbia.edu

Backing Up Your Data

Data on Engram labshare and locker levels is backed up to tape. We can back up your data on staging storage level if requested.

Tapes are encrypted and periodically moved to a secure offsite location.

Tape backup is the only data protection technology that protects your data against ransomware. Replication (syncing) solutions do not protect against ransomware.

Restoring Your Data

To request data restore please open a restore request by sending email to support@zi.columbia.edu and tell us:

1. The lab you are from
2. The directory from which you would like the files to be restored
3. A date range (for example, please restore these files from September 15, 2017)

Requesting Access to Existing Storage

If your lab already has a network drive, you can request access to it by emailing support@zi.columbia.edu.

Request should include:

1. Your Lab
2. Your full name and UNI
3. The name of the network drive you need access to

Requesting New Storage

Please email support@zi.columbia.edu and tell us:

1. How large a network drive you need
2. The storage level (Staging, Labshare, Locker)
3. List of UNIs that should have access to this network drive

Labshare and Locker storage levels are automatically backed up to tape. If you need the Staging storage level to be backed up on tape please mention this in your service request.

Please check the storage pricing above (Types of Storage and Rates).

If you need more information or would like to discuss your particular workflows and how you could best use Engram, we are happy to set up a meeting. Please contact us at support@zi.columbia.edu.

Requesting More Storage

If you would like to add additional storage capacity to your network drive, please email support@zi.columbia.edu. In your request include:

1. Your Lab
2. Your full name and UNI
3. The name of the network drive you would like more storage capacity added to
4. How much additional storage capacity you need, in 1 TB increments

Problems Connecting to Engram

If you have problems connecting to Engram, please follow these troubleshooting steps, in order:

1. Have you ever successfully connected to your Engram drive before?
2. Have you requested access to your Engram drive?
3. For username, make sure you put ADCU in front of your UNI (Windows): ADCU\Your_UNI
4. Are you typing your UNI password correctly? Can you log into other Columbia University services?
5. Make sure your computer is connected to "Columbia U Secure" Wi-Fi network. "Columbia University" Wi-Fi is open and insecure and cannot be used to to reach Engram
6. Reboot your computer

If, after all this, you still cannot connect to Engram, please open a ticket by sending email to support@zi.columbia.edu. Please include:

• Name of your lab
• Your full name and UNI
• Engram drive full path (example: locker-nfs.engram.rc.zi.columbia.edu\YourLab-locker)

Engram Photo

Engram is a 24-node storage cluster. All 24 nodes serve data. Data is backed up to tape. A copy of each tape is taken off-site to a secure facility.

Important Links

Set up Columbia University VPN (CUIT Service)
Change your UNI password (CUIT Service)