The Research Computing team has launched a dedicated, robust, large capacity scientific data storage system called Engram to support research and collaboration for Zuckerman researchers. The system has a capacity of 2.8 petabytes (2,800 terabytes) and is designed to accommodate growth of tens of thousands of terabytes if required.
Engram can be accessed remotely by using the CUIT Virtual Private Network (VPN) service. (More VPN setup details provided here (Limited access UNI login required))
Engram is a highly versatile scale-out network-attached storage (NAS) platform (EMC Isilon) that can provide fast access to massive amounts of unstructured data. Each of the 24 Engram storage nodes can respond to client requests and all the nodes are connected via a redundant 10 gigabit per second (Gbps) Ethernet fiber optic network. Your data on Engram is backed up to tape, and a copy is stored off-site in a safe and secure location, for disaster recovery cases.
Engram is designed as the primary storage for the Institute, and you are encouraged to centralize your data and workflows around it. You can do any or all of this on Engram:
We can help you find the best way to use Engram for your particular research - please contact us at rc@zi.columbia.edu.
Engram can be accessed remotely by using the CUIT Virtual Private Network (VPN) service. (More VPN setup details provided here (Limited access UNI login required))
Connect to Engram from Mac
Connect to Engram from Windows
Connect to Engram from Linux
Connect to Engram from Synology
Engram has three storage levels.
Each lab gets either:
Additional storage can be purchased in 1 TB increments.
Billing occurs every 3 months via your chart string.
Rates (in the table below) are per Terabyte (TB) per year.
Name | Purpose | What It's For | Rates |
---|---|---|---|
Staging | High performance | For computing and analytics on data from multiple servers at the same time. | $250 |
Labshare | General purpose | Good for simple/light data analytics. | $120 |
Locker | Longer-term | For daily use and sharing files within/between labs and external collaborators. | $75 |
We can help you decide which storage level to start with. Please contact us at rc@zi.columbia.edu.
You control the data stored on Engram. Your data will never be deleted. Only you can delete your data.
Engram is network-attached storage (NAS). It is just like a USB drive you plug into your computer - but instead of plugging the USB drive into your computer with a USB cable, Engram is connected to your server or computer using the network.
In Windows, Engram shows up as a network drive with a letter (for example P:). On Mac, you will see it as a folder in the Finder. In Linux, it shows up just like a USB drive, mounted under its own directory.
For all practical purposes, an Engram network drive has the same functionality as a local drive or folder. It is plugged into your computer or server. You can access data on it just like on a USB drive or a local directory.
And since it is a network drive, it can be accessed from multiple computers and servers. In other words, multiple computers can access the same data, at the same time.
Engram can be accessed:
Data on Engram labshare and locker levels is backed up to tape. We can back up your data on staging storage level if requested.
Tapes are encrypted and periodically moved to a secure offsite location.
Tape backup is the only data protection technology that protects your data against ransomware. Replication (syncing) solutions do not protect against ransomware.
To request data restore please open a restore request by sending email to rc@zi.columbia.edu and tell us:
If your lab already has a network drive and you need access to it, please send the following information to rc@zi.columbia.edu:
To request new storage please send the following information to rc@zi.columbia.edu:
Labshare and Locker storage levels are automatically backed up to tape. If you need the Staging storage level to be backed up on tape please mention this in your service request.
Please check the storage pricing above (under "Tiers of Storage and Rates").
Engram supports three different types of network drives:
We can help you decide on Engram size for your lab or project, please contact us at rc@zi.columbia.edu.
To add additional storage capacity to your network drive please send the following information to rc@zi.columbia.edu:
If you have problems connecting to Engram, please follow these troubleshooting steps, in order:
If, after all this, you still cannot connect to Engram, please contact us at rc@zi.columbia.edu
and include:
Engram is a 24-node storage cluster. All 24 nodes serve data. Data is backed up to tape. A copy of each tape is taken off-site to a secure facility.
Would you like to see what Engram looks like? Email us at rc@zi.columbia.edu and we will give you a tour.
Set up Columbia University VPN (CUIT Service)
Change your UNI password (CUIT Service)