Skip to main content

Storage Products

A number of different storage products and storage-related services are available to researchers at Monash University, each of which can be customised to accommodate specific criteria. These products include:



Market


Vault


Compute


Object

In addition to the above, there are also a number of services, tools and applications that offer data storage and related features such as:

Storage Product Overview​

Researchers can request Market, Vault, Compute and Object allocations and each storage product has different uses as well as advantages and disadvantages. New storage allocations can be requested via the Data Dashboard.

Market​

Security Classification Security Classification

Market storage (also known as Market-file) is a disk-based storage product that is designed to support research data that is accessed regularly (usually every day) by researchers. Market storage closely resembles the Monash University S:\ Drive and is suitable for data that changes frequently and needs to be available for daily work but is too large to be stored on a local machine. Market storage can be presented as a desktop share/network drive (SMB) or can be mounted on a server (NFS).

Market storage can be provided for large collections, but for very large collections (more than 10TB) a combination of Market and Vault storage is recommended. For such collections, it is recommended that a smaller Market allocation should be used for data that is currently being used or analysed while older, unused data should be stored in a Vault allocation and moved to Market as required. Market storage can also be made accessible to the Internet (not recommended for critical or sensitive data) using Aspera, a web-based service that is useful for sharing large amounts of data with external collaborators. Note that Market storage has less overall capacity than Vault storage and allocations are assigned individual quotas. Amendments and extensions for Market allocations can be requested through the Data Dashboard.

Vault​

Security Classification Security Classification

Vault storage is a tape-based storage product that is best used to store data that is not accessed frequently, or needs to be retained in the long-term. It is best used for archiving. Vault storage can be presented as a desktop share/network drive(SMB), can be mounted on a server (NFS) or can be accessed via SFTP or rsync.

The most common Vault storage use-cases are large collections of archived data that are accessed infrequently or backups of data that is stored locally. Vault allocations are often recommended as part of large Market allocations because data that is no longer required can be moved to Vault while data that needs to be used/analysed can be moved to and from the Market allocation as needed.

Due to the way in which magnetic tapes store data, it is recommended that users, especially those with large numbers of files (tens of thousands or more), split their data up into discrete elements (e.g. datasets) and bundle them up into archives (ZIP, TAR, etc) before storing them in the Vault. This will ensure that the data can be retrieved from tape efficiently and reduce wait time. Large amounts of data can be recalled by the RDSM team if necessary, which is known as a "bulk recall". Bulk recalls are uncommon and are assessed on a case-by-case basis.

Vault storage can be made accessible to the Internet (not recommended for critical or sensitive data) using Aspera, which is useful for sharing large amounts of data with external collaborators. There is a significant amount of Vault capacity available and allocations are assigned individual quotas. Amendments and extensions for Vault allocations can be requested through the Data Dashboard.

Compute​

Security Classification Security Classification

Compute storage (also known as volume storage) is a disk-based storage product that is best used as a scratch space when running parallel computational jobs on the Racmon cluster. Racmon is the Victorian node of the Nectar service. Compute storage can only be used by Nectar VMs (Virtual Machines) and, as a result, the storage is inextricably linked to specific VMs called "instances" which are created for specific projects. Nectar VMs can be requested through the Nectar Dashboard while Compute storage should be requested through the Data Dashboard. Additional information about compute storage can be found here. Note that amendments and extensions for Compute allocations must be submitted via the Nectar Dashboard.

Object​

Security Classification Security Classification

Object storage is a disk-based storage product that is best used for more static collections of data that will be accessed randomly and in any order by users. It is most commonly used by web-based services that provide access to large collections of data such as image galleries. Object storage is organised differently to more traditional storage and is generally recommended for unstructured collections that will be written once and then read many times in an unpredictable fashion. Nectar's Introduction to object storage page provides a good summary of Object storage's features as well as related information. Note that amendments and extensions for Object allocations must be submitted via the Nectar Dashboard.