Australia’s National Computational Infrastructure (NCI) has purchased two high-performance storage solutions to provide an advanced data platform for researchers.

The new storage systems, purchased from Fujitsu/NetApp and Hewlett Packard Enterprise (HPE) will replace NCI’s original 8 Petabyte Lustre filesystem, named gdata1, which was purchased in 2011 and has now reached the end of its operational life.

Construction of NCI’s global filesystems began in 2013 to meet researcher demands for a large, fast, persistent filesystem to support growing data sets required in High-Performance Computing and High-Performance Data analysis, made possible with the then newly deployed Raijin supercomputer.  

NCI says it wants to enable data to be accessed both on high-performance supercomputers and by researchers on NCI’s data-intensive cloud environment.

“This innovative integrated environment has delivered efficiency gains to researchers by negating the time-consuming process of copying data from one computer system to another,” the NCI said in a statement.

It says it is enabling “multiple research groups on different systems to access and work concurrently on the same shared data with the appropriate security permissions”.

The first stage of the gdata1 replacement from Fujitsu will use the NetApp E-series storage arrays to provide a proven and robust Lustre file system design in excess of 10 PB.

The second stage of the gdata1 replacement is from HPE utilising the HPE Apollo 4520 High Performance Computing storage and will provide a ZFS based Lustre file system with approximately 12 PB of usable storage.   

The new gdata1 system will have Mellanox EDR InfiniBand, providing performance of approximately 70GB/sec of bandwidth connection to NCI’s 83,068 core Raijin supercomputer. 

The additional systems will take NCI’s total data storage capacity to over 36 Petabytes, enabling NCI to continue to meet the demand for Australia’s rapidly expanding nationally significant data collections.