Research Infrastructure: CC* Data Storage: Multi-Petabyte Open Storage (MPOS) at the American Museum of Natural History

This project provides the American Museum of Natural History (AMNH) a major expansion of data management and large-scale centralized storage supporting a broad range of research projects and provides supporting services to facilitate the acquisition, processing, and sharing of data. This significant expansion of storage and additional services allows AMNH to address storage and data management needs institutionally and addresses long-standing problems acutely felt across the Museum’s research divisions and graduate school.

In this project, AMNH expands its data storage capabilities using Open Storage Network (OSN) storage pods, providing over 2 petabytes (PB) of centralized storage for active research data. The OSN pods are connected to the Museum’s Science DMZ, a high-performance network designed for research data flows, which provides connections to onsite high-performance computing (HPC) clusters, data transfer nodes (DTNs) in labs, and analytical systems, as well as to other institutions and resources in the cloud via a connection to the Internet2.

The storage provided by the OSN pods is directly accessible by both local and cloud-based resources such as compute nodes at various cloud services and supercomputing centers nationally. AMNH leverages the Integrated Rule-Oriented Data System (iRODS) for data management, indexing, automated ingest, storage tiering, compliance, data integrity, and publishing services.

The Clowder Framework provides a scalable data management framework supporting any data format across multiple research domains. Finally, AMNH makes available 20% of its allocated OSN storage to the shared OSN cloud, making it accessible to researchers nationally.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.