CC* Integration-Large: Prototyping a Secure Distributed Storage Infrastructure for Accelerating Big Science

Big science communities routinely publish and share vast amounts of data. Scalable, resilient, interoperable and controllable access to published data is essential to support data ­intensive science and engineering research. To facilitate scientific data publication and accessibility, this project aims to create a secure, resilient, and distributed data storage framework, Hydra, that enables scientific communities to build a loose federation of data repositories potentially owned by multiple administrative entities.

By taking a data-centric approach and leveraging the Named Data Networking (NDN) primitives, Hydra provides a generic software framework that allows scientists to publish datasets easily with data-centric security, automated access control, and automated data replication. It also facilitates access to data published on Hydra as well as other existing data repositories.

The project team aims to test Hydra using NSF's FABRIC testbed and utilize its built-in logging capability to verify and validate its usability, scalability, and security.

Hydra enables secure and scalable data access to alleviate the publication, data access, and security problems currently faced by a large number of scientific communities. Hydra has the potential to benefit scientific communities such as climate, meteorology, astrophysics, geology, and more through streamlined data publication, built-in data authenticity and access control, improved availability, and reduced network traffic.

The publicly available website for this project is https://hydra-repo.io. This website maintains pointers to various repositories that host up-to-date codebase, user documentation, hydra-repo operational data logs, and publications. This website will be available at least three years beyond the project's duration.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.