Simplifying Big-Data Infrastructure with a Name-based Ecosystem - Experiences and Lessons Learned

Abstract: This talk outlines our efforts in creating a name-based ecosystem for big science. It touches on various aspects of the big-data ecosystem, such as data naming, publication and discovery, distributed storage, and integration of these tools with scientific workflows. It discusses how such an ecosystem can simplify big-data management. The talk then focuses on one particular project - Hydra. Hydra aims to create a decentralized, federated, and geographically distributed data repository across different administrative domains based on names. Hydra is different than existing distributed databases. Hydra does not require a centralized controller, allows specific security policies based on local security constraints, and can automatically recover from failures. The talk discusses the design decisions, implementation choices, and our initial experience deploying it on FABRIC. Finally, we outline the lessons learned from our exercises in building Hydra and discuss open research questions.

Dr. Susmit Shannigrahi is an Assistant Professor in the Computer Science department at Tennessee Tech. His research interests include big-data management and movement over wide-area networks, networking for
AR/VR, vehicular networks, and 5G. Susmit obtained his Ph.D. from Colorado State University in 2019 where he was advised by Craig Partridge and Christos Papadopoulos. He has been on the Faculty of Tennessee Tech since 2019 where he runs the Next-Generation Networking Lab (tntech-ngin.net).