The Open Edge and HPC Initiative (OEHI) and the Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) have launched a collaborative Proof of Concept (PoC) project to explore data management in compute continuum infrastructures that seamlessly integrate HPC nodes, cloud, and edge environments. Traditionally, HPC nodes access data via a POSIX file system, while cloud and edge nodes interact through an object store interface. This initiative aims to develop and benchmark a unified software stack to enable HPC nodes to access data via a POSIX file system, while cloud and edge nodes interact through an object store interface. The primary goals are to enhance data management for complex workflows like AI applications deployed across diverse computing environments, foster a growing community of experts in compute continuum data management, and gain deeper insights into I/O patterns of AI workloads on HPC systems.

The project focuses on developing, implementing, and benchmarking workflows that showcase the potential of this unified architecture. Key objectives include documenting data management requirements, identifying system gaps, and optimising performance for AI tasks. Deliverables will feature technical white papers, scientific publications and presentations at major conferences. The collaboration also emphasizes community engagement through webinars, workshops, and knowledge-sharing events, aiming to cultivate an open ecosystem of researchers and companies specializing in data management.