BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CERN//INDICO//EN
BEGIN:VEVENT
SUMMARY:Converging Storage Technologies Using a Flexible HPC Storage Frame
 work
DTSTART;VALUE=DATE-TIME:20201202T101500Z
DTEND;VALUE=DATE-TIME:20201202T104500Z
DTSTAMP;VALUE=DATE-TIME:20260614T105044Z
UID:indico-contribution-1315@events.chpc.ac.za
DESCRIPTION:Speakers: Michael Kuhn (Otto von Guericke University Magdeburg
 )\nTraditionally\, file systems are mostly monolithic\, making it hard to 
 experiment with new approaches and technologies. Exchanging core functiona
 lity within a file system is a burdensome task\, leading to a lack of inno
 vation in this area. However\, data volumes are growing rapidly because th
 e ability to capture and produce data is increasing at an exponential rate
 . Rising core counts and data volumes present challenges for contemporary 
 storage systems\, especially regarding metadata performance and data manag
 ement. This makes it even more important to investigate new ways of storin
 g and managing data efficiently.\n\nHPC storage systems are typically desi
 gned around POSIX-compliant parallel distributed file systems that are acc
 essed using sophisticated I/O libraries. The file system and library layer
 s are strictly separated for portability reasons. While this allows exchan
 ging individual layers\, their complexities pose a high barrier of entry. 
 This is especially problematic for shorter research projects and presents 
 a significant hurdle for young researchers and students.\n\nWithin the JUL
 EA and CoSEMoS projects\, we are aiming to change this. JULEA is a flexibl
 e storage framework that can be used to prototype new ideas related to sto
 rage and file systems. It allows offering arbitrary I/O interfaces to appl
 ications and includes interfaces for object\, key-value and database stora
 ge. The framework has been designed to be easy to set up and run without a
 dministrative privileges\, so it can be used on a wide range of software a
 nd hardware environments. It also serves as the foundation of the CoSEMoS 
 project\, which explores the benefits of a coupled storage system for self
 -describing data formats\, such as HDF5 and ADIOS2. This allows the storag
 e system to manage file metadata found within these data formats and makes
  it possible to use structural information for selecting appropriate stora
 ge technologies. For instance\, metadata can be stored in database systems
  that can be queried efficiently. Moreover\, making use of established dat
 a formats allows running existing applications without modifications\, whi
 ch helps preserve past investments in software development.\n\nCoSEMoS ena
 bles novel data management approaches via a data analysis interface that g
 ives applications direct access to JULEA's backends\, eliminating the need
  to sift through large volumes of data to find relevant data points. Break
 ing up the strict separation has additional long-term benefits\, such as b
 eing able to take data migration decisions based on structural information
  found within the file formats.\n\nThis talk will briefly introduce the JU
 LEA and CoSEMoS projects\, and show the opportunities enabled by their nov
 el storage system design.\n\nhttps://events.chpc.ac.za/event/84/contributi
 ons/1315/
LOCATION:
URL:https://events.chpc.ac.za/event/84/contributions/1315/
END:VEVENT
END:VCALENDAR