HPC-Cloud-Big Data Convergent Architectures and Research Data Management: The LEXIS Approach

HPC-Cloud-Big Data Convergent Architectures and Research Data Management: The LEXIS Approach. Hachinger, S., Martinovič, J., Terzo, O., Levrier, M., Scionti, A., Magarielli, D., Goubier, T., Parodi, A., Harsh, P., Apopei, F., Munke, J., García-Hernández, R., Golasowski, M., Hayek, M., Donnat, F., Ganne, L., Koch-Hofer, C., Vitali, G., Viviani, P., Schorlemmer, D., Danovaro, E., Parodi, A., Murphy, S., & Dees, A. In Proceedings of International Symposium on Grids & Clouds 2021 — PoS(ISGC2021), volume 378, pages 004, 2021.
doi abstract bibtex

The LEXIS project (Large-scale EXecution for Industry & Society, H2020 GA825532) provides a platform for optimised execution of Cloud-HPC workflows, reducing computation time and increasing energy efficiency. The system will rely on advanced, distributed orchestration solutions (Atos YSTIA Suite, with Alien4Cloud and Yorc, based on TOSCA), the High-End Application Execution Middleware HEAppE, and new hardware capabilities for maximising efficiency in data processing, analysis and transfer (e.g. Burst Buffers with GPU- and FPGA-based data reprocessing). LEXIS handles computation tasks and data from three Pilots, based on representative and demanding HPC/Cloud-Computing use cases in Industry (SMEs) and Science: i) Simulations of complex turbomachinery and gearbox systems in Aeronautics, ii) Tsunami simulations and earthquake loss assessments which are time-constrained to enable immediate warnings and to support well-informed decisions, and iii) Weather and Climate simulations where massive amounts of in-situ data are assimilated to improve forecasts. A user-friendly LEXIS web portal, as a unique entry point, will provide access to data as well as workflow-handling and remote visualisation functionality. As part of its back-end, LEXIS builds an elaborate system for the handling of input, intermediate and result data. At its core, a Distributed Data Infrastructure (DDI) ensures the availability of LEXIS data at all participating HPC sites, which will be federated with a common LEXIS Authentication and Authorisation Infrastructure (with unified security model, user database and policies). The DDI leverages best of breed data-management solutions from EUDAT, such as B2SAFE (based on iRODS) and B2HANDLE. REST APIs on top of it will ensure a smooth interaction with LEXIS workflows and the orchestration layer. Last, but not least, the DDI will provide functionalities for Research Data Management following the FAIR principles (“Findable, Interoperable, Accessible, Reusable”), e.g. DOI acquisition, which helps to publish and disseminate open data products.

@inproceedings{21:lexis,
  author = "Hachinger, Stephan  and  Martinovič, Jan  and  Terzo, Olivier  and  Levrier, Marc  and  Scionti, Alberto  and  Magarielli, Donato  and  Goubier, Thierry  and  Parodi, Antonio  and  Harsh, Piyush  and  Apopei, Florin-Ionut  and  Munke, Johannes  and  García-Hernández, Rubén  and  Golasowski, Martin  and  Hayek, Mohamad  and  Donnat, Frédéric  and  Ganne, Laurent  and  Koch-Hofer, Cédric  and  Vitali, Giacomo  and  Viviani, Paolo  and  Schorlemmer, Danijel  and  Danovaro, Emanuele  and  Parodi, Andrea  and  Murphy, Seán  and  Dees, Aaron",
  title = "{HPC-Cloud-Big Data Convergent Architectures and Research Data Management: The LEXIS Approach}",
  doi = "10.22323/1.378.0004",
  booktitle = "Proceedings of International Symposium on Grids & Clouds 2021 {\textemdash} PoS(ISGC2021)",
  year = 2021,
  volume = "378",
  pages = "004",
  abstract     = {The LEXIS project (Large-scale EXecution for Industry & Society, H2020 GA825532) provides a platform for optimised execution of Cloud-HPC workflows, reducing computation time and increasing energy efficiency. The system will rely on advanced, distributed orchestration solutions (Atos YSTIA Suite, with Alien4Cloud and Yorc, based on TOSCA), the High-End Application Execution Middleware HEAppE, and new hardware capabilities for maximising efficiency in data processing, analysis and transfer (e.g. Burst Buffers with GPU- and FPGA-based data reprocessing). LEXIS handles computation tasks and data from three Pilots, based on representative and demanding HPC/Cloud-Computing use cases in Industry (SMEs) and Science: i) Simulations of complex turbomachinery and gearbox systems in Aeronautics, ii) Tsunami simulations and earthquake loss assessments which are time-constrained to enable immediate warnings and to support well-informed decisions, and iii) Weather and Climate simulations where massive amounts of in-situ data are assimilated to improve forecasts. A user-friendly LEXIS web portal, as a unique entry point, will provide access to data as well as workflow-handling and remote visualisation functionality. As part of its back-end, LEXIS builds an elaborate system for the handling of input, intermediate and result data. At its core, a Distributed Data Infrastructure (DDI) ensures the availability of LEXIS data at all participating HPC sites, which will be federated with a common LEXIS Authentication and Authorisation Infrastructure (with unified security model, user database and policies). The DDI leverages best of breed data-management solutions from EUDAT, such as B2SAFE (based on iRODS) and B2HANDLE. REST APIs on top of it will ensure a smooth interaction with LEXIS workflows and the orchestration layer. Last, but not least, the DDI will provide functionalities for Research Data Management following the FAIR principles (“Findable, Interoperable, Accessible, Reusable”), e.g. DOI acquisition, which helps to publish and disseminate open data products.}
}

Downloads: 0