HPC and AI are fated to meet for several reasons. This article will discuss some of them and argue why this will happen through the methods and technologies underpinning cloud computing. As a paradigmatic example, we present a new Federated Learning (FL) system that collaboratively trains a deep learning model in different supercomputing centers. The system is based on the StreamFlow workflow manager designed for hybrid cloud-HPC infrastructures.

Federated Learning meets HPC and cloud

Iacopo Colonnelli
First
;
Bruno Casella;Gianluca Mittone;Yasir Arfat;Barbara Cantalupo;Roberto Esposito;Alberto Riccardo Martinelli;Doriana Medic;Marco Aldinucci
Last
2023-01-01

Abstract

HPC and AI are fated to meet for several reasons. This article will discuss some of them and argue why this will happen through the methods and technologies underpinning cloud computing. As a paradigmatic example, we present a new Federated Learning (FL) system that collaboratively trains a deep learning model in different supercomputing centers. The system is based on the StreamFlow workflow manager designed for hybrid cloud-HPC infrastructures.
2023
International Conference on Machine Learning for Astrophysics (ML4ASTRO)
Catania, Italy
30 May - 1 June 2022
Astrophysics and Space Science Proceedings
Springer
60
193
199
978-3-031-34167-0
Iacopo Colonnelli, Bruno Casella, Gianluca Mittone, Yasir Arfat, Barbara Cantalupo, Roberto Esposito, Alberto Riccardo Martinelli, Doriana Medic, Ma...espandi
File in questo prodotto:
File Dimensione Formato  
ML4Astro.pdf

Accesso aperto

Descrizione: Preprint
Tipo di file: PREPRINT (PRIMA BOZZA)
Dimensione 284.73 kB
Formato Adobe PDF
284.73 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2318/1881080
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? ND
social impact