HPC and AI are fated to meet for several reasons. This article will discuss some of them and argue why this will happen through the methods and technologies underpinning cloud computing. As a paradigmatic example, we present a new Federated Learning (FL) system that collaboratively trains a deep learning model in different supercomputing centers. The system is based on the StreamFlow workflow manager designed for hybrid cloud-HPC infrastructures.
Federated Learning meets HPC and cloud
Iacopo Colonnelli
First
;Bruno Casella;Gianluca Mittone;Yasir Arfat;Barbara Cantalupo;Roberto Esposito;Alberto Riccardo Martinelli;Doriana Medic;Marco AldinucciLast
2023-01-01
Abstract
HPC and AI are fated to meet for several reasons. This article will discuss some of them and argue why this will happen through the methods and technologies underpinning cloud computing. As a paradigmatic example, we present a new Federated Learning (FL) system that collaboratively trains a deep learning model in different supercomputing centers. The system is based on the StreamFlow workflow manager designed for hybrid cloud-HPC infrastructures.
File in questo prodotto:
| File | Dimensione | Formato | |
|---|---|---|---|
|
ML4Astro.pdf
Accesso aperto
Descrizione: Preprint
Tipo di file:
PREPRINT (PRIMA BOZZA)
Dimensione
284.73 kB
Formato
Adobe PDF
|
284.73 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
