CINECA IRIS Institutional Research Information System
IRIS è il sistema di gestione integrata dei dati della ricerca (persone, progetti, pubblicazioni, attività) adottato dall'Università degli Studi di Torino.
AperTO è l'archivio istituzionale Open Access destinato a raccogliere, rendere visibile e conservare la produzione scientifica dell'Università degli Studi di Torino.
EnglishSolving a classification problem for a neural network means looking for a particular configuration of the internal parameters. This is commonly achieved by minimizing non-convex object functions. Hence, the same classification problem is likely to have several, different, equally valid solutions, depending on a number of factors like the initialization and the adopted optimizer. In this work, we propose an algorithm which looks for a zero-error path joining two solutions to the same classification problem. We witness that finding such a path is typically not a trivial problem; however, our heuristics is able to succeed in such a task. This is a step forward to explain why simple training heuristics (like SGD) are able to train complex neural networks: we speculate they focus on particular solutions, which belong to a connected solution sub-space. We work in two different scenarios: a synthetic, unbiased and totally-uncorrelated (hard) training problem, and MNIST. We empirically show that the algorithmically-accessible solutions space is connected, and we have hints suggesting it is a convex sub-space. © 2019, Springer Nature Switzerland AG.
| Titolo: | Take a Ramble into Solution Spaces for Classification Problems in Neural Networks |
| Autori Riconosciuti: | |
| Autori: | Tartaglione, Enzo; Grangetto, Marco |
| Data di pubblicazione: | 2019 |
| Abstract: | Solving a classification problem for a neural network means looking for a particular configuration of the internal parameters. This is commonly achieved by minimizing non-convex object functions. Hence, the same classification problem is likely to have several, different, equally valid solutions, depending on a number of factors like the initialization and the adopted optimizer. In this work, we propose an algorithm which looks for a zero-error path joining two solutions to the same classification problem. We witness that finding such a path is typically not a trivial problem; however, our heuristics is able to succeed in such a task. This is a step forward to explain why simple training heuristics (like SGD) are able to train complex neural networks: we speculate they focus on particular solutions, which belong to a connected solution sub-space. We work in two different scenarios: a synthetic, unbiased and totally-uncorrelated (hard) training problem, and MNIST. We empirically show that the algorithmically-accessible solutions space is connected, and we have hints suggesting it is a convex sub-space. © 2019, Springer Nature Switzerland AG. |
| Editore: | Springer Verlag |
| Titolo del libro: | International Conference on Image Analysis and Processing, ICIAP 2019 |
| Volume: | 11751 |
| Pagina iniziale: | 345 |
| Pagina finale: | 355 |
| Nome del convegno: | International Conference on Image Analysis and Processing, ICIAP 2019 |
| Luogo del convegno: | Trento |
| Anno del convegno: | 9/9/2019 |
| Digital Object Identifier (DOI): | 10.1007/978-3-030-30642-7_31 |
| ISBN: | 978-3-030-30641-0 978-3-030-30642-7 |
| Appare nelle tipologie: | 04A-Conference paper in volume |
File in questo prodotto:
| File | Descrizione | Tipologia | Licenza | |
|---|---|---|---|---|
| ICIAP19_takearamble.pdf | POSTPRINT (VERSIONE FINALE DELL’AUTORE) | Open Access Visualizza/Apri |
http://hdl.handle.net/2318/1714235
Copyright © 2021