Improving generalization is one of the main challenges for training deep neural networks on classification tasks. In particular, a number of techniques have been proposed, aiming to boost the performance on unseen data: from standard data augmentation techniques to the ℓ2 regularization, dropout, batch normalization, entropy-driven SGD and many more. In this work we propose an elegant, simple and principled approach: post-synaptic potential regularization (PSP). We tested this regularization on a number of different state-of-the-art scenarios. Empirical results show that PSP achieves a classification error comparable to more sophisticated learning strategies in the MNIST scenario, while improves the generalization compared to ℓ2 regularization in deep architectures trained on CIFAR-10.
Titolo: | Post-synaptic Potential Regularization Has Potential |
Autori Riconosciuti: | |
Autori: | Tartaglione, Enzo; Perlo, Daniele; Grangetto, Marco |
Data di pubblicazione: | 2019 |
Abstract: | Improving generalization is one of the main challenges for training deep neural networks on classification tasks. In particular, a number of techniques have been proposed, aiming to boost the performance on unseen data: from standard data augmentation techniques to the ℓ2 regularization, dropout, batch normalization, entropy-driven SGD and many more. In this work we propose an elegant, simple and principled approach: post-synaptic potential regularization (PSP). We tested this regularization on a number of different state-of-the-art scenarios. Empirical results show that PSP achieves a classification error comparable to more sophisticated learning strategies in the MNIST scenario, while improves the generalization compared to ℓ2 regularization in deep architectures trained on CIFAR-10. |
Editore: | Springer |
Titolo del libro: | Artificial Neural Networks and Machine Learning – ICANN 2019: Deep Learning. ICANN 2019 |
Volume: | 11728 |
Pagina iniziale: | 187 |
Pagina finale: | 200 |
Nome del convegno: | International Conference on Artificial Neural Networks |
Luogo del convegno: | Munich, Germany |
Anno del convegno: | 17-19/09/2019 |
Digital Object Identifier (DOI): | 10.1007/978-3-030-30484-3_16 |
ISBN: | 978-3-030-30483-6 978-3-030-30484-3 |
Appare nelle tipologie: | 04A-Conference paper in volume |