Deep Learning at Scale. Viviani, P., Drocco, M., Baccega, D., Colonnelli, I., & Aldinucci, M. In Proc. of 27th Euromicro Intl. Conference on Parallel Distributed and network-based Processing (PDP), pages 124–131, Pavia, Italy, 2019. IEEE.
Deep Learning at Scale [pdf]Paper  doi  abstract   bibtex   
This work presents a novel approach to distributed training of deep neural networks (DNNs) that aims to overcome the issues related to mainstream approaches to data parallel training. Established techniques for data parallel training are discussed from both a parallel computing and deep learning perspective, then a different approach is presented that is meant to allow DNN training to scale while retaining good convergence properties. Moreover, an experimental implementation is presented as well as some preliminary results.

Downloads: 0