Machine Learning Systems Are Stuck in a Rut. Barham, P. and Isard, M. In Proceedings of the Workshop on Hot Topics in Operating Systems, of HotOS '19, pages 177–183. ACM.
Machine Learning Systems Are Stuck in a Rut [link]Paper  doi  abstract   bibtex   
In this paper we argue that systems for numerical computing are stuck in a local basin of performance and programmability. Systems researchers are doing an excellent job improving the performance of 5-year-old benchmarks, but gradually making it harder to explore innovative machine learning research ideas. We explain how the evolution of hardware accelerators favors compiler back ends that hyper-optimize large monolithic kernels, show how this reliance on high-performance but inflexible kernels reinforces the dominant style of programming model, and argue these programming abstractions lack expressiveness, maintainability, and modularity; all of which hinders research progress. We conclude by noting promising directions in the field, and advocate steps to advance progress towards high-performance general purpose numerical computing systems on modern accelerators.
@inproceedings{barhamMachineLearningSystems2019,
  location = {{New York, NY, USA}},
  title = {Machine {{Learning Systems Are Stuck}} in a {{Rut}}},
  isbn = {978-1-4503-6727-1},
  url = {http://doi.acm.org/10.1145/3317550.3321441},
  doi = {10.1145/3317550.3321441},
  abstract = {In this paper we argue that systems for numerical computing are stuck in a local basin of performance and programmability. Systems researchers are doing an excellent job improving the performance of 5-year-old benchmarks, but gradually making it harder to explore innovative machine learning research ideas. We explain how the evolution of hardware accelerators favors compiler back ends that hyper-optimize large monolithic kernels, show how this reliance on high-performance but inflexible kernels reinforces the dominant style of programming model, and argue these programming abstractions lack expressiveness, maintainability, and modularity; all of which hinders research progress. We conclude by noting promising directions in the field, and advocate steps to advance progress towards high-performance general purpose numerical computing systems on modern accelerators.},
  booktitle = {Proceedings of the {{Workshop}} on {{Hot Topics}} in {{Operating Systems}}},
  series = {{{HotOS}} '19},
  publisher = {{ACM}},
  urldate = {2019-06-07},
  date = {2019},
  pages = {177--183},
  author = {Barham, Paul and Isard, Michael},
  file = {/home/dimitri/Nextcloud/Zotero/storage/KK5T5X4X/Barham and Isard - 2019 - Machine Learning Systems Are Stuck in a Rut.pdf},
  venue = {Bertinoro, Italy}
}
Downloads: 0