Scaling JupyterHub Using Kubernetes on Jetstream Cloud

Scaling JupyterHub Using Kubernetes on Jetstream Cloud. Sarajlic, S., Chastang, J., Marru, S., Fischer, J., & Lowe, M. In Proceedings of the Practice and Experience on Advanced Research Computing - PEARC '18, pages 1-4, 7, 2018. ACM Press.

Paper

Scaling JupyterHub Using Kubernetes on Jetstream Cloud [link]

Website doi abstract bibtex

Unidata, an NSF funded project that started in 1983, is a diverse community of education and research institutions with the common goal of sharing geoscience data and the tools to access and visualize that data. Unidata provides weather observations and other data, software tools, and support to enhance Earth-system education and research, and continuously examines ways of adapting their workflows for new technologies to maximize the reach of their education and research efforts. In support of Unidata objectives to host workshops for atmospheric data analysis using JupyterHub, we explore a cloud computing approach leveraging Kubernetes coupled with JupyterHub that when combined will provide a solution for researchers and students to pull data from Unidata and burst onto Jetstream cloud by requesting resources dynamically via easy to use JupyterHub. More specifically, on Jetstream, Kubernetes is used for automating deployment and scaling of domain specific containerized applications, and JupyterHub is used for spawning multiple hubs within the same Kubernetes cluster instance that will be used for supporting classroom settings. JupyterHub's modular kernel feature will support dynamic needs of classroom application requirements. The proposed approach will serve as an end-to-end solution for researchers to execute their workflows, with JupyterHub serving as a powerful tool for user training and next-generation workforce development in atmospheric sciences.

@inproceedings{
 title = {Scaling JupyterHub Using Kubernetes on Jetstream Cloud},
 type = {inproceedings},
 year = {2018},
 keywords = {Cloud Computing,JupyterHub,Kubernetes,Magnum,OpenStack,Unidata,Workforce Development},
 pages = {1-4},
 websites = {http://dl.acm.org/citation.cfm?doid=3219104.3229249},
 month = {7},
 publisher = {ACM Press},
 day = {22},
 city = {New York, New York, USA},
 id = {fc553946-212b-396c-83bf-e9b38820388d},
 created = {2019-10-01T17:20:23.818Z},
 accessed = {2019-08-19},
 file_attached = {true},
 profile_id = {42d295c0-0737-38d6-8b43-508cab6ea85d},
 last_modified = {2020-05-11T14:43:43.786Z},
 read = {false},
 starred = {false},
 authored = {true},
 confirmed = {true},
 hidden = {false},
 citation_key = {Sarajlic2018a},
 private_publication = {false},
 abstract = {Unidata, an NSF funded project that started in 1983, is a diverse community of education and research institutions with the common goal of sharing geoscience data and the tools to access and visualize that data. Unidata provides weather observations and other data, software tools, and support to enhance Earth-system education and research, and continuously examines ways of adapting their workflows for new technologies to maximize the reach of their education and research efforts. In support of Unidata objectives to host workshops for atmospheric data analysis using JupyterHub, we explore a cloud computing approach leveraging Kubernetes coupled with JupyterHub that when combined will provide a solution for researchers and students to pull data from Unidata and burst onto Jetstream cloud by requesting resources dynamically via easy to use JupyterHub. More specifically, on Jetstream, Kubernetes is used for automating deployment and scaling of domain specific containerized applications, and JupyterHub is used for spawning multiple hubs within the same Kubernetes cluster instance that will be used for supporting classroom settings. JupyterHub's modular kernel feature will support dynamic needs of classroom application requirements. The proposed approach will serve as an end-to-end solution for researchers to execute their workflows, with JupyterHub serving as a powerful tool for user training and next-generation workforce development in atmospheric sciences.},
 bibtype = {inproceedings},
 author = {Sarajlic, Semir and Chastang, Julien and Marru, Suresh and Fischer, Jeremy and Lowe, Mike},
 doi = {10.1145/3219104.3229249},
 booktitle = {Proceedings of the Practice and Experience on Advanced Research Computing - PEARC '18}
}

Downloads: 0