Twister2: TSet high-performance iterative dataflow. Wickramasinghe, P., Kamburugamuve, S., Govindarajan, K., Abeykoon, V., Widanage, C., Perera, N., Uyar, A., Gunduz, G., Akkas, S., & Fox, G. In 2019 International Conference on High Performance Big Data and Intelligent Systems, HPBD and IS 2019, pages 55-60, 5, 2019. Institute of Electrical and Electronics Engineers Inc..
Twister2: TSet high-performance iterative dataflow [pdf]Paper  doi  abstract   bibtex   
The dataflow model is slowly becoming the de facto standard for big data applications. While many popular frameworks are built around the dataflow model, very little research has been done on understanding the inner workings of the dataflow model. This has led to many inefficiencies in existing frameworks. It is important to note that understanding the relation between dataflow and HPC building blocks allows us to address and alleviate many of the fundamental inefficiencies in dataflow by learning from the extensive research literature in the HPC community. In this paper, we present TSet's, the dataflow abstraction of Twister2, which is a big data framework designed for high-performance dataflow and iterative computations. We discuss the dataflow model adopted by TSet's and the rationale behind implementing iteration handling at the worker level. Finally, we evaluate TSet's to show the performance of the framework.

Downloads: 0