Parallel Execution of Workflows driven by Distributed Database Techniques. Souza, R. & Mattoso, M. In Satelite Events of the 32nd Brazilian Symposium on Databases (SBBD), Thesis and Dissertations Contest, pages 163–168, Uberlândia, MG, October, 2017. Sociedade Brasileira de Computação. Best MS dissertation
Parallel Execution of Workflows driven by Distributed Database Techniques [pdf]Paper  abstract   bibtex   
Many-Task Computing (MTC) workflow executions involve thousands of parallel tasks that consume and produce large amounts of data and are scheduled on multiple nodes in a large HPC cluster. A complete run may last for weeks. Users have to analyze and steer the dataflow at runtime. This introduces several challenges for efficient data management without jeopardizing performance. This dissertation combines distributed data management techniques (ACID transactions, concurrency control, and database design) to propose a scalable solution for MTC workflows. Domain data, dataflow provenance, and workflow execution data are managed together in an in-memory distributed DBMS. As a result, a distributed scheduling via transactions in this database attains high scalability in a 1,008-cores HPC cluster, while maintaining runtime data analytical capabilities.

Downloads: 0