, pages 97:1–97:11, Los Alamitos, CA, USA, 2012. IEEE Computer Society Press\n
\n
@inproceedings{Bhatele:2012:MAC:2388996.2389128,\n author = {Bhatele, Abhinav and Gamblin, Todd and Langer, Steven H. and Bremer, Peer-Timo and Draeger, Erik W. and Hamann, Bernd and Isaacs, Katherine E. and Landge, Aaditya G. and Levine, Joshua A. and Pascucci, Valerio and Schulz, Martin and Still, Charles H.},\n title = {Mapping Applications with Collectives over Sub-communicators on Torus Networks},\n booktitle = {Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis},\n series = {SC '12},\n year = {2012},\n isbn = {978-1-4673-0804-5},\n location = {Salt Lake City, Utah},\n pages = {97:1--97:11},\n abstract={The placement of tasks in a parallel application on specific nodes of a supercomputer can significantly impact performance. Traditionally, this task mapping has focused on reducing the distance between communicating tasks on the physical network. This minimizes the number of hops that point-to-point messages travel and thus reduces link sharing between messages and contention. However, for applications that use collectives over sub-communicators, this heuristic may not be optimal. Many collectives can benefit from an increase in bandwidth even at the cost of an increase in hop count, especially when sending large messages. For example, placing communicating tasks in a cube configuration rather than a plane or a line on a torus network increases the number of possible paths messages might take. This increases the available bandwidth which can lead to significant performance gains.\nWe have developed Rubik, a tool that provides a simple and intuitive interface to create a wide variety of mappings for structured communication patterns. Rubik supports a number of elementary operations such as splits, tilts, or shifts, that can be combined into a large number of unique patterns. Each operation can be applied to disjoint groups of processes involved in collectives to increase the effective bandwidth. We demonstrate the use of Rubik for improving performance of two parallel codes, pF3D and Qbox, which use collectives over sub-communicators.},\n articleno = {97},\n numpages = {11},\n url = {papers/bhatele-rubik-mapping-sc12.pdf},\n acmid = {2389128},\n publisher = {IEEE Computer Society Press},\n address = {Los Alamitos, CA, USA},\n}\n\n
\n
\n The placement of tasks in a parallel application on specific nodes of a supercomputer can significantly impact performance. Traditionally, this task mapping has focused on reducing the distance between communicating tasks on the physical network. This minimizes the number of hops that point-to-point messages travel and thus reduces link sharing between messages and contention. However, for applications that use collectives over sub-communicators, this heuristic may not be optimal. Many collectives can benefit from an increase in bandwidth even at the cost of an increase in hop count, especially when sending large messages. For example, placing communicating tasks in a cube configuration rather than a plane or a line on a torus network increases the number of possible paths messages might take. This increases the available bandwidth which can lead to significant performance gains. We have developed Rubik, a tool that provides a simple and intuitive interface to create a wide variety of mappings for structured communication patterns. Rubik supports a number of elementary operations such as splits, tilts, or shifts, that can be combined into a large number of unique patterns. Each operation can be applied to disjoint groups of processes involved in collectives to increase the effective bandwidth. We demonstrate the use of Rubik for improving performance of two parallel codes, pF3D and Qbox, which use collectives over sub-communicators.\n