Efficient ranking and selection in parallel computing environments. Ni, E. C., Ciocan, D. F., Henderson, S. G., & Hunter, S. R. Operations Research, 65(3):821–836, May-June, 2017. Algorithm implemented in Simio Portal Edition (2017) and in Simio (2019).Paper doi abstract bibtex 3 downloads The goal of ranking and selection (R&S) procedures is to identify the best stochastic system from among a finite set of competing alternatives. Such procedures require constructing estimates of each system's performance, which can be obtained simultaneously by running multiple independent replications on a parallel computing platform. However, nontrivial statistical and implementation issues arise when designing R&S procedures for a parallel computing environment. Thus we propose several design principles for parallel R&S procedures that preserve statistical validity and maximize core utilization, especially when large numbers of alternatives or cores are involved. These principles are followed closely by our parallel Good Selection Procedure (GSP), which, under the assumption of normally distributed output, (i) guarantees to select a system in the indifference zone with high probability, (ii) runs efficiently on up to 1,024 parallel cores, and (iii) in an example uses smaller sample sizes compared to existing parallel procedures, particularly for large problems (over one million alternatives). In our computational study we discuss two methods for implementing GSP on parallel computers, namely the Message-Passing Interface (MPI) and Hadoop MapReduce and show that the latter provides good protection against core failures at the expense of a significant drop in utilization due to periodic unavoidable synchronization.
@article{2017nicioetal,
Year = {2017},
Author = {E. C. Ni and D. F. Ciocan and S. G. Henderson and S. R. Hunter},
Title = {Efficient ranking and selection in parallel computing environments},
Journal = {Operations Research},
volume = {65},
number = {3},
month = {May-June},
pages = {821--836},
doi = {10.1287/opre.2016.1577},
url_Paper = {https://arxiv.org/abs/1506.04986},
abstract = {The goal of ranking and selection (R\&S) procedures is to identify the best stochastic system from among a finite set of competing alternatives. Such procedures require constructing estimates of each system's performance, which can be obtained simultaneously by running multiple independent replications on a parallel computing platform. However, nontrivial statistical and implementation issues arise when designing R&S procedures for a parallel computing environment. Thus we propose several design principles for parallel R&S procedures that preserve statistical validity and maximize core utilization, especially when large numbers of alternatives or cores are involved. These principles are followed closely by our parallel Good Selection Procedure (GSP), which, under the assumption of normally distributed output, (i) guarantees to select a system in the indifference zone with high probability, (ii) runs efficiently on up to 1,024 parallel cores, and (iii) in an example uses smaller sample sizes compared to existing parallel procedures, particularly for large problems (over one million alternatives). In our computational study we discuss two methods for implementing GSP on parallel computers, namely the Message-Passing Interface (MPI) and Hadoop MapReduce and show that the latter provides good protection against core failures at the expense of a significant drop in utilization due to periodic unavoidable synchronization.},
keywords = {simulation optimization > single-objective > ranking and selection > parallel},
bibbase_note = {<span style="color: green">Algorithm implemented in Simio Portal Edition (2017) and in Simio (2019).</span>}}
Downloads: 3
{"_id":"n2qhjHpGnkCfSnGGL","bibbaseid":"ni-ciocan-henderson-hunter-efficientrankingandselectioninparallelcomputingenvironments-2017","downloads":3,"creationDate":"2017-03-14T16:52:18.216Z","title":"Efficient ranking and selection in parallel computing environments","author_short":["Ni, E. C.","Ciocan, D. F.","Henderson, S. G.","Hunter, S. R."],"year":2017,"bibtype":"article","biburl":"https://web.ics.purdue.edu/~hunter63/PAPERS/srhunterweb.bib","bibdata":{"bibtype":"article","type":"article","year":"2017","author":[{"firstnames":["E.","C."],"propositions":[],"lastnames":["Ni"],"suffixes":[]},{"firstnames":["D.","F."],"propositions":[],"lastnames":["Ciocan"],"suffixes":[]},{"firstnames":["S.","G."],"propositions":[],"lastnames":["Henderson"],"suffixes":[]},{"firstnames":["S.","R."],"propositions":[],"lastnames":["Hunter"],"suffixes":[]}],"title":"Efficient ranking and selection in parallel computing environments","journal":"Operations Research","volume":"65","number":"3","month":"May-June","pages":"821–836","doi":"10.1287/opre.2016.1577","url_paper":"https://arxiv.org/abs/1506.04986","abstract":"The goal of ranking and selection (R&S) procedures is to identify the best stochastic system from among a finite set of competing alternatives. Such procedures require constructing estimates of each system's performance, which can be obtained simultaneously by running multiple independent replications on a parallel computing platform. However, nontrivial statistical and implementation issues arise when designing R&S procedures for a parallel computing environment. Thus we propose several design principles for parallel R&S procedures that preserve statistical validity and maximize core utilization, especially when large numbers of alternatives or cores are involved. These principles are followed closely by our parallel Good Selection Procedure (GSP), which, under the assumption of normally distributed output, (i) guarantees to select a system in the indifference zone with high probability, (ii) runs efficiently on up to 1,024 parallel cores, and (iii) in an example uses smaller sample sizes compared to existing parallel procedures, particularly for large problems (over one million alternatives). In our computational study we discuss two methods for implementing GSP on parallel computers, namely the Message-Passing Interface (MPI) and Hadoop MapReduce and show that the latter provides good protection against core failures at the expense of a significant drop in utilization due to periodic unavoidable synchronization.","keywords":"simulation optimization > single-objective > ranking and selection > parallel","bibbase_note":"<span style=\"color: green\">Algorithm implemented in Simio Portal Edition (2017) and in Simio (2019).</span>","bibtex":"@article{2017nicioetal,\n\tYear = {2017},\n\tAuthor = {E. C. Ni and D. F. Ciocan and S. G. Henderson and S. R. Hunter},\n\tTitle = {Efficient ranking and selection in parallel computing environments},\n\tJournal = {Operations Research},\n\tvolume = {65},\n\tnumber = {3},\n\tmonth = {May-June},\n\tpages = {821--836},\n\tdoi = {10.1287/opre.2016.1577},\n\turl_Paper = {https://arxiv.org/abs/1506.04986},\n\tabstract = {The goal of ranking and selection (R\\&S) procedures is to identify the best stochastic system from among a finite set of competing alternatives. Such procedures require constructing estimates of each system's performance, which can be obtained simultaneously by running multiple independent replications on a parallel computing platform. However, nontrivial statistical and implementation issues arise when designing R&S procedures for a parallel computing environment. Thus we propose several design principles for parallel R&S procedures that preserve statistical validity and maximize core utilization, especially when large numbers of alternatives or cores are involved. These principles are followed closely by our parallel Good Selection Procedure (GSP), which, under the assumption of normally distributed output, (i) guarantees to select a system in the indifference zone with high probability, (ii) runs efficiently on up to 1,024 parallel cores, and (iii) in an example uses smaller sample sizes compared to existing parallel procedures, particularly for large problems (over one million alternatives). In our computational study we discuss two methods for implementing GSP on parallel computers, namely the Message-Passing Interface (MPI) and Hadoop MapReduce and show that the latter provides good protection against core failures at the expense of a significant drop in utilization due to periodic unavoidable synchronization.},\n\tkeywords = {simulation optimization > single-objective > ranking and selection > parallel},\n\tbibbase_note = {<span style=\"color: green\">Algorithm implemented in Simio Portal Edition (2017) and in Simio (2019).</span>}}\n\n","author_short":["Ni, E. C.","Ciocan, D. F.","Henderson, S. G.","Hunter, S. R."],"key":"2017nicioetal","id":"2017nicioetal","bibbaseid":"ni-ciocan-henderson-hunter-efficientrankingandselectioninparallelcomputingenvironments-2017","role":"author","urls":{" paper":"https://arxiv.org/abs/1506.04986"},"keyword":["simulation optimization > single-objective > ranking and selection > parallel"],"metadata":{"authorlinks":{"hunter, s":"https://web.ics.purdue.edu/~hunter63/"}},"downloads":3,"html":""},"search_terms":["efficient","ranking","selection","parallel","computing","environments","ni","ciocan","henderson","hunter"],"keywords":["simulation optimization > single-objective > ranking and selection > parallel"],"authorIDs":["2PAn3MHZBPgt7AWCw","2Z97GEaPa4YXAS8ny","2fb2S2XHAJ2t96GqF","2zh9DyNaWkygD5248","365LawuHG2j9DJRFe","3R63vuFS9Bfnm4SAW","3aJ4h5X54uMKyds8C","3dTRPZMyK6NQnkBNd","3fJLfDjoSLWT7fdsH","3wDvcSHgfLWgWtiCx","44WAdmxsAix3xNMiS","47j5kNH4LLRdyaGrh","4zadZEFLgF8nnfqBc","545708a08b01c81930000130","59NY6KQiaNfh8vHLK","5FBFKWvbGTDfY2rBw","5de6efe7eab4b7de010000cb","5de71da297054edf01000027","5de77dc6021482de010000f3","5de79fcfd9b46bde01000187","5de8f619978afbdf010001b9","5de95e87d574c6de0100012d","5de9cb9e5e5ac8de010000a8","5deac7d5cd660bde0100013b","5debba9fca1cdddf0100001f","5debc23aca1cdddf010000a3","5dec31c1d39dc7de01000079","5dec95d412c53edf01000100","5decfeeb3d02efdf010000c3","5dee96590ceb4cdf01000042","5deef47fdca291de010000b6","5def2332e83f7dde01000033","5defaf90d6a2dcde01000049","5defbb05d6a2dcde0100015e","5df0afd18367c8de0100005d","5df0b2dd8367c8de0100009d","5df0f25545b054df01000141","5df26ad663aac8df01000200","5df27fc327cd2fde0100013e","5df28f16cf8320de01000079","5df2f405b91ab0de01000180","5df3217b3b310cde010000ca","5df32ee83b310cde01000188","5df33739683178f301000067","5df36544d617e5de0100016e","5df385252b1f8ade01000008","5df396762b1f8ade0100011e","5df3df3eccbb4fde0100007a","5df4333396bfa7de01000047","5df4741a95416ade0100008e","5df4acfaa50a84df01000090","5df4c91a55b997de0100008a","5df4d56755b997de01000116","5df50a85daf598df01000126","5df516b7ea1457de01000068","5df5360afd245cde010000ba","5df5d11964d1d0df010000cf","5df631ba38e915de01000064","5df67c7a72bbd4df01000001","5df68f5a72bbd4df010000d6","5df7b85592a8e4df0100006b","5df7e5d6dc100cde010001b8","5df7f520b9bb17df010000bd","5df814fcd74ee7df010000df","5df85a56b99bcff301000001","5df88ca4db7d9ddf01000110","5df89f2610b1d1de010000a2","5df8f42b277e45de01000150","5df919f22e8c31df010001d6","5df92fe1d04b27df01000113","5df9503dccc001de0100013d","5dfbb3f6f6f0aede010000c8","5dfc0137b371afde0100005a","5dfd0d06ea1680de010000f3","5dfe83cc26fad1de0100001b","5dfedcd95dd8e7df01000011","5e015a766afa18de010000ae","5e02bf6319da8edf01000065","5e036b4cb1544ef2010000ab","5e04163d0fe3b7df01000099","5e07ed2cf1089ddf0100009a","5e08084ccdee3adf01000080","5e08ac387dc1dcdf0100006f","5e090c9079e131f3010000e7","5e093fc4934cacdf01000096","5e0a052252efb3de010000df","5e0a22ef52fbd9de010000e2","5e0acf4427625ede010000ec","5e0cd62c6762d1de01000093","5e0cf51b5631a6de01000094","5e0de73145d2fdde01000029","5e0e27de2b8028de0100007a","5e0e567fac7d11df01000039","5e0e86caa0f484df010000f1","5e0eb3bc03f891de01000054","5e0f244896e707df0100007a","5e0f4b622c4a31df01000129","5e0fbb33f350a2de010000ba","5e101e502ef76bdf0100004c","5e10701e71c264df0100003c","5e10b66fcfb06ddf010000a8","5e10d48b0192c6df010000ca","5e114ffcb59632f20100001f","5e1181d77da100de010001ab","5e119e1b91bc7ade0100013f","5e11ecc13e1c29de010000b5","5e120d52e9f185de01000132","5e122ffbc196d3de01000035","5e12599a31427bdf01000085","5e12b0303f181ade01000099","5e13249156233bde0100012b","5e139165a212e1de0100014d","5e13c558280ddede010000e2","5e13fa50f8aa5dde0100005b","5e149a2d830852de0100005f","5e14c260e55ed8de01000116","5e154833edfb1ede010000e4","5e15647d1e2528de01000046","5e15798df1f31adf01000019","5e1600f6efa1cddf01000257","5e168ec10ba191df010001ec","5e16af67dc7739de01000036","5e1764e44df69dde01000172","5e1770aa4df69dde0100021a","5e17893acf35a4de010001ad","5e17d5f74ba003df010000c7","5e187dc267b9ebde010000f1","5e18e6518fcbc2df010000a8","5e194b1e86b4aade010000c8","5e19adbf078978df0100004c","5e19fdebcde53bde01000028","5e1a11f1cde53bde0100011e","5e1aeaab5f3d2cdf01000100","5e1b901f7c0fe1df010000b8","5e1bb3c961cb16df010000b2","5e1c88124c9abfde010000ba","5e1cb0ab7723aadf010000f3","5e1d275afeb115df01000185","5e1dc86e8d71ddde01000151","5e1df7292cced5de010000af","5e1e0a93f6dca7f20100001a","5e1e11fdf6dca7f20100009c","5e1e2060f6dca7f2010001ba","5e1e38ab407a20de010001b9","5e1e3c44407a20de0100020c","5e1f6435e8f5ddde01000105","5e1f9b324fcd1cde01000079","5e20c5105c2065de010001aa","5e21011cc63e88df01000004","5e215a7e5a651cdf0100001c","5e217a0bc7842fde01000095","5e21ea7959a877de01000087","5e221004024c69df01000155","5e232f4f327a15de010000ce","5e24791636283cde010000a6","5e2488578c3885df01000036","5e2491cf8c3885df0100009d","5e250c0a2e79a1f2010000ea","5e25bd5af299d4de0100003d","5e25ec2aa6f19fde01000233","5e2621f3408641df01000124","5e2677d0581147f2010000f1","5e26af11f3bb7ddf010000d1","5e272f59557b88de010000c2","5e276b0758994fde010000f9","5e27854655fc50df0100000c","5e2790304c3b0dde010000db","5e27d25c68d625de01000019","5e28623f6ae365de01000052","5e28abc988416fde010001ee","5e28bbeb6acacbdf010000b4","5e28d839a3df5bdf01000092","5e2b1f0927ed83df01000022","5e2b3a70b9f2cade01000019","5e2b44c7b9f2cade0100009b","5e2ca0f8061fbfde010000d9","5e2d92a6481fd6de01000044","5e2e37b0185844df01000050","5e2f0b91e374eede010002d5","5e2f55e826e5cadf01000148","5e2f8afc48b7a4df0100010a","5e3056d557a222df01000151","5e30953bcb949bdf01000187","5e3334baa5c1fdde01000190","5e334d47e0067bde0100014d","5e33873f7a676dee010000e6","5e339d1717f2c9de0100008c","5e3481b9fae8b9de0100011d","5e35205e89e3d9de01000013","5e35dd2876dd53de0100010d","5e389020030bcadf010000f9","5e38adce645ed2de010000ff","5e38d7f581a46ade01000007","5e3944a39d05f2df0100016f","5e3a050baa2adade0100013e","5e3bf9a4ea028bde010000ed","5e3c7b17feacaede01000034","5e3d8b0c96e576de010001aa","5e3dbc1807ca74de010000ce","5e3dcf30d51253de0100005d","5e3ddfacd51253de01000142","5e3e00c4a4cc0ede0100014a","5e3ef14a382e41df010000ce","5e3f20d47da304de01000038","5e3f5f61cd8fe2de01000099","5e4071b8b531d7de01000038","5e416c4ed9f47bee0100003d","5e4189e08491fadf01000072","5e4193d78491fadf0100012d","5e41fdceebe241de01000085","5e437aceb5e412df01000117","5e44441ee5a34dde010001fb","5e446de3084293df010000d8","5e45623049667cde01000047","5e45678b49667cde010000aa","5e45be2e0920e8de010000ea","5e45f90fa5737fde01000093","5e4768d445a735de01000097","5e4770dc45a735de010000da","5e4776a127a0c8de0100001b","5e49d2feb63120f2010000a3","5e49ecd0885ce9df0100005b","5e4c2ef7c1eb51df0100000e","5e4c3272c1eb51df01000054","5e4c725ff2c6ddde010000f6","5e4c86cc5cc521f20100002f","5e4c89d75cc521f20100005d","5e4d3a95d43139de010000ce","5e4d673c08a8e5de0100007d","5e4e10f1d116fbde01000042","5e4f888342a908de01000070","5e4f8c7042a908de010000a1","5e4f8d9942a908de010000b2","5e4ffa78f5b214df01000102","5e500a19933046de0100003b","5e5023168c3a2cde0100000b","5e50725dcb6c3ede01000120","5e50e8199f7a6dde01000005","5e515d9aa04830de0100004c","5e54534d88d190df01000040","5e5481967f0f44de0100011b","5e555acee89e5fde010000d1","5e565ec06f0b61df01000107","5e56a5afe177dede0100008e","5e56afb3e177dede01000129","5e56c3da96127bde010000cf","5e56fe107840dfde010001e4","5e5700387840dfde010002e1","5e5771bf16d3d2f3010000be","5e57ffa6a38020de0100013b","5e583cdb1f3fc8de01000153","5e599a85ad6c7fde0100006d","5e5b0be96e568ade0100004f","5e5b7186502fdadf010000a9","5e5c6589f4282ddf010000c0","5e5cb1f5a9598ddf0100004e","5e5d36f373eb2edf01000035","5e5d4dc973eb2edf01000228","5e5de8f4863279df01000088","5e5e052b5c89fadf0100006d","5e5e8bcec0a53dde0100007e","5e5f802d5766d9df01000192","5e600a0913e3aede01000190","5e601becc064fcde010001b2","5e6080039119f0de0100012a","5e60a33bc28c0fde0100004a","5e61098731c7d3de010001fc","5e61223c1cc34ede010001f4","5e62afbe08ebcade010001a0","5e6443fdb8c607de01000034","5e651c7a9eed46de01000116","5e656cdede41b9df01000142","5e66eb4a85689bf301000069","5e679bbcd527f0de010001ad","5e67f5f00e29d3de0100030a","5e68f31f1a389bdf010003f3","5e69a88d23ebccde010000de","5e6a8bd10e8744de010000cf","5e6b7b12c024f9de01000161","5vvrLHqsgH9E2QPjP","5wgZYSuyAMqRBppg9","69Gb9G7uXuGC9CqrT","6BSaqwxscteLBtEJv","6XixHSsWpd8nxzP7K","6o7DA2usY7FENyPRa","6oAq3TGhqimHY2ber","6rFcASaST7YgKdxHc","6sGcbuXnEYZ8Wmuej","74SNnigtjBX48iG3J","77kA4icgJBShuH63W","7A7q37F5APXNrBhxq","7AjGyhK3iuwzff65n","7BAciBd9hbBEkJefR","7CKGZqSpvcj9u5Fsr","7TpbYvGuRf5yRNuMy","7XBRX4AfHzxd8Yhai","7bRE2A4kCENz7Y5qp","83kyS3bRfQKny6SY5","8BBCAgWbd5QXXgsKi","8D7teT2RWr8xuZFD7","8YF4BGcrFpTcZeybA","8ZTszaF4sYucLJTae","8csKr3DgDfXLDRe8a","9FWArSc8hhQ3zvuZp","9Lo5KJEgpgs2rS6tb","9bvDsXr3AkbNMTj3v","9eA3jvWTYH8FSkeBT","9oBgKFdzYg3aEn5oz","A3kcMtBKeBkRzJ4To","AxHmaYpiG48A4vPpB","AzCZNMFjGr5ZYibXh","B2khThQvkeNjzTrJo","B8jEGwLatXm2dJm4r","BD5J3ZmoyvJJJibjZ","BFZEw6tWHhQ29NZhW","BP7gWobLA4F44fwyS","BYCMaBqwqZrJaH8E6","CRGYszpgrsRcosfhy","CidHtLPczZxb4Zopq","CoPYFNeBaL9REPBcJ","Cy28TS23ZijFMJLGR","DD2ZkDRBYmppLTXZN","DJh92bpFSACASgRZj","DWqxxAYWrgCbhbye5","DbNtGNp7Q4SPb66vp","DbSgAa6dd5m5PeeM8","DyKSBkzDbpr7iNQuQ","ESLhYXSuLXx4bYQoE","EWoH3ji5LgDxk4rez","EXYASwpLGtAbmfwQn","EhtcjnZvWJEn8jsNp","EnATfkZY9bxYYQc2D","EsQJ3szXcmtN6EzF5","F2HtNNL3tjydiQNGC","FDv7QksjBPSGwx34t","FK4tgksyCD6q9EXmD","FRJc6p3hR43yvMLya","FkRCFp3njhTn9oH2q","FwmskpEqEi5e22No6","GcnKnvyQRvm7ufjvG","GmkdwRpnsTbRnGPon","Gx2aaKwAwexRyq922","H5iuFjLuqh9oBbyy3","HRhyNRNQq76ox2aZq","HSRa9AAqRKaduw928","HjG3JeRSfsFsFHFge","HqKgkHBhXPQiSz6Ax","HrnA7bqRRPp5yfKCW","J4DA5MLHaQaff2ivR","J7Kj5xZj7gvxGpGdR","JZX9c6cixBcSCcRGS","JwMCBP4YSNrfrxXFx","K4A22AsSSfeqBFrpT","KNXv3J5frDgBs4hzt","KPGd3pWTc37z2ezKa","KRS6anSw2zkcvuCZF","KSmJtfMob3tCnxBEH","KZPKXjxEb8DCTSPsm","KdDhHf9MgFpueLw4h","KoB4o2BiACqj5wZpu","KpRatkH36PEhxzWRh","KqYTsXZ2jaNM5xu48","KwBBk7q2r6thvDaEh","KyrmE4w34odibeaja","LY3g34CKzH8c4GLcn","LoEuQXkNFLqmk4q2D","LpJ8vbMGQRoiDYtTA","LvMApwCWwFjQuA9xu","MSPPZ5YXozD5hjb32","MXoqkWkTXa3FNWnb3","N8xvQbfJTTeDhmem4","NLvKfJtnLhY27LX8G","NWBGWbitviQJ7PMGx","NisS4NB2Z8XsoPiop","NqpDrTcQZStBfH7GP","NzRRX2tf5vZJTwTAT","P2qZ2h7fnEgXhP8wZ","PHoZQDYbZyCnFk4qx","PLA7e5v7eqSpa2srh","Pfyj9gd9vixBdW8Mz","Phrpv7TxDneNK9ZD6","PnP7X6JmJoortQxaE","Ps7nv28bsn83ovgJ5","PuxwagkSyCKBaaJjD","QFQkLLBNXhKq7zX3p","QFggoBXHygtZ9mrgA","QSRMyDaD5AMhsJoiT","QWoxEn67eLEERCcri","QckZeqhHc6NzEWjGQ","Qx2KiCGn5ZQj3Tm6D","R2bqLqzH37LSk3dEN","R9wEkBEHQhvitAeFK","RQPtRvHp3CHJYhx82","RWdXiPog4qrpiQhXM","RXbY8jQiFTwhFpHn5","RdcJRWAbJ6di8XL34","RmERscPN9dpBehM2r","RsPEvg4f4xNkZFiDz","S5rLyt5xWqri4Bm3B","SDXNeyucEqZJYzJPP","SMyENBGLwed9bg9dn","SPB7xjcD7dK5nag5i","SjPdLifsNLFiSPJnX","SonBkYjLbzfPdue6b","TqngjqzjEatA5mESz","WMMj2X9HmEfTPF4o6","WddAKna5uDTBSTL94","Wret9iQc82sz2AY32","WxkMEB7EpNhDhnDBb","WzzR4fueCTpsbR2e6","XhoedZd4vYHegRmzZ","Xk6BmXSsoNW6gQteS","YEDQuaQoMxfrEpZi3","YWG2Cg3nQeSruic2j","YioYfkr22LZsF4Bj4","ZAFn76SXremTQfSnC","ZGgpqGFXoZgyvbx3X","ZMQfYQC2HGHJpNDSW","ZcpScSR7rNarBsSKM","ZgNmEX9rJfSr3Bueo","ZorESQjDDeNSt2vqd","ZspbD9bauzj84c8bY","aZpJsemKeCgjXbTGe","aj9xZEAY5ehNe9wAv","anZ7x9gcmg7ZwnBy4","bHxtEXL9pz5BbQNWX","bQ7n7SYYoGvTu2BqW","bfbe5r9Efrvh3Abe9","c7gnKGitCDDFYXLu2","cJkKeEvMNpbG696wB","cPdTM2nhmScQHoDtd","cojpq7debEkuiAMKD","d8uKbua5DCY5YWL45","dCen7dYEkvh5gzLBu","dKbmMcMaBaMmHXs7R","dMH3mH6h7RBoq7n5r","dXDH3shpYKxXxiXjv","dwEz5g8oMXoQejtCY","eDH5nop7nDFFYbDyB","ev3EDPEzFJB99CmzN","f7FxbjPn83vKCNwNk","f8gt8jWwjAtdSpSmi","fMuYsPwdXuaHmhxsb","fMybk9pFkPsYfu2pm","fNbLXe22X3xL5Zyth","fgAXke2pPhQrFqn2v","fuQXWK67Sdygy9tFe","fzcyEogMMEjWWo9qh","gPqmewhWtsTPbgNPx","gd4XDSpYd52DJSLQz","ggitmEfdT5fdksrZR","h4qsJcBP4TfAyHrET","hN38yNSNKhwBpjoWY","hTSRyxPMhxL3PazAr","hvCmfbQcbGXvwgZdQ","iQ8pM7cN9k9ayFEgw","iS7zLee83q7RKyECX","iaeBcQcRdCBy6xkoB","icGnxiSs3QxJHPS4s","idQTxDet42qC3Jd2x","ioDPa62Yt5HtqzWBu","iu7X2yovAZ5LEffrz","jFyTMgvSAnuwd3Me5","jKxBw8nYpeW6HLL3c","jWS7s6FfkEdYCrXxx","jv7LiD4CaQ4tRPqme","kEmEjTDBS3hhFziWL","kKdYBKCnkJskTLAEq","kYiaRyZRtgDmgu88o","kZLyMfsQmTCHFo9LC","kdqbWGpRQqFBREEbt","m2qsJXBg6utwwaZc4","mXueAvNYgyizg5WYf","msQNMu4Jo4n23S4hR","mwxyQE8C3Y9Kzu34X","nEShNQfkQDLKnm8FJ","nHggQLJ5rWfWtTAMc","nbZJFgwHuyua9ZMzN","nwgMcwNRBJkaaSMYA","nz5DrStoskLqxcz3j","o5W2wMaBXz7AsMoFD","oEHyQe2wbwZNhgMfe","oS4CBDmTdRZ6zaBzN","oaXfRQgd56s8mLizu","ozRp6sPHyXbrvuGxz","pEn65TZJdJPSjfJ6o","pnn9hoHGt9ZB7Yh3C","ppTXsix8PiTptHPTv","pvwyq2FsvHLSD2mnu","pxX3fvGrCK6CfFdxR","qAEGfBZSyYw5g44aB","qARqJvwid5k5E4KbL","qG4DGzjdabcMZA6ZK","qaXjiXZy56D7eRK8s","qmTXWT4PjGCaFf3Qf","rKgbsFjQYxr4brzA9","rRmNDMhwxWb922jFP","rgiWPKgbncLaJAqdh","rw6eycCoLApNdcEK3","s26YgqwunsDmsFZ3d","s72o6CrDptDYGTpg7","sNaWwoyt3MQZcLBvN","sPWSimL3rqxPPbFmT","sPfFWugdNbn4tjcLK","saYcRnQds6rEnJcN6","sdNwxguoZiGQuzNnQ","shGSTLbNCQ2mQ7C6B","snGb73RqNP9yhN4MD","t9ndRegAnj4uBmXvf","tLxKsv6xZTqs7ZZfE","tY7P35MLkWM2aHYhn","u2tNnhyZqjS9d7iDG","uF8fHSMEGz6tMYypJ","uJ8Rrdfthg7CzYvvi","uSN2tqTyg3s86P9bD","uWvJWjog6W8qix2qB","uXozimonaRGCzaADW","uuyFnjBtxca5zh9Ky","uvrun9f6iXENKDt3n","uzhXQMSBLEiWwaSrF","vCGw9rPG9d8XMggop","vE5RpwhAoKjbjmx8s","vEnjk9djxnuHbCyTa","vLvEkskwie7Xk8dps","vMQgcRGs3htxFRYco","vkirKDjhjYWTQZj4m","wDoGhd6RWWE4o9CeX","wnMj9SHXnBYWtKQZp","wuPh3vGzrJ69cD83u","wxJH73fuSn9HdLMj9","xBYEKZNGX3wfQdTcT","xEm7baDDyTm33FzCK","xX3BFvy85tSBTuinW","xZC6uifPzee5wRWLB","xtey9QmYxZQ5cqYgy","xuM8ocmp3RXK6CQmE","xupeg6dJBotE9GBsm","xvKzadizesMZZntjF","yBPsWrbvZRvrQ9v3w","yggMy2DdHyArMfEP2","zeq65d2AxoMPqqqcE"],"dataSources":["ZEwmdExPMCtzAbo22","PkcXzWbdqPvM6bmCx"]}