Estimating individual treatment effect: generalization bounds and algorithms. Shalit, U., Johansson, F. D., & Sontag, D. In *Proceedings of the 34th International Conference on Machine Learning*, pages 3076-3085, 2017.

Paper abstract bibtex 8 downloads

Paper abstract bibtex 8 downloads

There is intense interest in applying machine learning to problems of causal inference in fields such as healthcare, economics and education. In particular, individual-level causal inference has important applications such as precision medicine. We give a new theoretical analysis and family of algorithms for predicting individual treatment effect (ITE) from observational data, under the assumption known as strong ignorability. The algorithms learn a "balanced" representation such that the induced treated and control distributions look similar. We give a novel, simple and intuitive generalization-error bound showing that the expected ITE estimation error of a representation is bounded by a sum of the standard generalization-error of that representation and the distance between the treated and control distributions induced by the representation. We use Integral Probability Metrics to measure distances between distributions, deriving explicit bounds for the Wasserstein and Maximum Mean Discrepancy (MMD) distances. Experiments on real and simulated data show the new algorithms match or outperform the state-of-the-art.

@inproceedings{ShalitEtAl_icml17, author = {Uri Shalit and Fredrik D. Johansson and David Sontag}, title = {Estimating individual treatment effect: generalization bounds and algorithms}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {3076-3085}, year = {2017}, keywords = {Machine learning, Causal inference, Deep learning}, url_Paper = {http://arxiv.org/pdf/1606.03976.pdf}, abstract = {There is intense interest in applying machine learning to problems of causal inference in fields such as healthcare, economics and education. In particular, individual-level causal inference has important applications such as precision medicine. We give a new theoretical analysis and family of algorithms for predicting individual treatment effect (ITE) from observational data, under the assumption known as strong ignorability. The algorithms learn a "balanced" representation such that the induced treated and control distributions look similar. We give a novel, simple and intuitive generalization-error bound showing that the expected ITE estimation error of a representation is bounded by a sum of the standard generalization-error of that representation and the distance between the treated and control distributions induced by the representation. We use Integral Probability Metrics to measure distances between distributions, deriving explicit bounds for the Wasserstein and Maximum Mean Discrepancy (MMD) distances. Experiments on real and simulated data show the new algorithms match or outperform the state-of-the-art.} }

Downloads: 8

{"_id":"qnBvk5Tn69FbqEeMf","bibbaseid":"shalit-johansson-sontag-estimatingindividualtreatmenteffectgeneralizationboundsandalgorithms-2017","downloads":8,"creationDate":"2017-09-25T03:37:20.547Z","title":"Estimating individual treatment effect: generalization bounds and algorithms","author_short":["Shalit, U.","Johansson, F. D.","Sontag, D."],"year":2017,"bibtype":"inproceedings","biburl":"http://people.csail.mit.edu/dsontag/papers/bibtex/david_sontag_papers_all.bib","bibdata":{"bibtype":"inproceedings","type":"inproceedings","author":[{"firstnames":["Uri"],"propositions":[],"lastnames":["Shalit"],"suffixes":[]},{"firstnames":["Fredrik","D."],"propositions":[],"lastnames":["Johansson"],"suffixes":[]},{"firstnames":["David"],"propositions":[],"lastnames":["Sontag"],"suffixes":[]}],"title":"Estimating individual treatment effect: generalization bounds and algorithms","booktitle":"Proceedings of the 34th International Conference on Machine Learning","pages":"3076-3085","year":"2017","keywords":"Machine learning, Causal inference, Deep learning","url_paper":"http://arxiv.org/pdf/1606.03976.pdf","abstract":"There is intense interest in applying machine learning to problems of causal inference in fields such as healthcare, economics and education. In particular, individual-level causal inference has important applications such as precision medicine. We give a new theoretical analysis and family of algorithms for predicting individual treatment effect (ITE) from observational data, under the assumption known as strong ignorability. The algorithms learn a \"balanced\" representation such that the induced treated and control distributions look similar. We give a novel, simple and intuitive generalization-error bound showing that the expected ITE estimation error of a representation is bounded by a sum of the standard generalization-error of that representation and the distance between the treated and control distributions induced by the representation. We use Integral Probability Metrics to measure distances between distributions, deriving explicit bounds for the Wasserstein and Maximum Mean Discrepancy (MMD) distances. Experiments on real and simulated data show the new algorithms match or outperform the state-of-the-art.","bibtex":"@inproceedings{ShalitEtAl_icml17,\n author = {Uri Shalit and\n Fredrik D. Johansson and\n David Sontag},\n title = {Estimating individual treatment effect: generalization bounds and\n algorithms},\n booktitle = {Proceedings of the 34th International Conference on Machine Learning},\n pages = {3076-3085},\n year = {2017},\n keywords = {Machine learning, Causal inference, Deep learning},\n url_Paper = {http://arxiv.org/pdf/1606.03976.pdf},\n abstract = {There is intense interest in applying machine learning to problems of causal inference in fields such as healthcare, economics and education. In particular, individual-level causal inference has important applications such as precision medicine. We give a new theoretical analysis and family of algorithms for predicting individual treatment effect (ITE) from observational data, under the assumption known as strong ignorability. The algorithms learn a \"balanced\" representation such that the induced treated and control distributions look similar. We give a novel, simple and intuitive generalization-error bound showing that the expected ITE estimation error of a representation is bounded by a sum of the standard generalization-error of that representation and the distance between the treated and control distributions induced by the representation. We use Integral Probability Metrics to measure distances between distributions, deriving explicit bounds for the Wasserstein and Maximum Mean Discrepancy (MMD) distances. Experiments on real and simulated data show the new algorithms match or outperform the state-of-the-art.}\n}\n\n","author_short":["Shalit, U.","Johansson, F. D.","Sontag, D."],"key":"ShalitEtAl_icml17","id":"ShalitEtAl_icml17","bibbaseid":"shalit-johansson-sontag-estimatingindividualtreatmenteffectgeneralizationboundsandalgorithms-2017","role":"author","urls":{" paper":"http://arxiv.org/pdf/1606.03976.pdf"},"keyword":["Machine learning","Causal inference","Deep learning"],"metadata":{"authorlinks":{"sontag, d":"https://people.csail.mit.edu/dsontag/papers/biblio_year.html"}},"downloads":8},"search_terms":["estimating","individual","treatment","effect","generalization","bounds","algorithms","shalit","johansson","sontag"],"keywords":["machine learning","causal inference","deep learning"],"authorIDs":["28jTNN3q6mLkHgWah","2B6sJqJczMbPNhoJr","2Lkpam56jEc3XGudp","2PRyndQegz8KR5jEL","2aALotdeNK7w3inyh","2ezjxH66CPZHKoDqR","2iQn5FvhRDLaa66hP","2iYucYeCgsNZPn6Wt","2vxZPoNCeHQhpW5d5","35FEjbbaWgpfMYjRc","35wDLtbttcvmroyZp","38d4AsrKpRQFHT5cw","3CFW5MC82K2dy8oqY","3R2FBomLP4E4ctEs8","3num84jtzmfdJ3Xip","3p7PQAcFw6KDC3waZ","439oYR7vzwYkkvPfx","46ywvi2F656nrzo33","48e3CTEtLiYZBJEYD","4BsKt5Q6Ro4d59go9","4CL6unvotGYYJmqNT","4EByPtEHXu8fQK5gT","4RH2Wu35DjCKSiik4","4TkxEg9eYJcNB69tJ","4iFi3edpdJnYC29iL","4ptybMdTd5R7hTgtB","4sF3hEQ7hhEnndw9K","4u4BgQD3cXf52E3zT","53Wo8iFFj7EykEGCR","59c879f08558facd23000014","5Cz8p2uy6Bjkb4Ty7","5PtoXJPGzmJRpAjiC","5Rix9PQKuCzxzjyNw","5XxHEMaPfFnAyh3wL","5Y3hNSkDg88FDBha6","5de6e76feab4b7de0100005a","5de6fbf6eab4b7de01000167","5de77319021482de01000051","5de7920dd9b46bde0100007c","5de7bc2fbc280fdf010001ba","5de87986e66c23df010000b9","5de882b0e66c23df01000124","5de8937f2c5eb2df0100009b","5de8afe63cfb74df0100009d","5de8c4829e80cdde01000045","5de8e2a0978afbdf0100006b","5de962b0d574c6de01000175","5de97d65768c31df01000172","5de98b1d59feb7f20100005d","5de9ef0c9f521ddf01000153","5dea2550b5dcc6df0100014e","5dea6395ddb5e6df01000205","5deabdb3cd660bde01000084","5deaeccfecff01ef0100012c","5deb246928b3ebde010000d3","5deb35d129c0bddf0100005c","5debc77dca1cdddf010000e9","5debf3a8188b03df0100002f","5dec2668f58df0de0100017b","5dec525ce16431f2010000bb","5dec968712c53edf01000104","5ded6a5b722536de010000b7","5ded78cd722536de010001a3","5dede29d0e26d5de01000126","5dee3bb80bde9edf010001d1","5dee44bfeaaee4df010000df","5deeb37e0ddb85de0100004e","5def0b39c5ae39f20100006d","5def1844c5ae39f20100012b","5def1b85c5ae39f201000162","5def7e2545114dde01000080","5defbc31d6a2dcde0100017e","5defc8bc090769df01000005","5defdbb7090769df01000167","5defeaa914db5cdf01000073","5defed1b14db5cdf0100009b","5defffde14db5cdf01000200","5df024835d7361de0100006c","5df06cc5e4ce32df01000146","5df09331f651f5df01000051","5df0a37bf651f5df0100013b","5df105ec092ae5df0100009b","5df11cc28fc423de0100002a","5df15305d177badf01000015","5df15f20d177badf010000f1","5df16aeed177badf010001c1","5df1bdd980f800de01000104","5df1d4df1070c8ef010000ef","5df21d3e1e4fe9df01000078","5df24c82480e6fde01000196","5df25c5763aac8df010000fe","5df28997cf8320de01000020","5df29154cf8320de010000ad","5df2a8740f18f1df01000061","5df2f71eb91ab0de010001a5","5df2feadbc9e6cde01000047","5df337b6683178f30100006e","5df37e2123fb6fdf0100015e","5df39c0d2b1f8ade01000189","5df3c4bd580920de01000081","5df3dc13ccbb4fde01000038","5df3e42bccbb4fde010000be","5df3f4c7ccbb4fde010001d0","5df405f6d1756cdf010000d8","5df409ccd1756cdf0100010a","5df423b49d2522de010000e0","5df42f1e96bfa7de01000018","5df44ad5b126eade01000014","5df4646cb126eade010000fa","5df46d1695416ade01000058","5df483b85ad889f001000017","5df48f125ad889f001000083","5df4b1aaa50a84df010000c1","5df4ce6b55b997de010000ce","5df4d693fc47dbde01000002","5df50858daf598df01000108","5df51789ea1457de0100007e","5df51f7cea1457de010000cf","5df532a3fd245cde01000082","5df53952fd245cde010000e5","5df53e0afd245cde0100011e","5df54a1dff5784de01000024","5df54c7fff5784de01000058","5df55211ff5784de01000095","5df566fcb3c244de0100002e","5df56f58b3c244de0100007e","5df57499b3c244de010000b2","5df578e7b3c244de01000102","5df580344734a4de01000012","5df5a244a0eab8de01000084","5df5a5cca0eab8de010000ab","5df5bef664d1d0df0100003e","5df5e990f65dd9de010000df","5df610f52b34d0de01000036","5df63d5938e915de010000e3","5df641f538e915de0100011f","5df662c3797ba9de0100001f","5df67762797ba9de01000119","5df6851172bbd4df0100005a","5df68a1272bbd4df0100008b","5df6b02b31a37ade0100016f","5df6b893f9318fdf0100003f","5df6cc68f9318fdf0100019a","5df6d129da9d91de01000007","5df6f77fdf3bb9f201000088","5df72a058efe8dde01000044","5df762bb952c76df01000055","5df7e977dc100cde010001fe","5df7f1d2b9bb17df0100008f","5df8e629877972de0100017b","5df8f638277e45de0100017a","5df942eeccc001de01000079","5df953b6ccc001de01000165","5df97e6b816c3de001000071","5dfb12f1a8dc13ee010000db","5dfbbdc7f6f0aede01000149","5dfc0449b371afde01000078","5dfcc4f77a3608de01000042","5dfceba72eb8b2de010000ce","5dfda651cdf7c8de0100002a","5dfee7405dd8e7df0100005d","5dff88efe1f8dade01000052","5dffc77c1ce6f1df010000d8","5e0088ded1b715de01000065","5e00f38b08c773de0100006b","5e0253726ffa15df010000a2","5e02bc6a19da8edf0100005b","5e054d161965a8de010000f3","5e05a8e3901debdf0100001a","5e0802b9cdee3adf01000061","5e085dbb1e4bbbdf010000a0","5e093c20934cacdf0100007a","5e0988dd83a3f3de01000038","5e09a0ba2203dedf01000025","5e0a4df4ce3ebce401000079","5e0a725d3eccf6e0010000c0","5e0ab08c610902df010000fa","5e0acbd827625ede010000c5","5e0b31cafc92e9de01000074","5e0c3057024014de0100003f","5e0ce1af6762d1de01000143","5e0d3b87ae5827df01000010","5e0def2e45d2fdde01000074","5e0e338b2b8028de01000103","5e0e7c8ba0f484df01000092","5e0eaf7603f891de0100002b","5e0f9d4d66a55ede01000178","5e0ff16c063b5cde010000b5","5e1025c32ef76bdf01000085","5e10c1460192c6df01000013","5e10e45145c12cde0100006f","5e11832f91bc7ade01000003","5e1244aac196d3de01000138","5e134b5c697554de0100007a","5e136ac9f16095df010000d0","5e137622f16095df01000198","5e1377eff16095df010001bc","5e139418a212e1de010001ab","5e141cafd64f35df010000aa","5e1465cc0467fede01000076","5e14c1fbe55ed8de01000111","5e14ed29b46f1bdf01000019","5e152e6a18da2edf010000ba","5e154dbeedfb1ede01000117","5e157a06f1f31adf01000020","5e164d5a28f1c0de010002e9","5e177458cf35a4de0100002c","5e17817fcf35a4de0100011b","5e17d6764ba003df010000d0","5e18bb6cbd0cf5de01000166","5e18cb38d06bffde01000062","5e18d5f8a382e2de01000081","5e18fd20a7672ede01000011","5e191c4cda91eae001000001","5e1a41773a430ede010000d7","5e1a981169523cde010000c5","5e1abce89da435de010000ba","5e1b4713749a45df01000123","5e1b6a5a71abfede0100003d","5e1c103f7530ebe80100000a","5e1c5776e556c6de01000108","5e1ce485abed9bde01000166","5e1d0761633f2ade01000148","5e1d3ddb6b18c4df010000e3","5e1d4d88e32442de0100006b","5e1d95f23a6d8cde01000243","5e1dd80e4986afdf010000bc","5e1ddc434986afdf01000129","5e1e05872cced5de01000209","5e1e1178f6dca7f201000097","5e1e1958f6dca7f201000128","5e1e3a74407a20de010001e4","5e1e727bce9ed9de010000fe","5e1ea366bedb58de01000090","5e1f1bd93cc57cde01000112","5e1f66c4e8f5ddde01000135","5e20072d453fb7df0100002c","5e208df380350bdf010001f3","5e20f4f445115cde01000112","5e210142c63e88df0100000a","5e213e003a0076de0100001e","5e21b7e096aea7de010000c3","5e220527024c69df010000b1","5e223bf675445bf201000041","5e225cab1f0592de0100004f","5e23500aaba105de01000012","5e244afc2fd95cde010000dc","5e25068e2e79a1f201000091","5e257566f58a5cde01000108","5e259596d31494de01000173","5e25c3bff299d4de010000f2","5e25dae1a6f19fde01000058","5e2614ef408641df01000074","5e267112581147f2010000b4","5e268944d297d6de01000053","5e26924fd297d6de010000d6","5e2700fd9667abde0100013d","5e2736e5557b88de0100016b","5e278c614c3b0dde0100008e","5e27c1e1328b80de0100009b","5e282756e6485dde0100002b","5e2858b1c4828cde01000199","5e285d246ae365de01000006","5e286c626ae365de01000119","5e28915967e11edf0100017f","5e28a32988416fde01000122","5e28d6faa3df5bdf01000083","5e296181661211de01000076","5e296edf661211de01000116","5e29c89dedf563df010001f3","5e29ea628fb0e6de0100003a","5e29fe358fb0e6de010001f7","5e2a2289dd9c08df0100003e","5e2a3c4a2d8494f201000010","5e2a6975621cc1df010000ac","5e2abb9a8319d5de010000da","5e2ad79a638921df01000104","5e2aff0b5da88ede01000151","5e2bbb1bdf2e50df0100008c","5e2bf9775e0743de01000086","5e2c8369ce5606de010000df","5e2e34e7185844df01000039","5e2eab95b84405df010000c7","5e2f41c878a7cedf010001b8","5e2f6bf60c34b9f201000118","5e2f89fc48b7a4df010000fb","5e2fa503a21e47de01000060","5e303ed546a666df01000208","5e30535457a222df010000e0","5e308c2ecb949bdf010000f3","5e30a5ddc99510de010000d4","5e30ddc35f9c1cf20100005b","5e318cd200e4e5de0100016a","5e3198326be690de0100000e","5e31a4426be690de01000104","5e31c392b5384fde01000134","5e31d0e9bba4d0de01000030","5e31f9797c8d24df010000db","5e3200527c8d24df01000162","5e3202dd7c8d24df010001a2","5e3265a85633c9de01000063","5e328d6c3957e3df010000ee","5e32f5dcc1389bde0100012d","5e32fdd3c1389bde01000301","5e334ef0e0067bde0100016f","5e3390e07a676dee01000175","5e346c81a8890cde01000195","5e3472a3a8890cde0100021e","5e348f4553b794de01000018","5e34f0995978bef201000072","5e353e00ba591ddf0100002e","5e354a01ba591ddf01000095","5e35e7c25cd57fde01000045","5e35fb305cd57fde01000139","5e360aad591ba4de010000c0","5e3662e74b25bbf2010000b5","5e36b8ff7b975dde0100007e","5e36fc2a6568afde0100002f","5e3763a6e25d29de01000143","5e377d15ac7fe8de010000f3","5e38070e918d4ede01000124","5e381d7a0691b8de01000107","5e3890d7030bcadf01000104","5e38b374645ed2de01000141","5e38bb45645ed2de01000196","5e38caf387bbe9de010000e5","5e392f139d05f2df0100001d","5e3945f89d05f2df01000177","5e398dd4d14579de01000187","5e39907ed14579de010001b3","5e39b2f3ad94f9de010001c6","5e3a2c63237cdede01000038","5e3a3f6b237cdede01000107","5e3a505d1469e8de0100006c","5e3a6a3eb291e7df01000046","5e3a97010b62daf301000111","5e3acbd1f2a00cdf01000198","5e3ae3051b85fadf0100014e","5e3afd9955f0f2df010000e6","5e3b03fd55f0f2df0100016d","5e3b6ce959b923df010000e5","5e3b88b4184d6ede010000b6","5e3ba05369c38bde01000070","5e3bbdf2657d0bdf0100007c","5e3c767a67788ede010001ab","5e3ca752f99721de01000120","5e3cca02948886de010000f8","5e3cf0e4ad8243de0100005c","5e3d3e3edc4cd0f301000011","5e3d64725f3240de0100009e","5e3da7d1f33211df0100016f","5e3f9fedcecf86de010001a4","5e4057a3aeea22df01000052","5e40c34a614963de01000015","5e410bacf47523de010000be","5e41946d8491fadf0100013b","5e41a1e90b4861de01000006","5e41b65c0b4861de01000171","5e41d5e7e7c67ade010001a9","5e42f416aefe1adf0100000c","5e4342018957d0de0100010a","5e437d67b5e412df01000133","5e43bede44c4f9df01000070","5e43c71944c4f9df01000189","5e43d4b59a621ade01000007","5e43ef4a9a621ade010001a4","5e444e3fdf3c2af3010000c7","5e455a1da96575df010001b7","5e45adef23a133e801000173","5e461033b57382df01000069","5e464b456a7198de01000096","5e46b5a48573d1de0100008a","5e46fa1f461d04f20100012b","5e476ade45a735de010000a6","5e48104bd4b913df01000085","5e48b0cdf1ed39de0100002d","5e49519d379644de010000d5","5e499d96552c77df010000c1","5e49f917885ce9df010000d2","5e4b0af4332a9bde010000fd","5e4b689bcf8e89df010000e1","5e4c05760dff2bde01000143","5e4c54b6271596df010000cc","5e4cc1ac0cc7d3de01000030","5e4d251d7d0ec8de0100017c","5e4d68de08a8e5de010000a9","5e4d8eb1d2986bde01000182","5e4dc181682c99de0100006f","5e4df804cc196bde010000bc","5e4e0879cc196bde010001bf","5e4e1c19d116fbde010000df","5e4e72a524a828df0100013f","5e4eb7f6d9cddadf01000036","5e4ee1d02962cadf01000136","5e4eedc6338acfde01000016","5e4f31a6aa67a8de01000092","5e4fac7d29097dde0100011a","5e500010f5b214df01000178","5e506a1acb6c3ede010000bf","5e507ad27e0827df01000019","5e5091d57e0827df010000df","5e518015f29c90df0100005d","5e5303b16d68b8df01000054","5e53e560d26e87df01000048","5e53fa21d26e87df010001ed","5e5417bc80c0b0de01000010","5e542ca480c0b0de01000195","5e5439187a758fde01000091","5e55102096ed20df0100014a","5e555387e89e5fde01000061","5e55a8907d0846de01000072","5e55da36c2c8a2df01000009","5e55e81dc2c8a2df01000080","5e56919deb2916df01000196","5e56da18a8dabede0100003e","5e574f5018f14bdf0100004d","5e5783ffcef9b7de0100003e","5e5824dd6a456fde010001a8","5e58272d6a456fde010001cd","5e58483a2c2732de0100004b","5e593e8ae60e02de010001bb","5e595fb956d60ade0100001e","5e596daa56d60ade0100016b","5e5a07bc3557f5df0100001c","5e5a1b273557f5df010000c3","5e5ab97f56d8d3de0100008d","5e5aef34038583de01000061","5e5b00ba038583de01000127","5e5b833a99fee0df010004e4","5e5c22b415d8f5de01000048","5e5ce536d12a0ade01000030","5e5d28f0168391de01000118","5e5d355373eb2edf01000017","5e5d7ce20b73f6de0100014f","5e5d96005726ecdf010000a3","5e5dd0cbc64f0ede010000ab","5e5e3417d3955dde0100001b","5e5e6b5614f49fde01000016","5e5ed8c78c261adf01000185","5e5f23d48ca867de010000c0","5e5f69a75766d9df01000014","5e60347f182590df01000062","5e604547182590df01000129","5e60c42e951697df0100007f","5e60f9e131c7d3de01000119","5e6114471cc34ede01000093","5e611d051cc34ede0100015e","5e623caaa84f4adf01000189","5e62608111ac5fde01000017","5e626d1011ac5fde01000158","5e628d196f352ede01000197","5e6314cce358e6de01000065","5e63caa474499adf0100014a","5e651b6b9eed46de0100010b","5e653365ee6356df01000120","5e656d50de41b9df01000146","5e6595065dd5c8de01000043","5e65ace4d92058de01000022","5e65bf07d92058de010000c2","5e66930644d2c4de0100009f","5e66f74e85689bf3010000e8","5e682715dfcfe3de0100025e","5e6837289cef79de010000b1","5e683c749cef79de01000141","5e689a72a79ee7de010001d7","5e6945086964dedf010002d3","5e695a93af718af2010004f3","5e69bcb86423dfde01000023","5e6a2b4938f351de0100025f","5e6ac58cd15181f3010000b4","5e6b4778e9cf32de01000018","5kBjGog9buZbavyt7","5p6HfsrqSnyGih83e","5pFY2BMQpEgM5LJC7","5tLATm6Zy4RroJcou","5wPiwAbCG2WdGmW3Q","5xPMPZR9GbP9qXgqG","6DNyfkCZAfpJv6Ybs","6TLce36kQ3j5CTgaE","6mmZWEPGud9XcnNdP","6pYxbredpmbdyaapJ","6vDXya94DEs4iy2R2","6wzR2fWuGvYvHaWZ5","73zX6CpdBjKWADgyZ","7Cy3NXw947zKN5xmR","7JmrecBwnyXHuTwK6","7khJEr82fEzPTqspF","7x2GyDgtuaT7dMAGQ","82tPP7hYSg8GJjo6E","83kKYP6RskKmZm5kx","889CCSHZyCjSqq7bu","89o9noNWJ8qTk6LDB","8CWqdeAXwY8ozPF3v","8FyjG8yh3ekPofxSW","8PkjQRx5HGnA7oLch","8RW876By2TuEJ8AkF","8YBhWRqiphCWfXBTi","8m2yk2jGY3aa3ZYSB","8nSWnk3e7RWkCgQFt","8s7hgMHZ7ycGXPhJG","8yTsL4QcA2dgpjyzL","9BWnrFnyZAnyRNRK8","9SaYTok2dawNwkQpE","9TmsBW2AjSy3aQkcE","9eWaR4aKndWyASbCq","9mHndA2Aev9WkR8Zk","9pz5HoTrFRMxB6fxe","9xDkMPWyThDcfqeXg","A8qEDT5BzBrfYKSnP","A9HiRvWseTZb8Cigv","AA9vAJZTwhXHvAD7q","AEP6SLSDtCQ5cNHtd","ASJ8mP44rWFWWox6N","ASr4LWBBfQgFbZpah","ATe7qJsgKGApkDoWN","AaZKtcD9zagcfb8Mi","AhdsbSmLqWmdwjonc","Aw8radhZ4c2L9Fomw","AwD8iGM9NqbLALYNk","BEcHnEoDoTuH69s9t","BF2Kfk9Nippq38Ee5","BJqsq75gCoH3ENvCr","BK2EQQxweQC3b2X58","BMbanCdB8P2XeKMmm","BSKMPX7urgatECwnv","BWy697nRWrAxBf4Z5","BbGDtZKXH8rqq4iTa","BnMvg4ZS5oDkxHTk5","Bqjw7FAot2CennHsj","C5E8L4abpnifgmvLP","CNvSsihFxZeCL3778","CmTaytLpdtM29YxAT","CsPj6hv4ywNcnmpFQ","CvBs3T7E7773DNqY7","DGHRHq8npWoPGZvoC","DNCfWe4iqrowcon3P","DeqpYgKvojx72gqc2","DibyWL3FE8PS6QPWX","DqFEtpLTxs3nyPjRM","Duwv2ugGFYxxZfdrL","Dx2N2j32RnmbmSA3H","E9fvRJeWEnuJcg9TN","ENSuXospfGA75Tbe7","EPJtaoberH57YagSr","ESwPWBFHGGtcgJBy4","EWtSA9Fvv6f6HbDuP","EevtjWvBBeG7Nxm2b","EjrmozMJru5bNYS9L","Ew2Msg6cytGF3TAHR","EzyY7MgEFAC6TS8Ku","FHG7bd6FWiRyYzdMs","FJEwzFq9LzS4KoX9v","FJfAFLFuE5WQw6cWp","FeBaBxLjDGMussusA","FxF7Fk5m35puGdbdn","G34JMdCoAoAJkEPQv","G4D3gTZXw2ZCqejeh","G67pHMqLRpXzGkoKK","GgvRMbLvyyzsHNPzu","Gok4BbNCmjKknYBMp","H7con4WLMetNGsEug","H8cp8Ei2MechsxyT8","HFtxBr5tWJEznza5b","HM3e8fTt33yhngoKM","HMpNFCGnbhNBJGhCX","HMtb9k6Xjr4KFeEh9","HZd83LT94P2tWJqKK","HjhcXGZZNDtGDjs6c","HpSi8ZdcP7eF4bMAD","HuJPazimkCL2C49ch","HwPccuty3CN4cKJvt","J2AjNWKDBgrRK4z49","J6AWJGp6rKEJqwtBQ","JPhheiwWhmLFsM6gC","JdBv6ie4o32ECMMpb","Jpy6SCj3LcwobRfM2","JukqADxabZkS2MqzB","K2qGBqGzvoQKJoh3J","K3AX87hRd8PuSfnrd","KCGCfwXiw2FZPNXAp","KJdsXpdjP8foZ37tm","KPrytv3sAeDM4zEZ8","KRb8sQ79h5qdgaWxy","KXxaadmqgEcQRBHHB","KgZNaxopDzWCaQqds","Ko72qLfeeenoTNqgJ","LC7HZdNEQypwyYxN2","LLwqxTh5v8omXoTMX","LNaLFJanCmJFFkXoS","LQhEZfHAa9yzejAxf","LZviizjXS5xk6CXE3","LdrtkSXbHncgTJyTD","LgShEumpNn2Tg7GRH","LiknuWePcyieGZoBb","LnPYiPEndeYDWpGCp","Lvjgt68o9JyXHYyAT","MBQwx2Gtkq7eAhsTF","MP4Lcp6RpdwJHmC5j","Mbxa55v3fm2aJci2o","Mu4ctyeWXKRx5mSzP","MvcdHQk29arWheHsw","N74adRKpJYZAbbt8f","N8dfbMQc6jE899dxA","N9hmyZRvGcYmfADzo","NKSDSQM8zK6cFgCte","NKdw4eqz95pwujZ6Z","NXQaciuKzaP255DuH","P49DmfJn3m9TeQAZo","P4rpEx9s7fx7C7Rk3","P9ErptsRYrk2KdLDB","PAbuQbymDQnBEEWKf","PFBjwybeq95QdAAn3","PPES5dtGYiGdjvu86","PbbdPbqnPQbiTrmr5","Pd2vwumN8KwZwWKTX","PgJiBmCF9TNgbELjW","PxSZvP8hSuBYskDQs","Q3Tt2nf7P46Y3qCxw","QMCDPn9TSBLbvMWGf","QX6P3yj6dN7BmSqjs","QbATkuZJTqG7JGAdk","QeWAPBcEha7Nw7xFN","QeiJ2zzHjSL2Gg5an","Qn7RMFEGo5nx7CmRt","QoHbibgTbYvvcAqHp","QqoR9RBGezMw2eyeA","QxXiQpZifXvtd52k4","R69EthH5wK28dWFL3","R8qazL59BygoiwX3s","RJNBYEfroTsYpkc9H","RJc7EpkABQg6NxKXN","RNyNNo8Lkv8aK3ofA","Rv63bsiq9qQEZeYHY","RzHKiY4nooBfKtmN3","S3zENY4m65Jvw6MKt","S4Aadh2LgJHzoAigg","SBY2vTMz95aQXWLxA","SGBvis6HPMYWn2PGc","SPaiXuhfF2dXi5WaT","Shw8Z86qWDS5jaCwD","SpQD5aqmQYQoL7vmQ","T9hRG2MKpfnJn9gRQ","TNpx3gisERnbT6euk","TP9XHmSaEqiMXLvcg","TauBNqqb59nF3cpWX","TbXtmFYzfcHSZsSCt","TemeA7upPQ4caDkLP","ThwuoPhyAEp3g4bXn","TkJhWfCH8hnqnxWWD","TznJL5yezoQ3xw8uj","W4AcgbuyYSBPnhMR5","W5YiE7oEGDdcb5art","W6Nq9Ppjn64Y9RcTK","WNxvaPYkuuQEqzxoN","WdtbWYRigDEJuixew","WgYMCW5W9hu3npj6r","Wm67sERMehRiDF9dx","WnZNScCxeuJkSBkbR","WpSux5LxQQp3gmWyi","X5nLwG4PzyPmBYYdF","X9uezBvb5RLv7DeFj","XDWyfXjuzDy99nwpx","XFJg2FNziNDQWZZif","XM3SivdnKp3ArPWH2","XYoGz7W43gRqdwMEd","XiaYPyQyBstgqSQAM","XovrtQ8uZBmA9Zs2d","XpWktgjenQkZNbvQ5","XrDFxWGBwB2zoeRo8","Xt4fxE8kLfYz8HxaQ","Y55skk5YCx93NG2Tq","Y6dgSEidBvEkkTTxw","YB4n7LfWDp6svy3gM","YZcWPSvyo5R8A3oLX","YkZkMSWAjfdrK6xcG","Yn8A5WsXM7F6dhtDp","YnYB86oPr8kDxXAte","YqoGdXfxAfZAQGRnX","YvW8pAe89MjLa7g72","Z8Td5XiRsG8oy3KTZ","ZMnih9LaXH8nthk2N","ZPqXdv7kFZfDqHGdK","ZXX5bd66Fp7FkZEhw","ZaNrpmfAxjWsAAHHH","Zbjr8NKGWw7ZwY5yN","ZmNzk6qkGgSSMkRSt","ZnbF2MX4eosSNScZf","a4RSLZS2L7bHmb3yJ","aB2dEEQh63hZpDqws","aBSAYpc3JpvnyKwvy","aFnRYmrkjciNggH5y","aWqityMBMbCPewcfd","aXrZ3GQA6Lm9pDLpb","arKYPAFgZcJqrWezg","arY67YYSa8Pjcy55y","b2fxdwu5hQj8j2ifc","b3dbZMf8ARd75QQXq","bCbe5GrbuR8XKbLBm","bCzQscXZxCYwtKcD3","bHDM8kz4yHY7BjY8W","bKdtj4LpYkF2jX4hS","bMNdScBQayvZSZ3Ca","bMa9wmcJ6twCjhJMD","bMnmC6JrQLLbPAXWp","bWPrEok7JNuRkqbF5","bdnTGosadQ7kSnNTi","bjQchasaM6DGh3RQ5","bkWanSWNHgn4zg5jm","bnNH48CbB6j7uns2B","bqHh9QzsMthKqzKk9","brNmxKHgBjbmLoSxm","btmtYgzpScZc7epbC","c4xhwabymWyxt33MA","c8fguowQHTAmx8sCQ","cRHMLPkN9QTy5XDbN","cakFvXPXCC4Y6Btk7","ccsWDQEoC5Lo76NRD","cgyw4CX8rTB2AeZ7r","cydkzEod83uwAfE5y","d3dEH3MZfjLKEHwHY","d9nSjx7jWzz4B52H3","dEaCbuP5aJK4TE4a4","dKKXsCtaS3CLJzmN4","dKrfNvkGosaBSvtBo","dPjMDwaaCHKogSQv2","dXfgsTouqXXsYi4Bt","dZqKEkwLWRPnrfSzY","dd5seF2YR5svHNNGq","deGCjQqAqxFuQRTem","dmKzCoguBXS6R3Siu","e2ojBQDuy27QMBHLn","eD8brpmQiMZws3syo","eEwgu4NJCLQSRLSce","ePcpsLSkdqspuXeod","eX53FQbeLQLNkCGBg","etBySHgMaS8PWmb9J","f2XwwCpYhet94Dg2v","f8qLCiGuNoWET6wCP","fBjttmQCY7JAS8djB","fEDH56hmvFhGNdtcP","fKM7wGujhjgXPXb7s","fQeT3Pv6n7X7eq7dH","fTr27xmrMb4T7QqRc","fWnEj77jrF4TXGadQ","fZ3aervy9QkcgArqG","fdrZMxvcfKrrQvS2g","fpYz9khyvpNFZGkA5","g4AnYEzFcsC4swLLN","gBPcEd4qscyAcj5jK","gboJRf5LwZ88hmnnT","gcjFgyMEgR9TXBgkP","ghZosEL3PJ55RMgsW","gx2DhujEFdQtvc3KQ","gyfBGx5ZaSocGs48z","h2j8pTraY8rzfEMzi","h7pLiwfEv8aGpq2Pm","hPhaLxc3C2qi4nShu","hZF3DLyNMFnkMF9Qd","haiB2TLh53hJsFdXt","hb874dvct7QDsmAbs","hfzHzgLmF6n3u9AEu","hpa7gEJEJ4fitCbvw","i6SeE4ZoXm6sMf8uJ","iCQaaxLdewqYYLEuZ","iFDvR8FuMtoDNNReK","iH5toY2Sk5ih5GH2S","iaKRLPLFcKwEZuR5G","in4rRt4ym5nsGfypP","itkwrSBHfF34XNpJu","iyRFrnNDqqvnxBpwP","j2MNPh6tBop3tixA5","jDCWjhdNcRvGzYi26","jE2zmi5zicRThJghq","jFoTYXc6ikY74RcdC","jKdsH4EFMdDtZG9fG","jLSqf9YtKR6PsjikH","jPrZwXQ4H2xR4beNL","jWR9cknAtujiG24nE","jbRhZKDuxDP3DmeJR","jbtpytFj2y8KLQNbM","jgW7xPco9rGpbutRT","kCwyuxFTJ2W6kSBS8","kMZFbZbAezyeeEc3p","kdbQBec6kqbekBmCm","kiqxH5pW93PQRocBp","ktgEe7dPDG24jSDpn","mGEMDoqduRwPFEXb8","mJkCJiF3QKjtakpN2","mb7DRMN9QjLi7y3da","mfimXySKESAPYAeHc","mgvaTtigSdm4w4cS2","msaNfmWtDWr6xZ5wS","n2DfnSKYnQiYNLQnD","n3Yk36u5KqTef6gd4","n3pEXTo7HFf4B25mz","n4keTykzBWMW6uzLj","n5NY7uyGNQpiw6Tcf","nQP76z3E4ups6vDBK","nXQJpMjGHDWuBmjM7","nv5ZW4F4KB9qcSGdx","o4y5eNk2QcutiXnEm","oFGq2ncQB7CHksotf","oJdYYXsiKSC9nf6hj","oTZNqJA82RDhS7SCf","oityvstCirpP3A5cQ","omCdNWm4wWRtZnxnm","ooHFaebPcYxtzzR2R","opuKmGD8oAPiA7szp","osMwyijihfQ6GRjqA","oxqbmDuxezfdaWNnD","oyHX4wmv7pAKvX7Rn","oz7wNmXRpHHPeddns","ozxEfpefATRisPqMs","pC5xR2xAhrb7NGFrc","pH9CHnyt8ML4McK2W","paBSuuKNRwybYWqPq","pfytuMCXzc9xzeEty","pr2qQxvF7fFjxh2rx","prsd5JrGMbxN4jAXJ","pse4QkKwmn8R8opC6","pvaS5WY4ry6EyKpN9","pwhb5Xh5CstEXtrez","pwtAzTwzGb6ZQt6Ay","q2S6KraJoX9JJJCfw","q5Ko6ypkzf5kDi33L","qH7iZ69HSmBsmjjfh","qieqsvpPQom6LyD42","qkw8mzyGXuctuYbvH","qydN3FWuThrzfSEzt","r7YvFKS8iQRxSkQfr","rPhBsi3J6Jey5v5m6","rRs7KsqyMrbPXc9a6","rYpRnr9fiYotY7wkq","rd32gznnSqtpTWKWL","rdCarzDQZEZgiicQA","rkpXeKXA6CHgrJiDc","rqNQyqgJNrJkDtiTD","rseunCNHSSYXCqtpo","rtbyJWfs6Twj7cMaF","shqiBTj9oxB3Mtvrg","t77CevojN8bDMhBhx","tH3zHxPnQjcJZCHRo","tbEtDLAbDLDvYzxcR","tbf8BEwXEDB7F9DSw","tiE4kpRQF4pyx73h9","tmnbYHB8oEknEFywM","toj72JP95DorN9SMu","trvjQopWpv4vXkgsz","uEswuoqzaiqja7MTN","uKkn58BXRuiZWWZsN","uMEHGy5n67E65GKmn","uTWzHhaAKksDeeDnd","uacxRNsYcQ73QAE3S","ucsuwkpn7SbLvD9QT","ujyFK9FiL8KQTKFJA","usQ4DNq3X9EaiyB6G","uzGeY4B8WmkLkiyDL","v3szQLBkCN3tx2HxW","v87MtauRqKvpmawB8","vPwTk2Gff8p6RRWoC","vmtj8fCiwdqRt6SJZ","wBrPAisfDHZPSQQAo","wH6sfouf525dyW92r","wJxSrHHqNiPLYqXFh","wPkTQockRzgtqF5DR","x5pB39qHmwjP9ZTwo","x7BTTLJHroPmgkHPE","xAAT6z7FfhqftLwcS","xAEjghYqEwcQihbvY","xoW8mHh6KyLgHYJan","xrG752QtKyxCTDpxn","y75YDJWJqH8DveweX","yE8QsWtfGYrwmp9B6","yXPM3WskiMAoowgwx","ynby82cQGRgNamXBG","ytzG88BKbE7GKmzNS","z4fCLL96RtzFyguoT","z5NWqnna4JyYkgxgL","z85MatuJgGksKaRBe","z93sw2BT6jN3DGCKZ","zQ5JxpuDKJWYFShw5","zQZEkvfgMBSWfFDea","zTfNWJLNEQTbwpW7d","zkyHTWokzxDHnvgnP","zoiym4hAeyMzfJaQo","zqCGPzpxAsRdT7qNa","zuRkee692GAoCZxmL","zx5sPqEvCtrw3Aq53"],"dataSources":["WWB362W3v645Y44ZD","g3ofqhxNQWsRWkCrp"]}