Multi-objective ranking and selection: Optimal sampling laws and tractable approximations via SCORE. Applegate, E. A., Feldman, G., Hunter, S. R., & Pasupathy, R. *Journal of Simulation*, 14(1):21–40, 2020. Recipient of the KD Tocher Medal, 2019-2020, The Operational Research Society.

Paper doi abstract bibtex

Paper doi abstract bibtex

Consider the multi-objective ranking and selection (MORS) problem in which we select the Pareto-optimal set from a finite set of systems evaluated on three or more stochastic objectives. Solving this problem is difficult because we must determine how to allocate a simulation budget among the systems to minimize the probability that any systems are misclassified. Toward determining such a simulation budget allocation, we characterize the exact asymptotically optimal sample allocation that maximizes the misclassification-probability decay rate, and we provide an implementable allocation called MO-SCORE. The MO-SCORE allocation has three salient features: (a) it simultaneously controls the probabilities of misclassification by exclusion and inclusion; (b) it uses a fast dimension-sweep algorithm to identify phantom Pareto systems crucial for computational efficiency; and (c) it models dependence between the objectives. The MO-SCORE allocation is fast and accurate for problems with three objectives or a small number of systems. For problems with four or more objectives and a large number of systems, where modeling dependence has diminishing returns relative to computational speed, we propose independent MO-SCORE (iMO-SCORE). Our numerical experience is extensive and promising: MO-SCORE and iMO-SCORE successfully solve MORS problems involving several thousand systems in three and four objectives.

@article{2020appfeletal, Year = {2020}, Author = {E. A. Applegate and G. Feldman and S. R. Hunter and R. Pasupathy}, Title = {Multi-objective ranking and selection: {O}ptimal sampling laws and tractable approximations via {SCORE}}, journal = {Journal of Simulation}, volume = {14}, number = {1}, pages = {21--40}, doi = {10.1080/17477778.2019.1633891}, url_Paper = {http://web.ics.purdue.edu/~hunter63/PAPERS/pre2019appfeletal.pdf}, abstract = {Consider the multi-objective ranking and selection (MORS) problem in which we select the Pareto-optimal set from a finite set of systems evaluated on three or more stochastic objectives. Solving this problem is difficult because we must determine how to allocate a simulation budget among the systems to minimize the probability that any systems are misclassified. Toward determining such a simulation budget allocation, we characterize the exact asymptotically optimal sample allocation that maximizes the misclassification-probability decay rate, and we provide an implementable allocation called MO-SCORE. The MO-SCORE allocation has three salient features: (a) it simultaneously controls the probabilities of misclassification by exclusion and inclusion; (b) it uses a fast dimension-sweep algorithm to identify phantom Pareto systems crucial for computational efficiency; and (c) it models dependence between the objectives. The MO-SCORE allocation is fast and accurate for problems with three objectives or a small number of systems. For problems with four or more objectives and a large number of systems, where modeling dependence has diminishing returns relative to computational speed, we propose independent MO-SCORE (iMO-SCORE). Our numerical experience is extensive and promising: MO-SCORE and iMO-SCORE successfully solve MORS problems involving several thousand systems in three and four objectives.}, keywords = {simulation optimization > multi-objective > ranking and selection}, bibbase_note = {<span style="color: green">Recipient of the KD Tocher Medal, 2019-2020, The Operational Research Society.</span>}}

Downloads: 0

{"_id":"PBa56NjSCG5LLkA7P","bibbaseid":"applegate-feldman-hunter-pasupathy-multiobjectiverankingandselectionoptimalsamplinglawsandtractableapproximationsviascore-2020","authorIDs":["2PAn3MHZBPgt7AWCw","2Z97GEaPa4YXAS8ny","2fb2S2XHAJ2t96GqF","2zh9DyNaWkygD5248","365LawuHG2j9DJRFe","3R63vuFS9Bfnm4SAW","3aJ4h5X54uMKyds8C","3dTRPZMyK6NQnkBNd","3fJLfDjoSLWT7fdsH","3wDvcSHgfLWgWtiCx","44WAdmxsAix3xNMiS","47j5kNH4LLRdyaGrh","4zadZEFLgF8nnfqBc","59NY6KQiaNfh8vHLK","5FBFKWvbGTDfY2rBw","5e139165a212e1de0100014d","5e13c558280ddede010000e2","5e13fa50f8aa5dde0100005b","5e149a2d830852de0100005f","5e14c260e55ed8de01000116","5e154833edfb1ede010000e4","5e15647d1e2528de01000046","5e15798df1f31adf01000019","5e1600f6efa1cddf01000257","5e168ec10ba191df010001ec","5e16af67dc7739de01000036","5e1764e44df69dde01000172","5e17893acf35a4de010001ad","5e17d5f74ba003df010000c7","5e187dc267b9ebde010000f1","5e18e6518fcbc2df010000a8","5e194b1e86b4aade010000c8","5e19adbf078978df0100004c","5e19fdebcde53bde01000028","5e1a11f1cde53bde0100011e","5e1aeaab5f3d2cdf01000100","5e1b901f7c0fe1df010000b8","5e1bb3c961cb16df010000b2","5e1c88124c9abfde010000ba","5e1cb0ab7723aadf010000f3","5e1d275afeb115df01000185","5e1dc86e8d71ddde01000151","5e1df7292cced5de010000af","5e1e0a93f6dca7f20100001a","5e1e11fdf6dca7f20100009c","5e1e2060f6dca7f2010001ba","5e1e38ab407a20de010001b9","5e1e3c44407a20de0100020c","5e1f6435e8f5ddde01000105","5e1f9b324fcd1cde01000079","5e20c5105c2065de010001aa","5e21011cc63e88df01000004","5e215a7e5a651cdf0100001c","5e217a0bc7842fde01000095","5e21ea7959a877de01000087","5e221004024c69df01000155","5e232f4f327a15de010000ce","5e24791636283cde010000a6","5e2488578c3885df01000036","5e2491cf8c3885df0100009d","5e250c0a2e79a1f2010000ea","5e25bd5af299d4de0100003d","5e25ec2aa6f19fde01000233","5e2621f3408641df01000124","5e2677d0581147f2010000f1","5e26af11f3bb7ddf010000d1","5e272f59557b88de010000c2","5e276b0758994fde010000f9","5e27854655fc50df0100000c","5e2790304c3b0dde010000db","5e27d25c68d625de01000019","5e28623f6ae365de01000052","5e28abc988416fde010001ee","5e28bbeb6acacbdf010000b4","5e28d839a3df5bdf01000092","5e2b1f0927ed83df01000022","5e2b3a70b9f2cade01000019","5e2b44c7b9f2cade0100009b","5e2ca0f8061fbfde010000d9","5e2d92a6481fd6de01000044","5e2e37b0185844df01000050","5e2f0b91e374eede010002d5","5e2f55e826e5cadf01000148","5e2f8afc48b7a4df0100010a","5e3056d557a222df01000151","5e30953bcb949bdf01000187","5e3334baa5c1fdde01000190","5e334d47e0067bde0100014d","5e33873f7a676dee010000e6","5e339d1717f2c9de0100008c","5e3481b9fae8b9de0100011d","5e35205e89e3d9de01000013","5e35dd2876dd53de0100010d","5e389020030bcadf010000f9","5e38adce645ed2de010000ff","5e38d7f581a46ade01000007","5e3944a39d05f2df0100016f","5e3a050baa2adade0100013e","5e3bf9a4ea028bde010000ed","5e3c7b17feacaede01000034","5e3d8b0c96e576de010001aa","5e3dbc1807ca74de010000ce","5e3dcf30d51253de0100005d","5e3ddfacd51253de01000142","5e3e00c4a4cc0ede0100014a","5e3ef14a382e41df010000ce","5e3f20d47da304de01000038","5e3f5f61cd8fe2de01000099","5e4071b8b531d7de01000038","5e416c4ed9f47bee0100003d","5e4189e08491fadf01000072","5e4193d78491fadf0100012d","5e41fdceebe241de01000085","5e437aceb5e412df01000117","5e44441ee5a34dde010001fb","5e446de3084293df010000d8","5e45623049667cde01000047","5e45678b49667cde010000aa","5e45be2e0920e8de010000ea","5e45f90fa5737fde01000093","5e4768d445a735de01000097","5e4770dc45a735de010000da","5e4776a127a0c8de0100001b","5e49d2feb63120f2010000a3","5e49ecd0885ce9df0100005b","5e4c2ef7c1eb51df0100000e","5e4c3272c1eb51df01000054","5e4c725ff2c6ddde010000f6","5e4c86cc5cc521f20100002f","5e4c89d75cc521f20100005d","5e4d3a95d43139de010000ce","5e4d673c08a8e5de0100007d","5e4e10f1d116fbde01000042","5e4f888342a908de01000070","5e4f8c7042a908de010000a1","5e4f8d9942a908de010000b2","5e4ffa78f5b214df01000102","5e500a19933046de0100003b","5e5023168c3a2cde0100000b","5e50725dcb6c3ede01000120","5e50e8199f7a6dde01000005","5e515d9aa04830de0100004c","5e54534d88d190df01000040","5e5481967f0f44de0100011b","5e555acee89e5fde010000d1","5e565ec06f0b61df01000107","5e56a5afe177dede0100008e","5e56afb3e177dede01000129","5e56c3da96127bde010000cf","5e56fe107840dfde010001e4","5e5700387840dfde010002e1","5e5771bf16d3d2f3010000be","5e57ffa6a38020de0100013b","5e583cdb1f3fc8de01000153","5e599a85ad6c7fde0100006d","5e5b0be96e568ade0100004f","5e5b7186502fdadf010000a9","5e5c6589f4282ddf010000c0","5e5cb1f5a9598ddf0100004e","5e5d36f373eb2edf01000035","5e5d4dc973eb2edf01000228","5e5de8f4863279df01000088","5e5e052b5c89fadf0100006d","5e5e8bcec0a53dde0100007e","5e5f802d5766d9df01000192","5e600a0913e3aede01000190","5e601becc064fcde010001b2","5e6080039119f0de0100012a","5e60a33bc28c0fde0100004a","5e61098731c7d3de010001fc","5e61223c1cc34ede010001f4","5e62afbe08ebcade010001a0","5e6443fdb8c607de01000034","5e651c7a9eed46de01000116","5e656cdede41b9df01000142","5e66eb4a85689bf301000069","5e679bbcd527f0de010001ad","5e67f5f00e29d3de0100030a","5e68f31f1a389bdf010003f3","5e69a88d23ebccde010000de","5e6a8bd10e8744de010000cf","5e6b7b12c024f9de01000161","5vvrLHqsgH9E2QPjP","5wgZYSuyAMqRBppg9","69Gb9G7uXuGC9CqrT","6BSaqwxscteLBtEJv","6XixHSsWpd8nxzP7K","6o7DA2usY7FENyPRa","6oAq3TGhqimHY2ber","6rFcASaST7YgKdxHc","6sGcbuXnEYZ8Wmuej","74SNnigtjBX48iG3J","77kA4icgJBShuH63W","7A7q37F5APXNrBhxq","7AjGyhK3iuwzff65n","7BAciBd9hbBEkJefR","7CKGZqSpvcj9u5Fsr","7TpbYvGuRf5yRNuMy","7XBRX4AfHzxd8Yhai","7bRE2A4kCENz7Y5qp","83kyS3bRfQKny6SY5","8BBCAgWbd5QXXgsKi","8D7teT2RWr8xuZFD7","8YF4BGcrFpTcZeybA","8ZTszaF4sYucLJTae","8csKr3DgDfXLDRe8a","9FWArSc8hhQ3zvuZp","9Lo5KJEgpgs2rS6tb","9bvDsXr3AkbNMTj3v","9eA3jvWTYH8FSkeBT","9oBgKFdzYg3aEn5oz","A3kcMtBKeBkRzJ4To","AxHmaYpiG48A4vPpB","AzCZNMFjGr5ZYibXh","B2khThQvkeNjzTrJo","B8jEGwLatXm2dJm4r","BD5J3ZmoyvJJJibjZ","BFZEw6tWHhQ29NZhW","BP7gWobLA4F44fwyS","BYCMaBqwqZrJaH8E6","CRGYszpgrsRcosfhy","CidHtLPczZxb4Zopq","CoPYFNeBaL9REPBcJ","Cy28TS23ZijFMJLGR","DD2ZkDRBYmppLTXZN","DJh92bpFSACASgRZj","DWqxxAYWrgCbhbye5","DbNtGNp7Q4SPb66vp","DbSgAa6dd5m5PeeM8","DyKSBkzDbpr7iNQuQ","ESLhYXSuLXx4bYQoE","EWoH3ji5LgDxk4rez","EXYASwpLGtAbmfwQn","EhtcjnZvWJEn8jsNp","EnATfkZY9bxYYQc2D","EsQJ3szXcmtN6EzF5","F2HtNNL3tjydiQNGC","FDv7QksjBPSGwx34t","FK4tgksyCD6q9EXmD","FRJc6p3hR43yvMLya","FkRCFp3njhTn9oH2q","FwmskpEqEi5e22No6","GcnKnvyQRvm7ufjvG","GmkdwRpnsTbRnGPon","Gx2aaKwAwexRyq922","H5iuFjLuqh9oBbyy3","HRhyNRNQq76ox2aZq","HSRa9AAqRKaduw928","HjG3JeRSfsFsFHFge","HqKgkHBhXPQiSz6Ax","HrnA7bqRRPp5yfKCW","J4DA5MLHaQaff2ivR","J7Kj5xZj7gvxGpGdR","JZX9c6cixBcSCcRGS","JwMCBP4YSNrfrxXFx","K4A22AsSSfeqBFrpT","KNXv3J5frDgBs4hzt","KPGd3pWTc37z2ezKa","KRS6anSw2zkcvuCZF","KSmJtfMob3tCnxBEH","KZPKXjxEb8DCTSPsm","KdDhHf9MgFpueLw4h","KoB4o2BiACqj5wZpu","KpRatkH36PEhxzWRh","KqYTsXZ2jaNM5xu48","KwBBk7q2r6thvDaEh","KyrmE4w34odibeaja","LY3g34CKzH8c4GLcn","LoEuQXkNFLqmk4q2D","LpJ8vbMGQRoiDYtTA","LvMApwCWwFjQuA9xu","MSPPZ5YXozD5hjb32","MXoqkWkTXa3FNWnb3","N8xvQbfJTTeDhmem4","NLvKfJtnLhY27LX8G","NWBGWbitviQJ7PMGx","NisS4NB2Z8XsoPiop","NqpDrTcQZStBfH7GP","NzRRX2tf5vZJTwTAT","P2qZ2h7fnEgXhP8wZ","PHoZQDYbZyCnFk4qx","PLA7e5v7eqSpa2srh","Pfyj9gd9vixBdW8Mz","Phrpv7TxDneNK9ZD6","PnP7X6JmJoortQxaE","Ps7nv28bsn83ovgJ5","PuxwagkSyCKBaaJjD","QFQkLLBNXhKq7zX3p","QFggoBXHygtZ9mrgA","QSRMyDaD5AMhsJoiT","QWoxEn67eLEERCcri","QckZeqhHc6NzEWjGQ","Qx2KiCGn5ZQj3Tm6D","R2bqLqzH37LSk3dEN","R9wEkBEHQhvitAeFK","RQPtRvHp3CHJYhx82","RWdXiPog4qrpiQhXM","RXbY8jQiFTwhFpHn5","RdcJRWAbJ6di8XL34","RmERscPN9dpBehM2r","RsPEvg4f4xNkZFiDz","S5rLyt5xWqri4Bm3B","SDXNeyucEqZJYzJPP","SMyENBGLwed9bg9dn","SPB7xjcD7dK5nag5i","SjPdLifsNLFiSPJnX","SonBkYjLbzfPdue6b","TqngjqzjEatA5mESz","WMMj2X9HmEfTPF4o6","WddAKna5uDTBSTL94","Wret9iQc82sz2AY32","WxkMEB7EpNhDhnDBb","WzzR4fueCTpsbR2e6","XhoedZd4vYHegRmzZ","Xk6BmXSsoNW6gQteS","YEDQuaQoMxfrEpZi3","YWG2Cg3nQeSruic2j","YioYfkr22LZsF4Bj4","ZAFn76SXremTQfSnC","ZGgpqGFXoZgyvbx3X","ZMQfYQC2HGHJpNDSW","ZcpScSR7rNarBsSKM","ZgNmEX9rJfSr3Bueo","ZorESQjDDeNSt2vqd","ZspbD9bauzj84c8bY","aZpJsemKeCgjXbTGe","aj9xZEAY5ehNe9wAv","anZ7x9gcmg7ZwnBy4","bHxtEXL9pz5BbQNWX","bQ7n7SYYoGvTu2BqW","bfbe5r9Efrvh3Abe9","c7gnKGitCDDFYXLu2","cJkKeEvMNpbG696wB","cPdTM2nhmScQHoDtd","cojpq7debEkuiAMKD","d8uKbua5DCY5YWL45","dCen7dYEkvh5gzLBu","dKbmMcMaBaMmHXs7R","dMH3mH6h7RBoq7n5r","dXDH3shpYKxXxiXjv","dwEz5g8oMXoQejtCY","eDH5nop7nDFFYbDyB","ev3EDPEzFJB99CmzN","f7FxbjPn83vKCNwNk","f8gt8jWwjAtdSpSmi","fMuYsPwdXuaHmhxsb","fMybk9pFkPsYfu2pm","fNbLXe22X3xL5Zyth","fgAXke2pPhQrFqn2v","fuQXWK67Sdygy9tFe","fzcyEogMMEjWWo9qh","gPqmewhWtsTPbgNPx","gd4XDSpYd52DJSLQz","ggitmEfdT5fdksrZR","h4qsJcBP4TfAyHrET","hN38yNSNKhwBpjoWY","hTSRyxPMhxL3PazAr","hvCmfbQcbGXvwgZdQ","iQ8pM7cN9k9ayFEgw","iS7zLee83q7RKyECX","iaeBcQcRdCBy6xkoB","icGnxiSs3QxJHPS4s","idQTxDet42qC3Jd2x","ioDPa62Yt5HtqzWBu","iu7X2yovAZ5LEffrz","jFyTMgvSAnuwd3Me5","jKxBw8nYpeW6HLL3c","jWS7s6FfkEdYCrXxx","jv7LiD4CaQ4tRPqme","kEmEjTDBS3hhFziWL","kKdYBKCnkJskTLAEq","kYiaRyZRtgDmgu88o","kZLyMfsQmTCHFo9LC","kdqbWGpRQqFBREEbt","m2qsJXBg6utwwaZc4","mXueAvNYgyizg5WYf","msQNMu4Jo4n23S4hR","mwxyQE8C3Y9Kzu34X","nEShNQfkQDLKnm8FJ","nHggQLJ5rWfWtTAMc","nbZJFgwHuyua9ZMzN","nwgMcwNRBJkaaSMYA","nz5DrStoskLqxcz3j","o5W2wMaBXz7AsMoFD","oEHyQe2wbwZNhgMfe","oS4CBDmTdRZ6zaBzN","oaXfRQgd56s8mLizu","ozRp6sPHyXbrvuGxz","pEn65TZJdJPSjfJ6o","pnn9hoHGt9ZB7Yh3C","ppTXsix8PiTptHPTv","pvwyq2FsvHLSD2mnu","pxX3fvGrCK6CfFdxR","qAEGfBZSyYw5g44aB","qARqJvwid5k5E4KbL","qG4DGzjdabcMZA6ZK","qaXjiXZy56D7eRK8s","qmTXWT4PjGCaFf3Qf","rKgbsFjQYxr4brzA9","rRmNDMhwxWb922jFP","rgiWPKgbncLaJAqdh","rw6eycCoLApNdcEK3","s26YgqwunsDmsFZ3d","s72o6CrDptDYGTpg7","sNaWwoyt3MQZcLBvN","sPWSimL3rqxPPbFmT","sPfFWugdNbn4tjcLK","saYcRnQds6rEnJcN6","sdNwxguoZiGQuzNnQ","shGSTLbNCQ2mQ7C6B","snGb73RqNP9yhN4MD","t9ndRegAnj4uBmXvf","tLxKsv6xZTqs7ZZfE","tY7P35MLkWM2aHYhn","u2tNnhyZqjS9d7iDG","uF8fHSMEGz6tMYypJ","uJ8Rrdfthg7CzYvvi","uSN2tqTyg3s86P9bD","uWvJWjog6W8qix2qB","uXozimonaRGCzaADW","uuyFnjBtxca5zh9Ky","uvrun9f6iXENKDt3n","uzhXQMSBLEiWwaSrF","vCGw9rPG9d8XMggop","vE5RpwhAoKjbjmx8s","vEnjk9djxnuHbCyTa","vLvEkskwie7Xk8dps","vMQgcRGs3htxFRYco","vkirKDjhjYWTQZj4m","wDoGhd6RWWE4o9CeX","wnMj9SHXnBYWtKQZp","wuPh3vGzrJ69cD83u","wxJH73fuSn9HdLMj9","xBYEKZNGX3wfQdTcT","xEm7baDDyTm33FzCK","xX3BFvy85tSBTuinW","xZC6uifPzee5wRWLB","xtey9QmYxZQ5cqYgy","xuM8ocmp3RXK6CQmE","xupeg6dJBotE9GBsm","xvKzadizesMZZntjF","yBPsWrbvZRvrQ9v3w","yggMy2DdHyArMfEP2","zeq65d2AxoMPqqqcE"],"author_short":["Applegate, E. A.","Feldman, G.","Hunter, S. R.","Pasupathy, R."],"bibdata":{"bibtype":"article","type":"article","year":"2020","author":[{"firstnames":["E.","A."],"propositions":[],"lastnames":["Applegate"],"suffixes":[]},{"firstnames":["G."],"propositions":[],"lastnames":["Feldman"],"suffixes":[]},{"firstnames":["S.","R."],"propositions":[],"lastnames":["Hunter"],"suffixes":[]},{"firstnames":["R."],"propositions":[],"lastnames":["Pasupathy"],"suffixes":[]}],"title":"Multi-objective ranking and selection: Optimal sampling laws and tractable approximations via SCORE","journal":"Journal of Simulation","volume":"14","number":"1","pages":"21–40","doi":"10.1080/17477778.2019.1633891","url_paper":"http://web.ics.purdue.edu/~hunter63/PAPERS/pre2019appfeletal.pdf","abstract":"Consider the multi-objective ranking and selection (MORS) problem in which we select the Pareto-optimal set from a finite set of systems evaluated on three or more stochastic objectives. Solving this problem is difficult because we must determine how to allocate a simulation budget among the systems to minimize the probability that any systems are misclassified. Toward determining such a simulation budget allocation, we characterize the exact asymptotically optimal sample allocation that maximizes the misclassification-probability decay rate, and we provide an implementable allocation called MO-SCORE. The MO-SCORE allocation has three salient features: (a) it simultaneously controls the probabilities of misclassification by exclusion and inclusion; (b) it uses a fast dimension-sweep algorithm to identify phantom Pareto systems crucial for computational efficiency; and (c) it models dependence between the objectives. The MO-SCORE allocation is fast and accurate for problems with three objectives or a small number of systems. For problems with four or more objectives and a large number of systems, where modeling dependence has diminishing returns relative to computational speed, we propose independent MO-SCORE (iMO-SCORE). Our numerical experience is extensive and promising: MO-SCORE and iMO-SCORE successfully solve MORS problems involving several thousand systems in three and four objectives.","keywords":"simulation optimization > multi-objective > ranking and selection","bibbase_note":"<span style=\"color: green\">Recipient of the KD Tocher Medal, 2019-2020, The Operational Research Society.</span>","bibtex":"@article{2020appfeletal,\n\tYear = {2020},\n\tAuthor = {E. A. Applegate and G. Feldman and S. R. Hunter and R. Pasupathy},\n\tTitle = {Multi-objective ranking and selection: {O}ptimal sampling laws and tractable approximations via {SCORE}},\n\tjournal = {Journal of Simulation},\n\tvolume = {14},\n\tnumber = {1},\n\tpages = {21--40},\n\tdoi = {10.1080/17477778.2019.1633891},\n\turl_Paper = {http://web.ics.purdue.edu/~hunter63/PAPERS/pre2019appfeletal.pdf},\n\tabstract = {Consider the multi-objective ranking and selection (MORS) problem in which we select the Pareto-optimal set from a finite set of systems evaluated on three or more stochastic objectives. Solving this problem is difficult because we must determine how to allocate a simulation budget among the systems to minimize the probability that any systems are misclassified. Toward determining such a simulation budget allocation, we characterize the exact asymptotically optimal sample allocation that maximizes the misclassification-probability decay rate, and we provide an implementable allocation called MO-SCORE. The MO-SCORE allocation has three salient features: (a) it simultaneously controls the probabilities of misclassification by exclusion and inclusion; (b) it uses a fast dimension-sweep algorithm to identify phantom Pareto systems crucial for computational efficiency; and (c) it models dependence between the objectives. The MO-SCORE allocation is fast and accurate for problems with three objectives or a small number of systems. For problems with four or more objectives and a large number of systems, where modeling dependence has diminishing returns relative to computational speed, we propose independent MO-SCORE (iMO-SCORE). Our numerical experience is extensive and promising: MO-SCORE and iMO-SCORE successfully solve MORS problems involving several thousand systems in three and four objectives.},\n\tkeywords = {simulation optimization > multi-objective > ranking and selection},\n\tbibbase_note = {<span style=\"color: green\">Recipient of the KD Tocher Medal, 2019-2020, The Operational Research Society.</span>}} \n","author_short":["Applegate, E. A.","Feldman, G.","Hunter, S. R.","Pasupathy, R."],"key":"2020appfeletal","id":"2020appfeletal","bibbaseid":"applegate-feldman-hunter-pasupathy-multiobjectiverankingandselectionoptimalsamplinglawsandtractableapproximationsviascore-2020","role":"author","urls":{" paper":"http://web.ics.purdue.edu/~hunter63/PAPERS/pre2019appfeletal.pdf"},"keyword":["simulation optimization > multi-objective > ranking and selection"],"metadata":{"authorlinks":{"hunter, s":"https://web.ics.purdue.edu/~hunter63/"}}},"bibtype":"article","biburl":"http://web.ics.purdue.edu/~hunter63/PAPERS/srhunterweb.bib","creationDate":"2020-01-06T22:01:54.397Z","downloads":56,"keywords":["simulation optimization > multi-objective > ranking and selection"],"search_terms":["multi","objective","ranking","selection","optimal","sampling","laws","tractable","approximations","via","score","applegate","feldman","hunter","pasupathy"],"title":"Multi-objective ranking and selection: Optimal sampling laws and tractable approximations via SCORE","year":2020,"dataSources":["PkcXzWbdqPvM6bmCx","ZEwmdExPMCtzAbo22"]}