Born-again tree ensembles

Born-again tree ensembles. Vidal, T. & Schiffer, M. In III, H., D. & Singh, A., editors, ICML'20, volume 119, of Proceedings of Machine Learning Research, pages 9743-9753, 2020. PMLR.

Paper

Website abstract bibtex 2 downloads

The use of machine learning algorithms in finance, medicine, and criminal justice can deeply impact human lives. As a consequence, research into interpretable machine learning has rapidly grown in an attempt to better control and fix possible sources of mistakes and biases. Tree ensembles, in particular, offer a good prediction quality in various domains, but the concurrent use of multiple trees reduces the interpretability of the ensemble. Against this background, we study born-again tree ensembles, i.e., the process of constructing a single decision tree of minimum size that reproduces the exact same behavior as a given tree ensemble in its entire feature space. To find such a tree, we develop a dynamic-programming based algorithm that exploits sophisticated pruning and bounding rules to reduce the number of recursive calls. This algorithm generates optimal born-again trees for many datasets of practical interest, leading to classifiers which are typically simpler and more interpretable without any other form of compromise.

@inproceedings{
 title = {Born-again tree ensembles},
 type = {inproceedings},
 year = {2020},
 pages = {9743-9753},
 volume = {119},
 websites = {http://proceedings.mlr.press/v119/vidal20a.html},
 publisher = {PMLR},
 city = {Virtual},
 series = {Proceedings of Machine Learning Research},
 id = {c20200b9-20b6-3024-b8e5-da5ef302e63a},
 created = {2020-12-11T17:34:06.153Z},
 file_attached = {true},
 profile_id = {5e3d1dc4-cb58-3af5-aff1-4d943d2eaf6a},
 last_modified = {2024-03-03T18:42:12.142Z},
 read = {true},
 starred = {false},
 authored = {true},
 confirmed = {true},
 hidden = {false},
 citation_key = {Vidal2020a},
 folder_uuids = {a320750a-d11c-40da-85cb-b8658f813237},
 private_publication = {false},
 abstract = {The use of machine learning algorithms in finance, medicine, and criminal justice can deeply impact human lives. As a consequence, research into interpretable machine learning has rapidly grown in an attempt to better control and fix possible sources of mistakes and biases. Tree ensembles, in particular, offer a good prediction quality in various domains, but the concurrent use of multiple trees reduces the interpretability of the ensemble. Against this background, we study born-again tree ensembles, i.e., the process of constructing a single decision tree of minimum size that reproduces the exact same behavior as a given tree ensemble in its entire feature space. To find such a tree, we develop a dynamic-programming based algorithm that exploits sophisticated pruning and bounding rules to reduce the number of recursive calls. This algorithm generates optimal born-again trees for many datasets of practical interest, leading to classifiers which are typically simpler and more interpretable without any other form of compromise.},
 bibtype = {inproceedings},
 author = {Vidal, T. and Schiffer, M.},
 editor = {III, Hal Daumé and Singh, Aarti},
 booktitle = {ICML'20}
}

Downloads: 2

{"_id":"vu7b5h4HKCsQ8RcG5","bibbaseid":"vidal-schiffer-bornagaintreeensembles-2020","authorIDs":["awoTsndRPACdYbwLY"],"author_short":["Vidal, T.","Schiffer, M."],"bibdata":{"title":"Born-again tree ensembles","type":"inproceedings","year":"2020","pages":"9743-9753","volume":"119","websites":"http://proceedings.mlr.press/v119/vidal20a.html","publisher":"PMLR","city":"Virtual","series":"Proceedings of Machine Learning Research","id":"c20200b9-20b6-3024-b8e5-da5ef302e63a","created":"2020-12-11T17:34:06.153Z","file_attached":"true","profile_id":"5e3d1dc4-cb58-3af5-aff1-4d943d2eaf6a","last_modified":"2024-03-03T18:42:12.142Z","read":"true","starred":false,"authored":"true","confirmed":"true","hidden":false,"citation_key":"Vidal2020a","folder_uuids":"a320750a-d11c-40da-85cb-b8658f813237","private_publication":false,"abstract":"The use of machine learning algorithms in finance, medicine, and criminal justice can deeply impact human lives. As a consequence, research into interpretable machine learning has rapidly grown in an attempt to better control and fix possible sources of mistakes and biases. Tree ensembles, in particular, offer a good prediction quality in various domains, but the concurrent use of multiple trees reduces the interpretability of the ensemble. Against this background, we study born-again tree ensembles, i.e., the process of constructing a single decision tree of minimum size that reproduces the exact same behavior as a given tree ensemble in its entire feature space. To find such a tree, we develop a dynamic-programming based algorithm that exploits sophisticated pruning and bounding rules to reduce the number of recursive calls. This algorithm generates optimal born-again trees for many datasets of practical interest, leading to classifiers which are typically simpler and more interpretable without any other form of compromise.","bibtype":"inproceedings","author":"Vidal, T. and Schiffer, M.","editor":"III, Hal Daumé and Singh, Aarti","booktitle":"ICML'20","bibtex":"@inproceedings{\n title = {Born-again tree ensembles},\n type = {inproceedings},\n year = {2020},\n pages = {9743-9753},\n volume = {119},\n websites = {http://proceedings.mlr.press/v119/vidal20a.html},\n publisher = {PMLR},\n city = {Virtual},\n series = {Proceedings of Machine Learning Research},\n id = {c20200b9-20b6-3024-b8e5-da5ef302e63a},\n created = {2020-12-11T17:34:06.153Z},\n file_attached = {true},\n profile_id = {5e3d1dc4-cb58-3af5-aff1-4d943d2eaf6a},\n last_modified = {2024-03-03T18:42:12.142Z},\n read = {true},\n starred = {false},\n authored = {true},\n confirmed = {true},\n hidden = {false},\n citation_key = {Vidal2020a},\n folder_uuids = {a320750a-d11c-40da-85cb-b8658f813237},\n private_publication = {false},\n abstract = {The use of machine learning algorithms in finance, medicine, and criminal justice can deeply impact human lives. As a consequence, research into interpretable machine learning has rapidly grown in an attempt to better control and fix possible sources of mistakes and biases. Tree ensembles, in particular, offer a good prediction quality in various domains, but the concurrent use of multiple trees reduces the interpretability of the ensemble. Against this background, we study born-again tree ensembles, i.e., the process of constructing a single decision tree of minimum size that reproduces the exact same behavior as a given tree ensemble in its entire feature space. To find such a tree, we develop a dynamic-programming based algorithm that exploits sophisticated pruning and bounding rules to reduce the number of recursive calls. This algorithm generates optimal born-again trees for many datasets of practical interest, leading to classifiers which are typically simpler and more interpretable without any other form of compromise.},\n bibtype = {inproceedings},\n author = {Vidal, T. and Schiffer, M.},\n editor = {III, Hal Daumé and Singh, Aarti},\n booktitle = {ICML'20}\n}","author_short":["Vidal, T.","Schiffer, M."],"editor_short":["III, H., D.","Singh, A."],"urls":{"Paper":"https://bibbase.org/service/mendeley/1465671/file/4d1e6c64-6cf6-fab1-57cf-0edc8620c476/Vidal_Schiffer___2020___Born_Again_Tree_Ensembles.pdf.pdf","Website":"http://proceedings.mlr.press/v119/vidal20a.html"},"biburl":"https://bibbase.org/service/mendeley/1465671","bibbaseid":"vidal-schiffer-bornagaintreeensembles-2020","role":"author","metadata":{"authorlinks":{"vidal, t":"https://bibbase.org/show?bib=https%3A%2F%2Fw1.cirrelt.ca%2F~vidalt%2Fresources%2FMy%2520Collection.bib"}},"downloads":2},"bibtype":"inproceedings","biburl":"https://bibbase.org/service/mendeley/1465671","creationDate":"2021-02-18T20:21:05.663Z","downloads":2,"keywords":[],"search_terms":["born","again","tree","ensembles","vidal","schiffer"],"title":"Born-again tree ensembles","year":2020,"dataSources":["yinfondEAJRbDM9sJ","sempRA6PhmAdGk3yG","2252seNhipfTmjEBQ","Cfgnp5s4HQSBd8tAf"]}