Design and Development of a Provenance Capture Platform for Data Science

Design and Development of a Provenance Capture Platform for Data Science. Gregori, L., Missier, P., Stidolph, M., Torlone, r., & Wood, A. In Procs. 3rd DATAPLAT workshop, co-located with ICDE 2024, Utrecht, NL, May, 2024. IEEE.

Paper abstract bibtex 3 downloads

As machine learning and AI systems become more prevalent, understanding how their decisions are made is key to maintaining their trust. To solve this problem, it is widely accepted that fundamental support can be provided by the knowledge of how data are altered in the pre-processing phase, using data provenance to track such changes. This paper focuses on the design and development of a system for collecting and managing data provenance of data preparation pipelines in data science. An investigation of publicly available machine learning pipelines is conducted to identify the most important features required for the tool to achieve impact on a broad selection of pre-processing data manipulation. This reveals that the operations that are used in practice can be implemented by combining a rather limited set of basic operators. We then illustrate and test implementation choices aimed at supporting the provenance capture for those operations efficiently and with minimal effort for data scientists.

@inproceedings{gregori_design_2024,
	address = {Utrecht, NL},
	title = {Design and {Development} of a {Provenance} {Capture} {Platform} for {Data} {Science}},
	abstract = {As machine learning and AI systems become more prevalent, understanding how their decisions are made is key to maintaining their trust. To solve this problem, it is widely accepted that fundamental support can be provided by the knowledge of how data are altered in the pre-processing phase, using data provenance to track such changes.
This paper focuses on the design and development of a system for collecting and managing data provenance of data preparation pipelines in data science. An investigation of publicly available machine learning pipelines is conducted to identify the most important features required for the tool to achieve impact on a broad selection of pre-processing data manipulation. This reveals that the operations that are used in practice can be implemented by combining a rather limited set of basic operators. We then illustrate and test implementation choices aimed at supporting the provenance capture for those operations efficiently and with minimal effort for data scientists.},
	booktitle = {Procs. 3rd {DATAPLAT} workshop, co-located with {ICDE} 2024},
	publisher = {IEEE},
	author = {Gregori, Luca and Missier, Paolo and Stidolph, Matthew and Torlone, riccardo and Wood, Alessandro},
	month = may,
	year = {2024},
	url={https://www.dropbox.com/scl/fi/plz8egd5wdvb5bp5vra09/840300a285.pdf?rlkey=gitqo6jzveh915g9fhbsqpqyn&st=8pk9vluh&dl=0}
}

Downloads: 3

{"_id":"7nnD9ZfEHKogY3k6s","bibbaseid":"gregori-missier-stidolph-torlone-wood-designanddevelopmentofaprovenancecaptureplatformfordatascience-2024","author_short":["Gregori, L.","Missier, P.","Stidolph, M.","Torlone, r.","Wood, A."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","address":"Utrecht, NL","title":"Design and Development of a Provenance Capture Platform for Data Science","abstract":"As machine learning and AI systems become more prevalent, understanding how their decisions are made is key to maintaining their trust. To solve this problem, it is widely accepted that fundamental support can be provided by the knowledge of how data are altered in the pre-processing phase, using data provenance to track such changes. This paper focuses on the design and development of a system for collecting and managing data provenance of data preparation pipelines in data science. An investigation of publicly available machine learning pipelines is conducted to identify the most important features required for the tool to achieve impact on a broad selection of pre-processing data manipulation. This reveals that the operations that are used in practice can be implemented by combining a rather limited set of basic operators. We then illustrate and test implementation choices aimed at supporting the provenance capture for those operations efficiently and with minimal effort for data scientists.","booktitle":"Procs. 3rd DATAPLAT workshop, co-located with ICDE 2024","publisher":"IEEE","author":[{"propositions":[],"lastnames":["Gregori"],"firstnames":["Luca"],"suffixes":[]},{"propositions":[],"lastnames":["Missier"],"firstnames":["Paolo"],"suffixes":[]},{"propositions":[],"lastnames":["Stidolph"],"firstnames":["Matthew"],"suffixes":[]},{"propositions":[],"lastnames":["Torlone"],"firstnames":["riccardo"],"suffixes":[]},{"propositions":[],"lastnames":["Wood"],"firstnames":["Alessandro"],"suffixes":[]}],"month":"May","year":"2024","url":"https://www.dropbox.com/scl/fi/plz8egd5wdvb5bp5vra09/840300a285.pdf?rlkey=gitqo6jzveh915g9fhbsqpqyn&st=8pk9vluh&dl=0","bibtex":"@inproceedings{gregori_design_2024,\n\taddress = {Utrecht, NL},\n\ttitle = {Design and {Development} of a {Provenance} {Capture} {Platform} for {Data} {Science}},\n\tabstract = {As machine learning and AI systems become more prevalent, understanding how their decisions are made is key to maintaining their trust. To solve this problem, it is widely accepted that fundamental support can be provided by the knowledge of how data are altered in the pre-processing phase, using data provenance to track such changes.\nThis paper focuses on the design and development of a system for collecting and managing data provenance of data preparation pipelines in data science. An investigation of publicly available machine learning pipelines is conducted to identify the most important features required for the tool to achieve impact on a broad selection of pre-processing data manipulation. This reveals that the operations that are used in practice can be implemented by combining a rather limited set of basic operators. We then illustrate and test implementation choices aimed at supporting the provenance capture for those operations efficiently and with minimal effort for data scientists.},\n\tbooktitle = {Procs. 3rd {DATAPLAT} workshop, co-located with {ICDE} 2024},\n\tpublisher = {IEEE},\n\tauthor = {Gregori, Luca and Missier, Paolo and Stidolph, Matthew and Torlone, riccardo and Wood, Alessandro},\n\tmonth = may,\n\tyear = {2024},\n\turl={https://www.dropbox.com/scl/fi/plz8egd5wdvb5bp5vra09/840300a285.pdf?rlkey=gitqo6jzveh915g9fhbsqpqyn&st=8pk9vluh&dl=0}\n}\n\n","author_short":["Gregori, L.","Missier, P.","Stidolph, M.","Torlone, r.","Wood, A."],"key":"gregori_design_2024","id":"gregori_design_2024","bibbaseid":"gregori-missier-stidolph-torlone-wood-designanddevelopmentofaprovenancecaptureplatformfordatascience-2024","role":"author","urls":{"Paper":"https://www.dropbox.com/scl/fi/plz8egd5wdvb5bp5vra09/840300a285.pdf?rlkey=gitqo6jzveh915g9fhbsqpqyn&st=8pk9vluh&dl=0"},"metadata":{"authorlinks":{}},"downloads":3},"bibtype":"inproceedings","biburl":"https://bibbase.org/f/MTSG9SdhWPisKNpZX/MyPublications-bibbase.bib","dataSources":["ze2X9uz8Dcv2oGipf","afppXLgSuddAzAL9e","wJE4ynGem9MRsXBRn"],"keywords":[],"search_terms":["design","development","provenance","capture","platform","data","science","gregori","missier","stidolph","torlone","wood"],"title":"Design and Development of a Provenance Capture Platform for Data Science","year":2024,"downloads":3}