International Journal of Metadata, Semantics and Ontologies, 13(1):9–19, Inderscience Publishers (IEL), 2018. Paper doi abstract bibtex
The popularity of applications based on artificial intelligence creates the need for making them able to explain their behaviour and be accountable for their decisions. This is a challenge mainly if applications are distributed, being composed of multiple autonomous agents, forming a Multi-Agent System (MAS). A key means of making these systems explainable is to track agent behaviour, that is, to record the provenance of their actions and reasoning. Although the idea of provenance has been explored in some contexts, it has been little explored in the context of MAS, leaving many open issues that must be understood and addressed. Our goal in this paper is to make a case for the importance of the data provenance to MAS, discussing what questions can be answered regarding MAS behaviour using provenance and, with a case study, demonstrating the benefits that provenance provides to answer these questions. This study involves the use of a framework, namely FProvW3C, which collects and stores the provenance of data produced by MAS. These data can be analysed to answer a wide variety of questions to understand the MAS behaviour. Our case study thus demonstrates that the use of data provenance in MAS is a potential solution to making the agent reasoning process transparent.