Expected ratio of relevant units: \A\ measure of structured information retrieval. Piwowarski, B. & Gallinari, P. In Fuhr, N., Lalmas, M., & Malik, S., editors, Proceedings of INEX 2003, pages 158–166, 2004.
abstract   bibtex   
Since the 60's, evaluation has been a key problem for Information Retrieval (IR) systems and has been extensively discussed in the IR community. New IR paradigms, like Structured Information Retrieval (SIR), make classical evaluation measures inappropriate. A few tentative extensions to these measures have been proposed but are also inadequate. We propose in this paper a new measure which is a generalisation of recall. This measure takes into account the specicity of SIR, when elements to be retrieved are linked by structural relationships. We show an instantiation of this measure on the INEX database and present experiments to show how well it is adapted to SIR evaluation.
@inproceedings{Piwowarski/Gallinari:04,
	title = {Expected ratio of relevant units: \{{A}\} measure of structured information retrieval},
	abstract = {Since the 60's, evaluation has been a key problem for
Information Retrieval (IR) systems and has been
extensively discussed in the IR community. New IR
paradigms, like Structured Information Retrieval (SIR),
make classical evaluation measures inappropriate. A few
tentative extensions to these measures have been
proposed but are also inadequate. We propose in this
paper a new measure which is a generalisation of
recall. This measure takes into account the specicity
of SIR, when elements to be retrieved are linked by
structural relationships. We show an instantiation of
this measure on the INEX database and present
experiments to show how well it is adapted to SIR
evaluation.},
	booktitle = {Proceedings of {INEX} 2003},
	author = {Piwowarski, Benjamin and Gallinari, Patrick},
	editor = {Fuhr, Norbert and Lalmas, Mounia and Malik, Saadia},
	year = {2004},
	pages = {158--166},
}

Downloads: 0