Evaluating spoken language systems. Kamm, C.; Walker, M. A; and Litman, D. J In AVIOS 1999. Proceedings of the annual meeting of the American Voice Input/Output Society, pages 187-197, San José, CA.
Evaluating spoken language systems [link]Paper  abstract   bibtex   
Spoken language systems (SLSs) for accessing information sources or services through the telephone network and the Internet are currently being trialed and deployed for a variety of tasks. Evaluating the usability of different interface designs requires a method for comparing performance of different versions of the SLS. Recently, Walker et al (1997) proposed PARADISE (PARAdigm for DIalogue System Evaluation) as a general methodology for evaluating SLSs. The PARADISE framework models user satisfaction with an SLS as a linear combination of measures reflecting both task success and dialogue costs. As a test of this methodology, we applied PARADISE to dialogues collected with three SLSs. This paper describes the salient measures identified using PARADISE within and across the three SLSs, and discusses the generalizability of PARADISE performance models.
@inproceedings{kamm_evaluating_1999,
	Address = {San José, CA},
	Author = {Kamm, Candace and Walker, Marilyn A and Litman, Diane J},
	Booktitle = {AVIOS 1999. Proceedings of the annual meeting of the American Voice Input/Output Society},
	Date = {1999},
	Date-Modified = {2016-09-24 18:56:06 +0000},
	Keywords = {assessment, dialogue systems, speech technology},
	Pages = {187-197},
	Title = {Evaluating spoken language systems},
	Url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.13.9747&rep=rep1&type=pdf},
	Abstract = {Spoken language systems (SLSs) for accessing information sources or services through the telephone network and the Internet are currently being trialed and deployed for a variety of tasks. Evaluating the usability of different interface designs requires a method for comparing performance of different versions of the SLS. Recently, Walker et al (1997) proposed PARADISE (PARAdigm for DIalogue System Evaluation) as a general methodology for evaluating SLSs. The PARADISE framework models user satisfaction with an SLS as a linear combination of measures reflecting both task success and dialogue costs. As a test of this methodology, we applied PARADISE to dialogues collected with three SLSs. This paper describes the salient measures identified using PARADISE within and across the three SLSs, and discusses the generalizability of PARADISE performance models.},
	Bdsk-Url-1 = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.13.9747&rep=rep1&type=pdf}}
Downloads: 0