Evaluating Advanced Search Interfaces using Established Information-Seeking Models

Evaluating Advanced Search Interfaces using Established Information-Seeking Models. Wilson, M., schraefel, & White, R. In the Journal of the American Society for Information Science and Technology, 60(7):1407--1422, July, 2009.

Paper abstract bibtex

When users have poorly defined or complex goals search interfaces offering only keyword searching facilities provide inadequate support to help them reach their information-seeking objectives. The emergence of interfaces with more advanced capabilities such as faceted browsing and result clustering can go some way to some way toward addressing such problems. The evaluation of these interfaces, however, is challenging since they generally offer diverse and versatile search environments that introduce overwhelming amounts of independent variables to user studies; choosing the interface object as the only independent variable in a study would reveal very little about why one design out-performs another. Nonetheless if we could effectively compare these interfaces we would have a way to determine which was best for a given scenario and begin to learn why. In this article we present a formative framework for the evaluation of advanced search interfaces through the quantification of the strengths and weaknesses of the interfaces in supporting user tactics and varying user conditions. This framework combines established models of users, user needs, and user behaviours to achieve this. The framework is applied to evaluate three search interfaces and demonstrates the potential value of this approach to interactive IR evaluation.

@article{ ecs14301,
  author    = {Max Wilson and  schraefel and Ryen White},
  title     = {Evaluating Advanced Search Interfaces using Established Information-Seeking Models},
  journal   = {In the Journal of the American Society for Information Science and Technology}, 
  abstract   = {When users have poorly defined or complex goals search interfaces offering only keyword searching facilities provide inadequate support to help them reach their information-seeking objectives. The emergence of interfaces with more advanced capabilities such as faceted browsing and result clustering can go some way to some way toward addressing such problems. The evaluation of these interfaces, however, is challenging since they generally offer diverse and versatile search environments that introduce overwhelming amounts of independent variables to user studies; choosing the interface object as the only independent variable in a study would reveal very little about why one design out-performs another. Nonetheless if we could effectively compare these interfaces we would have a way to determine which was best for a given scenario and begin to learn why. In this article we present a formative framework for the evaluation of advanced search interfaces through the quantification of the strengths and weaknesses of the interfaces in supporting user tactics and varying user conditions. This framework combines established models of users, user needs, and user behaviours to achieve this. The framework is applied to evaluate three search interfaces and demonstrates the potential value of this approach to interactive IR evaluation.},
  month   = {July},
  pages   = {1407--1422},
  volume   = {60},
  number   = {7},
  url   = {http://eprints.ecs.soton.ac.uk/14301/} ,
  year   = {2009}
}

Downloads: 0

{"_id":{"_str":"53423ffc0e946d920a0002e9"},"__v":43,"authorIDs":["5456f0fe8b01c8193000008e","545722342abc8e9f370000c7","545b4c8bb43425b77200138a"],"author_short":["Wilson, M.","schraefel","White, R."],"bibbaseid":"wilson-schraefel-white-evaluatingadvancedsearchinterfacesusingestablishedinformationseekingmodels-2009","bibdata":{"bibtype":"article","type":"article","author":[{"firstnames":["Max"],"propositions":[],"lastnames":["Wilson"],"suffixes":[]},{"firstnames":[],"propositions":[],"lastnames":["schraefel"],"suffixes":[]},{"firstnames":["Ryen"],"propositions":[],"lastnames":["White"],"suffixes":[]}],"title":"Evaluating Advanced Search Interfaces using Established Information-Seeking Models","journal":"In the Journal of the American Society for Information Science and Technology","abstract":"When users have poorly defined or complex goals search interfaces offering only keyword searching facilities provide inadequate support to help them reach their information-seeking objectives. The emergence of interfaces with more advanced capabilities such as faceted browsing and result clustering can go some way to some way toward addressing such problems. The evaluation of these interfaces, however, is challenging since they generally offer diverse and versatile search environments that introduce overwhelming amounts of independent variables to user studies; choosing the interface object as the only independent variable in a study would reveal very little about why one design out-performs another. Nonetheless if we could effectively compare these interfaces we would have a way to determine which was best for a given scenario and begin to learn why. In this article we present a formative framework for the evaluation of advanced search interfaces through the quantification of the strengths and weaknesses of the interfaces in supporting user tactics and varying user conditions. This framework combines established models of users, user needs, and user behaviours to achieve this. The framework is applied to evaluate three search interfaces and demonstrates the potential value of this approach to interactive IR evaluation.","month":"July","pages":"1407--1422","volume":"60","number":"7","url":"http://eprints.ecs.soton.ac.uk/14301/","year":"2009","bibtex":"@article{ ecs14301,\n author = {Max Wilson and schraefel and Ryen White},\n title = {Evaluating Advanced Search Interfaces using Established Information-Seeking Models},\n journal = {In the Journal of the American Society for Information Science and Technology}, \n abstract = {When users have poorly defined or complex goals search interfaces offering only keyword searching facilities provide inadequate support to help them reach their information-seeking objectives. The emergence of interfaces with more advanced capabilities such as faceted browsing and result clustering can go some way to some way toward addressing such problems. The evaluation of these interfaces, however, is challenging since they generally offer diverse and versatile search environments that introduce overwhelming amounts of independent variables to user studies; choosing the interface object as the only independent variable in a study would reveal very little about why one design out-performs another. Nonetheless if we could effectively compare these interfaces we would have a way to determine which was best for a given scenario and begin to learn why. In this article we present a formative framework for the evaluation of advanced search interfaces through the quantification of the strengths and weaknesses of the interfaces in supporting user tactics and varying user conditions. This framework combines established models of users, user needs, and user behaviours to achieve this. The framework is applied to evaluate three search interfaces and demonstrates the potential value of this approach to interactive IR evaluation.},\n month = {July},\n pages = {1407--1422},\n volume = {60},\n number = {7},\n url = {http://eprints.ecs.soton.ac.uk/14301/} ,\n year = {2009}\n}\n\n\n","author_short":["Wilson, M.","schraefel","White, R."],"key":"ecs14301","id":"ecs14301","bibbaseid":"wilson-schraefel-white-evaluatingadvancedsearchinterfacesusingestablishedinformationseekingmodels-2009","role":"author","urls":{"Paper":"http://eprints.ecs.soton.ac.uk/14301/"},"downloads":0},"bibtype":"article","biburl":"http://data.bibbase.org/author/schraefel/?format=bibtex","downloads":0,"keywords":[],"search_terms":["evaluating","advanced","search","interfaces","using","established","information","seeking","models","wilson","schraefel","white"],"title":"Evaluating Advanced Search Interfaces using Established Information-Seeking Models","year":2009,"dataSources":["XkNCQfnmetiAMsZiF"]}