A Systematic Literature Review on the Usage of Eye-tracking in Software Engineering

A Systematic Literature Review on the Usage of Eye-tracking in Software Engineering. Sharafi, Z., Soh, Z., & Gu�h�neuc, Y. Journal of Information and Software Technology (IST), 67:79–107, Elsevier, November, 2015. 28 pages.

Paper abstract bibtex

Context. Eye-tracking is a mean to collect evidence regarding some participants' cognitive processes. Eye-trackers monitor participants' visual attention by collecting eye-movement data. These data are useful to get insights into participants' cognitive processes during reasoning tasks. Objective. The Evidence-based Software Engineering (EBSE) paradigm has been proposed in 2004 and, since then, has been used to provide detailed insights regarding different topics in software engineering research and practice. Systematic Literature Reviews (SLR) are also useful in the context of EBSE by bringing together all existing evidence of research and results about a particular topic. This SLR evaluates the current state of the art of using eye-trackers in software engineering and provides evidence on the uses and contributions of eye-trackers to empirical studies in software engineering. Method. We perform a SLR covering eye-tracking studies in software engineering published from 1990 up to the end of 2014. To search all recognised resources, instead of applying manual search, we perform an extensive automated search using Engineering Village. We identify 36 relevant publications, including nine journal papers, two workshop papers, and 25 conference papers. Results. The software engineering community started using eye-trackers in the 1990s and they have become increasingly recognised as useful tools to conduct empirical studies from 2006. We observe that researchers use eye-trackers to study model comprehension, code comprehension, debugging, collaborative interaction, and traceability. Moreover, we find that studies use different metrics based on eye-movement data to obtain quantitative measures. We also report the limitations of current eye-tracking technology, which threaten the validity of previous studies, along with suggestions to mitigate these limitations. Conclusion. However, not withstanding these limitations and threats, we conclude that the advent of new eye-trackers makes the use of these tools easier and less obtrusive and that the software engineering community could benefit more from this technology.

@ARTICLE{Sharafi15-IST-SLREyeTracking,
   AUTHOR       = {Zohreh Sharafi and Z�phyrin Soh and Yann-Ga�l Gu�h�neuc},
   JOURNAL      = {Journal of Information and Software Technology (IST)},
   TITLE        = {A Systematic Literature Review on the Usage of 
      Eye-tracking in Software Engineering},
   YEAR         = {2015},
   MONTH        = {November},
   NOTE         = {28 pages.},
   OPTNUMBER    = {},
   PAGES        = {79--107},
   VOLUME       = {67},
   EDITOR       = {Guenther Ruhe and Guilherme Horta Travassos and 
      Laurie Williams},
   KEYWORDS     = {Topic: <b>Program comprehension</b>, Venue: <b>IST</b>},
   PUBLISHER    = {Elsevier},
   URL          = {http://www.ptidej.net/publications/documents/IST15.doc.pdf},
   ABSTRACT     = { \textit{Context.} Eye-tracking is a mean to collect 
      evidence regarding some participants' cognitive processes. 
      Eye-trackers monitor participants' visual attention by collecting 
      eye-movement data. These data are useful to get insights into 
      participants' cognitive processes during reasoning tasks. 
      \textit{Objective.} The Evidence-based Software Engineering (EBSE) 
      paradigm has been proposed in 2004 and, since then, has been used to 
      provide detailed insights regarding different topics in software 
      engineering research and practice. Systematic Literature Reviews 
      (SLR) are also useful in the context of EBSE by bringing together all 
      existing evidence of research and results about a particular topic. 
      This SLR evaluates the current state of the art of using eye-trackers 
      in software engineering and provides evidence on the uses and 
      contributions of eye-trackers to empirical studies in software 
      engineering. \textit{Method.} We perform a SLR covering eye-tracking 
      studies in software engineering published from 1990 up to the end of 
      2014. To search all recognised resources, instead of applying manual 
      search, we perform an extensive automated search using Engineering 
      Village. We identify 36 relevant publications, including nine journal 
      papers, two workshop papers, and 25 conference papers. 
      \textit{Results.} The software engineering community started using 
      eye-trackers in the 1990s and they have become increasingly 
      recognised as useful tools to conduct empirical studies from 2006. We 
      observe that researchers use eye-trackers to study model 
      comprehension, code comprehension, debugging, collaborative 
      interaction, and traceability. Moreover, we find that studies use 
      different metrics based on eye-movement data to obtain quantitative 
      measures. We also report the limitations of current eye-tracking 
      technology, which threaten the validity of previous studies, along 
      with suggestions to mitigate these limitations. \textit{Conclusion.} 
      However, not withstanding these limitations and threats, we conclude 
      that the advent of new eye-trackers makes the use of these tools 
      easier and less obtrusive and that the software engineering community 
      could benefit more from this technology. }
}

Downloads: 0

{"_id":"LQb6Qa9TLiE98BSRC","bibbaseid":"sharafi-soh-guhneuc-asystematicliteraturereviewontheusageofeyetrackinginsoftwareengineering-2015","downloads":0,"creationDate":"2018-01-17T20:29:42.216Z","title":"A Systematic Literature Review on the Usage of Eye-tracking in Software Engineering","author_short":["Sharafi, Z.","Soh, Z.","Gu�h�neuc, Y."],"year":2015,"bibtype":"article","biburl":"http://www.yann-gael.gueheneuc.net/Work/Publications/Biblio/complete-bibliography.bib","bibdata":{"bibtype":"article","type":"article","author":[{"firstnames":["Zohreh"],"propositions":[],"lastnames":["Sharafi"],"suffixes":[]},{"firstnames":["Z�phyrin"],"propositions":[],"lastnames":["Soh"],"suffixes":[]},{"firstnames":["Yann-Ga�l"],"propositions":[],"lastnames":["Gu�h�neuc"],"suffixes":[]}],"journal":"Journal of Information and Software Technology (IST)","title":"A Systematic Literature Review on the Usage of Eye-tracking in Software Engineering","year":"2015","month":"November","note":"28 pages.","optnumber":"","pages":"79–107","volume":"67","editor":[{"firstnames":["Guenther"],"propositions":[],"lastnames":["Ruhe"],"suffixes":[]},{"firstnames":["Guilherme","Horta"],"propositions":[],"lastnames":["Travassos"],"suffixes":[]},{"firstnames":["Laurie"],"propositions":[],"lastnames":["Williams"],"suffixes":[]}],"keywords":"Topic: Program comprehension, Venue: IST","publisher":"Elsevier","url":"http://www.ptidej.net/publications/documents/IST15.doc.pdf","abstract":"Context. Eye-tracking is a mean to collect evidence regarding some participants' cognitive processes. Eye-trackers monitor participants' visual attention by collecting eye-movement data. These data are useful to get insights into participants' cognitive processes during reasoning tasks. Objective. The Evidence-based Software Engineering (EBSE) paradigm has been proposed in 2004 and, since then, has been used to provide detailed insights regarding different topics in software engineering research and practice. Systematic Literature Reviews (SLR) are also useful in the context of EBSE by bringing together all existing evidence of research and results about a particular topic. This SLR evaluates the current state of the art of using eye-trackers in software engineering and provides evidence on the uses and contributions of eye-trackers to empirical studies in software engineering. Method. We perform a SLR covering eye-tracking studies in software engineering published from 1990 up to the end of 2014. To search all recognised resources, instead of applying manual search, we perform an extensive automated search using Engineering Village. We identify 36 relevant publications, including nine journal papers, two workshop papers, and 25 conference papers. Results. The software engineering community started using eye-trackers in the 1990s and they have become increasingly recognised as useful tools to conduct empirical studies from 2006. We observe that researchers use eye-trackers to study model comprehension, code comprehension, debugging, collaborative interaction, and traceability. Moreover, we find that studies use different metrics based on eye-movement data to obtain quantitative measures. We also report the limitations of current eye-tracking technology, which threaten the validity of previous studies, along with suggestions to mitigate these limitations. Conclusion. However, not withstanding these limitations and threats, we conclude that the advent of new eye-trackers makes the use of these tools easier and less obtrusive and that the software engineering community could benefit more from this technology. ","bibtex":"@ARTICLE{Sharafi15-IST-SLREyeTracking,\r\n AUTHOR = {Zohreh Sharafi and Z�phyrin Soh and Yann-Ga�l Gu�h�neuc},\r\n JOURNAL = {Journal of Information and Software Technology (IST)},\r\n TITLE = {A Systematic Literature Review on the Usage of \r\n Eye-tracking in Software Engineering},\r\n YEAR = {2015},\r\n MONTH = {November},\r\n NOTE = {28 pages.},\r\n OPTNUMBER = {},\r\n PAGES = {79--107},\r\n VOLUME = {67},\r\n EDITOR = {Guenther Ruhe and Guilherme Horta Travassos and \r\n Laurie Williams},\r\n KEYWORDS = {Topic: Program comprehension, Venue: IST},\r\n PUBLISHER = {Elsevier},\r\n URL = {http://www.ptidej.net/publications/documents/IST15.doc.pdf},\r\n ABSTRACT = { \\textit{Context.} Eye-tracking is a mean to collect \r\n evidence regarding some participants' cognitive processes. \r\n Eye-trackers monitor participants' visual attention by collecting \r\n eye-movement data. These data are useful to get insights into \r\n participants' cognitive processes during reasoning tasks. \r\n \\textit{Objective.} The Evidence-based Software Engineering (EBSE) \r\n paradigm has been proposed in 2004 and, since then, has been used to \r\n provide detailed insights regarding different topics in software \r\n engineering research and practice. Systematic Literature Reviews \r\n (SLR) are also useful in the context of EBSE by bringing together all \r\n existing evidence of research and results about a particular topic. \r\n This SLR evaluates the current state of the art of using eye-trackers \r\n in software engineering and provides evidence on the uses and \r\n contributions of eye-trackers to empirical studies in software \r\n engineering. \\textit{Method.} We perform a SLR covering eye-tracking \r\n studies in software engineering published from 1990 up to the end of \r\n 2014. To search all recognised resources, instead of applying manual \r\n search, we perform an extensive automated search using Engineering \r\n Village. We identify 36 relevant publications, including nine journal \r\n papers, two workshop papers, and 25 conference papers. \r\n \\textit{Results.} The software engineering community started using \r\n eye-trackers in the 1990s and they have become increasingly \r\n recognised as useful tools to conduct empirical studies from 2006. We \r\n observe that researchers use eye-trackers to study model \r\n comprehension, code comprehension, debugging, collaborative \r\n interaction, and traceability. Moreover, we find that studies use \r\n different metrics based on eye-movement data to obtain quantitative \r\n measures. We also report the limitations of current eye-tracking \r\n technology, which threaten the validity of previous studies, along \r\n with suggestions to mitigate these limitations. \\textit{Conclusion.} \r\n However, not withstanding these limitations and threats, we conclude \r\n that the advent of new eye-trackers makes the use of these tools \r\n easier and less obtrusive and that the software engineering community \r\n could benefit more from this technology. }\r\n}\r\n\r\n","author_short":["Sharafi, Z.","Soh, Z.","Gu�h�neuc, Y."],"editor_short":["Ruhe, G.","Travassos, G. H.","Williams, L."],"key":"Sharafi15-IST-SLREyeTracking","id":"Sharafi15-IST-SLREyeTracking","bibbaseid":"sharafi-soh-guhneuc-asystematicliteraturereviewontheusageofeyetrackinginsoftwareengineering-2015","role":"author","urls":{"Paper":"http://www.ptidej.net/publications/documents/IST15.doc.pdf"},"keyword":["Topic: Program comprehension","Venue: IST"],"metadata":{"authorlinks":{"gu�h�neuc, y":"https://bibbase.org/show?bib=http%3A%2F%2Fwww.yann-gael.gueheneuc.net%2FWork%2FPublications%2FBiblio%2Fcomplete-bibliography.bib&msg=embed","guéhéneuc, y":"https://bibbase.org/show?bib=http://www.yann-gael.gueheneuc.net/Work/BibBase/guehene%20(automatically%20cleaned).bib"}},"downloads":0},"search_terms":["systematic","literature","review","usage","eye","tracking","software","engineering","sharafi","soh","gu�h�neuc"],"keywords":["topic: program comprehension","venue: ist"],"authorIDs":["AfJhKcg96muyPdu7S","xkviMnkrGBneANvMr"],"dataSources":["Sed98LbBeGaXxenrM","8vn5MSGYWB4fAx9Z4"]}