Error Leakage and Wasted Time: Sensitivity and Effort Analysis of a Requirements Consistency Checking Process

Error Leakage and Wasted Time: Sensitivity and Effort Analysis of a Requirements Consistency Checking Process. Li, W., Hayes, J. H., Antoniol, G., Gu�h�neuc, Y., & Adams, B. Journal of Software: Evolution and Process (JSEP), 28(12):1061–1080, Wiley, October, 2016. 20 pages.

Paper abstract bibtex

Several techniques are used by requirements engineering practitioners to address difficult problems such as specifying precise requirements while using inherently ambiguous natural language text and ensuring the consistency of requirements. Often, these problems are addressed by building processes/tools that combine multiple techniques where the output from 1 technique becomes the input to the next. While powerful, these techniques are not without problems. Inherent errors in each technique may leak into the subsequent step of the process. We model and study 1 such process, for checking the consistency of temporal requirements, and assess error leakage and wasted time. We perform an analysis of the input factors of our model to determine the effect that sources of uncertainty may have on the final accuracy of the consistency checking process. Convinced that error leakage exists and negatively impacts the results of the overall consistency checking process, we perform a second simulation to assess its impact on the analysts' efforts to check requirements consistency. We show that analyst's effort varies depending on the precision and recall of the subprocesses and that the number and capability of analysts affect their effort. We share insights gained and discuss applicability to other processes built of piped techniques.

@ARTICLE{Li16-JSEP-ErrorLeakage,
   AUTHOR       = {Wenbin Li and Jane Huffman Hayes and Giulio Antoniol and 
      Yann-Ga�l Gu�h�neuc and Bram Adams},
   JOURNAL      = {Journal of Software: Evolution and Process (JSEP)},
   TITLE        = {Error Leakage and Wasted Time: Sensitivity and Effort 
      Analysis of a Requirements Consistency Checking Process},
   YEAR         = {2016},
   MONTH        = {October},
   NOTE         = {20 pages.},
   NUMBER       = {12},
   PAGES        = {1061--1080},
   VOLUME       = {28},
   EDITOR       = {Marouane Kessentini and Mel � Cinn�ide},
   KEYWORDS     = {Topic: <b>Requirements and features</b>, 
      Venue: <b>JSME</b>, Venue: <b>JSEP</b>},
   PUBLISHER    = {Wiley},
   URL          = {http://www.ptidej.net/publications/documents/JSEP16.doc.pdf},
   ABSTRACT     = {Several techniques are used by requirements engineering 
      practitioners to address difficult problems such as specifying 
      precise requirements while using inherently ambiguous natural 
      language text and ensuring the consistency of requirements. Often, 
      these problems are addressed by building processes/tools that combine 
      multiple techniques where the output from 1 technique becomes the 
      input to the next. While powerful, these techniques are not without 
      problems. Inherent errors in each technique may leak into the 
      subsequent step of the process. We model and study 1 such process, 
      for checking the consistency of temporal requirements, and assess 
      error leakage and wasted time. We perform an analysis of the input 
      factors of our model to determine the effect that sources of 
      uncertainty may have on the final accuracy of the consistency 
      checking process. Convinced that error leakage exists and negatively 
      impacts the results of the overall consistency checking process, we 
      perform a second simulation to assess its impact on the analysts' 
      efforts to check requirements consistency. We show that analyst's 
      effort varies depending on the precision and recall of the 
      subprocesses and that the number and capability of analysts affect 
      their effort. We share insights gained and discuss applicability to 
      other processes built of piped techniques.}
}

Downloads: 0

{"_id":"87zbfk9ixcfTj4e8i","bibbaseid":"li-hayes-antoniol-guhneuc-adams-errorleakageandwastedtimesensitivityandeffortanalysisofarequirementsconsistencycheckingprocess-2016","downloads":0,"creationDate":"2018-01-17T20:29:42.181Z","title":"Error Leakage and Wasted Time: Sensitivity and Effort Analysis of a Requirements Consistency Checking Process","author_short":["Li, W.","Hayes, J. H.","Antoniol, G.","Gu�h�neuc, Y.","Adams, B."],"year":2016,"bibtype":"article","biburl":"http://www.yann-gael.gueheneuc.net/Work/Publications/Biblio/complete-bibliography.bib","bibdata":{"bibtype":"article","type":"article","author":[{"firstnames":["Wenbin"],"propositions":[],"lastnames":["Li"],"suffixes":[]},{"firstnames":["Jane","Huffman"],"propositions":[],"lastnames":["Hayes"],"suffixes":[]},{"firstnames":["Giulio"],"propositions":[],"lastnames":["Antoniol"],"suffixes":[]},{"firstnames":["Yann-Ga�l"],"propositions":[],"lastnames":["Gu�h�neuc"],"suffixes":[]},{"firstnames":["Bram"],"propositions":[],"lastnames":["Adams"],"suffixes":[]}],"journal":"Journal of Software: Evolution and Process (JSEP)","title":"Error Leakage and Wasted Time: Sensitivity and Effort Analysis of a Requirements Consistency Checking Process","year":"2016","month":"October","note":"20 pages.","number":"12","pages":"1061–1080","volume":"28","editor":[{"firstnames":["Marouane"],"propositions":[],"lastnames":["Kessentini"],"suffixes":[]},{"firstnames":["Mel","�"],"propositions":[],"lastnames":["Cinn�ide"],"suffixes":[]}],"keywords":"Topic: Requirements and features, Venue: JSME, Venue: JSEP","publisher":"Wiley","url":"http://www.ptidej.net/publications/documents/JSEP16.doc.pdf","abstract":"Several techniques are used by requirements engineering practitioners to address difficult problems such as specifying precise requirements while using inherently ambiguous natural language text and ensuring the consistency of requirements. Often, these problems are addressed by building processes/tools that combine multiple techniques where the output from 1 technique becomes the input to the next. While powerful, these techniques are not without problems. Inherent errors in each technique may leak into the subsequent step of the process. We model and study 1 such process, for checking the consistency of temporal requirements, and assess error leakage and wasted time. We perform an analysis of the input factors of our model to determine the effect that sources of uncertainty may have on the final accuracy of the consistency checking process. Convinced that error leakage exists and negatively impacts the results of the overall consistency checking process, we perform a second simulation to assess its impact on the analysts' efforts to check requirements consistency. We show that analyst's effort varies depending on the precision and recall of the subprocesses and that the number and capability of analysts affect their effort. We share insights gained and discuss applicability to other processes built of piped techniques.","bibtex":"@ARTICLE{Li16-JSEP-ErrorLeakage,\r\n AUTHOR = {Wenbin Li and Jane Huffman Hayes and Giulio Antoniol and \r\n Yann-Ga�l Gu�h�neuc and Bram Adams},\r\n JOURNAL = {Journal of Software: Evolution and Process (JSEP)},\r\n TITLE = {Error Leakage and Wasted Time: Sensitivity and Effort \r\n Analysis of a Requirements Consistency Checking Process},\r\n YEAR = {2016},\r\n MONTH = {October},\r\n NOTE = {20 pages.},\r\n NUMBER = {12},\r\n PAGES = {1061--1080},\r\n VOLUME = {28},\r\n EDITOR = {Marouane Kessentini and Mel � Cinn�ide},\r\n KEYWORDS = {Topic: Requirements and features, \r\n Venue: JSME, Venue: JSEP},\r\n PUBLISHER = {Wiley},\r\n URL = {http://www.ptidej.net/publications/documents/JSEP16.doc.pdf},\r\n ABSTRACT = {Several techniques are used by requirements engineering \r\n practitioners to address difficult problems such as specifying \r\n precise requirements while using inherently ambiguous natural \r\n language text and ensuring the consistency of requirements. Often, \r\n these problems are addressed by building processes/tools that combine \r\n multiple techniques where the output from 1 technique becomes the \r\n input to the next. While powerful, these techniques are not without \r\n problems. Inherent errors in each technique may leak into the \r\n subsequent step of the process. We model and study 1 such process, \r\n for checking the consistency of temporal requirements, and assess \r\n error leakage and wasted time. We perform an analysis of the input \r\n factors of our model to determine the effect that sources of \r\n uncertainty may have on the final accuracy of the consistency \r\n checking process. Convinced that error leakage exists and negatively \r\n impacts the results of the overall consistency checking process, we \r\n perform a second simulation to assess its impact on the analysts' \r\n efforts to check requirements consistency. We show that analyst's \r\n effort varies depending on the precision and recall of the \r\n subprocesses and that the number and capability of analysts affect \r\n their effort. We share insights gained and discuss applicability to \r\n other processes built of piped techniques.}\r\n}\r\n\r\n","author_short":["Li, W.","Hayes, J. H.","Antoniol, G.","Gu�h�neuc, Y.","Adams, B."],"editor_short":["Kessentini, M.","Cinn�ide, M. �"],"key":"Li16-JSEP-ErrorLeakage","id":"Li16-JSEP-ErrorLeakage","bibbaseid":"li-hayes-antoniol-guhneuc-adams-errorleakageandwastedtimesensitivityandeffortanalysisofarequirementsconsistencycheckingprocess-2016","role":"author","urls":{"Paper":"http://www.ptidej.net/publications/documents/JSEP16.doc.pdf"},"keyword":["Topic: Requirements and features","Venue: JSME","Venue: JSEP"],"metadata":{"authorlinks":{"gu�h�neuc, y":"https://bibbase.org/show?bib=http%3A%2F%2Fwww.yann-gael.gueheneuc.net%2FWork%2FPublications%2FBiblio%2Fcomplete-bibliography.bib&msg=embed","guéhéneuc, y":"http://www.yann-gael.gueheneuc.net/"}}},"search_terms":["error","leakage","wasted","time","sensitivity","effort","analysis","requirements","consistency","checking","process","li","hayes","antoniol","gu�h�neuc","adams"],"keywords":["topic: requirements and features","venue: jsme","venue: jsep"],"authorIDs":["AfJhKcg96muyPdu7S","xkviMnkrGBneANvMr"],"dataSources":["Sed98LbBeGaXxenrM","8vn5MSGYWB4fAx9Z4"]}