RATEN: An Efficient Robustness Analysis and Test Enhancement Framework for State Machines

RATEN: An Efficient Robustness Analysis and Test Enhancement Framework for State Machines. Babaei, M. & Gu�h�neuc, Y. Software and Systems Modeling (SoSym), Springer, 2026. 20 pages.

Paper abstract bibtex

Robustness analysis is part of the validation process that includes testing the behavior of a system against its specifications under unexpected runtime conditions to check whether the system meets robustness requirements. Unlike traditional approaches that focus on design-time model verification, this paper proposes a runtime robustness analysis and test enhancement framework called RATEN for systems whose behavior is specified using finite-state machines. RATEN takes as input the behavioral model and property model of the system under test and evaluates the robustness of a system based on a quantitative notion of cost computed for every transition that leads to an unexpected state during execution. In this study, we consider three common runtime robustness failures: wrong messages, wrong payload, and missing messages, which represent unexpected environmental deviations that can occur during system operation. Through different scenarios, we evaluate the efficiency and effectiveness of our approach using both simple and complex state machine models. The experimental results show that compared to traditional trace annotation approaches, RATEN detects almost all non-robust instances while incurring similar runtime overhead (1.02x to 1.28x). Furthermore, we demonstrate that the integration of RATEN into model-based testing approaches that implement Record & Replay mechanisms reduces the size of the test suite required to detect regressions by an average of 47% across different failure scenarios.

@ARTICLE{Babei26-SoSym-RATEN,
   AUTHOR       = {Majid Babaei and Yann-Ga�l Gu�h�neuc},
   JOURNAL      = {Software and Systems Modeling (SoSym)},
   TITLE        = {RATEN: An Efficient Robustness Analysis and Test 
      Enhancement Framework for State Machines},
   YEAR         = {2026},
   OPTMONTH     = {},
   NOTE         = {20 pages.},
   OPTNUMBER    = {},
   OPTPAGES     = {},
   OPTVOLUME    = {},
   EDITOR       = {Marsha Chechik},
   KEYWORDS     = {Topic: <b>Evolution patterns</b>, 
      Rubrique : <b>patrons d'�volution</b>, 
      Topic: <b>Requirements and features</b>, 
      Rubrique : <b>besoins et fonctionalit�s</b>, Journal: <b>SoSym</b>},
   PUBLISHER    = {Springer},
   URL          = {http://www.ptidej.net/publications/documents/SoSym26.doc.pdf},
   ABSTRACT     = {Robustness analysis is part of the validation process 
      that includes testing the behavior of a system against its 
      specifications under unexpected runtime conditions to check whether 
      the system meets robustness requirements. Unlike traditional 
      approaches that focus on design-time model verification, this paper 
      proposes a runtime robustness analysis and test enhancement framework 
      called RATEN for systems whose behavior is specified using 
      finite-state machines. RATEN takes as input the behavioral model and 
      property model of the system under test and evaluates the robustness 
      of a system based on a quantitative notion of cost computed for every 
      transition that leads to an unexpected state during execution. In 
      this study, we consider three common runtime robustness failures: 
      wrong messages, wrong payload, and missing messages, which represent 
      unexpected environmental deviations that can occur during system 
      operation. Through different scenarios, we evaluate the efficiency 
      and effectiveness of our approach using both simple and complex state 
      machine models. The experimental results show that compared to 
      traditional trace annotation approaches, RATEN detects almost all 
      non-robust instances while incurring similar runtime overhead (1.02x 
      to 1.28x). Furthermore, we demonstrate that the integration of RATEN 
      into model-based testing approaches that implement Record &amp; 
      Replay mechanisms reduces the size of the test suite required to 
      detect regressions by an average of 47% across different failure 
      scenarios.}
}

Downloads: 0

{"_id":"yoBTFewYrFRnWe7tu","bibbaseid":"babaei-guhneuc-ratenanefficientrobustnessanalysisandtestenhancementframeworkforstatemachines-2026","author_short":["Babaei, M.","Gu�h�neuc, Y."],"bibdata":{"bibtype":"article","type":"article","author":[{"firstnames":["Majid"],"propositions":[],"lastnames":["Babaei"],"suffixes":[]},{"firstnames":["Yann-Ga�l"],"propositions":[],"lastnames":["Gu�h�neuc"],"suffixes":[]}],"journal":"Software and Systems Modeling (SoSym)","title":"RATEN: An Efficient Robustness Analysis and Test Enhancement Framework for State Machines","year":"2026","optmonth":"","note":"20 pages.","optnumber":"","optpages":"","optvolume":"","editor":[{"firstnames":["Marsha"],"propositions":[],"lastnames":["Chechik"],"suffixes":[]}],"keywords":"Topic: Evolution patterns, Rubrique : patrons d'�volution, Topic: Requirements and features, Rubrique : besoins et fonctionalit�s, Journal: SoSym","publisher":"Springer","url":"http://www.ptidej.net/publications/documents/SoSym26.doc.pdf","abstract":"Robustness analysis is part of the validation process that includes testing the behavior of a system against its specifications under unexpected runtime conditions to check whether the system meets robustness requirements. Unlike traditional approaches that focus on design-time model verification, this paper proposes a runtime robustness analysis and test enhancement framework called RATEN for systems whose behavior is specified using finite-state machines. RATEN takes as input the behavioral model and property model of the system under test and evaluates the robustness of a system based on a quantitative notion of cost computed for every transition that leads to an unexpected state during execution. In this study, we consider three common runtime robustness failures: wrong messages, wrong payload, and missing messages, which represent unexpected environmental deviations that can occur during system operation. Through different scenarios, we evaluate the efficiency and effectiveness of our approach using both simple and complex state machine models. The experimental results show that compared to traditional trace annotation approaches, RATEN detects almost all non-robust instances while incurring similar runtime overhead (1.02x to 1.28x). Furthermore, we demonstrate that the integration of RATEN into model-based testing approaches that implement Record & Replay mechanisms reduces the size of the test suite required to detect regressions by an average of 47% across different failure scenarios.","bibtex":"@ARTICLE{Babei26-SoSym-RATEN,\r\n AUTHOR = {Majid Babaei and Yann-Ga�l Gu�h�neuc},\r\n JOURNAL = {Software and Systems Modeling (SoSym)},\r\n TITLE = {RATEN: An Efficient Robustness Analysis and Test \r\n Enhancement Framework for State Machines},\r\n YEAR = {2026},\r\n OPTMONTH = {},\r\n NOTE = {20 pages.},\r\n OPTNUMBER = {},\r\n OPTPAGES = {},\r\n OPTVOLUME = {},\r\n EDITOR = {Marsha Chechik},\r\n KEYWORDS = {Topic: Evolution patterns, \r\n Rubrique : patrons d'�volution, \r\n Topic: Requirements and features, \r\n Rubrique : besoins et fonctionalit�s, Journal: SoSym},\r\n PUBLISHER = {Springer},\r\n URL = {http://www.ptidej.net/publications/documents/SoSym26.doc.pdf},\r\n ABSTRACT = {Robustness analysis is part of the validation process \r\n that includes testing the behavior of a system against its \r\n specifications under unexpected runtime conditions to check whether \r\n the system meets robustness requirements. Unlike traditional \r\n approaches that focus on design-time model verification, this paper \r\n proposes a runtime robustness analysis and test enhancement framework \r\n called RATEN for systems whose behavior is specified using \r\n finite-state machines. RATEN takes as input the behavioral model and \r\n property model of the system under test and evaluates the robustness \r\n of a system based on a quantitative notion of cost computed for every \r\n transition that leads to an unexpected state during execution. In \r\n this study, we consider three common runtime robustness failures: \r\n wrong messages, wrong payload, and missing messages, which represent \r\n unexpected environmental deviations that can occur during system \r\n operation. Through different scenarios, we evaluate the efficiency \r\n and effectiveness of our approach using both simple and complex state \r\n machine models. The experimental results show that compared to \r\n traditional trace annotation approaches, RATEN detects almost all \r\n non-robust instances while incurring similar runtime overhead (1.02x \r\n to 1.28x). Furthermore, we demonstrate that the integration of RATEN \r\n into model-based testing approaches that implement Record & \r\n Replay mechanisms reduces the size of the test suite required to \r\n detect regressions by an average of 47% across different failure \r\n scenarios.}\r\n}\r\n\r\n","author_short":["Babaei, M.","Gu�h�neuc, Y."],"editor_short":["Chechik, M."],"key":"Babei26-SoSym-RATEN","id":"Babei26-SoSym-RATEN","bibbaseid":"babaei-guhneuc-ratenanefficientrobustnessanalysisandtestenhancementframeworkforstatemachines-2026","role":"author","urls":{"Paper":"http://www.ptidej.net/publications/documents/SoSym26.doc.pdf"},"keyword":["Topic: Evolution patterns","Rubrique : patrons d'�volution","Topic: Requirements and features","Rubrique : besoins et fonctionalit�s","Journal: SoSym"],"metadata":{"authorlinks":{}}},"bibtype":"article","biburl":"http://www.yann-gael.gueheneuc.net/Work/Publications/Biblio/complete-bibliography.bib","dataSources":["8vn5MSGYWB4fAx9Z4"],"keywords":["topic: evolution patterns","rubrique : patrons d'�volution","topic: requirements and features","rubrique : besoins et fonctionalit�s","journal: sosym"],"search_terms":["raten","efficient","robustness","analysis","test","enhancement","framework","state","machines","babaei","gu�h�neuc"],"title":"RATEN: An Efficient Robustness Analysis and Test Enhancement Framework for State Machines","year":2026}