IoT Systems Testing: Taxonomy, Empirical Findings, and Recommendations

IoT Systems Testing: Taxonomy, Empirical Findings, and Recommendations. Minani, J. B., El Fellah, Y., Sabir, F., Moha, N., Gu�h�neuc, Y., Kuradusenge, M., & Masuda, T. Journal of Systems and Software (JSS), 226:112408, Elsevier, August, 2025. 24 pages.

Paper abstract bibtex

The Internet of Things (IoT) is reshaping our lives, increasing the need for thorough pre-deployment testing. However, traditional software testing may not address the testing requirements of IoT systems, leading to quality challenges. A specific testing taxonomy is crucial, yet no widely recognized taxonomy exists for IoT system testing. We introduced an IoT-specific testing taxonomy that categorizes aspects of IoT systems testing into seven distinct categories. We mined testing aspects from 83 primary studies in IoT systems testing and built an initial taxonomy. This taxonomy was refined and validated through two rounds of surveys involving 16 and then 204 IoT industry practitioners. We assessed its effectiveness by conducting an empirical evaluation on two separate IoT systems, each involving 12 testers. Our findings categorize seven testing aspects: (1) testing objectives, (2) testing tools and artifacts, (3) testers, (4) testing stage, (5) testing environment, (6) Object Under Test (OUT) and metrics, and (7) testing approaches. The evaluation showed that testers equipped with the taxonomy could more effectively identify diverse test cases and scenarios. Additionally, we recommend new research opportunities to enhance the testing of IoT systems.

@ARTICLE{Minani25-JSS-TaxonomyIoTTesting,
   AUTHOR       = {Jean Bapstiste Minani and El Fellah, Yahia and 
      Fatima Sabir and Naouel Moha and Yann-Ga�l Gu�h�neuc and 
      Martin Kuradusenge and Tomoaki Masuda},
   JOURNAL      = {Journal of Systems and Software (JSS)},
   TITLE        = {IoT Systems Testing: Taxonomy, Empirical Findings, and 
      Recommendations},
   YEAR         = {2025},
   MONTH        = {August},
   NOTE         = {24 pages.},
   OPTNUMBER    = {},
   PAGES        = {112408},
   VOLUME       = {226},
   EDITOR       = {Paris Avgeriou and David C. Shepherd},
   KEYWORDS     = {Topic: <b>ESE for the IoT</b>, 
      Rubrique : <b>GL empirique pour l'IdO</b>, 
      Topic: <b>Test case generation</b>, 
      Rubrique : <b>g�n�ration de cas de test</b>, Journal: <b>JSS</b>},
   PUBLISHER    = {Elsevier},
   URL          = {http://www.ptidej.net/publications/documents/JSS25a.doc.pdf},
   ABSTRACT     = {The Internet of Things (IoT) is reshaping our lives, 
      increasing the need for thorough pre-deployment testing. However, 
      traditional software testing may not address the testing requirements 
      of IoT systems, leading to quality challenges. A specific testing 
      taxonomy is crucial, yet no widely recognized taxonomy exists for IoT 
      system testing. We introduced an IoT-specific testing taxonomy that 
      categorizes aspects of IoT systems testing into seven distinct 
      categories. We mined testing aspects from 83 primary studies in IoT 
      systems testing and built an initial taxonomy. This taxonomy was 
      refined and validated through two rounds of surveys involving 16 and 
      then 204 IoT industry practitioners. We assessed its effectiveness by 
      conducting an empirical evaluation on two separate IoT systems, each 
      involving 12 testers. Our findings categorize seven testing aspects: 
      (1) testing objectives, (2) testing tools and artifacts, (3) testers, 
      (4) testing stage, (5) testing environment, (6) Object Under Test 
      (OUT) and metrics, and (7) testing approaches. The evaluation showed 
      that testers equipped with the taxonomy could more effectively 
      identify diverse test cases and scenarios. Additionally, we recommend 
      new research opportunities to enhance the testing of IoT systems.}
}

Downloads: 0

{"_id":"ZhiDBMCZ4cxYyz3ch","bibbaseid":"minani-elfellah-sabir-moha-guhneuc-kuradusenge-masuda-iotsystemstestingtaxonomyempiricalfindingsandrecommendations-2025","author_short":["Minani, J. B.","El Fellah, Y.","Sabir, F.","Moha, N.","Gu�h�neuc, Y.","Kuradusenge, M.","Masuda, T."],"bibdata":{"bibtype":"article","type":"article","author":[{"firstnames":["Jean","Bapstiste"],"propositions":[],"lastnames":["Minani"],"suffixes":[]},{"propositions":[],"lastnames":["El","Fellah"],"firstnames":["Yahia"],"suffixes":[]},{"firstnames":["Fatima"],"propositions":[],"lastnames":["Sabir"],"suffixes":[]},{"firstnames":["Naouel"],"propositions":[],"lastnames":["Moha"],"suffixes":[]},{"firstnames":["Yann-Ga�l"],"propositions":[],"lastnames":["Gu�h�neuc"],"suffixes":[]},{"firstnames":["Martin"],"propositions":[],"lastnames":["Kuradusenge"],"suffixes":[]},{"firstnames":["Tomoaki"],"propositions":[],"lastnames":["Masuda"],"suffixes":[]}],"journal":"Journal of Systems and Software (JSS)","title":"IoT Systems Testing: Taxonomy, Empirical Findings, and Recommendations","year":"2025","month":"August","note":"24 pages.","optnumber":"","pages":"112408","volume":"226","editor":[{"firstnames":["Paris"],"propositions":[],"lastnames":["Avgeriou"],"suffixes":[]},{"firstnames":["David","C."],"propositions":[],"lastnames":["Shepherd"],"suffixes":[]}],"keywords":"Topic: ESE for the IoT, Rubrique : GL empirique pour l'IdO, Topic: Test case generation, Rubrique : g�n�ration de cas de test, Journal: JSS","publisher":"Elsevier","url":"http://www.ptidej.net/publications/documents/JSS25a.doc.pdf","abstract":"The Internet of Things (IoT) is reshaping our lives, increasing the need for thorough pre-deployment testing. However, traditional software testing may not address the testing requirements of IoT systems, leading to quality challenges. A specific testing taxonomy is crucial, yet no widely recognized taxonomy exists for IoT system testing. We introduced an IoT-specific testing taxonomy that categorizes aspects of IoT systems testing into seven distinct categories. We mined testing aspects from 83 primary studies in IoT systems testing and built an initial taxonomy. This taxonomy was refined and validated through two rounds of surveys involving 16 and then 204 IoT industry practitioners. We assessed its effectiveness by conducting an empirical evaluation on two separate IoT systems, each involving 12 testers. Our findings categorize seven testing aspects: (1) testing objectives, (2) testing tools and artifacts, (3) testers, (4) testing stage, (5) testing environment, (6) Object Under Test (OUT) and metrics, and (7) testing approaches. The evaluation showed that testers equipped with the taxonomy could more effectively identify diverse test cases and scenarios. Additionally, we recommend new research opportunities to enhance the testing of IoT systems.","bibtex":"@ARTICLE{Minani25-JSS-TaxonomyIoTTesting,\r\n AUTHOR = {Jean Bapstiste Minani and El Fellah, Yahia and \r\n Fatima Sabir and Naouel Moha and Yann-Ga�l Gu�h�neuc and \r\n Martin Kuradusenge and Tomoaki Masuda},\r\n JOURNAL = {Journal of Systems and Software (JSS)},\r\n TITLE = {IoT Systems Testing: Taxonomy, Empirical Findings, and \r\n Recommendations},\r\n YEAR = {2025},\r\n MONTH = {August},\r\n NOTE = {24 pages.},\r\n OPTNUMBER = {},\r\n PAGES = {112408},\r\n VOLUME = {226},\r\n EDITOR = {Paris Avgeriou and David C. Shepherd},\r\n KEYWORDS = {Topic: ESE for the IoT, \r\n Rubrique : GL empirique pour l'IdO, \r\n Topic: Test case generation, \r\n Rubrique : g�n�ration de cas de test, Journal: JSS},\r\n PUBLISHER = {Elsevier},\r\n URL = {http://www.ptidej.net/publications/documents/JSS25a.doc.pdf},\r\n ABSTRACT = {The Internet of Things (IoT) is reshaping our lives, \r\n increasing the need for thorough pre-deployment testing. However, \r\n traditional software testing may not address the testing requirements \r\n of IoT systems, leading to quality challenges. A specific testing \r\n taxonomy is crucial, yet no widely recognized taxonomy exists for IoT \r\n system testing. We introduced an IoT-specific testing taxonomy that \r\n categorizes aspects of IoT systems testing into seven distinct \r\n categories. We mined testing aspects from 83 primary studies in IoT \r\n systems testing and built an initial taxonomy. This taxonomy was \r\n refined and validated through two rounds of surveys involving 16 and \r\n then 204 IoT industry practitioners. We assessed its effectiveness by \r\n conducting an empirical evaluation on two separate IoT systems, each \r\n involving 12 testers. Our findings categorize seven testing aspects: \r\n (1) testing objectives, (2) testing tools and artifacts, (3) testers, \r\n (4) testing stage, (5) testing environment, (6) Object Under Test \r\n (OUT) and metrics, and (7) testing approaches. The evaluation showed \r\n that testers equipped with the taxonomy could more effectively \r\n identify diverse test cases and scenarios. Additionally, we recommend \r\n new research opportunities to enhance the testing of IoT systems.}\r\n}\r\n\r\n","author_short":["Minani, J. B.","El Fellah, Y.","Sabir, F.","Moha, N.","Gu�h�neuc, Y.","Kuradusenge, M.","Masuda, T."],"editor_short":["Avgeriou, P.","Shepherd, D. C."],"key":"Minani25-JSS-TaxonomyIoTTesting","id":"Minani25-JSS-TaxonomyIoTTesting","bibbaseid":"minani-elfellah-sabir-moha-guhneuc-kuradusenge-masuda-iotsystemstestingtaxonomyempiricalfindingsandrecommendations-2025","role":"author","urls":{"Paper":"http://www.ptidej.net/publications/documents/JSS25a.doc.pdf"},"keyword":["Topic: ESE for the IoT","Rubrique : GL empirique pour l'IdO","Topic: Test case generation","Rubrique : g�n�ration de cas de test","Journal: JSS"],"metadata":{"authorlinks":{}}},"bibtype":"article","biburl":"http://www.yann-gael.gueheneuc.net/Work/Publications/Biblio/complete-bibliography.bib","dataSources":["8vn5MSGYWB4fAx9Z4"],"keywords":["topic: ese for the iot","rubrique : gl empirique pour l'ido","topic: test case generation","rubrique : g�n�ration de cas de test","journal: jss"],"search_terms":["iot","systems","testing","taxonomy","empirical","findings","recommendations","minani","el fellah","sabir","moha","gu�h�neuc","kuradusenge","masuda"],"title":"IoT Systems Testing: Taxonomy, Empirical Findings, and Recommendations","year":2025}