A Multi-Method Study of Internet of Things Systems Testing in Industry

A Multi-Method Study of Internet of Things Systems Testing in Industry. Minani, J. B., Sabir, F., Moha, N., & Gu�h�neuc, Y. Internet of Things Journal (IoTJ), 11(1):1662–1684, IEEE CS Press, January, 2023. 23 pages.

Paper abstract bibtex

As the Internet of Things (IoT) grows, its failures may have dramatic consequences on the lives of people who depend on it. Yet, it is hard to test IoT systems before they are deployed. Several researchers have provided state-of-the-art approaches for testing IoT systems. However, many of those approaches are based on academia rather than industry. Therefore, we conducted a multi-method study of IoT systems testing in the industry with IoT practitioners. We used three methods: 1 an industry survey, 2 practitioners interviews, and 3 analysis of Eclipse IoT surveys. This study focuses on testing IoT systems by industry practitioners. The findings show that 1 testing focuses more on the device, network, and application layers. IoT testing gives more importance to integration testing than acceptance testing. Test coverage is the most important metric, but metrics may vary depending on the project. 2 IoT system testing mainly uses the model-based approach and is often manual or semi-automated, with low adoption of white box testing. Node-RED is commonly used in testing IoT systems, while Amazon AWS IoT is popular for cloud platform testing of IoT devices. 3 Log analysis is the main approach to analyzing the root cause of bugs. 4 The main challenges in IoT testing include the lack of standards, security, connectivity, and reference architecture. Generating test cases and establishing a standard test approach are recommended for further research. This studyfs findings can help IoT practitioners and researchers to identify and tackle challenges in IoT system testing, leading to future research opportunities.

@ARTICLE{Minani23-IoTJ-TestingIndustry,
   AUTHOR       = {Jean Baptiste Minani and Fatima Sabir and Naouel Moha and 
      Yann-Ga�l Gu�h�neuc},
   JOURNAL      = {Internet of Things Journal (IoTJ)},
   TITLE        = {A Multi-Method Study of Internet of Things Systems 
      Testing in Industry},
   YEAR         = {2023},
   MONTH        = {January},
   NOTE         = {23 pages.},
   NUMBER       = {1},
   PAGES        = {1662--1684},
   VOLUME       = {11},
   EDITOR       = {Nei Kato},
   KEYWORDS     = {Topic: <b>ESE for the IoT</b>, Venue: <b>IoTJ</b>},
   PUBLISHER    = {IEEE CS Press},
   URL          = {http://www.ptidej.net/publications/documents/IoTJ23.doc.pdf},
   ABSTRACT     = {As the Internet of Things (IoT) grows, its failures may 
      have dramatic consequences on the lives of people who depend on it. 
      Yet, it is hard to test IoT systems before they are deployed. Several 
      researchers have provided state-of-the-art approaches for testing IoT 
      systems. However, many of those approaches are based on academia 
      rather than industry. Therefore, we conducted a multi-method study of 
      IoT systems testing in the industry with IoT practitioners. We used 
      three methods: 1 an industry survey, 2 practitioners interviews, and 
      3 analysis of Eclipse IoT surveys. This study focuses on testing IoT 
      systems by industry practitioners. The findings show that 1 testing 
      focuses more on the device, network, and application layers. IoT 
      testing gives more importance to integration testing than acceptance 
      testing. Test coverage is the most important metric, but metrics may 
      vary depending on the project. 2 IoT system testing mainly uses the 
      model-based approach and is often manual or semi-automated, with low 
      adoption of white box testing. Node-RED is commonly used in testing 
      IoT systems, while Amazon AWS IoT is popular for cloud platform 
      testing of IoT devices. 3 Log analysis is the main approach to 
      analyzing the root cause of bugs. 4 The main challenges in IoT 
      testing include the lack of standards, security, connectivity, and 
      reference architecture. Generating test cases and establishing a 
      standard test approach are recommended for further research. This 
      studyfs findings can help IoT practitioners and researchers to 
      identify and tackle challenges in IoT system testing, leading to 
      future research opportunities.}
}

Downloads: 0

{"_id":"zBisM6e2JeZwHZpg4","bibbaseid":"minani-sabir-moha-guhneuc-amultimethodstudyofinternetofthingssystemstestinginindustry-2023","author_short":["Minani, J. B.","Sabir, F.","Moha, N.","Gu�h�neuc, Y."],"bibdata":{"bibtype":"article","type":"article","author":[{"firstnames":["Jean","Baptiste"],"propositions":[],"lastnames":["Minani"],"suffixes":[]},{"firstnames":["Fatima"],"propositions":[],"lastnames":["Sabir"],"suffixes":[]},{"firstnames":["Naouel"],"propositions":[],"lastnames":["Moha"],"suffixes":[]},{"firstnames":["Yann-Ga�l"],"propositions":[],"lastnames":["Gu�h�neuc"],"suffixes":[]}],"journal":"Internet of Things Journal (IoTJ)","title":"A Multi-Method Study of Internet of Things Systems Testing in Industry","year":"2023","month":"January","note":"23 pages.","number":"1","pages":"1662–1684","volume":"11","editor":[{"firstnames":["Nei"],"propositions":[],"lastnames":["Kato"],"suffixes":[]}],"keywords":"Topic: ESE for the IoT, Venue: IoTJ","publisher":"IEEE CS Press","url":"http://www.ptidej.net/publications/documents/IoTJ23.doc.pdf","abstract":"As the Internet of Things (IoT) grows, its failures may have dramatic consequences on the lives of people who depend on it. Yet, it is hard to test IoT systems before they are deployed. Several researchers have provided state-of-the-art approaches for testing IoT systems. However, many of those approaches are based on academia rather than industry. Therefore, we conducted a multi-method study of IoT systems testing in the industry with IoT practitioners. We used three methods: 1 an industry survey, 2 practitioners interviews, and 3 analysis of Eclipse IoT surveys. This study focuses on testing IoT systems by industry practitioners. The findings show that 1 testing focuses more on the device, network, and application layers. IoT testing gives more importance to integration testing than acceptance testing. Test coverage is the most important metric, but metrics may vary depending on the project. 2 IoT system testing mainly uses the model-based approach and is often manual or semi-automated, with low adoption of white box testing. Node-RED is commonly used in testing IoT systems, while Amazon AWS IoT is popular for cloud platform testing of IoT devices. 3 Log analysis is the main approach to analyzing the root cause of bugs. 4 The main challenges in IoT testing include the lack of standards, security, connectivity, and reference architecture. Generating test cases and establishing a standard test approach are recommended for further research. This studyfs findings can help IoT practitioners and researchers to identify and tackle challenges in IoT system testing, leading to future research opportunities.","bibtex":"@ARTICLE{Minani23-IoTJ-TestingIndustry,\r\n AUTHOR = {Jean Baptiste Minani and Fatima Sabir and Naouel Moha and \r\n Yann-Ga�l Gu�h�neuc},\r\n JOURNAL = {Internet of Things Journal (IoTJ)},\r\n TITLE = {A Multi-Method Study of Internet of Things Systems \r\n Testing in Industry},\r\n YEAR = {2023},\r\n MONTH = {January},\r\n NOTE = {23 pages.},\r\n NUMBER = {1},\r\n PAGES = {1662--1684},\r\n VOLUME = {11},\r\n EDITOR = {Nei Kato},\r\n KEYWORDS = {Topic: ESE for the IoT, Venue: IoTJ},\r\n PUBLISHER = {IEEE CS Press},\r\n URL = {http://www.ptidej.net/publications/documents/IoTJ23.doc.pdf},\r\n ABSTRACT = {As the Internet of Things (IoT) grows, its failures may \r\n have dramatic consequences on the lives of people who depend on it. \r\n Yet, it is hard to test IoT systems before they are deployed. Several \r\n researchers have provided state-of-the-art approaches for testing IoT \r\n systems. However, many of those approaches are based on academia \r\n rather than industry. Therefore, we conducted a multi-method study of \r\n IoT systems testing in the industry with IoT practitioners. We used \r\n three methods: 1 an industry survey, 2 practitioners interviews, and \r\n 3 analysis of Eclipse IoT surveys. This study focuses on testing IoT \r\n systems by industry practitioners. The findings show that 1 testing \r\n focuses more on the device, network, and application layers. IoT \r\n testing gives more importance to integration testing than acceptance \r\n testing. Test coverage is the most important metric, but metrics may \r\n vary depending on the project. 2 IoT system testing mainly uses the \r\n model-based approach and is often manual or semi-automated, with low \r\n adoption of white box testing. Node-RED is commonly used in testing \r\n IoT systems, while Amazon AWS IoT is popular for cloud platform \r\n testing of IoT devices. 3 Log analysis is the main approach to \r\n analyzing the root cause of bugs. 4 The main challenges in IoT \r\n testing include the lack of standards, security, connectivity, and \r\n reference architecture. Generating test cases and establishing a \r\n standard test approach are recommended for further research. This \r\n studyfs findings can help IoT practitioners and researchers to \r\n identify and tackle challenges in IoT system testing, leading to \r\n future research opportunities.}\r\n}\r\n\r\n","author_short":["Minani, J. B.","Sabir, F.","Moha, N.","Gu�h�neuc, Y."],"editor_short":["Kato, N."],"key":"Minani23-IoTJ-TestingIndustry","id":"Minani23-IoTJ-TestingIndustry","bibbaseid":"minani-sabir-moha-guhneuc-amultimethodstudyofinternetofthingssystemstestinginindustry-2023","role":"author","urls":{"Paper":"http://www.ptidej.net/publications/documents/IoTJ23.doc.pdf"},"keyword":["Topic: ESE for the IoT","Venue: IoTJ"],"metadata":{"authorlinks":{}}},"bibtype":"article","biburl":"http://www.yann-gael.gueheneuc.net/Work/Publications/Biblio/complete-bibliography.bib","dataSources":["8vn5MSGYWB4fAx9Z4"],"keywords":["topic: ese for the iot","venue: iotj"],"search_terms":["multi","method","study","internet","things","systems","testing","industry","minani","sabir","moha","gu�h�neuc"],"title":"A Multi-Method Study of Internet of Things Systems Testing in Industry","year":2023}