A Time Series Classification Pipeline for Detecting Interaction Ruptures in HRI Based on User Reactions. Wachowiak, L., Tisnikar, P., Coles, A., Canal, G., & Celiktutan, O. In Proceedings of the 26th International Conference on Multimodal Interaction, of ICMI '24, pages 657–665, New York, NY, USA, 2024. Association for Computing Machinery. ![A Time Series Classification Pipeline for Detecting Interaction Ruptures in HRI Based on User Reactions [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex To be able to react to interaction ruptures such as errors, a robot needs a way of realizing such a rupture occurred. We test whether it is possible to detect interaction ruptures from the user’s anonymized speech, posture, and facial features. We showcase how to approach this task, presenting a time series classification pipeline that works well with various machine learning models. A sliding window is applied to the data and the continuously updated predictions make it suitable for detecting ruptures in real-time. Our best model, an ensemble of MiniRocket classifiers, is the winning approach to the ICMI ERR@HRI challenge. A feature importance analysis shows that the model heavily relies on speaker diarization data that indicates who spoke when. Posture data, on the other hand, impedes performance. Our code is available online1.
@inproceedings{10.1145/3678957.3688386,
author = {Wachowiak, Lennart and Tisnikar, Peter and Coles, Andrew and Canal, Gerard and Celiktutan, Oya},
title = {A Time Series Classification Pipeline for Detecting Interaction Ruptures in HRI Based on User Reactions},
year = {2024},
isbn = {9798400704628},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3678957.3688386},
doi = {10.1145/3678957.3688386},
abstract = {To be able to react to interaction ruptures such as errors, a robot needs a way of realizing such a rupture occurred. We test whether it is possible to detect interaction ruptures from the user’s anonymized speech, posture, and facial features. We showcase how to approach this task, presenting a time series classification pipeline that works well with various machine learning models. A sliding window is applied to the data and the continuously updated predictions make it suitable for detecting ruptures in real-time. Our best model, an ensemble of MiniRocket classifiers, is the winning approach to the ICMI ERR@HRI challenge. A feature importance analysis shows that the model heavily relies on speaker diarization data that indicates who spoke when. Posture data, on the other hand, impedes performance. Our code is available online1.},
booktitle = {Proceedings of the 26th International Conference on Multimodal Interaction},
pages = {657–665},
numpages = {9},
keywords = {HRI, machine learning, robot errors, social robotics},
location = {San Jose, Costa Rica},
series = {ICMI '24}
}
Downloads: 0
{"_id":"5v8GicEpHtpqYSDnQ","bibbaseid":"wachowiak-tisnikar-coles-canal-celiktutan-atimeseriesclassificationpipelinefordetectinginteractionrupturesinhribasedonuserreactions-2024","author_short":["Wachowiak, L.","Tisnikar, P.","Coles, A.","Canal, G.","Celiktutan, O."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","author":[{"propositions":[],"lastnames":["Wachowiak"],"firstnames":["Lennart"],"suffixes":[]},{"propositions":[],"lastnames":["Tisnikar"],"firstnames":["Peter"],"suffixes":[]},{"propositions":[],"lastnames":["Coles"],"firstnames":["Andrew"],"suffixes":[]},{"propositions":[],"lastnames":["Canal"],"firstnames":["Gerard"],"suffixes":[]},{"propositions":[],"lastnames":["Celiktutan"],"firstnames":["Oya"],"suffixes":[]}],"title":"A Time Series Classification Pipeline for Detecting Interaction Ruptures in HRI Based on User Reactions","year":"2024","isbn":"9798400704628","publisher":"Association for Computing Machinery","address":"New York, NY, USA","url":"https://doi.org/10.1145/3678957.3688386","doi":"10.1145/3678957.3688386","abstract":"To be able to react to interaction ruptures such as errors, a robot needs a way of realizing such a rupture occurred. We test whether it is possible to detect interaction ruptures from the user’s anonymized speech, posture, and facial features. We showcase how to approach this task, presenting a time series classification pipeline that works well with various machine learning models. A sliding window is applied to the data and the continuously updated predictions make it suitable for detecting ruptures in real-time. Our best model, an ensemble of MiniRocket classifiers, is the winning approach to the ICMI ERR@HRI challenge. A feature importance analysis shows that the model heavily relies on speaker diarization data that indicates who spoke when. Posture data, on the other hand, impedes performance. Our code is available online1.","booktitle":"Proceedings of the 26th International Conference on Multimodal Interaction","pages":"657–665","numpages":"9","keywords":"HRI, machine learning, robot errors, social robotics","location":"San Jose, Costa Rica","series":"ICMI '24","bibtex":"@inproceedings{10.1145/3678957.3688386,\nauthor = {Wachowiak, Lennart and Tisnikar, Peter and Coles, Andrew and Canal, Gerard and Celiktutan, Oya},\ntitle = {A Time Series Classification Pipeline for Detecting Interaction Ruptures in HRI Based on User Reactions},\nyear = {2024},\nisbn = {9798400704628},\npublisher = {Association for Computing Machinery},\naddress = {New York, NY, USA},\nurl = {https://doi.org/10.1145/3678957.3688386},\ndoi = {10.1145/3678957.3688386},\nabstract = {To be able to react to interaction ruptures such as errors, a robot needs a way of realizing such a rupture occurred. We test whether it is possible to detect interaction ruptures from the user’s anonymized speech, posture, and facial features. We showcase how to approach this task, presenting a time series classification pipeline that works well with various machine learning models. A sliding window is applied to the data and the continuously updated predictions make it suitable for detecting ruptures in real-time. Our best model, an ensemble of MiniRocket classifiers, is the winning approach to the ICMI ERR@HRI challenge. A feature importance analysis shows that the model heavily relies on speaker diarization data that indicates who spoke when. Posture data, on the other hand, impedes performance. Our code is available online1.},\nbooktitle = {Proceedings of the 26th International Conference on Multimodal Interaction},\npages = {657–665},\nnumpages = {9},\nkeywords = {HRI, machine learning, robot errors, social robotics},\nlocation = {San Jose, Costa Rica},\nseries = {ICMI '24}\n}\n\n","author_short":["Wachowiak, L.","Tisnikar, P.","Coles, A.","Canal, G.","Celiktutan, O."],"key":"10.1145/3678957.3688386","id":"10.1145/3678957.3688386","bibbaseid":"wachowiak-tisnikar-coles-canal-celiktutan-atimeseriesclassificationpipelinefordetectinginteractionrupturesinhribasedonuserreactions-2024","role":"author","urls":{"Paper":"https://doi.org/10.1145/3678957.3688386"},"keyword":["HRI","machine learning","robot errors","social robotics"],"metadata":{"authorlinks":{}}},"bibtype":"inproceedings","biburl":"https://bibbase.org/network/files/EW3s6Cf225dQ5cj4b","dataSources":["NrhQD3tZfM3gHzBpf"],"keywords":["hri","machine learning","robot errors","social robotics"],"search_terms":["time","series","classification","pipeline","detecting","interaction","ruptures","hri","based","user","reactions","wachowiak","tisnikar","coles","canal","celiktutan"],"title":"A Time Series Classification Pipeline for Detecting Interaction Ruptures in HRI Based on User Reactions","year":2024}