Pantomime: Mid-Air Gesture Recognition with Sparse Millimeter-Wave Radar Point Clouds. Palipana, S., Salami, D., Leiva, L., & Sigg, S. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 5(1):1-27, ACM New York, NY, USA, 2021. abstract bibtex We introduce Pantomime, a novel mid-air gesture recognition system exploiting spatio-temporal properties of millimeter-wave radio frequency (RF) signals. Pantomime is positioned in a unique region of the RF landscape: mid-resolution mid-range high-frequency sensing, which makes it ideal for motion gesture interaction. We configure a commercial frequency-modulated continuous-wave radar device to promote spatial information over temporal resolution by means of sparse 3D point clouds, and contribute a deep learning architecture that directly consumes the point cloud, enabling real-time performance with low computational demands. Pantomime achieves 95% accuracy and 99% AUC in a challenging set of 21 gestures articulated by 45 participants in two indoor environments, outperforming four state-of-the-art 3D point cloud recognizers. We also analyze the effect of environment, articulation speed, angle, and distance to the sensor. We conclude that Pantomime is resilient to various input conditions and that it may enable novel applications in industrial, vehicular, and smart home scenarios.
@article{Sameera_2021_IMWUT,
author={Sameera Palipana and Dariush Salami and Luis Leiva and Stephan Sigg},
journal={Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)},
title={Pantomime: Mid-Air Gesture Recognition with Sparse Millimeter-Wave Radar Point Clouds},
year={2021},
abstract={We introduce Pantomime, a novel mid-air gesture recognition system exploiting spatio-temporal properties of millimeter-wave radio frequency (RF) signals. Pantomime is positioned in a unique region of the RF landscape: mid-resolution mid-range high-frequency sensing, which makes it ideal for motion gesture interaction. We configure a commercial frequency-modulated continuous-wave radar device to promote spatial information over temporal resolution by means of sparse 3D point clouds, and contribute a deep learning architecture that directly consumes the point cloud, enabling real-time performance with low computational demands. Pantomime achieves 95\% accuracy and 99\% AUC in a challenging set of 21 gestures articulated by 45 participants in two indoor environments, outperforming four state-of-the-art 3D point cloud recognizers. We also analyze the effect of environment, articulation speed, angle, and distance to the sensor. We conclude that Pantomime is resilient to various input conditions and that it may enable novel applications in industrial, vehicular, and smart home scenarios.
},
issue_date = {March 2021},
publisher = {ACM New York, NY, USA},
volume = {5},
number = {1},
pages = {1-27},
group = {ambience},
project = {radiosense,windmill}
}
Downloads: 0
{"_id":"eDFp5CmBQFu9xtf3Y","bibbaseid":"palipana-salami-leiva-sigg-pantomimemidairgesturerecognitionwithsparsemillimeterwaveradarpointclouds-2021","authorIDs":["9sdTym9dh2BsmwNqn","dnF6WY6ZnxxYfGnJd"],"author_short":["Palipana, S.","Salami, D.","Leiva, L.","Sigg, S."],"bibdata":{"bibtype":"article","type":"article","author":[{"firstnames":["Sameera"],"propositions":[],"lastnames":["Palipana"],"suffixes":[]},{"firstnames":["Dariush"],"propositions":[],"lastnames":["Salami"],"suffixes":[]},{"firstnames":["Luis"],"propositions":[],"lastnames":["Leiva"],"suffixes":[]},{"firstnames":["Stephan"],"propositions":[],"lastnames":["Sigg"],"suffixes":[]}],"journal":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)","title":"Pantomime: Mid-Air Gesture Recognition with Sparse Millimeter-Wave Radar Point Clouds","year":"2021","abstract":"We introduce Pantomime, a novel mid-air gesture recognition system exploiting spatio-temporal properties of millimeter-wave radio frequency (RF) signals. Pantomime is positioned in a unique region of the RF landscape: mid-resolution mid-range high-frequency sensing, which makes it ideal for motion gesture interaction. We configure a commercial frequency-modulated continuous-wave radar device to promote spatial information over temporal resolution by means of sparse 3D point clouds, and contribute a deep learning architecture that directly consumes the point cloud, enabling real-time performance with low computational demands. Pantomime achieves 95% accuracy and 99% AUC in a challenging set of 21 gestures articulated by 45 participants in two indoor environments, outperforming four state-of-the-art 3D point cloud recognizers. We also analyze the effect of environment, articulation speed, angle, and distance to the sensor. We conclude that Pantomime is resilient to various input conditions and that it may enable novel applications in industrial, vehicular, and smart home scenarios. ","issue_date":"March 2021","publisher":"ACM New York, NY, USA","volume":"5","number":"1","pages":"1-27","group":"ambience","project":"radiosense,windmill","bibtex":"@article{Sameera_2021_IMWUT,\nauthor={Sameera Palipana and Dariush Salami and Luis Leiva and Stephan Sigg},\njournal={Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)},\ntitle={Pantomime: Mid-Air Gesture Recognition with Sparse Millimeter-Wave Radar Point Clouds},\nyear={2021},\nabstract={We introduce Pantomime, a novel mid-air gesture recognition system exploiting spatio-temporal properties of millimeter-wave radio frequency (RF) signals. Pantomime is positioned in a unique region of the RF landscape: mid-resolution mid-range high-frequency sensing, which makes it ideal for motion gesture interaction. We configure a commercial frequency-modulated continuous-wave radar device to promote spatial information over temporal resolution by means of sparse 3D point clouds, and contribute a deep learning architecture that directly consumes the point cloud, enabling real-time performance with low computational demands. Pantomime achieves 95\\% accuracy and 99\\% AUC in a challenging set of 21 gestures articulated by 45 participants in two indoor environments, outperforming four state-of-the-art 3D point cloud recognizers. We also analyze the effect of environment, articulation speed, angle, and distance to the sensor. We conclude that Pantomime is resilient to various input conditions and that it may enable novel applications in industrial, vehicular, and smart home scenarios.\n},\nissue_date = {March 2021},\npublisher = {ACM New York, NY, USA},\nvolume = {5},\nnumber = {1},\npages = {1-27},\ngroup = {ambience},\nproject = {radiosense,windmill}\n}\n\n\n","author_short":["Palipana, S.","Salami, D.","Leiva, L.","Sigg, S."],"key":"Sameera_2021_IMWUT","id":"Sameera_2021_IMWUT","bibbaseid":"palipana-salami-leiva-sigg-pantomimemidairgesturerecognitionwithsparsemillimeterwaveradarpointclouds-2021","role":"author","urls":{},"metadata":{"authorlinks":{"sigg, s":"https://thepreciousproject.eu/team_stephan.php","palipana, s":"https://thepreciousproject.eu/team_sameera.php"}},"downloads":0},"bibtype":"article","biburl":"http://ambientintelligence.aalto.fi/bibtex/LiteraturAll","creationDate":"2021-01-08T06:29:38.223Z","downloads":0,"keywords":[],"search_terms":["pantomime","mid","air","gesture","recognition","sparse","millimeter","wave","radar","point","clouds","palipana","salami","leiva","sigg"],"title":"Pantomime: Mid-Air Gesture Recognition with Sparse Millimeter-Wave Radar Point Clouds","year":2021,"dataSources":["aPfcTvMp5jE2KuS7H","a6QYyvmdLfrsx7DiL"]}