Enhancing Affective Representations Of Music-Induced Eeg Through Multimodal Supervision And Latent Domain Adaptation. Avramidis, K., Garoufis, C., Zlatintsi, A., & Maragos, P. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4588–4592, May, 2022. ISSN: 2379-190Xdoi abstract bibtex The study of Music Cognition and neural responses to music has been invaluable in understanding human emotions. Brain signals, though, manifest a highly complex structure that makes processing and retrieving meaningful features challenging, particularly of abstract constructs like affect. Moreover, the performance of learning models is undermined by the limited amount of available neuronal data and their severe inter-subject variability. In this paper we extract efficient, personalized affective representations from EEG signals during music listening. To this end, we employ music signals as a supervisory modality to EEG, aiming to project their semantic correspondence onto a common representation space. We utilize a bi-modal framework by combining an LSTM-based attention model to process EEG and a pre-trained model for music tagging, along with a reverse domain discriminator to align the distributions of the two modalities, further constraining the learning process with emotion tags. The resulting framework can be utilized for emotion recognition both directly, by performing supervised predictions from either modality, and indirectly, by providing relevant music samples to EEG input queries. The experimental findings show the potential of enhancing neuronal data through stimulus information for recognition purposes and yield insights into the distribution and temporal variance of music-induced affective features.
@inproceedings{avramidis_enhancing_2022,
title = {Enhancing {Affective} {Representations} {Of} {Music}-{Induced} {Eeg} {Through} {Multimodal} {Supervision} {And} {Latent} {Domain} {Adaptation}},
doi = {10.1109/ICASSP43922.2022.9746643},
abstract = {The study of Music Cognition and neural responses to music has been invaluable in understanding human emotions. Brain signals, though, manifest a highly complex structure that makes processing and retrieving meaningful features challenging, particularly of abstract constructs like affect. Moreover, the performance of learning models is undermined by the limited amount of available neuronal data and their severe inter-subject variability. In this paper we extract efficient, personalized affective representations from EEG signals during music listening. To this end, we employ music signals as a supervisory modality to EEG, aiming to project their semantic correspondence onto a common representation space. We utilize a bi-modal framework by combining an LSTM-based attention model to process EEG and a pre-trained model for music tagging, along with a reverse domain discriminator to align the distributions of the two modalities, further constraining the learning process with emotion tags. The resulting framework can be utilized for emotion recognition both directly, by performing supervised predictions from either modality, and indirectly, by providing relevant music samples to EEG input queries. The experimental findings show the potential of enhancing neuronal data through stimulus information for recognition purposes and yield insights into the distribution and temporal variance of music-induced affective features.},
booktitle = {{ICASSP} 2022 - 2022 {IEEE} {International} {Conference} on {Acoustics}, {Speech} and {Signal} {Processing} ({ICASSP})},
author = {Avramidis, Kleanthis and Garoufis, Christos and Zlatintsi, Athanasia and Maragos, Petros},
month = may,
year = {2022},
note = {ISSN: 2379-190X},
keywords = {Brain modeling, Cross-Modal Learnin, Electroencephalograph, Electroencephalography, Emotion Recognitio, Music, Music Cognitio, Semantics, Signal processing, Speech recognition, Tagging},
pages = {4588--4592},
}
Downloads: 0
{"_id":"HG5tNCazrrFiBygf6","bibbaseid":"avramidis-garoufis-zlatintsi-maragos-enhancingaffectiverepresentationsofmusicinducedeegthroughmultimodalsupervisionandlatentdomainadaptation-2022","author_short":["Avramidis, K.","Garoufis, C.","Zlatintsi, A.","Maragos, P."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","title":"Enhancing Affective Representations Of Music-Induced Eeg Through Multimodal Supervision And Latent Domain Adaptation","doi":"10.1109/ICASSP43922.2022.9746643","abstract":"The study of Music Cognition and neural responses to music has been invaluable in understanding human emotions. Brain signals, though, manifest a highly complex structure that makes processing and retrieving meaningful features challenging, particularly of abstract constructs like affect. Moreover, the performance of learning models is undermined by the limited amount of available neuronal data and their severe inter-subject variability. In this paper we extract efficient, personalized affective representations from EEG signals during music listening. To this end, we employ music signals as a supervisory modality to EEG, aiming to project their semantic correspondence onto a common representation space. We utilize a bi-modal framework by combining an LSTM-based attention model to process EEG and a pre-trained model for music tagging, along with a reverse domain discriminator to align the distributions of the two modalities, further constraining the learning process with emotion tags. The resulting framework can be utilized for emotion recognition both directly, by performing supervised predictions from either modality, and indirectly, by providing relevant music samples to EEG input queries. The experimental findings show the potential of enhancing neuronal data through stimulus information for recognition purposes and yield insights into the distribution and temporal variance of music-induced affective features.","booktitle":"ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":[{"propositions":[],"lastnames":["Avramidis"],"firstnames":["Kleanthis"],"suffixes":[]},{"propositions":[],"lastnames":["Garoufis"],"firstnames":["Christos"],"suffixes":[]},{"propositions":[],"lastnames":["Zlatintsi"],"firstnames":["Athanasia"],"suffixes":[]},{"propositions":[],"lastnames":["Maragos"],"firstnames":["Petros"],"suffixes":[]}],"month":"May","year":"2022","note":"ISSN: 2379-190X","keywords":"Brain modeling, Cross-Modal Learnin, Electroencephalograph, Electroencephalography, Emotion Recognitio, Music, Music Cognitio, Semantics, Signal processing, Speech recognition, Tagging","pages":"4588–4592","bibtex":"@inproceedings{avramidis_enhancing_2022,\n\ttitle = {Enhancing {Affective} {Representations} {Of} {Music}-{Induced} {Eeg} {Through} {Multimodal} {Supervision} {And} {Latent} {Domain} {Adaptation}},\n\tdoi = {10.1109/ICASSP43922.2022.9746643},\n\tabstract = {The study of Music Cognition and neural responses to music has been invaluable in understanding human emotions. Brain signals, though, manifest a highly complex structure that makes processing and retrieving meaningful features challenging, particularly of abstract constructs like affect. Moreover, the performance of learning models is undermined by the limited amount of available neuronal data and their severe inter-subject variability. In this paper we extract efficient, personalized affective representations from EEG signals during music listening. To this end, we employ music signals as a supervisory modality to EEG, aiming to project their semantic correspondence onto a common representation space. We utilize a bi-modal framework by combining an LSTM-based attention model to process EEG and a pre-trained model for music tagging, along with a reverse domain discriminator to align the distributions of the two modalities, further constraining the learning process with emotion tags. The resulting framework can be utilized for emotion recognition both directly, by performing supervised predictions from either modality, and indirectly, by providing relevant music samples to EEG input queries. The experimental findings show the potential of enhancing neuronal data through stimulus information for recognition purposes and yield insights into the distribution and temporal variance of music-induced affective features.},\n\tbooktitle = {{ICASSP} 2022 - 2022 {IEEE} {International} {Conference} on {Acoustics}, {Speech} and {Signal} {Processing} ({ICASSP})},\n\tauthor = {Avramidis, Kleanthis and Garoufis, Christos and Zlatintsi, Athanasia and Maragos, Petros},\n\tmonth = may,\n\tyear = {2022},\n\tnote = {ISSN: 2379-190X},\n\tkeywords = {Brain modeling, Cross-Modal Learnin, Electroencephalograph, Electroencephalography, Emotion Recognitio, Music, Music Cognitio, Semantics, Signal processing, Speech recognition, Tagging},\n\tpages = {4588--4592},\n}\n\n","author_short":["Avramidis, K.","Garoufis, C.","Zlatintsi, A.","Maragos, P."],"key":"avramidis_enhancing_2022","id":"avramidis_enhancing_2022","bibbaseid":"avramidis-garoufis-zlatintsi-maragos-enhancingaffectiverepresentationsofmusicinducedeegthroughmultimodalsupervisionandlatentdomainadaptation-2022","role":"author","urls":{},"keyword":["Brain modeling","Cross-Modal Learnin","Electroencephalograph","Electroencephalography","Emotion Recognitio","Music","Music Cognitio","Semantics","Signal processing","Speech recognition","Tagging"],"metadata":{"authorlinks":{}}},"bibtype":"inproceedings","biburl":"https://api.zotero.org/groups/4553800/items?key=BfP7bN7FF9dJwtyiLBORewdg&format=bibtex&limit=100","dataSources":["5EcJ4DjTLGyJLTga3"],"keywords":["brain modeling","cross-modal learnin","electroencephalograph","electroencephalography","emotion recognitio","music","music cognitio","semantics","signal processing","speech recognition","tagging"],"search_terms":["enhancing","affective","representations","music","induced","eeg","through","multimodal","supervision","latent","domain","adaptation","avramidis","garoufis","zlatintsi","maragos"],"title":"Enhancing Affective Representations Of Music-Induced Eeg Through Multimodal Supervision And Latent Domain Adaptation","year":2022}