Consistent but Modest: A Meta-analysis on Unimodal and Multimodal Affect Detection Accuracies from 30 Studies. D'Mello, S. & Kory, J. In Proceedings of the 14th ACM International Conference on Multimodal Interaction, of ICMI '12, pages 31--38, New York, NY, USA, 2012. ACM.
Consistent but Modest: A Meta-analysis on Unimodal and Multimodal Affect Detection Accuracies from 30 Studies [link]Paper  doi  abstract   bibtex   
The recent influx of multimodal affect classifiers raises the important question of whether these classifiers yield accuracy rates that exceed their unimodal counterparts. This question was addressed by performing a meta-analysis on 30 published studies that reported both multimodal and unimodal affect detection accuracies. The results indicated that multimodal accuracies were consistently better than unimodal accuracies and yielded an average 8.12% improvement over the best unimodal classifiers. However, performance improvements were three times lower when classifiers were trained on natural or seminatural data (4.39% improvement) compared to acted data (12.1% improvement). Importantly, performance of the best unimodal classifier explained an impressive 80.6% (cross-validated) of the variance in multimodal accuracy. The results also indicated that multimodal accuracies were substantially higher than accuracies of the second-best unimodal classifiers (an average improvement of 29.4%) irrespective of the naturalness of the training data. Theoretical and applied implications of the findings are discussed.
@inproceedings{dmello_consistent_2012,
	address = {New York, NY, USA},
	series = {{ICMI} '12},
	title = {Consistent but {Modest}: {A} {Meta}-analysis on {Unimodal} and {Multimodal} {Affect} {Detection} {Accuracies} from 30 {Studies}},
	isbn = {978-1-4503-1467-1},
	shorttitle = {Consistent but {Modest}},
	url = {http://doi.acm.org/10.1145/2388676.2388686},
	doi = {10.1145/2388676.2388686},
	abstract = {The recent influx of multimodal affect classifiers raises the important question of whether these classifiers yield accuracy rates that exceed their unimodal counterparts. This question was addressed by performing a meta-analysis on 30 published studies that reported both multimodal and unimodal affect detection accuracies. The results indicated that multimodal accuracies were consistently better than unimodal accuracies and yielded an average 8.12\% improvement over the best unimodal classifiers. However, performance improvements were three times lower when classifiers were trained on natural or seminatural data (4.39\% improvement) compared to acted data (12.1\% improvement). Importantly, performance of the best unimodal classifier explained an impressive 80.6\% (cross-validated) of the variance in multimodal accuracy. The results also indicated that multimodal accuracies were substantially higher than accuracies of the second-best unimodal classifiers (an average improvement of 29.4\%) irrespective of the naturalness of the training data. Theoretical and applied implications of the findings are discussed.},
	urldate = {2014-06-05TZ},
	booktitle = {Proceedings of the 14th {ACM} {International} {Conference} on {Multimodal} {Interaction}},
	publisher = {ACM},
	author = {D'Mello, Sidney and Kory, Jacqueline},
	year = {2012},
	pages = {31--38}
}

Downloads: 0