Multi-modal Markers for Meaning: using behavioural, acoustic and textual cues for automatic, context dependent summarization of lectures. Wegener, R. & Cassens, J. In Cassens, J., Wegener, R., & Kofod-Petersen, A., editors, Proceedings of the Eighth International Workshop on Modelling and Reasoning in Context, 2016.
Multi-modal Markers for Meaning: using behavioural, acoustic and textual cues for automatic, context dependent summarization of lectures [pdf]Paper  abstract   bibtex   
Meaning making is most often multi-modal and it is this feature that we make use of in outlining a model for an automatic and context dependent note-taking system for academic lectures. Drawing on semiotic models of gesture and behaviour, linguistic models of text structure and sound, and a rich model of context, we argue that the combination of information from all of these through data triangulation provides a better basis for information extraction and summarization than each alone. Further, we suggest that by using a rich model of context that maps the unfolding of the text in real time with features of the context, we can produce query driven summarization. While this outlines research on academic lectures, future work will focus on optimizing this for different domains such as tele-medicine, minuting for meetings and services for the Deaf.

Downloads: 0