Embodied Language Understanding with a Multiple Timescale Recurrent Neural Network. Heinrich, S., Weber, C., & Wermter, S.
Embodied Language Understanding with a Multiple Timescale Recurrent Neural Network [pdf]Paper  Embodied Language Understanding with a Multiple Timescale Recurrent Neural Network [link]Website  abstract   bibtex   
How the human brain understands natural language and what we can learn for intelligent systems is open research. Recently, re-searchers claimed that language is embodied in most – if not all – sensory and sensorimotor modalities and that the brain's architecture favours the emergence of language. In this paper we investigate the characteris-tics of such an architecture and propose a model based on the Multiple Timescale Recurrent Neural Network, extended by embodied visual per-ception. We show that such an architecture can learn the meaning of utterances with respect to visual perception and that it can produce verbal utterances that correctly describe previously unknown scenes.

Downloads: 0