Extending multimedia languages to support multimodal user interactions. Guedes, Á. L. V., Azevedo, R. G. d. A., & Barbosa, S. D. J. 76(4):5691–5720, 2017. ![Extending multimedia languages to support multimodal user interactions [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex 41 downloads Historically, the Multimedia community research has focused on output modalities, through studies on timing and multimedia processing. The Multimodal Interaction community, on the other hand, has focused on user-generated modalities, through studies on Multimodal User Interfaces (MUI). In this paper, aiming to assist the development of multimedia applications with MUIs, we propose the integration of concepts from those two communities in a unique high-level programming framework. The framework integrates user modalities —both user-generated (e.g., speech, gestures) and user-consumed (e.g., audiovisual, haptic)— in declarative programming languages for the specification of interactive multimedia applications. To illustrate our approach, we instantiate the framework in the NCL (Nested Context Language) multimedia language. NCL is the declarative language for developing interactive applications for Brazilian Digital TV and an ITU-T Recommendation for IPTV services. To help evaluate our approach, we discuss a usage scenario and implement it as an NCL application extended with the proposed multimodal features. Also, we compare the expressiveness of the multimodal NCL against existing multimedia and multimodal languages, for both input and output modalities.
@article{guedes_extending_2017,
title = {Extending multimedia languages to support multimodal user interactions},
volume = {76},
issn = {1573-7721},
url = {http://dx.doi.org/10.1007/s11042-016-3846-8},
doi = {10.1007/s11042-016-3846-8},
abstract = {Historically, the Multimedia community research has focused on output modalities, through studies on timing and multimedia processing. The Multimodal Interaction community, on the other hand, has focused on user-generated modalities, through studies on Multimodal User Interfaces ({MUI}). In this paper, aiming to assist the development of multimedia applications with {MUIs}, we propose the integration of concepts from those two communities in a unique high-level programming framework. The framework integrates user modalities —both user-generated (e.g., speech, gestures) and user-consumed (e.g., audiovisual, haptic)— in declarative programming languages for the specification of interactive multimedia applications. To illustrate our approach, we instantiate the framework in the {NCL} (Nested Context Language) multimedia language. {NCL} is the declarative language for developing interactive applications for Brazilian Digital {TV} and an {ITU}-T Recommendation for {IPTV} services. To help evaluate our approach, we discuss a usage scenario and implement it as an {NCL} application extended with the proposed multimodal features. Also, we compare the expressiveness of the multimodal {NCL} against existing multimedia and multimodal languages, for both input and output modalities.},
pages = {5691--5720},
number = {4},
journaltitle = {Multimedia Tools and Applications},
author = {Guedes, Álan Lívio Vasconcelos and Azevedo, Roberto Gerson de Albuquerque and Barbosa, Simone Diniz Junqueira},
year = {2017},
}
Downloads: 41
{"_id":"WyKhABCDNpHkCuuJq","bibbaseid":"guedes-azevedo-barbosa-extendingmultimedialanguagestosupportmultimodaluserinteractions-2017","downloads":41,"creationDate":"2017-07-16T16:15:21.639Z","title":"Extending multimedia languages to support multimodal user interactions","author_short":["Guedes, Á. L. V.","Azevedo, R. G. d. A.","Barbosa, S. D. J."],"year":2017,"bibtype":"article","biburl":"http://www.telemidia.puc-rio.br/files/biblio/all.bib","bibdata":{"bibtype":"article","type":"article","title":"Extending multimedia languages to support multimodal user interactions","volume":"76","issn":"1573-7721","url":"http://dx.doi.org/10.1007/s11042-016-3846-8","doi":"10.1007/s11042-016-3846-8","abstract":"Historically, the Multimedia community research has focused on output modalities, through studies on timing and multimedia processing. The Multimodal Interaction community, on the other hand, has focused on user-generated modalities, through studies on Multimodal User Interfaces (MUI). In this paper, aiming to assist the development of multimedia applications with MUIs, we propose the integration of concepts from those two communities in a unique high-level programming framework. The framework integrates user modalities —both user-generated (e.g., speech, gestures) and user-consumed (e.g., audiovisual, haptic)— in declarative programming languages for the specification of interactive multimedia applications. To illustrate our approach, we instantiate the framework in the NCL (Nested Context Language) multimedia language. NCL is the declarative language for developing interactive applications for Brazilian Digital TV and an ITU-T Recommendation for IPTV services. To help evaluate our approach, we discuss a usage scenario and implement it as an NCL application extended with the proposed multimodal features. Also, we compare the expressiveness of the multimodal NCL against existing multimedia and multimodal languages, for both input and output modalities.","pages":"5691–5720","number":"4","journaltitle":"Multimedia Tools and Applications","author":[{"propositions":[],"lastnames":["Guedes"],"firstnames":["Álan","Lívio","Vasconcelos"],"suffixes":[]},{"propositions":[],"lastnames":["Azevedo"],"firstnames":["Roberto","Gerson","de","Albuquerque"],"suffixes":[]},{"propositions":[],"lastnames":["Barbosa"],"firstnames":["Simone","Diniz","Junqueira"],"suffixes":[]}],"year":"2017","bibtex":"@article{guedes_extending_2017,\n\ttitle = {Extending multimedia languages to support multimodal user interactions},\n\tvolume = {76},\n\tissn = {1573-7721},\n\turl = {http://dx.doi.org/10.1007/s11042-016-3846-8},\n\tdoi = {10.1007/s11042-016-3846-8},\n\tabstract = {Historically, the Multimedia community research has focused on output modalities, through studies on timing and multimedia processing. The Multimodal Interaction community, on the other hand, has focused on user-generated modalities, through studies on Multimodal User Interfaces ({MUI}). In this paper, aiming to assist the development of multimedia applications with {MUIs}, we propose the integration of concepts from those two communities in a unique high-level programming framework. The framework integrates user modalities —both user-generated (e.g., speech, gestures) and user-consumed (e.g., audiovisual, haptic)— in declarative programming languages for the specification of interactive multimedia applications. To illustrate our approach, we instantiate the framework in the {NCL} (Nested Context Language) multimedia language. {NCL} is the declarative language for developing interactive applications for Brazilian Digital {TV} and an {ITU}-T Recommendation for {IPTV} services. To help evaluate our approach, we discuss a usage scenario and implement it as an {NCL} application extended with the proposed multimodal features. Also, we compare the expressiveness of the multimodal {NCL} against existing multimedia and multimodal languages, for both input and output modalities.},\n\tpages = {5691--5720},\n\tnumber = {4},\n\tjournaltitle = {Multimedia Tools and Applications},\n\tauthor = {Guedes, Álan Lívio Vasconcelos and Azevedo, Roberto Gerson de Albuquerque and Barbosa, Simone Diniz Junqueira},\n\tyear = {2017},\n}\n\n","author_short":["Guedes, Á. L. V.","Azevedo, R. G. d. A.","Barbosa, S. D. J."],"key":"guedes_extending_2017","id":"guedes_extending_2017","bibbaseid":"guedes-azevedo-barbosa-extendingmultimedialanguagestosupportmultimodaluserinteractions-2017","role":"author","urls":{"Paper":"http://dx.doi.org/10.1007/s11042-016-3846-8"},"metadata":{"authorlinks":{"azevedo, r":"http://www.telemidia.puc-rio.br/~roberto/publications"}},"downloads":41},"search_terms":["extending","multimedia","languages","support","multimodal","user","interactions","guedes","azevedo","barbosa"],"keywords":[],"authorIDs":["22qdanuzGpjfebYp2","244dmL4NwKXTfTqFB","24yoNLHLsDa4x9D2M","2wkrqiNDpnJQeYFvw","3fArXRQLQoc8ZwrMq","5508feeedf05089b6e0001ce","5YSSXn6odxExxuoFA","5de8c5c19e80cdde01000072","5dee8365fc4c19df01000114","5df5f4a0a37a40df01000028","5df60bec2b34d0de01000007","5df7cf16dc100cde01000018","5e06bd583cb7b3df010000f7","5e123dd0c196d3de010000e6","5e135e67697554de010001c9","5e1361bbf16095df01000016","5e136472f16095df01000072","5e1517f788b10dde01000104","5e151a5088b10dde0100011c","5e1763eb4df69dde01000162","5e191ef430dc56df01000005","5e1c63b492587bde01000005","5e271e10f51e02de0100018a","5e2d83fd556d50df010000c5","5e32bb75466076df010000b9","5e3d85b096e576de01000140","5e3db84407ca74de0100008b","5e414225b7d170df010000fb","5e41459eb7d170df01000121","5e45df99ad0603df0100010a","5e45e07cad0603df0100011c","5e4ae41441072bdf01000071","5e4bffff0dff2bde010000c4","5e4f0949e5389bde0100000b","5e501ee3933046de0100016e","5e501f2e933046de01000179","5e5e8b5ec0a53dde01000071","5e5ff99113e3aede0100003d","5e63aeed5e3c57de01000146","5yMsKZKLBnZKYvSHM","6AYHMrhkKKxQmxubb","6jHF2A9kwyWmNFCCd","6wryNCAx7Zo8fbwAB","7hPsR4oepA7Xc4HaE","884pdHmj7ppG8iMk9","8yyq55iMiNu7KsgmJ","9JGhhPKtYaFCnMcTC","AED8rhcj6RNJLxXaL","AinPa5MKBTARKabGR","BAt34XTbdkvxhXgcC","CKx8XtftEtEkTS9XX","CWCixdwNu6XxFiHB5","EZ5k5yT5wLDS5oY2k","EiPFBrZGk8szdGvdM","EnvpRHSpqMg8PPvFL","EyTjgNaspXXTjLnRA","FeT67z5eszt53SB9m","G4TmkruogoSArYmz8","G6dsN8GBTgKb4zgHK","GdSNk5rbGcsuazQAA","Gg6FyBxS4c9crJQWY","HK6F7nRGfMHsZwRQn","JBPNdykuQFNRwkJKm","Jq4oRAA6kc2cEWEMx","Js8D8d5fYQiaZimyM","K3h2ZWfWMtS5NWpEB","KzsLTkoPMJ6gNJo64","L8qBKy36xDRcmE8oH","MfBYf6SeReriSdD5q","NNShBA8E6jTJ9pc49","NwmD6quP7sfJgs4cE","P4HhxCoz5c2Bn55xN","RBufWNXJd7ArN6gPp","RKoz2F7cr9jhRzBSA","SKn7QDikeo6LsniwG","Tu33b2iijsccpNwNy","WSi47kKxoBL9zxMPk","XcDM74dsEkKXvGtSE","ZLyJ3SvsZAgkSmMhA","aHoWYt7KttYy4xsyb","dT4Q9vZmW8XW3rssQ","dYyNfMq6QDvMwqEmg","hN8avpNdk8F9Bdmih","hy6qqBWeQBxrfZQkQ","iZrZ562QLES379AeJ","kbbhu3riRm4GbuxXr","mYnbCr4EQk3aayLEc","oZTNoGwQwN6Lwjkyt","pRdcpdf3i5Fm32gWn","pfHvmrarHFaCnXruD","qfTGajRsvSg96C5Lj","s6NMQeuXCvahtn5xD","sTvD7E6KuhsRZRD3E","srPxnveRYPn3cx52j","thzkvvp2oSxPQeiE7","tshc376gy7go5omwM","vnp2utbXmofWRR7MA","wiggSGkKjJvNyXH6j","x2ZvzHk267HfqDhwM","xi7msFqQ4BePcjteq","xjLLYibFy9e2YPqtu","z2zDhcNq9rbP2GbuG","zwSxNcwtdzLJCX6eB"],"dataSources":["g2kK7LGtY6BSAGWXZ","gXSBTZhj3xCWydoZF","fzQqRdpBjaqFvMqtM","2GMp8PAJ4r2b8svJX"]}