Audiovisual signals shape our perception of materials in virtual reality. Koppisetty, H., Allison, R., & Wilcox, L. M. In International Multisensory Research Forum, pages 73. 2024. abstract bibtex We investigated how visual and auditory properties affect our material perception when these attributes conflict, and whether the outcome depends on our interaction with the object. A VR headset displayed a target object that was struck with a rod and made an associated impact sound. The participant classified the target material using four options: glass, metal, plastic, or wood. All the sounds were presented with each of the visual textures resulting in 16 conditions. To study the effect of agency, in half the trials the participant struck the target using a rendered rod and a VR controller; in the rest they observed an agent striking the target. Participants (N=42) tended to classify the materials by sound over appearance, but relied more on appearance when the sounds were similar. Agency (direct interaction) did not significantly change the classifications. Finally, there were 3 conditions in which participants frequently classified the target as a third option that matched neither appearance nor sound – suggesting a material-based audiovisual illusion. To date we have used a metal rod to strike the target. Would changing the rod affect participant response? Would these illusions be more likely if sound quality was degraded? We will present the results of ongoing experiments that investigate these questions. Modern technology enables realistic (faux) appearances. Here we show that dissonance with the visual appearance may produce non-intuitive intersensory interactions. These results highlight interesting intersensory dependencies in material perception that should be considered in applications from flooring design to creation of virtual environments.
@incollection{Koppisetty:2024ez,
abstract = {We investigated how visual and auditory properties affect our material perception when these attributes conflict,
and whether the outcome depends on our interaction with the object. A VR headset displayed a target object that
was struck with a rod and made an associated impact sound. The participant classified the target material using four
options: glass, metal, plastic, or wood. All the sounds were presented with each of the visual textures resulting in 16
conditions. To study the effect of agency, in half the trials the participant struck the target using a rendered rod and a
VR controller; in the rest they observed an agent striking the target. Participants (N=42) tended to classify the materials
by sound over appearance, but relied more on appearance when the sounds were similar. Agency (direct interaction)
did not significantly change the classifications. Finally, there were 3 conditions in which participants frequently
classified the target as a third option that matched neither appearance nor sound -- suggesting a material-based
audiovisual illusion. To date we have used a metal rod to strike the target. Would changing the rod affect participant
response? Would these illusions be more likely if sound quality was degraded? We will present the results of ongoing
experiments that investigate these questions. Modern technology enables realistic (faux) appearances. Here we
show that dissonance with the visual appearance may produce non-intuitive intersensory interactions. These results
highlight interesting intersensory dependencies in material perception that should be considered in applications from
flooring design to creation of virtual environments.},
author = {Koppisetty, H. and Allison, R.S. and Wilcox, L. M.},
booktitle = {International Multisensory Research Forum},
date-added = {2024-05-07 01:13:34 -0400},
date-modified = {2024-07-31 10:00:51 -0400},
keywords = {Augmented & Virtual Reality},
pages = {73},
title = {Audiovisual signals shape our perception of materials in virtual reality},
year = {2024}}
Downloads: 0
{"_id":"noPQ49oKTD9ngAokq","bibbaseid":"koppisetty-allison-wilcox-audiovisualsignalsshapeourperceptionofmaterialsinvirtualreality-2024","author_short":["Koppisetty, H.","Allison, R.","Wilcox, L. M."],"bibdata":{"bibtype":"incollection","type":"incollection","abstract":"We investigated how visual and auditory properties affect our material perception when these attributes conflict, and whether the outcome depends on our interaction with the object. A VR headset displayed a target object that was struck with a rod and made an associated impact sound. The participant classified the target material using four options: glass, metal, plastic, or wood. All the sounds were presented with each of the visual textures resulting in 16 conditions. To study the effect of agency, in half the trials the participant struck the target using a rendered rod and a VR controller; in the rest they observed an agent striking the target. Participants (N=42) tended to classify the materials by sound over appearance, but relied more on appearance when the sounds were similar. Agency (direct interaction) did not significantly change the classifications. Finally, there were 3 conditions in which participants frequently classified the target as a third option that matched neither appearance nor sound – suggesting a material-based audiovisual illusion. To date we have used a metal rod to strike the target. Would changing the rod affect participant response? Would these illusions be more likely if sound quality was degraded? We will present the results of ongoing experiments that investigate these questions. Modern technology enables realistic (faux) appearances. Here we show that dissonance with the visual appearance may produce non-intuitive intersensory interactions. These results highlight interesting intersensory dependencies in material perception that should be considered in applications from flooring design to creation of virtual environments.","author":[{"propositions":[],"lastnames":["Koppisetty"],"firstnames":["H."],"suffixes":[]},{"propositions":[],"lastnames":["Allison"],"firstnames":["R.S."],"suffixes":[]},{"propositions":[],"lastnames":["Wilcox"],"firstnames":["L.","M."],"suffixes":[]}],"booktitle":"International Multisensory Research Forum","date-added":"2024-05-07 01:13:34 -0400","date-modified":"2024-07-31 10:00:51 -0400","keywords":"Augmented & Virtual Reality","pages":"73","title":"Audiovisual signals shape our perception of materials in virtual reality","year":"2024","bibtex":"@incollection{Koppisetty:2024ez,\n\tabstract = {We investigated how visual and auditory properties affect our material perception when these attributes conflict,\nand whether the outcome depends on our interaction with the object. A VR headset displayed a target object that\nwas struck with a rod and made an associated impact sound. The participant classified the target material using four\noptions: glass, metal, plastic, or wood. All the sounds were presented with each of the visual textures resulting in 16\nconditions. To study the effect of agency, in half the trials the participant struck the target using a rendered rod and a\nVR controller; in the rest they observed an agent striking the target. Participants (N=42) tended to classify the materials\nby sound over appearance, but relied more on appearance when the sounds were similar. Agency (direct interaction)\ndid not significantly change the classifications. Finally, there were 3 conditions in which participants frequently\nclassified the target as a third option that matched neither appearance nor sound -- suggesting a material-based\naudiovisual illusion. To date we have used a metal rod to strike the target. Would changing the rod affect participant\nresponse? Would these illusions be more likely if sound quality was degraded? We will present the results of ongoing\nexperiments that investigate these questions. Modern technology enables realistic (faux) appearances. Here we\nshow that dissonance with the visual appearance may produce non-intuitive intersensory interactions. These results\nhighlight interesting intersensory dependencies in material perception that should be considered in applications from\nflooring design to creation of virtual environments.},\n\tauthor = {Koppisetty, H. and Allison, R.S. and Wilcox, L. M.},\n\tbooktitle = {International Multisensory Research Forum},\n\tdate-added = {2024-05-07 01:13:34 -0400},\n\tdate-modified = {2024-07-31 10:00:51 -0400},\n\tkeywords = {Augmented & Virtual Reality},\n\tpages = {73},\n\ttitle = {Audiovisual signals shape our perception of materials in virtual reality},\n\tyear = {2024}}\n\n","author_short":["Koppisetty, H.","Allison, R.","Wilcox, L. M."],"key":"Koppisetty:2024ez","id":"Koppisetty:2024ez","bibbaseid":"koppisetty-allison-wilcox-audiovisualsignalsshapeourperceptionofmaterialsinvirtualreality-2024","role":"author","urls":{},"keyword":["Augmented & Virtual Reality"],"metadata":{"authorlinks":{}},"html":""},"bibtype":"incollection","biburl":"www.cse.yorku.ca/percept/papers/self.bib","dataSources":["2KKYxJNEDKp35ykmq","BPKPSXjrbMGteC59J"],"keywords":["augmented & virtual reality"],"search_terms":["audiovisual","signals","shape","perception","materials","virtual","reality","koppisetty","allison","wilcox"],"title":"Audiovisual signals shape our perception of materials in virtual reality","year":2024}