The effect of modality on social presence, presence and performance in collaborative virtual environments

Sammanfattning: Humans rely on all their senses when interacting with others in order to communicate and collaborate efficiently. In mediated interaction the communication channel is more or less constrained, and humans have to cope with the fact that they cannot get all the information that they get in face-to-face interaction. The particular concern in this thesis is how humans are affected by different multimodal interfaces when they are collaborating with another person in a shared virtual environment. One aspect considered is how different modalities affect social presence, i.e. people’s ability to perceive the other person’s intentions and emotions. Another aspect investigated is how different modalities affect people’s notion of being present in a virtual environment that feels realistic and meaningful. Finally, this thesis attempts to understand how human behavior and efficiency in task performance are affected when using different modalities for collaboration.In the experiment presented in articles A and B, a shared virtual environment that provided touch feedback was used, making it possible to feel the shape, weight and softness of objects as well as collisions between objects and forces produced by another person. The effects of touch feedback on people’s task performance, perceived social presence, perceived presence and perceived task performance were investigated in tasks where people manipulated objects together. Voice communication was possible during the collaboration. Touch feedback improved task performance significantly, making it both faster and more precise. People reported significantly higher levels of presence and perceived performance, but no difference was found in the perceived social presence between the visual only condition and the condition with touch feedback.In article C an experiment is presented, where people performed a decision making task in a collaborative virtual environment (CVE) using avatar representations. They communicated either by text-chat, a telephone connection or a video conference system when collaborating in the CVE. Both perceived social presence and perceived presence were significantly lower in the CVE text-chat condition than in the CVE telephone and CVE video conference conditions. The number of words and the tempo in the dialogue as well as the task completion time differed significantly for persons that collaborated using CVE text-chat compared to those that used a telephone or a video conference in the CVE. The tempo in the dialogue was also found to be significantly higher when people communicated using a telephone compared to a video conference system in CVEs. In a follow-up experiment people performed the same task using a website instead, with no avatar but with the same information content as before. Subjects communicated either by telephone or a video conference iv system. Results from the follow-up experiment showed that people that used a telephone completed tasks significantly faster than those that used a video conference system, and that the tempo in the dialogue was significantly higher in the web environments than in the CVEs. Handing over objects is a common event during collaboration in face-to face interaction. In the experiment presented in article D and E, the effects of providing touch feedback was investigated in a shared virtual environment in which subjects passed a series of cubic objects to each other and tapped them at target areas. Subjects could not communicate verbally during the experiment. The framework of Fitts’ law was applied and it was hypothesized that object hand off constituted a collaboratively performed Fitts’ law task, with target distance to target size ratio as a fundamental performance determinant.Results showed that task completion time indeed linearly increased with Fitts’ index of difficulty, both with and without touch feedback. The error rate was significantly lower in the condition with touch feedback than in the condition with only visual feedback. It was also found that touch feedback significantly increased people’s perceived presence, social presence and perceived performance in the virtual environment. The results presented in article A and E analyzed together, suggest that when voice communication is provided the effect of touch feedback on social presence might be overshadowed. However, when verbal communication is not possible, touch proves to be important for social presence.