I was leading a discussion of Meri Nana-Ana Danquah’s Willow Weep for Me: A Black Woman’s Journey Through Depression in February 2020 in a 3D simulation of a seminar meeting when I found myself talking to a stick of butter and a chili pepper (Figure 1). These meetings were part of my ongoing research into the pedagogical possibilities of 3D discussion-based meetings in Humanities courses using Oculus headsets. I met each week with a group of five students in a 3D classroom space that had been created for me by my industry partner XpertVR (https://xpertvr.ca). XpertVR had taken photographs of the students and created avatars that mimicked as far as possible their real-life appearance in addition to creating a meeting room that simulated a typical seminar space. Students retained their original realistic avatars for the first couple of meetings, but once they discovered that they could change their avatar appearance at will, I abruptly found myself discussing Willow Weep for Me with a variety of avatars who spoke with my students’ voices but who did not correspond with their real-life appearance. The experience raises questions about the effect of nonhuman avatars on student social engagement that need further research.
Methodology
I have long been interested in the possibilities of 3D interactions as an alternative for online teaching using the University’s text-based Learning Management System (LMS), and in 2018 I won a Brock University Chancellor’s Chair for Teaching Excellence that funded my research for three years. My research proposal focused on the “bodily presence” that 3D devices like the Oculus Quest enabled because they track the movements of the user and translate them into gestures by an avatar in a 3D environment (see Slater and Sanchez-Vives 2016; Han et al. 2023). I was interested in assessing the way that devices like the Oculus Quest can enhance the connection between the user and an avatar by mimicking bodily movements and thus promote student social engagement in a virtual environment.
The students involved in the 3D discussion were enrolled in a course on Lifewriting in which they read first-person texts and wrote critical analyses in an open-ended assignment in which they were free to choose their own topic. I chose this course not because of the content but because of the hybrid mode of delivery: one face-to-face meeting and one online meeting each week in which students communicated through writing. The class of twenty was organized into four groups of five students for the online component with one student in the group tasked with uploading an analysis of the assigned text by the beginning of the class into the LMS; the other four students then responded to this essay in a real-time written discussion with the author lasting for the class period of 50 minutes. The essay received a grade, and the students responding to the author received credit for participation.
My pilot experimental project involved two small groups of five students from the course who met for six weeks once a week in a 3D seminar room using the Oculus headsets in 2019 and 2020. The students were given the headsets to use for the duration of the course and returned them at the end of the course (either in person or via courier during the pandemic).1 These students were volunteers who did not receive extra credit for participation. The group substituted meeting with me in 3D for the online written response in the LMS. Students kept a journal of their experiences during the semester and submitted a final report of their experiences at the end of the course. The journals and final report were open ended with no page number or topic specified; they were designed to obtain raw initial reactions to interacting in what was for these students a completely new medium. My model for these journals was Maeva Veerapen’s first-person account (2011) of the process of creating an avatar and interacting with others in Second Life. I received approximately twenty-five pages of double-spaced journal entries and summarizing comments from the students involved. I then extracted comments from each student under the headings of avatars, engagement, play and problems.
Creating a 3D Seminar Room
My industry partner XpertVR created a 3D seminar room in 2018 before the start of the course. XpertVR is a company started by two Brock University graduates that at the time of this experiment was housed in the Brock LINC, an incubator for joint faculty and industry research.2 XpertVR helped choose the platform and built the 3D space for seminar meetings and provided remote technical support for all the seminar meetings. When consulting with XpertVR on the design of the 3D space, I stressed that I wanted a realistic meeting room to avoid suggestions that students were playing a game, initially following the same approach as Grinberg et al. (2014) in assuming that a realistic environment would enhance student engagement. Games are of course not automatically incompatible with learning as the “serious games” genre shows (Annetta and Bronack 2011), but my original hypothesis was that a game-like atmosphere would impede discussion. XpertVR built a classroom for me in VRChat3 with chairs, tables and windows. The virtual classroom had tables arranged in a square in the middle of the room to facilitate discussion, a functioning whiteboard, a lectern and table with a fake PC, and a “window” that showed simulated trees. This was as close to a replica of the actual room at the University where I held face-to-face meetings of the “Lifewriting” class as could be achieved in VRChat.
3D Research and the Humanities
My research is based in courses in the Humanities, but much academic research into 3D interactions focuses on STEM courses (Bartiz 2017; Cinto et al. 2016; Ke et al. 2016; Kleven 2014; Němec and Kratochvíl 2017; Nordin et al. 2008; Ortelt and Ruider 2017). As devices such as the Oculus Quest become cheaper and more ubiquitous, it will be possible to integrate them much more widely into online interactions (Szabo 2019). As a Humanities scholar my primary interest is in how effective such devices are in facilitating online the kind of discussions that take place in my face-to-face classes as I encourage students to articulate and extend their analyses of texts. Three-dimensional environments are immersive, and immersion in a task enables students to retain information better than video instruction (Lan and Liao 2018). These 3D environments also have been recognized as promoting a greater sense of social connection in response to COVID-19 (Dialani 2020; Barreda-Ángeles and Hartmann 2022). Recent studies also find that the use of 3D technology and 3D environments for instruction provide students with immersive learning experiences by increasing social presence (Huang et al. 2020; Jin 2012; Kawulich and D'Alba 2019; Ke et al. 2016), supporting the use of these tools to promote learning in discussion-based online contexts with extensive interaction (Porter et al. 2021).
Much previous 3D educational research has been conducted in Second Life (Kawulich and D’Alba 2019; Machado et al. 2016; Omale et al. 2009; Ozkan 2017; Triberti and Chirico 2017; Zold 2014) because it is free and does not require equipment beyond a computer and internet connection. For example, Grinberg et al. (2014) in their study of user social engagement (479) created a realistic environment with tables, chairs and a lounge (481) and tracked user movements within the virtual space hosted in Second Life. They also created realistic avatars for participants (481). While I share the constructivist approach of such research and a commitment to “presence pedagogy” online (Bronack et al. 2008; Vygotsky 1978), I was frustrated by Second Life’s lack of the turn-taking protocols that govern interactions in face-to-face meetings.
The experiment conducted by Grinberg et al. is representative of the kind of study conducted in Second Life. The experiment was designed to measure social engagement by monitoring the movements of a subject in a virtual space and their engagement with an avatar. An environment was created to “heighten the sense of realism” by including “appropriate décor such as tables and chairs in the dining room, a host stand outside the dining room, art hanging in the art gallery, couches and chairs in the lounge, barstools in the bar, books in the library, sinks and mirrors in the bathrooms, and a front desk in the lobby” (Grinberg et al. 2014, 481). Student participants entered the virtual space individually for fifteen minutes each to explore the space and engage in text-based conversation with the avatars of three research assistants and the principal investigator. Research assistants monitored the students’ movements and the time spent talking to avatars to measure “interest, spatial exploration and social engagement” (Grinberg et al. 2014, 482). Their conclusion, based on these measures, was that “participants who exhibited high levels of social engagement had a high sense of immersion and participants who exhibited high levels of spatial exploration had a low sense of immersion” (Grinberg et al. 2014, 484). In other words, when avatars moved around the space more, they were less immersed in the experience. The research assistants reached this conclusion because they equated time spent in conversation with being immersed in the environment, but this seems to measure social interaction more than immersion.
The research by Grinberg et al. makes a number of assumptions that I am questioning in my experiment with avatars in discussion-based Humanities courses conducted in 3D environments. Firstly, “realism” may well not be necessary for student engagement. Secondly, a student may be immersed in an environment while not engaging socially: the equivalent of a student who sits quietly in a seminar without speaking but is following the discussion. Finally, the experiment did not duplicate the context of a discussion in a seminar but rather exploration of a virtual space by participants who were not familiar with the environment which is different from the experience of a student who meets with the same group of people in the same environment each week. I followed a similar model to Grinberg et al. in creating a realistic environment but using Oculus Quest headsets and voice communication rather than Second Life, because the bodily movement of avatars allows for the kind of turn-taking protocols used in face-to-face meetings, such as the raising of a hand to indicate that someone wishes to participate in the conversation.
Oculus Quest Headsets and 3D Environments
In more recent research using Oculus Quest headsets, Han et al. (2023) studied the effect of avatars and environment on users in 3D environments. Their study used different environments: a panoramic outdoor, a panoramic indoor, a constrained outdoor, and a constrained indoor environment (Han et al. 2023, 11). In the avatar component of their study, participants could use a “self-avatar” in which they chose their own appearance or a “uniform avatar” where they could not customize the avatar appearance. Not surprisingly, when participants interacted with the uniform, they reported less “self-presence” because the uniformity of avatars “may lower an individual’s sense of ownership of their self and embodiment, affecting their sense of self as an individual” (Han et al. 2023, 11). This would seem to indicate that choosing an avatar is important for a sense of social presence. On the other hand, the uniform avatars may avoid some of the negative associations that are carried over from offline attitudes identified by Szolin et al. (2022). In my own experiment the students interacted with each other in face-to-face meetings as well as in the 3D seminar room where, at least initially, their avatars resembled their offline appearance. The question for my study was whether the sense that they were talking to a real person would carry over into the 3D space as a sense of social engagement.
Social Engagement in Online Interactions
Because of the COVID-19 pandemic, all courses at Brock University went fully online in the middle of the term in March 2020. The switch from a hybrid face-to-face and online to a fully online format forced a reorientation of my research. All my interactions with my students were now virtual which raised the question as to which was more effective in terms of “social engagement”—online discussion in Microsoft Teams, via text in the LMS, or in 3D using the Oculus Quest. “Social engagement” is defined as the “perceptions of others as “real people” (Gunawardena 1997, 151), the ability of participants to project themselves “socially and emotionally” (Garrison 2016, 94; Whiteside 2017), and the “feeling, perception, and reaction of being connected” (Tu and McIsaac 2002, 140) even when they are in a virtual environment and operating as avatars. This definition can be applied to any face-to-face interaction, but achieving social engagement is more complicated in online interactions where participant communication is mediated by technology. The problem of creating social engagement is particularly acute for online instruction where face-to-face discussion is not possible and as a result there has been an increase in academic research in this area (Akcaoglu and Lee 2016; Bronack et al. 2008; Chau et al. 2013; Cheney and Bronack 2011; Dos Santos et al. 2016; Lowenthal and Dunlap 2010; Richardson et al. 2012; Rogers and Lea 2005).
The broad theoretical issue at stake is the identification between participant and avatar. The question of the relationship between “offline” and “online” identities has been the subject of research since the publication of Sherry Turkle’s The Second Self: Computers and the Human Spirit (1984) and is itself connected to the wider issue of the “performance” of identity analyzed by Erving Goffman in The Presentation of Self in Everyday Life (1959). The metaphors of performance used by Goffman are particularly apt for 3D interactions because users construct a visual version of their offline identities through which to interact with others. Previous research has tracked user conduct in 3D environments with real world behavior and found close correlation suggesting that users behave similarly in face-to-face and virtual interactions (Bayraktar and Amca 2012; Yee and Bailenson 2007; Yee and Ducheneaut 2011). More data is needed to compare the effects on behavior of realistic versus non-realistic avatars and environments to build on previous research on virtual communication (Chau et al. 2013; Gwo-Dong Chen et al. 2012; Cinto et al. 2016; Lee 2014; Morrison 2010; Triberti and Chirico 2017). This small-scale experiment is designed to add to this research.
My own research parallels that of Chae et al. (2016) in their exploration of the effect of an avatar on what they termed “trust” in the instructor. The authors used “social engagement” which is a term that avoids some of the moral overtones of “trust.” In their experiment students interacted with two types of avatars that they termed “expert” and “attractive” with both male and female versions. The authors acknowledge that “attractive” is culturally determined (387) but based their choice of avatar on a poll of the students in the course (379). Their conclusion after having participants complete a survey was that “avatar instructors designed to display human characteristics are more likely to invoke social relationships with users than are simple avatars with no human characteristics” (379).
Student Social Engagement
I began my experiment with the premise that a realistic setting and realistic avatars would be the most effective to promote social engagement, and for that reason I asked XpertVR to create a room that mimicked as far as possible the seminar meeting spaces at my university and also had them create realistic avatars for the students. Overall, based on student comments, this approach proved successful. One student said, “The Oculus allows for much smoother conversation as it feels like you are right there in the room with everyone.”4 Another observed, “The interactive experience of VRChat allowed for me to connect with the other students and the professor on a social level.” Another student wrote:
By being in the VR class with them, instead of when I see these students in the Microsoft Teams group they feel more real to me. Their comments seem more valuable, and valid. I notice especially their voices now and mannerisms, rather than their appearance. It was almost like their avatars had became their “real,” selves within my head, as they are the ones I interact more meaningfully with.
Such comments indicate that, although we were interacting via avatars, students still had a sense of “social connection.” This was especially valuable during the year in which all courses at my university were online, and the students interacted only virtually. As one student commented, “Another advantage was it gave a sense of community from the comfort of your own home. Plus, it had a more interactive feel, being able to actually talk instead of typing at a screen.” Such statements underscore how the immediacy of voice and the ability of the avatar to mimic bodily movements enhanced the sense of social engagement. This held true when students compared using the avatar to meeting in Microsoft Teams because the avatar provided a buffer from direct exposure on camera making one student “a bit more relaxed than a seminar over Microsoft Teams would be, which I think is because I don’t need to have those shy feelings of being on the camera, but still have that movement aspect.”
Overall student satisfaction with the interactions in the 3D seminar room hinged on the degree to which they felt social presence. Rogers and Lea (2005) argue that “a feeling of belongingness to a group” which results in “perceptual immersion” is crucial in creating a sense of social presence (153). The immersion in a 3D environment helps explain why the students who could only communicate remotely during COVID pandemic restrictions felt a stronger sense of connection in the 3D seminar room than on Microsoft Teams, corroborating the findings of Barreda-Ángeles and Hartmann (2022) on the psychological benefits of participation in virtual interactions during the pandemic. Even though in a videoconference the students could see the faces of their interlocutors, they were aware that it was a face on a screen, whereas the bodily movement and voice communication in the 3D environment conveyed a sense of “being there” (Rogers and Lea 2005, 151). In addition, because their presence was mediated through an avatar, they felt less inhibited and could express themselves more freely, as the student indicates in the quotation above. Overall, the 3D interactions helped create a sense of community and social engagement more fully than either my University’s LMS or a session via videoconference.
Avatars and Play
Some students kept the same avatar throughout the meetings, but others started experimenting with the range of possible nonhuman avatars available in VRChat—which is why I found myself talking to a stick of butter. One of the students who switched to a nonhuman avatar enjoyed being able to take on a more playful appearance in the virtual space:
I really got a kick, once I figured it out, playing with the alternative avatars, but some were quite dark and bizarre. I loved the bottle of wine, the chili pepper, and the butter, and appreciated being able to play around with those ‘characters’!
That some students started to feel that they needed more enjoyment by manipulating their avatars accords with the finding of Han et al. (2023) that “wearing a self-avatar increased nonverbal synchrony, self-presence, and realism, but decreased enjoyment” (1). In other words, while the realistic avatars promoted social engagement, students enjoyed the interactions with others more when they could choose nonhuman avatars. I wanted to avoid any suggestion that our meetings were a “game” fearing that it would interfere with the seriousness of the discussion, but this participant felt that they would have preferred more playful encounters. This was a surprising and unanticipated reaction and conflicted with my assumption that realistic avatars rather than playful ones would promote social engagement.
Another student praised the immersive experience of interacting in a 3D space because “interestingly enough, it is harder to be distracted when using VR because you are fully immersed.” In conversation students admitted that while in meetings on Microsoft Teams they would look at other sites on the web and find themselves distracted but while wearing the headsets they were fully immersed in the class. One student commented, “I am forced to really focus on the discussion and do not have very many distractions (such as looking at other stuff on my laptop or looking out the window).” The pilot project overall suggested that meeting in 3D environments was preferable in terms of social engagement in small groups, but given my experience with students choosing non-human avatars and expressing a preference for a more playful experience, I have had to rethink my commitment to realism in seminar meetings.
3D as a “Game Space”
Although I tried to provide visual cues through the seminar setting and realistic avatars that we were engaged in a serious discussion, the students still perceived elements of a game in our interactions and wished for a more playful environment:
I think that because we are essentially in a “game” space, we should explore and experiment with the different settings we can work in. We mentioned it would be cool to work in a room like something at Hogwarts or an old academia space. One student said it would be interesting for the space to reflect the texts we are looking at for the week which would also be engaging and fun!
Different virtual environments create different levels of immersion, as Dillon and Cai (2022) found when they exposed participants to virtual greenspaces that enabled higher levels of engagement and restoration than urban virtual spaces. However, a virtual replica of Hogwarts from the “Harry Potter” series of novels and films as the student suggested would make the meeting space explicitly a fantasy environment aimed at entertainment. I like to create a relaxed atmosphere in my classes where students feel free to express their opinions, but from my perspective the nonhuman avatars or a more fantastical environment tip things too far toward the “fun” end of the spectrum, especially when we were discussing serious issues like racism in access to therapy for writers like Danquah in Willow Weep for Me. The contrast between the subject matter and the playfulness was a little jarring. However, after acknowledging the new playful avatars at the beginning of the seminar, I was able to keep the discussion of Willow Weep for Me on track.
The students’ desire for playfulness reflects an ambiguity in current 3D environments. Sites like ENGAGE5 and Spatial.io6 feature a combination of realistic spaces like meeting rooms and museums and more playful ones such as the surface of Mars or a “Mouse Town.” They are poised ambiguously between virtual environments as a serious tool for social engagement and a play space. The students felt this tension as well which is why some of them, once they became accustomed to interacting in the virtual seminar room, began to swap the realistic avatars that XpertVR created for them with a stick of butter, a character from The Nightmare Before Christmas, or a wine bottle. I approached the seminar room as a serious pedagogical space that replicated my role as a teacher in face-to-face meetings, but students who sensed the game-like possibilities even within the constraints that I had established chafed at the realism. This is an ongoing issue for me in meetings in 3D and one of the current drawbacks as a pedagogical tool.
Disadvantages of 3D Interactions
While some students enjoyed the game-like potential, for another the nonhuman avatars interfered with the discussion:
Some avatars are more distracting/eye catching than others…. For example today when a student was a wine bottle, its features continually baffled me (i.e. the flexibility of its skinny cork-neck, its hands, what type of wine it was).
For this student the nonhuman avatars interfered with the immersion that others found so advantageous and created a distraction from the discussion. In this case, the avatar disrupted social engagement and made them aware of the avatar as a construct rather than as a representation of a real person with whom they were having a discussion. More seriously, one student was particularly troubled by aspects of the virtual experience because of a feeling of loss of control over their avatar as a representation of their self:
I feel a loss of control in the virtual realm of being. It is uncomfortable. As someone who struggles with anxiety, it brings a new dimension in which to stress about; one I am unable to control.
For a student with anxiety issues, the experience of 3D only reaffirmed their discomfort with social situations and obviously also with social engagement. In their final summary of their experience of the 3D meetings, this student addressed me directly about the drawbacks for students with disabilities, including anxiety issues:
The Professor is still very clear in his positive perspective of VR’s potential to accommodate students. I really don’t know if I support that opinion. I feel like there is a lot of potential for VR but it needs improvement in accommodating students of different physical needs, and maybe even mental health conditions.
This was a salutary and necessary corrective to my own enthusiasm for the potential of virtual interactions to provide an alternative to videoconferencing that would be more immersive and engaging. As the student points out there are barriers to access for students with disabilities, mental health challenges, or a propensity for vertigo, which some 3D environments can exacerbate.
Another student who was generally positive about the 3D experience noted a time when the virtual environment acted in ways that belied the realistic appearance of the seminar meeting, writing that it was distracting “when your teacher is talking clearly and coherently while walking through tables and chairs and people.” Once again, my effort to create a realistic environment was subverted by the virtual space which allowed for such things as walking through tables and even other avatars, thus highlighting the artificiality of the environment and disrupting immersion in the experience. The same effect was produced for me and other students when a participant would lose their internet connection and abruptly vanish from the room. Other issues such as having to update software before a meeting, having a headset run out of battery power, or needing new batteries for a controller in the middle of a meeting showed the disadvantage of having to rely on the mediation of technology and internet connections.
Conclusion
My experiment with Oculus headsets and a virtual seminar room involved a small number of students and was intended as a pilot project to explore the feasibility of using Oculus Quest headsets for discussion-based Humanities courses. Like other researchers I relied on a small sample size (Lanier et al. 2019) which limits my ability to make generalizations about virtual interactions. My use of the Oculus Quest in this course was also unusual because I was using it solely as a communications device; the possibilities of manipulating 3D objects in virtual environments may open up other ways in which to use the technology. However, based on this small sample, my preliminary conclusion is that virtual interactions in 3D environments provide many benefits over a seminar meeting based on text or video but the cost and current barriers to participation for students with health or mental challenges need to be addressed before it can be widely adopted.
The students’ perception of a 3D environment as a game space also presents challenges if the object of the meeting is to have a discussion of serious issues. This tension between discussion of serious issues and play remains unresolved after my analysis of the student experience. Studies by Fox and Bailenson (2009), Lugrin et al. (2015), and Silva and Moioli (2017) have examined the effect of realistic versus non-realistic avatars on users, but the experiments were not designed to measure extended verbal interactions in an educational setting. Interacting in 3D environments using non-realistic avatars for some students suggests that they were participating in a game-like atmosphere with different norms of behavior from a face-to-face seminar meeting. I approached the meetings as a simulacrum of in-person discussions and expected the same behaviors to govern the interaction in the virtual seminar room, but as some of the students noted, this did not exploit the possible creative range of both avatars and environments in 3D settings. The potential of virtual environments is still being explored and the tension between serious uses of 3D and play is a current feature of the technology, as sites like ENGAGE and Spatial.io show. Mark Zuckerberg has indicated that he believes that the Metaverse will become a widely adopted social space by rebranding Facebook and his company as Meta. It remains to be seen whether 3D environments will be a part of peoples’ everyday lives, a space for serious interactions for business and education, an extension of the market for video games, or perhaps ultimately a combination of all three.
