Paper: Design insights into embedding virtual reality content into life skills training for people with intellectual disability

Just had paper accepted for OzCHI 2016 - "Design insights into embedding virtual reality content into life skills training for people with intellectual disability" with my colleagues Laurianne Sitbon, Lauren Fell, Margot Brereton (QUT), and Stewart Koplick and Chris Beaument (Endeavour Foundation).  Paper can be found here.

Abstract - This paper presents some lessons learnt from the co-creation, within a support organisation, of a virtual world dedicated to practicing life skills for people with intellectual disability. The progression of the project throughout successive stages has indicated what people would want to see and practice in a such a virtual world, which would not have been found after a single development iteration. These desires include the representation of public spaces, social interactions and disruptions. Based on initial trials, we propose some new design insights for further implementation of such virtual worlds in training sessions with the use of immersive headsets. While immersive headsets have been demonstrated to enhance the learning experience, they also visually isolate the person in training from the trainer. To overcome this isolation, a new design concept is presented which allows the teacher to communicate, using gestures, with the immersed student.

Ross

Journal: On the (virtual) getting of wisdom: immersive 3D interfaces for eliciting spatial information from experts, J. Spat. Stat.

Have had a journal paper accepted for Spatial Statistics (Elsevier).  Product of my collaboration with Kerrie Mengersen and Peter Bruza.  Paper can be found here. 

Abstract: Expert information is a valuable resource in developing robust spatial models to improve understanding and prediction of systems in health, environment, business and society. However, getting experts to reliably activate and encode what they know has proven to be an elusive goal. A root cause of this elusiveness is the fact that expert knowledge is largely tacit, i.e., experts struggle to describe what they know. The imperative to address this goal is increasing. There is a constant need for better models of expert knowledge in organizations, much observational data is sparse and inadequate for spatial modelling, and many domains have knowledgeable workers leaving in numbers. Interviews are often used for eliciting expert knowledge, due to their ease of implementation. However, there is evidence that a lack of appropriate stimuli reduces the quality of knowledge elicited. This paper explores the use of immersive 3D virtual worlds for improved knowledge elicitation, due to their priming effects on memory recall. A case study on habitat prediction for a rock wallaby species is presented, in which the new approach is trialed. This paper is one of the first that aims to combine new virtual spatial technology with expert elicitation for improved spatial statistical modelling.

Paper: A 3D visualization approach for process training in office environments, COOPIS 2016

Have had a paper accepted for COOPIS 2016, part of a collaboration with my friends Banu Aysolmaz and Hajo Reijers, at VU Amsterdam. Paper can be found here.

Abstract: Process participants need to learn how to perform in the context of their business processes. Process training is challenging due to cognitive difficulties in relating process model elements to real world concepts. In this paper we present a 3D Virtual World (VW) process training approach for office environments. In this VW, process participants can experience a process in an immersive environment. They inter- act with VW representations of process elements in changing locations, based on process activities. By means of embodied 3D representation, deep immersion and engagement in this environment, enhancements in long term memory learning and episodic memory usage for knowledge retrieval are expected. Our illustration of an example process model shows the applicability of the approach. We list a number of future directions to extend the use and the benefits.

Ross

Video: Australian TV segment on our virtual reality elicitation work

Video link here to a Qld TV segment on our Virtual Elicitation work with Kerrie Mengersen @ ACEMS.

Ross

Review: The Effect of Interactivity on Learning Physical Actions in Virtual Reality

#Title#

The Effect of Interactivity on Learning Physical Actions in Virtual Reality

#Authors#

Jeremy Bailenson, Kayur Patel, Alexia Nielsen, Ruzena Bajscy, Sang-Hack Jung & Gregorij Kurillo

#Venue#

Media Psychology, Volume 11, Issue 3, 2008

#DOI#

http://dx.doi.org/10.1080/15213260802285214

#Abstract#

Virtual reality (VR) offers new possibilities for learning, specifically for training individuals to perform physical movements such as physical therapy and exercise. The current article examines two aspects of VR that uniquely contribute to media interactivity: the ability to capture and review physical behaviour and the ability to see one's avatar rendered in real time from third person points of view. In two studies, we utilised a state-of-the-art, image-based tele-immersive system, capable of tracking and rendering many degrees of freedom of human motion in real time. In Experiment 1, participants learned better in VR than in a video learning condition according to self-report measures, and the cause of the advantage was seeing one's avatar stereoscopically in the third person. In Experiment 2, we added a virtual mirror in the learning environment to further leverage the ability to see oneself from novel angles in real time. Participants learned better in VR than in video according to objective performance measures. Implications for learning via interactive digital media are discussed.

#Comments#

The interesting idea here to me is the possibility of understanding how the movements are better encoded due to visual and proprioceptive feedback.  The question then, is this a two way mechanism?  If the same stimuli are presented, will it bring back better memory.  In addition, is it a movement encoding.  Can the encoding be accessed better by an expert.  Here they are novices in training, what happens if the person already knows the moves, but cannot explain them.  Can they explain them better if they have an immersive 3D view of their actions?  No brainer, it should work in this case clearly.  The question is, will it work as well if the visualisation is not the user of the tool at that moment in time.

They use the term "Sea of Cameras," would have thought a forest would be more apt. :-)

Experiment 1 

They rate the VR learning experience more positively.  Found the VR trainer to be more credible.

Experiment 2

VR version of learning system outperformed the video version, with significant results.

Interesting Quote:

"In fact, there is ample research dedicated to the discordance among self report measures and behavioural measures when measuring behaviour in virtual reality (Bailenson et al., 2004, Bailenson,

Swinth, et al., 2005;  Slater, 2005), concluding that neither self-report nor behavioural measures are sufficient, and that only by examining a host of measures can one assess virtual behaviour"

Shows that we need to use repeated versions of both behavioural and subjective to obtain insights into the effectiveness of such VR interfaces.  Need to keep this in mind for future experiments.

The image reproduction methods used in the paper are poor (think Kinect image level) so the quality of the images in the Tai Chi lessons may have had an effects on results.  Also, they had to look forward to compare themselves in the VR case; a cave or HMD might be better in this case due to a universal viewpoint capability.  Using an Oculus might introduce better results due to ego centre being situated in the training space.

#ImportantRefs#

Bailenson, J. N., Aharoni, E., Beall, A. C., Guadagno, R. E., Dimov, A., & Blascovich,

J. (2004). Comparing behavioral and self-report measures of embodied agents’

social presence in immersive virtual environments. Proceedings of the 7th Annual

International Workshop on PRESENCE, October 13–15, Valencia, Spain.

(Semi) Virtual Goods

Have just read this set of slides on the Asian economy in Virtual Goods by Benjamin Joffe. It seems there is a $7 Billion business in the area of selling things via virtual environments, ranging from Online Game Gold, to Mobile Apps. and all Avatars in between. The US market is hitting $1 billion, so this is a world wide phenomenon reputedly at $52 Billion in value.

This gets me thinking about the nature of virtual goods, and their value to a consumer. I buy virtual goods in a form, they are iPhone apps., iTune songs and movies. In my research, I also buy the odd virtual product in Second Life, as I am not prepared to spend time modelling chairs and desks for my business process demonstrations.

While this virtual goods economy is growing, it is also bounded somewhat by the usage of such goods in virtual environments, and a lot of people don't spend that much time in such spaces. What will be interesting is the merging of such worlds via forms of augmented reality and newer display systems that are emerging, including forms of paper-like electronic displays. Thus virtual goods become more and more integrated with actual physical environments. I would then upgrade the walls of my home with a downloaded purchase.

The logical extreme is the extension to 3D printers. Thus you will buy the virtual product (eg. a cup) and print it out for use in the real world. In fact, I wonder if such integration might indeed decrease the popularity of pure virtual worlds, as the integration of real and virtual opens up much more exciting possibilities.

New iPhone/Android Augmented Reality applications, such as Layor, are appearing in a viable commercial form, and so I expect to see a maturation of the integration between real and virtual both in representation, and economies.

Ross

CFP: Massively Multiuser Virtual Environments (MMVE 2009)

CALL FOR PAPERS

The 2nd International Workshop on Massively Multiuser Virtual Environments (MMVE 2009) at the IEEE Conference on Virtual Reality 2009 (VR09)

March 15th, 2009
Lafayette, Louisiana, USA
http://peers-at-play.org/MMVE09/
--------------------------------------------------------

With the increasing number of potential users of virtual and augmented reality systems, as indicated by the recent booms of massively multiuser online societies, the design of distributed, massively multiuser virtual environments

(MMVEs) becomes increasingly important, posing new requirements on both distribution platforms and virtual reality systems. Facing these challenges is a community-spanning effort that requires pooling the resources and experiences of both the virtual reality and the networking/ distributed computing communities.

This workshop hopes to provide a link between these communities to foster the development of highly distributed, flexible and robust virtual environments. We aim to gather both practitioners and researchers under one roof to discuss their findings, incite collaborations, and move the state of the art forward.

Ross