Paper: Augmenting process elicitation with visual priming: An empirical exploration of user behaviour and modelling outcomes

We have just recently published a new journal paper in the Information Systems Journal on our virtual worlds elicitation work in BPM - "Augmenting process elicitation with visual priming: An empirical exploration of user behaviour and modelling outcomes."

QUT Eprints is here, with Journal DOI.  The work was performed by my PhD student Joel Harman along with my collaborators at QUT, Metasonic and University of Vienna.

Well done Joel!


Abstract

Business process models have become an effective way of examining business practices to identify areas for improvement. While common information gathering approaches are generally efficacious, they can be quite time consuming and have the risk of developing inaccuracies when information is forgotten or incorrectly interpreted by analysts. In this study, the potential of a role-playing approach to process elicitation and specification has been examined. This method allows stakeholders to enter a virtual world and role-play actions similarly to how they would in reality. As actions are completed, a model is automatically developed, removing the need for stakeholders to learn and understand a modelling grammar. An empirical investigation comparing both the modelling outputs and participant behaviour of this virtual world role-play elicitor with an S-BPM process modelling tool found that while the modelling approaches of the two groups varied greatly, the virtual world elicitor may not only improve both the number of individual process task steps remembered and the correctness of task ordering, but also provide a reduction in the time required for stakeholders to model a process view.

Paper: Model as you do : engaging an S-BPM vendor on process modelling in 3D virtual worlds

Have recently had a book chapter "Model as you do : engaging an S-BPM vendor on process modelling in 3D virtual worlds," accepted.  The paper was written with Joel Harman, my Honours student from QUT, and Udo Kannengiesser, Nils Meyer and Thomas Rothschaedl from Metasonic GmbH.  It describes the processes Joel and I went through to implement a virtual world process elicitation tool in conjunction with Metasonic in Germany.  The chapter will be published in "In S-BPM in the Wild – Value Creating Practice in the Field," Springer, Berlin Heidelberg.

QUT eprints entry is here, email me on r.brown@qut.edu.au if you want a copy.

Abstract: Accurate process model elicitation continues to be a time consuming task, requiring skill on the part of the interviewer to extract explicit and tacit process information from the interviewee. Many errors occur in this elicitation stage that would be avoided by better activity recall, more consistent specification methods and greater engagement in the elicitation process by interviewees. Metasonic GmbH has developed a process elicitation tool for their process suite. As part of a research engagement with Metasonic, staff from QUT, Australia have developed a 3D virtual world approach to the same problem, viz. eliciting process models from stakeholders in an intuitive manner. This book chapter tells the story of how QUT staff developed a 3D Virtual World tool for process elicitation, took the outcomes of their research project to Metasonic for evaluation, and finally, Metasonic’s response to the initial proof of concept.

Ross

Paper: Evidence that virtual worlds improve business process elicitation

Our paper "Virtual Business Role-play: Leveraging Familiar Environments to Prime Stakeholder Memory during Process Elicitation," has been accepted for CaISE 2015 in Stockholm, Sweden.  This paper is a product of an Honours thesis by my student Joel Harman, in collaboration with Stefanie Rinderle-Ma (Uni. Vienna), Daniel Johnson (QUT) and Udo Kannengiesser (Metasonic GmbH).  

The paper is stored here at QUT eprints, contact me on r.brown@qut.edu.au if you want a pdf copy.

Abstract. Business process models have traditionally been an effective way of examining business practices to identify areas for improvement. While common information gathering approaches are generally efficacious, they can be quite time consuming and have the risk of developing inaccuracies when information is forgotten or incorrectly interpreted by analysts. In this study, the potential of a role-playing approach for process elicitation and specification has been examined. This method allows stakeholders to enter a virtual world and role-play actions as they would in reality. As actions are completed, a model is automatically developed, removing the need for stakeholders to learn and understand a modelling grammar. Empirical data obtained in this study suggests that this approach may not only improve both the number of individual process task steps remembered and the correctness of task ordering, but also provide a reduction in the time required for stakeholders to model a process view.

Not only is this is a great achievement by Joel, CaISE is a very competitive conference, but the preliminary evidence is very encouraging.  Virtual worlds do indeed work well as a process elicitation tool, especially, we believe, for naive stakeholders.

Ross

Review: Virtually True: Children’s Acquisition of False Memories in Virtual Reality

#Title#

Virtually True: Children’s Acquisition of False Memories in Virtual Reality

#Authors#

KATHRYN Y. SEGOVIA and JEREMY N. BAILENSON

#Venue#

Journal of Media Psychology, 12:371–393, 2009

#DOI#

DOI: 10.1080/15213260903287267

#Abstract#

Previous work on human memory has shown that prompting participants with false events and self-relevant information via different types of media such as narratives, edited 2-dimensional images, and mental imagery creates false memories. This study tested a new form of media for studying false memory formation: Immersive Virtual Environment Technology (IVET). Using this tool, we examined how memory was affected by viewing dynamic simulations of avatars performing novel actions. In the study, 55 preschool and elementary children were randomly assigned to 1 of 4 memory prompt conditions (idle, mental imagery, IVET simulation of another child, or IVET simulation of self). Each child was questioned 3 different times: once before the memory prompt, once immediately after the memory prompt, and once approximately 5 days after the memory prompt. Results showed that preschool children were equally likely to develop false memories regardless of memory prompt condition. But, for elementary children, the mental imagery and IVET self conditions caused significantly more false memories than the idle condition. Implications regarding the use of digital media in courtroom settings, clinical therapy settings, entertainment, and other applications are discussed.

#Comments#

Interesting, another acronym - Immersive Virtual Environment Technology (IVET).  I don't like it, prefer 3D virtual worlds.  But I guess the use of immersive is pertinent here.

Note here that experiments were performed with children, not adults.  They state that the vulnerability of preschool children to suggestive influences is very high, so may not apply to adults.

They also bring up the source monitoring judgement as an experimental task; viz. the identification of sources of memories, and how it is actually poorly managed on the part of the brain.

They also state that narratives are commonly used to test false memories.  Are these analogous to workflows, and the place of false memories in elicitation?!?!?

Conjecture: Is the imposition of a process a form of false memory imputation over the actuality of the work processes in an enterprise?  Or, for any elicitation process for that matter?  Thus the more real the stimuli, the more likely to get the truth?  Possible research question here.

They note that narrative information previously used in experiments is not that rich.  The media richness theory suggests a greater effect from IVETs.

Interesting passage:

"In another set of studies, college students heard simple action statements and in some conditions either performed or imagined the action as well (Goff & Roediger, 1998). In a second session, participants imagined performing actions (some of which came from the first session and some of which were new) either one, three, or five times. In the third session, participants were asked to identify actions only if they had occurred in the first session and, if identified, to tell whether the action statement had been carried out, imagined, or merely heard. The main finding was that increasing the number of imaginings during the second session caused participants to later remember that they had performed an action during the first session when in fact the participant had not."

Note the influence of imagination over the actual memory of a performed event.  This seems to hold across familiar and non-familiar actions - relevant to knowledge elicitation.  If you get people to imagine badly, they will report badly.

They also note the power of imaging past actions; very close and personal stimuli and feedback interactions in the person's head.  May be part of a feedback loop.

Another key factor here:

"We believe that the manipulation explored by Strange and colleagues (2008) is important and would like to offer a complementary hypothesis to explain the reported effect. We propose that personalized photographs were more powerful stimuli because they fulfilled the personal focus criterion in- volved in media richness. The personal focus involved in the media richness criteria is influential in the memory process because it causes humans to self-reference; people encode more attributes when they are engaged in self-referent processing than other types of processing (Symons & Johnson, 1997). The greater the amount of information encoded the more similar the memory becomes to a memory of a physical world event, and the greater the likelihood that a source monitoring error will occur."

This is the key here to me, not regarding false memories, but regarding actual memories.  Such an idea lends further weight to the fact that elicitation is enhanced by personal recollection, but may be further confounded if the personal recollection has edits introduced.  So, we have a mechanism here for the noise involved in recollection, especially that which is false, not just knocked out due to functional problems, such as random memory lapses.

They note that use of media may provide a low-cognitive organisation environment, compared to abstract representations, such as text.  This may lead, in fact, to more inaccuracy, as errors are amplified by the nature of the immersion involved.

Their summation indicates that children are affected by false narratives, even in a passive viewing, non-interactive manner, with the IVET.  This indicates a predictable effect on people of experiencing false stories in IVETs, and leads to the conclusion that such environments should be used with caution, but that they hold promise in enabling better recall, if the stimuli are accurate.

#ImportantRefs#

Paper: Augmenting and assisting model elicitation tasks with 3D virtual world context metadata

Just had a paper I have written with Stefanie Rinderle-Ma, Simone Kriglstein and Sonja Kabicher-Fuchs accepted for COOPIS 2014.  Paper is found here.

This is the fourth paper I have had published this year with Simone, part of a productive collaboration with my Austrian colleagues.

Abstract. Accurate process model elicitation continues to be a time consuming task, requiring skill on the part of the interviewer to extract explicit and tacit process information from the interviewee. Many errors occur in this elicitation stage that would be avoided by better activity recall, more consistent specification methods and greater engagement in the elicitation process by interviewees. Theories of situated cognition indicate that interactive 3D representations of real work environments engage and prime the cognitive state of the viewer. In this paper, our major contribution is to augment a previous process elicitation methodology with virtual world context metadata, drawn from a 3D simulation of the workplace. We present a conceptual and formal approach for representing this contextual metadata, integrated into a process similarity measure that provides hints for the business analyst to use in later modelling steps. Finally, we conclude with examples from two use cases to illustrate the potential abilities of this approach.

Ross

Info: 3D Virtual Worlds to Save Qld Wallabies!

Attached is a picture of me wearing an Occulus Rift wandering around the (virtual) hills of South-east Queensland. In collaboration with Prof. Kerrie Mengersen@QUT, Justine Murray@CSIRO and Prof Peter Bruza@QUT, we have developed a prototype tool for eliciting expert knowledge from environmental managers using elevation models of natural environments.

Research hypothesis is that the Bayesian wallaby population models will be of a better quality if extracted with greater situational information provided by the interactive virtual world. Normally the process is performed with 2D GIS systems.

Will be testing the prototype soon with environmental experts. Thanks to Kerrie Mengersen, Michael Rosemann and Peter Bruza for the support.  Thanks to the bods at QUT VISER for the usual Star Wars equipment.

BTW, the prototype has been developed by a third year QUT games undergraduate RA, Wesley Heard.

Ross

Video: Virtual World S-BPM Elicitator

Video of S-BPM based virtual world modelling tool developed by my Honours student Joel Harman with the financial assistance of Metasonic, AG - http://www.metasonic.de/

The tool allows a stakeholder to use a 3D representation of their work place to provide activity and message information to create an S-BPM model.

This was presented and tested in focus groups at the S-BPM One conference in Eichstaett over Easter.

Well done Joel! Thanks again to Metasonic for the provision of a scholarship for Joel and travel money to Eichstaett.

Ross

Paper Review: The effect of priming pictures and videos on a question-answer dialog scenario in a virtual environment

#Title#The effect of priming pictures and videos on a question-answer dialog scenario in a virtual environment.
#Authors#Qu, C., Brinkman, W.-P., Wiggers, P., & Heynderickx, I.
#Venue#Presence Teleoperators and Virtual Environments, 22(2), 91-109
#Doi#doi:10.1162/PRES_a_00111

#Abstract#

Having a free speech conversation with avatars in a virtual environment can be desirable in
virtual reality applications such as virtual therapy and serious games. However, recognizing and
processing free speech seems too ambitious to realize with the current technology. As an
alternative, pre-scripted conversations with keyword detection can handle a number of
goal-oriented situations as well as some scenarios in which the conversation content is of
secondary importance. This is, for example, the case in virtual exposure therapy for the
treatment of people with social phobia, where conversation is for exposure and anxiety arousal
only. A drawback of pre-scripted dialog is the limited scope of user’s answers. The system cannot
handle a user’s response, which does not match the pre-defined content, other than by providing
a default reply. A new method which uses priming material to restrict the possibility of the
user’s response is proposed in this paper to solve this problem. Two studies were conducted to
investigate whether people can be guided to mention specific keywords with video and/or picture
primings. Study 1 was a two by two experiment in which participants (n = 20) were asked to
answer a number of open questions. Prior to the session, participants watched priming videos or
unrelated videos. During the session, they could see priming pictures or unrelated pictures on a
whiteboard behind the person who asked the questions. Results showed that participants tended
to mention more keywords both with priming videos and pictures. Study 2 shared the same
experimental setting but was carried out in virtual reality instead of in the real world.
Participants (n = 20) were asked to answer questions of an avatar when they were exposed to
priming material before and/or during the conversation session. The same results were found:
the surrounding media content had a guidance effect. Furthermore, when priming pictures
appeared in the environment, people sometimes forgot to mention the content they typically
would mention.

#Comments#

Keyword dialogs with priming as pragmatic solution to NLP short comings.

Seeks to use the virtual world to give an illusion of free will, ironically, primed by visual content.  What does this say about our sense of free will.  Hmmmm.

Useful social phobia application, doe it hold that visual priming will work when obtaining information from experts in the field?  I think it does, due to the obvious priming effect.  If the avatar is an expert, then one can surmise that a person in the world could use the virtual world priming to provoke relevant questions in world, and extract better information from the participant in the interview.

They state they are the first in this place; my work on this topic is a close second! Yay!

VRET is considered as good as "in-vivo"; I prefer the word "in-situ" from my perspective.

They do not seem to be across the latest developments in NLP and speech recog.; their main citation is from 2000!

Igroup Presence Questionnaire (IPQ) (Schubert, Friedmann, & Regenbrecht, 2001). can be used to assess immersion levels for effects by oculus rifts etc. with an online database for comparison - NOTE THIS FOR FUTURE WORK!

They have found a significant effect with priming via images and videos - more keywords were mentioned with real setting and with virtual setting.  Results were significant.

Note, their results are with an immersive HMD; need to see if effects transfer to less immersive environments such as laptops, tablets etc.  Would be an interesting experiment.

They note issues with the lack of realism of the virtual world, and its possible noise effect on the differences.  However, one would assume that a poorly modelled world giving a good outcome for this work should indicate that the realism is above threshold for the priming effect.

They found that people still gave common answers to questions, even with the priming effects with videos.  Pictures seemed to suppress common answers from the participants.

#ImportantRefs#

So, from my perspective, this work supports strongly the idea that conversations can be affected by the priming effect of virtual world content.  Thus, I can hypothesise that using virtual worlds for process model elicitation should be positively affected by a representation of the interviewee's workplace.  The expected priming effects should occur.

Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society of London - Series B: Biological Sciences, 364 (1535), 3549–3557.

Pena, J., Hancock, J. T., & Merola, N. a. (2009, September). The Priming E↵ects of Avatars in Virtual Settings. Communication Research, 36 (6), 838–856.
Schubert, T., Friedmann, F., & Regenbrecht, H. (2001). The Experience of Presence: Factor Analytic Insights. Presence Teleoperators Virtual Environments, 10 (3), 266–281.

Video: Virtual World as a Process Modelling Cultural Probe

This is a video of a virtual world we have developed for eliciting expert information from stakeholders. The intention is that the virtual world prompts the user to remember more about their work processes. Our example shows a sparse visualisation of the University of Vienna Department of Computer Science.  This part of a collaboration with Simone Kriglstein, Sonja Kabicher-Fuchs and Stefanie Rinderle-Ma at the University of Vienna.

The video can be found here.

Ross