Thursday, May 14, 2015

Paper Review: Situating Cognition within the Virtual World

Situating Cognition within the Virtual World

Paul R. Smart and Katia Sycara

6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the 
Affiliated Conferences, AHFE 2015


Cognitive architectures and virtual environments have a long history of use within the cognitive science community. Few studies, however, have sought to combine the use of these technologies to support computational studies into embodied, extended, situated and distributed (EESD) cognition. Here, we explore the extent to which the ACT-R cognitive architecture and the Unity game engine can be used for these purposes. A range of issues are discussed including the respective responsibilities that the cognitive architecture and game engine have for the implementation of specific processes, the extent to which the representational and computational capabilities of cognitive architectures are suited to the modeling of EESD cognitive systems, and the extent to which the kind of embodiment seen in the case of so-called ‘embodied virtual agents’ resembles that seen in the case of real-world bio-cognitive systems. These issues are likely to inform the focus of future research efforts concerning the integrative use of virtual environments and cognitive architectures for the computational modeling and simulation of EESD cognitive processes. 

An interesting little paper covering some issues around integrating Unity with present situated cognition systems such as ACT-R and THESEUS - their own simulation framework.

Not much theory here, but worth noting regarding the utility of processing visual features in the environment into abstract representations suitable for ACT-R to process.

Interesting observation, they quote Clark "Clark [1] thus distinguishes between ‘mere embodiment’, ‘basic embodiment’ and ‘profound embodiment’. He suggests that profound embodiment is primarily a feature of bio-cognitive systems, and that this form of embodiment is unlike that seen in the case of synthetic agents."

My point here is that, this applies to humans in the space.  Using certain interfaces should increase this level of embodiment.  A good idea would be to create a scale for this, so that researchers can assess the experience of embodiment, to maybe predict responses in humans.

They note the difficulty of integrating EESD forms of situated cognition into games systems.  Which I find interesting.  Why don't they form embodied scripts in the environments which are trained to interact with each other in a similar way to models of situated cognition, regarding our bodily movements?  Sounds like PhD project to me.  Surely these have been used in robotics, so should translate to a computational architecture easily, and, with Unity's nice scripting setup, should be able to be executed relatively easily, but only on a powerful machine. :-)


1. Clark, A. Supersizing the Mind: Embodiment, Action, and Cognitive Extension, Oxford University Press, New York, New York, USA, 2008

2. Smart, P. R., Scutt, T., Sycara, K., Shadbolt, N. R. in: Turner, J. O., Nixon, M., Bernardet, U., DiPaola, S. (Eds.) Integrating Cognitive Architectures into Virtual Character Design, IGI Global, Hershey, Pennsylvania, USA, in press.

Thursday, April 30, 2015

2nd CFP: TAProViz 2015 : 4th International Workshop on Theory and Application of Visualizations and Human-centric Aspects in Processes

TAProViz’15 4th International Workshop on Theory and Application of Visualizations and Human-centric Aspects in Processes, Innsbruck, Austria - 31 August 2015

In conjunction with the 12th International Conference on Business Process Management BPM2015 - at Innsbruck, Austria.

Call for Papers

Visualizations can make the structure and dependencies between elements in processes accessible in order to support users who need to analyze process models and their instances.

However, effectively visualizing processes in a user-friendly way is often a big challenge, especially for complex process models which can consist of hundreds of process components (e.g., process activities, data flows, and resources) and thousands of running process instances in different execution states.

Many challenges remain to be addressed within the broad area of process visualization, human interaction and user led design such as: scalability, human-computer interaction, cognitive aspects, applicability of different approaches, collaboration, process evolution, run-time requirements of process instances and applications, user-engagement etc.

Topics of interest include (but are not limited to):
* Visual Metaphors in Processes
* Visual Design and Aesthetics for Processes
* Visualization of Dynamic Data in Processes
* Change Visualization for Processes
* Interface and Interaction Techniques for Process Visualization
* Visualization Techniques for Collaboration and Distributed Processes
* Visualization of Large-scale Processes
* Cognition and Perception in Process Visualization
* Evaluation and User Studies of Process Visualization
* Evaluation Methods for Human Aspects in PAIS
* Visual Modeling Languages
* Analysis Techniques and Visualization for Processes
* Process Visualization of Large Screens
* Mobile Process Visualization
* Visualization Tools and Systems for Processes
* Visualization Techniques for Processes
* Process Visualization and Sonification
* Virtual World Process Visualization
* Immersive Process Modeling Approaches
* Human Computer Interaction Design Applied to Process Systems
* 3D Process Visualization Approaches
* Human-centric aspects in business process management
* User-centered design for BPM
* User Interface design for Processes

Format of the Workshop

The half day workshop will comprise accepted papers and tool evaluations. Papers should be submitted in advance and will be reviewed by at least three members of the program committee.

This year will also include a new innovation in the programme. Part of the workshop time (depending on the number of prototype submissions) will be set aside for focus group assessments of tools. We will be requesting tool report authors, successful workshop paper authors and panel members attending BPM, to assist in the assessment of demonstration visualization techniques and software. This evaluation process will be a service to attendees, as these heuristic assessments can be written up later as separate papers, or by the workshop chairs as an aggregated workshop outcome. Such evaluations will be an exciting addition to the workshop, as people experienced in Information Visualization, BPM, HCI and related fields, will provide detailed feedback on your prototypes. The evaluation approach is largely in the hands of the tool report writers, but at a minimum, should involve direct interaction with your software and some form of validation via a questionnaire.

All accepted papers will appear in the workshop proceedings published by Springer in the Lecture Notes in Business Information Processing (LNBIP) series. There will be a single LNBIP volume dedicated to the proceedings of all BPM workshops. As this volume will appear after the conference, there will be informal proceedings during the workshop. At least one author for each accepted paper should register for the workshop and present the paper.

Important Dates

* Deadline for workshop paper submissions: 29 May 2015
* Notification of Acceptance: 29 June 2015
* Camera-ready version: 20 July 2015
* TAProViz Workshop: 31 August 2015

Paper Submission

Prospective authors are invited to submit papers for presentation in any of the areas listed above.

Three types of submissions are possible:

* (1) full papers (12 pages long) reporting mature research results
* (2) position papers reporting research that may be in preliminary stage that has not yet been evaluated
* (3) tool reports, to be evaluated at the workshop

Position papers and tool reports should be no longer than 6 pages. Tool reports should include a brief evaluation plan as an appendix, for the evaluation session at the workshop on the day.

Papers must be in English and must present original research contributions not concurrently submitted elsewhere. Papers should be submitted in the LNBIP format. The title page must contain a short abstract, a classification of the topics covered, preferably using the list of topics above, and an indication of the submission category (regular paper/position paper/tool report).

All accepted workshop papers will be published by Springer as a post-workshop proceedings volume in the series Lecture Notes in Business Information Processing (LNBIP). Hard copies of these proceedings will be shipped to all registered participants approximately four months after the workshops, while preliminary proceedings will be distributed during the workshop.

Submitted papers will be evaluated, in a double blind manner, on the basis of significance, originality, technical quality, and exposition. Papers should clearly establish their research contribution and the relation to the theory and application of process visualization.


Accepted papers imply that at least one of the authors will register for BPM2015 and present the paper at the TAProViz workshop.

Further workshop information is available from the website:

Hope to see you at TAProViz'15!

Thanks and best regards,

Ross Brown
Simone Kriglstein
Stefanie Rinderle-Ma

TAProViz Organising Committee

Wednesday, April 8, 2015

Paper: Process visualization techniques for multi-perspective process comparisons

Have just had the paper "Process visualization techniques for multi-perspective process comparisons" accepted for AP-BPM, written with Azzurra Pini (Politecnico Milano), and Moe Wynn (QUT).  This is an outcome of Azzurra's work as an intern with us at QUT in 2014.  the paper is stored here, please contact me if you want a copy.

Abstract - Organizations executing similar business processes need to understand the differences and similarities in activities performed across work environments. Presently, research interest is directed towards the potential of visualization for the display of process models, to support users in their analysis tasks. Although recent literature in process mining and comparison provide several methods and algorithms to perform process and log comparison, few contributions explore novel visualization techniques. This paper analyzes process comparison from a design perspective, providing some practical visualization techniques as analysis solutions. In order to support the needs of business analysts the design of the visual comparison has been tackled via three different points of view: the general model, the superimposed model and the side-by-side comparison. A case study is presented showing a preliminary evaluation of the application of process mining and visualization techniques to patient treatment across two Australian hospitals.


Sunday, April 5, 2015

Paper Review: A Quantum Information Retrieval Approach to Memory

A Quantum Information Retrieval Approach to Memory

Kirsty Kitto, Peter Bruza, Liane Gabora

Neural Networks (IJCNN), The 2012 International Joint Conference on

DOI: 10.1109/IJCNN.2012.6252492

Abstract—As computers approach the physical limits of in- formation storable in memory, new methods will be needed to further improve information storage and retrieval. We propose a quantum inspired vector based approach, which offers a contextually dependent mapping from the subsymbolic to the symbolic representations of information. If implemented computationally, this approach would provide exceptionally high density of information storage, without the traditionally required physical increase in storage capacity. The approach is inspired by the structure of human memory and incorporates elements of Gaerdenfors’ Conceptual Space approach and Humphreys et al.’s matrix model of memory.

This paper detail's Bruza, Kitto and Gabora's matrix model of memory.  Peter Bruza is also a colleague of mine at QUT. :-)

The key components I find interesting are the relationship between symbolic and subsymbolic levels, and its movement towards a distributed overlaid model of memory, that causes excitations in related symbolic entities/terms.  The matrix notation allows for a representation of context as a matrix of features, which via tensor product formalism,  allows for memory to be a distributed process, with activations of related terms.

This is of interest to me, as we can start to utilise visual features in a computational model of subsymbolic components contributing to memory recall via similar inputs of features.

While this is strongly related to an NN associated matrix approach, it makes the relationships between the components explicit, and is thus a candidate as a cognitive model of priming in expert elicitation sessions.  This approach can then be used to modulate user interfaces in virtual world elicitation systems.


Thursday, March 19, 2015

Videos/Code: Free Unity Shader Demos from my Rendering Course @ QUT

I've made videos and Unity project files from my teaching at QUT available on my Youtube channel  The playlist is here.  Enjoy and feel free to pass onto interested people.

NB: If you want to know about the theory, come and do my course INB382 Real-time Rendering Techniques at QUT. ;-)


Sunday, March 8, 2015

Review: Imagination Inflation: Imagining a Childhood Event Inflates Confidence that it Occurred

Imagination Inflation: Imagining a Childhood Event Inflates Confidence that it Occurred

Maryanne Garry, Charles G. Manning, Elizabeth F. Loftus

Psychonomic Bulletin & Review
3 (2), 208-214


Counterfactual imaginings are known to have far reaching implications. In the present experiment, we ask if imagining events from one's past can affect memory for childhood events. We draw on the social psychology literature showing that imagining a future event increases the subjective likelihood that the event will occur. The concepts of cognitive availability and the source monitoring framework provide reasons to expect that imagination may inflate confidence that a childhood event occurred. However, people routinely produce myriad counterfactual imaginings (i.e., daydreams and fantasies) but usually do not confuse them with past experiences. To determine the effects of imagining a childhood event, we pretested subjects on how confident they were that a number of childhood events had happened, asked them to imagine some of those events, and then gathered new confidence measures. For each of the target items, imagination inflated confidence that the event had occurred in childhood. We discuss implications for situations in which imagination is used as an aid in searching for presumably lost memories.

Am strongly interested in the effect of false memories on the process of expert elicitation.  In particular, the stimulus used in the process of extracting the information from experts.

So, in this paper, I am interested in the quality of the inputs into a session.  Could one plant false memories of the work or knowledge using such systems.  Could it be that using a virtual world to perform the elicitation, if incorrectly configured, will introduce more errors, due to the creation of powerful false memories from the visuals created.  Hmmmm.

Interesting to note that the effect is easier with early childhood memories - attributed to vagueness of distant memory.  This is also possibly related to the credulity of young children; do you become a little childlike by remembering your childhood?  If the effect is consistent, this opens up all sorts of possibilities for creation of beneficial false memories, or to reduce the effects of bad environments (GTA 5 comes to mind) by making sure the rules in such environments make moral sense, removing the effect of such false memories induced by gameplay.

So what is War Thunder doing to me?  Do I know have false memories of driving a Tiger I?

The weird part would be the eerie familiarity of driving the Tiger in real life, the eerie familiarity is my false memory of driving in a game, but I experience familiarity in real life.  Would I discern the difference, can I, as the familiarity is beyond my control to an extent.

Interestingly enough, this experimental method almost reads like an elicitation session.  The experiment manager states to the participant: "What are you likely to do next" while imagining the false event.  They are, in concept, creating a false sequence of events or episodes in the memory.

Note, the early estimate of past memory was repeated after the experimenter has faked losing their results - might be dodgy, could people see through this.  I wonder if they controlled for insight into the ruse; it is not noted in the description of results.

So, the data showed a consistent increase in confidence of remembering the fake event, especially after imagining the event (personal VR :-) ).  They controlled for big jumps, conjectured to be actual priming of actual lost memories; this is important in elicitation.  They also note that the number of big jumps is small.  Also note, that people who did not imagine between tests still went up, but not as much.  Another effect in play, maybe regression to the mean, or just a familiarity effect.  These effects are important in any elicitation test; just repeating questions may bring about a false memory - this is what cops and psychologists do.

They also bring up the issue of self being in the imagination session.  Brings up the idea that an avatar should represent the person who is doing the elicitation, to bring the participant a sense of performing the task in world; might increase the concept of presence.


Paper: Model as you do : engaging an S-BPM vendor on process modelling in 3D virtual worlds

Have recently had a book chapter "Model as you do : engaging an S-BPM vendor on process modelling in 3D virtual worlds," accepted.  The paper was written with Joel Harman, my Honours student from QUT, and Udo Kannengiesser, Nils Meyer and Thomas Rothschaedl from Metasonic GmbH.  It describes the processes Joel and I went through to implement a virtual world process elicitation tool in conjunction with Metasonic in Germany.  The chapter will be published in "In S-BPM in the Wild – Value Creating Practice in the Field," Springer, Berlin Heidelberg.

QUT eprints entry is here, email me on if you want a copy.

Abstract: Accurate process model elicitation continues to be a time consuming task, requiring skill on the part of the interviewer to extract explicit and tacit process information from the interviewee. Many errors occur in this elicitation stage that would be avoided by better activity recall, more consistent specification methods and greater engagement in the elicitation process by interviewees. Metasonic GmbH has developed a process elicitation tool for their process suite. As part of a research engagement with Metasonic, staff from QUT, Australia have developed a 3D virtual world approach to the same problem, viz. eliciting process models from stakeholders in an intuitive manner. This book chapter tells the story of how QUT staff developed a 3D Virtual World tool for process elicitation, took the outcomes of their research project to Metasonic for evaluation, and finally, Metasonic’s response to the initial proof of concept.