Within the last decade, we have seen the likes of Oculus and HTC wipe out the existence of head mounted displays (HMDs) such as the VST-2002 (COASTAR) and MREAL (HM-A1) by Canon or the xSightTM HMDs by Sensics. What was meant purely for research at Universities or at large companies with high end budgets has suddenly become a consumer device, with costs dropping from the tens of thousands into the hundreds. When the power and entertainment value of Virtual Reality (VR) technology is coupled with affordability, it seemed only natural to see a widespread adoption of the technology – one of Facebook’s rationale for acquiring Oculus at ~$2B. But 4 years on, despite promises and the belief that “the VR wave is really here”, we are yet to see applications beyond gaming in the consumer world, and a trickle of adoption for training in the business world. While experts and people will view the reasons for this through different lenses (most of which are likely true and hard to argue against), it remains a fact that we are social beings and any technology that targets mass adoption must yield an increased social value while preserving convenience. The radio, the television, the smart phones, the internet and a host of other technologies have consistently proven their value by evolving to increase social connectivity while preserving convenience. And hardware manufacturers of VR technologies are striving to do the same. The difference however lies in the fact that they are dependent on the effectiveness of the content that is best suited to and delivered via VR. There is an added complication - while most other technologies augment our senses, VR attempts to effectively replace some of them. And there-in lies the challenge.
I recall developing applications for VR during my academic days and studying their effectiveness over other mediums. Hardware advancements clearly helped create better experiences. But the term “better” and “effectiveness” only had meaning when the context of the application was taken into account. The effectiveness of a VR experience is quantified by the degree to which each of our senses (sight, sound, smell, touch & taste) are encompassed by the experience. The illusion of reality must be created, preserved, and fostered by the experience. When a VR experience attempts to replace any of our real-world senses, it is critical that this replacement be nearly as good as what we may experience in the real world. A personal favorite that captures this concept is Mel Slater’s work on Situational Plausibility and Place Illusion.
​
When designing a VR experience with an established purpose, ask yourself the following questions:

• Does your experience require at least one of the 5 senses to be fully replaced?
• Which remaining senses have a co-dependence on the replaced sense?
• Does the technology affordance allow you to effectively replace the primary and the co-dependent senses?

To best illustrate the above, I can’t help but refer to one of the earliest experiments in Virtual Reality termed “The Pit”. I have experienced several variations of this, but probably the best rendition I have ever witnessed is at the Stanford Virtual Human Interaction Lab run by friend and mentor, Jeremy Bailenson. The essence of “The Pit” is this – would you be willing to walk off a plank, at an unknown height, into a deep dark pit, knowing fully well that the experience is being created in virtual reality?

What makes this particular rendition so effective is the attention to detail that Jeremy and his colleagues have paid to the replacement of the senses. The technical barriers to achieving this are many including but not limited to sensor tracking rates, sensor spatial accuracy, latency, refresh rates of the rendering device, resolution, field of view and frame rate among others. At VHIL, all of these have not only been overcome, but also augmented by the addition of synchronized haptic floor shakers, auditory cues and stunning visuals using a variety of VR headsets. In essence, when you are experiencing “The Pit” in VR at VHIL, your sight, sound and touch have effectively been replaced. And there is synchrony and co-dependence between them in the context of the experience. You can see the pit, you can hear the floorboards under you creak as you get closer to the edge, and you may even feel the difference in haptics under your feet as you walk along the edge of the pit – all reinforcing the belief that there is a real pit only a few inches away from where you stand. This is what makes the experience seem nerve-wrecking and very real – in other words, situationally plausible while preserving place illusion.
At Mursion, we follow similar design principles when recreating the most challenging situations encountered on the job. Only, Mursion is not an isolated experience like “The Pit”. It is social, allowing people to interact with avatars, just as they would with other people – through spoken dialogue. Artificial Intelligence (AI) in general and Natural Language Processing in specific is simply not nuanced enough to mimic human conversation (I’ll write more about this in future blogs). We overcome this by augmenting AI with human reasoning captured via live human performance. We focus on the effective replacement of sight and sound in addition to adding a social component to create, foster and preserve the intricacy of human interaction. The term effective is associated with technological nuances beyond the scope of this blog. But, the end result is a highly engaging platform that places learners in different situations – unique events specified by location, time, and circumstance. This exposure to various situations (called multi-exemplar training) facilitates the acquisition of soft skills through intentional practice – or experiential learning. And we are beginning to study just how effective these simulations can be at helping enhance interpersonal skills across various industries.
​
Virtual Reality can be very exciting and powerful - provided we take the time to thoroughly understand its purpose, recognize the limitations and respect the design constraints given our natural ways of interaction with the world. When experiencing a situation in virtual reality, your conscious self is completely aware that the experience is not real. But is it strong enough to overcome your sub-conscious instincts, driven in part by the processing of sensory signals by your brain? If you’re a VR enthusiast and are in the Bay Area, do yourself a favor and sign up for a tour at VHIL to find out. And while you’re at it, come visit us at Mursion, so we can show you what we’re up to.