Immersive Shadowplay
Immersive Shadowplay
Concept
Immersive Shadowplay explores the possibilities of hand interactions in VR with tracked objects in a mixed reality setting and the opportunities of including the audience in an exhibition environment or a public space. Therefore it investigates the HCI in a virtual realm, as well as the reactions from the audience outside.
Immersive Shadowplay explores the possibilities of hand interactions in VR with tracked objects in a mixed reality setting and the opportunities of including the audience in an exhibition environment or a public space. Therefore it investigates the HCI in a virtual realm, as well as the reactions from the audience outside.
About
Immersive Shadowplay is a mixed reality prototype that introduces people to hand interactions with objects in a VR environment.
It explores how people respond to seeing their own hands and interact with physical objects in VR. The shadow play is bridging the gap between the audience outside the installation and the person interacting inside.
It explores how people respond to seeing their own hands and interact with physical objects in VR. The shadow play is bridging the gap between the audience outside the installation and the person interacting inside.
UX challenges
Even with an immersive technology like VR, quite often there is a lack of tactility. One of the challenges of this project was to bring that tactility into the virtual realm while having very natural hand interactions with the object. Another hurdle was to bridge the gap between the audience outside and the person inside. Quite often, this is done by mirroring screens. Therefore I wanted to find a more playful approach to reach that goal while keeping the surprise of what is happening inside the VR.
Exhibit on plinth
Hand interactions
Prototyping
For prototyping purposes, I used a leap motion sensor attached to an HTC Vive headset and Vive trackers with different base station setups.
With that framework, I’ve been setting up a variety of experiments and test environments in Unity.
With that framework, I’ve been setting up a variety of experiments and test environments in Unity.
Mirror to encourage interactions
Hand trackingInteraction design
I’ve been trying different experiments – being able to see your own hands, putting a mirror in the VR world, and having linked objects – a real one that links to a virtual one. I wanted to have a physical element, where people can touch things – this is a problem in museums as well as in VR. I would like to have a version of a real object, linked with a myth or story, that you can interact with.
I’ve been trying different experiments – being able to see your own hands, putting a mirror in the VR world, and having linked objects – a real one that links to a virtual one. I wanted to have a physical element, where people can touch things – this is a problem in museums as well as in VR. I would like to have a version of a real object, linked with a myth or story, that you can interact with.
Solution
To bring that physical element into the VR experience, I attached a Vive tracker onto an object. A 3D object with the same measures is mapped and linked onto the physical artefact. Being able to see your own hands, allows people to interact intuitively in a virtual environment without any introductions. Therefore people were enabled to pick up the object, place it on a plinth or move it around with their hands. The virtual mirror inside the VR experience just reflects the hand movements and encourages the people to move their hands. That casts interesting shadows to draw other visitors into the VR experience.
To bring that physical element into the VR experience, I attached a Vive tracker onto an object. A 3D object with the same measures is mapped and linked onto the physical artefact. Being able to see your own hands, allows people to interact intuitively in a virtual environment without any introductions. Therefore people were enabled to pick up the object, place it on a plinth or move it around with their hands. The virtual mirror inside the VR experience just reflects the hand movements and encourages the people to move their hands. That casts interesting shadows to draw other visitors into the VR experience.
Set design
Visiting many VR experiences, I noticed that something is missing. Public VR experiences can feel quite exclusive and intimidating, sometimes shared through a screen-based, flattened representation of what the person in the headset is experiencing. So I thought about how these interactions could become more attractive, by combining it with something straightforward – shadow play. I like the idea of questioning who the audience and performer is. That is very playful and takes away the feeling that you’re on a tableau because you can just see the HCI. It’s more playful approach and brings in a performative aspect connecting the person inside the VR experience with those outside.
Visiting many VR experiences, I noticed that something is missing. Public VR experiences can feel quite exclusive and intimidating, sometimes shared through a screen-based, flattened representation of what the person in the headset is experiencing. So I thought about how these interactions could become more attractive, by combining it with something straightforward – shadow play. I like the idea of questioning who the audience and performer is. That is very playful and takes away the feeling that you’re on a tableau because you can just see the HCI. It’s more playful approach and brings in a performative aspect connecting the person inside the VR experience with those outside.
Exhibition
Royal College of Art - Work In Progress Show 2019
Royal College of Art - Work In Progress Show 2019