Missing the Point: An Exploration of How to Guide Users’ Attention During Cinematic Virtual Reality
2016
Unity · Maya · Oculus · MATLAB
A published study of attention guidance in cinematic VR, comparing a firefly, forced rotation and free viewing with 45 participants.
- My role
- Research design, Unity development and statistical analysis
- Context
- Academic research
- Status
- Published research · ACM VRST 2016
How do you guide attention through a story when the viewer controls where they look? I co-authored a study at Aalborg University Copenhagen exploring this question in cinematic virtual reality. The work was published as a short paper in the Proceedings of the 22nd ACM Symposium on Virtual Reality Software and Technology (ACM VRST 2016).
We developed a taxonomy of attention cues and built a Unity experience to compare three conditions: guidance by a firefly, forced rotation of the viewer’s virtual body, and no guidance. Forty-five participants experienced one condition each, with 15 participants per group. We evaluated presence, perceived guidance and recall of story elements.
Explore the scene in 360°
For the study, participants explored the Unity experience using an Oculus Rift DK2. After the study, we made this 360° recording to share the experience with people without a headset. The recording is monoscopic, without the stereoscopic depth of the headset version.
The recording shows the Firefly condition. The firefly first approaches the viewer to attract their attention through visual and auditory cues, then flies towards the next story event. The experience also uses spatial audio, lighting and animation to guide attention within the narrative.
Look around in the YouTube app
For 360° viewing on a phone or tablet, use the YouTube app. Move your device or swipe to look around. If the link opens in your browser, choose “Open in app” when offered.
Watch on YouTubeDrag the control to start the video and look around.
Building and testing the VR experience
I contributed to the research question, study design, data collection, testing, statistical analysis and co-authorship of the paper. My implementation work connected the experience participants saw with the data we could analyse.
I developed the Unity mechanics used in the experiment and wrote MATLAB code to process headset-tracking data for analysis. I also created 3D models and shading assets in Maya, and filmed and processed the green-screen footage integrated into the scene.
Findings and relevance
The firefly was rated significantly more helpful than forced rotation for guiding attention. Presence ratings tended to be lower with forced rotation, suggesting a possible trade-off between directing attention and preserving the sense of being there, although the differences were not statistically significant.
We found no statistically significant differences in perceived ease of following the story, and recall of story elements was similar across conditions. Higher helpfulness ratings did not translate into better recall of story elements in this study.
One practical lesson was the importance of timing: if a cue arrived too early, viewers could look away again before the story event began. Guiding attention meant coordinating the cue, the action and competing elements in the scene.
I still use several of these techniques in XR development today, combining spatial audio, lighting and animation to guide attention within an immersive environment.