Storytelling in VR - The Paradox of Presence

Body: 

The goal for most VR practitioners is 'presence'. This is the feeling of being 'there', in other words being-in-a-world, which from a technical VR perspective means being immersed in 360 degrees of 3D information with ambisonic sound, and perhaps even a 4th dimension, such as touch or smell.

Yet despite these efforts, the emotional engagement produced by narrative VR remains, in general, less than in traditional film, where key aspects of character and plot can be conveyed in visceral close-ups or sequences of medium shots that inject pace and drama into a scene.

Somehow, when it comes to storytelling, presence comes up short.

In traditional cinema, one looks forward (toward the screen) and is told a story that plays out within a rectangular frame. By contrast, VR has been described as ‘reverse theater-in-the-round' with the viewer standing at the center of a 360° radius of action. The difference is profound.

In VR, the viewer feels in the scene, not apart from it, inhabiting something akin to a permanent point-of-view shot. Until something happens to contradict this sensation, the default narrative stance of VR is therefore 'first-person'. One feels like a participant in the story because one is literally 'there', and seemingly capable of doing the main things humans have done since time immemorial, which is to look around and investigate.

Many VR filmmakers play to this intuition, by having secondary characters address the viewer/camera, or by plot devices that treat the VR user’s point-of-view (the camera position) as an explicit point-of-view shot. Most VR films, for instance, play out at head height, simulating a human point-of-view. But even when the camera is raised or lowered, one still feels appreciably 'there' - a literal fly-on-the-wall.

From a narrative point-of-view, there are some problems with this. Firstly, in traditional cinema, a POV shot is never under one’s control. When a character addresses you, you cannot ‘look away’ as you can in a VR film. You can look away from the screen, of course, whilst seated inside the cinema theatre (effectively opting out of the movie) but this is not an option available to the VR user.

This sets up a basic cognitive dissonance that VR has yet to resolve, namely that one feels a physiological sense of presence at the same time as a mechanic of detachment that allows you to act (move/look) as if you were NOT truly present. In a VR film, one can ignore social interactions or significant plot points without any appreciable social cost, which is not like real life at all.

Moreover, this dissonance does not disappear when the VR filmmaker attempts to shift narrative stance from the first to the third person. Even as a fly-on-the-wall (or a mouse-on-the-floor), the viewer feels like a participant in the scene, albeit an invisible one. Some VR pundits talk about the Swayze or Ghost effect in VR, which is the feeling that one is a disembodied ghost, floating (usually at head height) in each scene, wondering why no one is looking in one’s direction.

Another factor contributing to this uneasy storytelling paradigm is the basic identification process with other characters. In a linear 2D film told in the third-person, one lives vicariously through the characters depicted. There is a clear sense of separation that allows for situations-in-extremis. One empathises with the hero. And although one experiences their trials and tribulations second-hand, their jeopardy – whether hanging from cliff-face or squaring off with an assassin - becomes our own, through well established psychological processes of empathy and identification.

But if we are watching the story unfold from a position that is simultaneously IN the story, we experience a strange conjunction of first- and third-person perspectives. We are ‘there’ and yet we cannot help or interact with the hero. We also may feel vulnerable ourselves – for what if we take a tumble off the cliff, or the assassin turns on us?

This dissonance works against engagement, rather than in its favour, promoting a kind of existential confusion that may actually inhibit empathy.

Presence therefore seems to carry these built-in contradictions. If one is there, capable of looking around, implicitly capable of agency, then why is no one acknowledging you? Or why, in a story that is purportedly unfolding in front of you, does one feel vulnerable and ‘in jeopardy’. Besides the lack of pace and drama that I have argued is principally due to restrictions on editing, VR is also bedeviled by this second storytelling problem, which may be described as the paradox of presence - being there but not being there.

The solution to this contradiction is to find a way to return an authentic third-person narrative perspective to VR. This idea is counter-intuitive, working at cross-purposes to the ideal of presence that guides so many VR storytellers. But stories are not ‘life’ – they are mediated experiences, and depend on detachment and vicarious identification with other characters (typically a central ‘hero’), or they are not stories at all.

This is not to say that a VR film should not contain moments of presence, merely that these should be used selectively, to emphasize and enhance the central story, with it’s archetypal hero’s journey. Key plot points and dramatic turning points (perhaps corresponding to a Jungian story structure with it's various gateways and transformations) are prime candidates for 'presence'.

In my next article, I will discuss how a new, hybrid form of VR might return the ‘frame’ to VR storytelling, achieving both an increase in pace, drama and jeopardy, while not sacrificing presence as a powerful phenomenon to be selectively employed.

Storytelling in VR - on Framing

Body: 

Grammatically speaking, a key difference between VR and traditional film is the absence of a frame in VR. This relates to the problem of pace and drama (discussed in a previous blog), insofar as the frame allows for editing – specifically, sequences of wide, medium and close-up shots that tell the story.

But cutting in VR is inherently problematic. Besides the viewer needing more time to adjust to their surroundings (imposing a slower pace on every scene) the mere act of cutting is often disorienting. A sudden change, even within a scene, may not map aesthetically onto the previous shot. Depending on where the viewer has been looking and the rate and direction of their head-movement, a change in perspective is likely to jar. Even when this is kept to a minimum, each cut necessarily imposes a new 360 degree view, meaning that the viewer will need to make sense of context all over again.

Another related problem is that of the close-up. In traditional film, key pieces of information are usually conveyed in close-ups – guns, tear-stained cheeks, etc. But there are no close-ups in VR (or even mid-shots), except as a side-effect of how the actors are blocked or objects placed within a scene. Here we are only talking about greater proximity to the camera, rather than a true close-up per se. For even if someone (or something) is positioned close enough to the front-facing lens so that the initial framing approximates a close-up (say, in size and detail), the shot is still, effectively, a wide-shot, just with something in the way (the foregrounded object). These kind of close-ups can feel obtrusive rather than revelatory.

All of which is why most VR films dispense with editing in the normal sense.

This lack of edit points has some unfortunate knock-on effects for production. For one thing, actors have to rehearse more intensively because their scenes play out over fewer, longer shots. In a complex scene with several actors, each actor has to nail the performance for the shot to be a success. The editor cannot hide weak moments by cutting to a reaction shot, for instance. In this respect, VR has much in common with theatre, as is often pointed out.

But crucially, even when the performers nail it, the resulting scenes still tend to lack pace and tension. Google Spotlight’s big budget action VR short ‘Help’, features a crash-landed alien monster on a rampage - a chase sequence replete with explosions and high-end CGI. But there is a perceptible absence of drama and energy. This is ironic since the immersion-in-a-world that is touted as the raison d'être of virtual reality is supposed to promote emotional engagement.

Conventional cinema, where key aspects of character and plot can be conveyed in visceral close-ups or medium shots that flash across the screen, still trumps VR when it comes to stirring the blood or sparking emotion.

To my mind, there are two solutions to this storytelling problem. One is to continue to use visual and audio cues to direct attention, enhanced by ever more elaborate sound design and exotic visual environments. The other is to find a way to return the ‘frame’ to the VR film, enabling something that approximates cinematic editing and thereby returning power to the editor and storyteller. This will be the subject of a later blog.

Storytelling in VR - The Problem of Pacing

Body: 

A central problem for VR storytellers is how to advance the plot of a VR film when the audience has the freedom to look anywhere they please.

In real life, when we find ourselves in new surroundings, we look around, and the same is true in VR. Because VR is immersive, usually encompassing a 360 degree view, it takes several seconds (at least) to acclimatize to our virtual environment.

VR storytellers have responded by making scenes longer than in traditional film, choosing not to compress the action too much, and certainly not to commence each scene, as eminent screenwriter William Goldman once suggested, as late as possible.

Instead, VR scenes typically play out in 360 degree ‘wide’ shots, often in real-time, with plenty of lead-in before the main action, so there’s time for the viewer to absorb the context of the scene, and not be distracted when important plot information is transmitted.

These shots are usually static because a moving camera in VR can result in motion sickness.

The result is long, slow, static shots that sacrifice pace and drama for orientation and comprehension.

Besides longer lead-in times, a second, related problem is how to ensure the viewer’s attention does not wander once the story-telling part of the scene has commenced. In the absence of a traditional frame, there's simply no telling where the VR user might be looking.

This challenge is usually resolved with the help of visual and audio cues, effectively guiding the viewer’s attention. Creaking doors, characters gasping and pointing, sudden shafts of light, or perhaps the curl of an animated brushstroke (as in the excellent Dear Angelica) are used to control our focus point, so we don’t miss important plot information.

To some extent, mise en scène (the arrangement of props and scenery) can perform the same function, creating ‘action areas’ for performance, sometimes accentuated by lighting, which likewise act as 'attractors' for our attention.

When there is no frame, the choice of location, camera position, lighting and set dressing all play a role in where the viewer looks, which is why the roles of Art Director, DOP and Director overlap to a greater degree in VR than in traditional film.

Such is the current state of visual grammar in VR film-making. But could there be alternative rules or grammatical principles, or even an entirely different grammar available to VR storytellers?

This will be the subject of future blogs.

Some other entries

Body: 

Sentient Pictures is a full service content strategy consultancy and media production facilitator. Current and past clients include Unilever, Ford, SAB Miller, Tiger Brands, Primedia, Media 24 (SA), Joule, Red Skin Media, Oasis Aquila Housing, One Degree, Work Ready (UK) and FireID Payments (US).

The company specialises in content strategy and oversight of media content generation, including above-the-line campaigns, integrated web-based campaigns (display, search, social), content marketing, PR and mobile apps marketing.

Historically, the company has specialized in corporate communications and inbound marketing with a bias toward social business projects, charities, NGO's and causes.

About Steven Cholerton

Body: 

My background is in television broadcasting (directing & producing) and digital strategy consulting – specifically, conceiving and directing large multi-channel media projects, often with a strong video element, across a range of digital channels.

International clients include Unilever, Heineken, SAB Miller, Ford and Naspers.

I’ve also co-founded two technology startups (in mobile payments) leading interdisciplinary teams through the product lifecycle. Whether as a television practitioner, digital consultant or entrepreneur, I’m an advocate for customer-centric thinking, incorporating the proactive use of intelligent video content to engage and inspire.