The Taxonomy of Experience: Part 1
A universal language for categorizing IRL and XR experiences.
Introduction
People are talking more and more about AR, VR and other immersive experiences. And you may have noticed that, when discussing these extended reality (XR) experiences, the existing language we use to talk about point of view falls short.
We can do more in XR than we’ve ever been able to do in other media or even in real life. And we need new words to describe these new capabilities.
This post introduces the four main categories of point of view (POV), two of which you may be familiar with and two of which you most likely are not. (They were identified and named in 2018).
Narrative Point of View
POV is most commonly thought of in terms of narrative point-of-view: first, second or third person, which is the point of view from which the story is being told.
1st person narrative POV:
“My mom once told me that my dad had given me an alliterative name, Wade Watts, because he thought it sounded like the secret identity of a superhero.”
- Ernest Cline’s Ready Player One
2nd person narrative POV:
“You stare at the envelope with your name written in faded ink. The lawyer handed it to you an hour ago after he read your grandfather’s will.”
- R.A. Montgomery’s The Trail of Lost Time
3rd person narrative POV:
“Harry’s mind wandered a long way from the marquee, back to afternoons spent alone with Ginny in lonely parts of the school grounds.”
- J.K. Rowling’s Harry Potter & the Deathly Hallows
Every XR experience has narrative POV, and it may or may not be consistent the visual POV of that same experience. After a brief review of Visual POV, we’ll discuss how visual and narrative POV may or may not “agree” in XR.
Visual Point of View
Film and other visual media such as television and video have visual point-of-view, the POV from which the story is being seen. This is similarly broken down into three categories:
1st person visual POV:
The viewer is seeing the action from the POV of a character in the film, show or game, as demonstrated in the above still from Duck Hunt, in which the player is the hunter in the game, seeing the crosshairs of the rifle the hunter is looking through.
2nd person visual POV:
The viewer is acknowledged by the medium:
“Oh hey, yup, this has an audience and we know it!”
Sometimes this is done fantastically, by the character breaking the fourth wall and talking to the audience, sometimes it’s done logically with the conceit of a documentary film crew: as is the case in the television show The Office and even in the video game Mario 64, where the player sees action from the POV of Latiku, a monster and cameraman, while still controlling Mario. The viewer is straddling the worlds.
3rd person visual POV:
The viewer is not meant to be a character in the film, show or game, nor are they acknowledged by the medium. They are on the outside looking in.
Effectual Point of View
When we start talking about extended reality (XR) content—which includes video games, interactive television, virtual reality and more—there is the question of what effect the viewer’s presence has on the scene.
In the simplest terms, can the person experiencing an immersive work impact it? If so, to what extent? This impact is described by an experience’s effectual point of view, which can be broken down into three categories:
Non-Entity:
The viewer is an invisible observer. The viewer is present in the scene, but n one and nothing in the scene can tell. The viewer has no impact on the scene and it’s as if the viewer is not even there.
One example is Baobab Studios’ Crow: The Legend. In this story, the viewer is not Crow nor any of the characters Crow encounters. Rather, the viewer is sometimes standing on the ground, sometimes floating in the air, and never looked at. The viewer is not a character; moreover, the viewer is not acknowledged, addressed or given any indication that they exist.
Entity:
The viewer is acknowledged and interacted with but has no agency. Characters in the story or scene are aware of the viewer’s presence — they see the viewer and they may interact with the viewer. In this way, the viewer has an impact on the scene even though they can’t actively make choices that affect it.
One example of entity POV is Exelauno’s Human/Art/Object, in which the viewer gets to experience what it is like to be a work of art in an art gallery.
Participant:
The viewer has agency. They feel like — and are— somebody or something that can do things. And these things are perceived by or affect the scene. The viewer is no longer just a “viewer.”
One example of participant POV is the popular VR rhythm game Beat Saber. The player uses their saber to hit the boxes and, when they do, the box goes away, the player’s score goes up and, ultimately, the player can win or lose.
Experiential Point of View
With extended reality (XR) content, there is also the question of whether the viewer can impact how they experience the scene or story. This is described by experiential point of view, which can be broken down into three categories:
Robot:
The viewer’s experience is programmed. The viewer can’t control it. This includes any experience where the viewer feels like they’re moving but isn’t actually walking or in/on a vehicle.
One example of this experiential POV is Breaking Fourth’s BroBots, a four-part series available on SamsungVR in which two British robots, Otis and Roberto, arrive in New York and join the NYPD.
A close approximation of robot experiential POV in real life would be when you’re on a moving sidewalk—you’re “walking” but you’re not actually walking.
Mortal:
The viewer can control their experience much as they could in real life. An example of this is the winner of DreamlandXR’s Best Project for Television 2020, The BizNest.
In the mortal experiential POV, the viewer has control over all of the experience’s provided Degrees of Freedom (DOF), which may include anywhere from 1–6 DOF.
Deity:
The viewer has control over more than six degrees of freedom. Since the six degrees of freedom cover all the possible independent motions in three-dimensional space, to get more than six degrees of freedom, one needs to be able to move in or through more dimensions.
The fourth dimension is time, so if an experience allows the viewer to travel through time, it is giving the viewer more than six degrees of freedom and is therefore deity experiential POV.
Superstring theory posits ten dimensions (Williams, 2014), but without getting too deep into that, a simple way to think about deity POV is that, with it, the viewer can experience the world as various people or things, teleport or time travel. The characters in Bewitched, Freaky Friday, and Back to the Future have deity experiential POV.
Discussion of this taxonomy continues in Part 2,
which uses the relationship between narrative and visual POVs to defines “embodied” and “disembodied” experiences.
The author of this post is also the author of the taxonomy and can be reached with comments, questions or consulting requests on social media at @eveweston across platforms.
