Doctor's Note - Issue 26 - Meta and the paradox of VR
In this issue: Meta and the paradox of VR / 50% off Endineering / Power of Ten / Book Corner / The Linkhole
Welcome to Issue 26 of Doctor’s Note. If you were forwarded this by a friend, you can sign up here. It’s still free, but if you love it, you can buy me a coffee and I’ll love you back even more.
Can Meta overcome the paradox of VR?
I sat through Mark Zuckerberg's painfully stilted Meta Connect Keynote about The Metaverse pondering two questions. The first was why the team that puppet Zuckerberg's arms are so out of harmony with those who puppet his head. His cartoon avatar was weirdly more human.
The second was why their version of a metaverse1 is so dull. Then, when the partnerships with Accenture2 and Microsoft were announced, it all made sense. Meta's Metaverse is the VR design equivalent of PowerPoint templates and corporate design thinking spaces.
Dullness is a problem for the metaverse, because of a central paradox at the heart of VR that it appears Meta doesn't understand. We use the word immersion to mean two different types of experiences and they pull against each other. To explain, let's go back to 1978.
This is VisiCalc, the world's first spreadhseet, invented by Dan Bricklin and Bob Frankston. Five years later, Ben Schneiderman published a paper called Direct Manipulation: a step beyond programming languages in which he described the wonder of VisiCalc and how it allowed display editors that “display the document in its final form.” Schneiderman was used to working with single-line text editors, which only allowed the user to view and edit one line at a time, “like seeing the world through a narrow cardboard tube,” he wrote in his paper. Later, he uses more recent examples from the early 80s of interfaces with rudimentary graphic elements reminiscent of early Atari games. In all of these, the key ingredient, according to Schneiderman, is that they allow users to “directly” interact with and manipulate the data to hand.
If your world was single-line text editors, it must have been a leap to see a whole calculation on one page, but obviously we look at this with modern eyes and think, "That's not a direct manipulation interface, take a look at what I can do on my smartphone!" The thing is, hardly any digital interfaces are direct manipulation interfaces. They all use metaphors of varying levels to trick us. We're not really pinching a picture larger or smaller on our smartphones, we're simply wiping our fingers in a pinching motion across a piece of glass.
The clever slight of hand of the iPhone was to turn the entire device into a interactive metaphor for whatever tool you are using. Instead of it being a phone with some apps, Apple made the phone one of several apps. When you are using the phone app, your smartphone is a phone. When you're using a web browser, your smartphone is a web browser, the same with the camera and almost all other apps. That was the conceit at the heart of Steve Jobs's unveiling of the iPhone. It's such a clever slight of hand that we barely notice what's going on. We simply call it "intuitive".
Why is it that an interface metaphor like VisiCalc can seem intuitive "direct manipulation" at one time and then 25 years later feel awkward and clumsy? We might think it is simply the processing power of modern machines, but there is a mystery at the centre of this. While many old app interfaces can feel extremely clumsy, retro videogames are still engaging. I can still buy, play and enjoy the entire Atari back catalogue on the latest Playstation. Indeed, when you play these games, you can become as immersed in the game as in a modern game like Horizon Zero Dawn. Sometimes more so, since I seem to spend a lot of time just running around in Horizon Zero Dawn a little bored.
I spent several years writing a PhD about playfulness and interactivity to try and answer this apparently simple question. The tl;dr version is that playful interfaces lead to playful interactions and we easily become engrossed in the play – we are immersed in the same way a child playing with building blocks can become immersed in the activity. These interactions don't have to be games either, though games can clearly immerse the player through role-playing, combat, chance and vertigo – the various modes of play that Roger Caillois argued all games take in his seminal work Man, Play, Games. Most modern videogames are, at heart, either role-playing combat, competition, or mazes and puzzles.
This playful kind of immersion brings us back to VR and the metaverse. The paradox at the heart of VR is that the experience of having your senses immersed constantly distracts you from being immersed in the content.
Occasionally in TV shows or movies, you would see the boom mic drop in from the top of the frame. These days you might spot a coffee cup or water bottle in the background of a historical film. This immediately breaks your willing suspension of disbelief—the idea that you know a film or theatre play is not real, but willingly pretend it is in order to enjoy it3. It pulls you out of the immersion of the story or game.
The same happens when games are glitchy or have janky controls4. VR does this to you all the time, even when the graphics are pretty smooth and high quality. There's no getting around the fact that, despite the illusion, part of your brain knows you're wearing a headset. It's worse if you wear spectacles like me. Apart from often getting headaches, it always takes me about 5-10 minutes for my eyes to get back to normal focusing afterwards.
Now, there is compelling VR content. Some of it uses clever illusions that do trick your brain into believing you're there, though actual physical reality brings it into check pretty quickly. What you'll notice in this kind of content is how much effort goes into convincing you you're in a real space. Games are, of course, great at this—the gameplay is compelling content. But here's the thing: compelling content is not enhanced by VR. Compelling content helps overcome the constant physical reminder that you are in VR. It's the content that helps you suspend your disbelief, not the technology.
From Zuckerberg's keynote, it appears that Meta see success in VR as a technology issue to be solved. Better, lighter headsets may indeed help with the distractions of physical immersion, but I see plenty of people deeply immersed in the tiny, flat square of glass of their smartphones. Immersion is not a VR technology problem, it's a content problem. That's why boring scenarios in VR are a non-starter and there's nothing more boring than meetings, the prime use-case examples from Meta and co.
Yes, there are occasions in which work meetings in virtual space might be useful, compelling even. Workshops, perhaps. Walking around objects or data in different dimensions, like studying molecular structures or moving through some data visualisation, maybe. Yet these are not the scenarios Meta is envisaging. In their virtual world, people inhabiting their LEGO-friends style avatars in a tasteless corporate meeting room meet with others who can't be there in VR streamed onto a 3D rendering of a boardroom video conferencing set-up. Yawn.
Yes, you can work this way in VR, but why would anyone possibly want to? People turn their cameras off on group calls to tune out and check their e-mail. Hell, I've tuned out in plenty of real-life face-to-face meetings. Why would anyone want to don a headset to sit in a virtual conference room pretending to video chat with someone? The same goes for using a spreadsheet on a virtual screen. Sure, it's possible in VR. I might fill half my luggage with a VR headset so I can have wrap around VR monitors in my hotel room, but I probably won't. I have a smartphone in my pocket and a laptop in my backpack and I'll use those. Excel's UX is awful as it is, I don't want to add to that.
What was missing from Zuckerberg's keynote was imagination and creativity. Why are these people not having a meeting in the forest of Zuckerberg's avatar hair or wandering around the arteries from a patient's 3D rendered medical scan or even simply having a meeting outside under the oldest tree in the world instead of in an ugly conference room?
This doesn’t even begin to touch upon the most compelling content in virtual worlds – other people. The other place we talk about being immersed is in conversation. As Mark Pesce and Tony Parisi explain in their must-listen series, A Brief History of the Metaverse, it's not enough to just make a world and let people inhabit it and all will be well. That's what Second Life tried and all is not well there. People need a reason to be the space and to interact and that's the other reason games are so compelling. Fortnite's Battle Royale storm that forces randomly dropped players together is a brilliant mechanic for this.
Office work is not a compelling reason to meet in a virtual space. If we've learned anything in the past two years, it's that office work is only seldom a compelling reason to meet in real-world space. Like so many folks in tech, Meta appear to view the history of the medium as worthless. No wonder they're doomed in their repetition of it.
50% off Endineering training
I have long said that the "leave" stage of a service is the lost touchpoint (along with error messages). It's astonishing how many platforms don't have a designed offboarding experience. This is even worse with physical products in which the asymmetry between the lush onboarding and "it's up to you to dispose of" offboarding is central to our current sustainability crisis.
In his excellent books, Ends and Endineering, Joe Macleod has developed an entire thesis and approach to the problem and now Joe is training folks in endineering.
Very generously, Joe is offering Doctor's Note subscribers 50% off his next course, starting Friday 13th January 2023. You can sign up here and use the code 50offcohortall for your discount. Feel free to share with your colleagues and friends (though maybe don't broadcast it across social media—Joe deserves to make a living).
Power of Ten
Jason Mesut was my most recent guest on Power of Ten, talking design’s role in helping cope with the complex wicked problems facing us, design leadership, design teams and how they self-reflect, direct their future and develop others.
For the new year, I have episodes already in the can with three super smart women, Natalie Dunbar, Abby Covert and Alex Schmidt.
You can subscribe in the all the usual places you get your podcasts or go here for all the links to services.
Guff of the Week
For years I've been collecting vacuous statements from people in business and consulting when they really don't know what they're talking about, but want to sound like they do. It's called The Big Book of Guff. This issue's one is actually public, from Accenture's Productivity in Uncertain Times through the Elastic Digital Workplace. Apparently, one of the important ways to "manage now" is:
Design a change management plan, including leadership communications that encourage employees to adopt key technologies that will facilitate connections to their team members and leaders in this new environment, internal meetings and even large-scale external events.
So, send out company-wide e-mails to tell staff they must only use Microsoft Teams, yeah? 🤷
Book Corner
I very much appreciated Alex Schmidt's Deliberate Intervention: Using Policy and Design to Blunt the Harms of New Technology (I'm cheating, I linked to this last note as well, but everyone should read it).
If you haven't read Abby Covert's How to Make Sense of Any Mess you should. And if you have, you'll enjoy Stuck? Diagrams Help..
Having read Tamsyn Muir's Gideon the Ninth and Harrow the Ninth, I thought I had reached the end with Nona the Ninth, but there will be one more. It's well worth reading the series, though they get progressively more confusing as Muir plays with time and memory.
I recently had to drive to the UK and back on my own. Ian Mcewan's Lessons was a brilliant travel companion as an audiobook. I also listened to On Chesil Beach and The Cockroach on another recent road trip. For a view on living with AI, his Machines Like Me is a must-read.
The Linkhole
The now much-read thought piece from Ben Evans on ways to think about a metaverse
Meta is having a midlife crisis, and Zuckerberg’s stale vision is to blame
A Brief History of the Metaverse by Mark Pesce and Tony Parisi.
Cal Newport on learning from the deep history of work
Regret can come up in all sorts of ways in my coaching practice. Here's good article in Psyche about how to deal with regret.
Since Twitter has become what it has become, you’ll find me over on Mastodon at @apolaine@pkm.social.
That’s it for this issue. Thanks for reading, sharing and listening.
If you liked it, please consider sharing it with a friend or colleague to sign-up. All the shares, feedback and new subscriptions apply a healthy pressure to write more frequently, which I clearly need.
Happy Holidays and see you in the New Year!
Andy
I'm using capitalised "Metaverse" for Facebook's Meta's branded version, but metaverse for the general concept, since Neal Stephenson came up with the name.
Disclosure: my previous employer.
A phrase coined by 18th Century poet, Samuel Taylor Coleridge
If you really want to understand the relationship between immersion and janky games controls, Janky Controls and Embodied Play: Disrupting the Cybernetic Gameplay Circuit by M. D. Schmalzer is a great read.