FACE TO FACE WITH AN ICON

Using Virtual Reality to Preserve a Legacy

How do you use brand-new technology to record a 94-year-old legend in a way that will be meaningful generations from now?

Earlier this year Legion M made a bet with a once-in-a-lifetime interview of Stan Lee. And while it may be decades before we know how our gamble pays off, this is the story of what we did, why we did it, and how we're doubling down on the notion that the future of VR is the present.

ICONS: FACE TO FACE

Virtual Reality is, without a doubt, one of the most paradoxical spaces in entertainment today. It's a nascent industry with the potential to revolutionize the way we communicate. The opportunity is staggering, the state of the art is rudimentary, and the landscape is constantly evolving. VR is, in essence, where the movie industry was 100 years ago as maverick showmen "invented the close up" and figured out how to tell stories in an entirely new medium. Given that the successful startups of that era are now some of the biggest companies on the planet, it's no surprise to see some of Hollywood's best and brightest (not to mention billions of dollars) joining the rush to pioneer this new frontier. 

Legion M decided to throw its hat in the ring with a series called ICONS: Face to Face. The premise is simple -- use the magic of virtual reality to create "virtual time capsules" that put YOU face to face with the luminaries, titans and leaders of our age. Imagine if you could go back and spend an hour with William Shakespeare, Joan of Arc, or Martin Luther King Jr. Those icons are lost forever, but VR technology gives us an unprecedented opportunity to capture and preserve the legacy of today's icons -- told in their own voice and defined on their own terms. 

Stan Lee is the perfect example. As the former President of Marvel Comics and co-creator of "the pantheon of modern gods," Stan is an undisputed icon of his age. At 94 years old, he's also one of the most charismatic, generous, heartwarming and inspiring voices of our time. 

People travel from around the world and wait in line for hours for a handshake and photograph with Stan. What if they could sit down with Stan -- in his home -- and hear his story in his own words? We wanted to capture that experience and package it in a way that could inspire fans in every corner of the globe. And do it in a way that will be meaningful for generations to come.

FUTURE-PROOF

As we started to plan our shoot, it immediately became apparent that we would need to blaze a new trail. It's not that there hadn't been high profile VR interviews before -- it's just that none of them had been done with the same lofty goal we had for Stan. Most VR experiences today are like magazines -- they entertain, but aren't expected to stand the test of time. There's not a lot of incentive to build for technology and marketplaces that don't yet exist.

Unless you're shooting a 94-year-old man. 

This was a once-in-a-lifetime opportunity. Stan had not only agreed to the interview, but he agreed to do it in his home. And we had Kevin Smith to boot. We had one shot to capture this magical moment in space and time.

We didn't want to create a VR magazine, we wanted build a VR monument. We needed footage that would stand the test of time. This couldn't be just another Stan Lee interview--we needed THE Stan Lee interview. The one that's worth watching 100 years from now. 

CHASING HISTORY

When you strap on one of today's VR headsets, two things become immediately apparent. The first is that immersion works. Even the headsets of today have a remarkable ability to transport you to a virtual world. When people in VR walk up to a virtual cliff, most find it extremely hard (if not impossible) to step off. The technology is good enough to trick millions of years of evolutionary biology and protect your virtual well-being.

However, the second thing you notice is that the visual quality of today's headsets sucks. You can see individual pixels, making it feel like you're looking through a screen door. In a scuba mask. And 360° video is the worst. In a world where people are accustomed to HD, VR has the visual quality of television circa 1987. If you've ever wondered what it's like to be trapped in a Sony Trinitron, today's VR will give you an answer.

But that will change. Advancements in display technology, processing power, memory, and data streaming are as inevitable as the next installment of Halo. Soon VR will become HD, and at some point it may even become indistinguishable from real life. But how do you design to a spec that doesn't exist yet?

We decided to take a cue from Steve Jobs.

THE LIMITS OF HUMAN PERCEPTION

In 2010, Apple introduced Retina displays with great fanfare. What made these displays special, as Steve Jobs explained onstage, was not just that they were higher quality than had ever been seen in a mobile phone, but that they were specifically designed to be at the very limit of human perception. When you view an iPhone 4 ten to twelve inches from your eyes, the resolution is so high that the text on the screen is indistinguishable from a printed book. There's no reason for more pixels because the human eye can't perceive the difference.

The optics of a VR headset are different than those of an iPhone, but the basic science remains. There's a threshold of human perception beyond which additional spatial resolution (i.e. pixels) becomes irrelevant--a point at which the resolution of a display becomes virtually indistinguishable from reality. We dubbed it Li-Fi -- the fidelity of real life.

The precise measurement of that threshold is difficult to pin down. Biology is messy--not everyone has the same vision, and not all images are created equally. And video quality is measured in more than just pixels. In addition to spatial resolution, you also need to consider things like temporal resolution (frames per second), color resolution (color depth), and dynamic range. Ultimately we created our own spec for Li-Fi: 60 FPS, high dynamic range, and the generally accepted value of 75 pixels per degree (PPD) of viewing angle. The first Retina displays were about 58 PPD--if that was good enough for Steve Jobs, 75 PPD would work for us!

CHASING Li-Fi

Today's state-of-the-art cameras for professional-grade 360° video are made by Google and Jaunt. These cameras -- an array of outwardly facing wide angle cameras (16 GoPros in the case of the Google Odyssey) -- are used to shoot most high quality VR experiences. Algorithmic stitching allows you stitch together the output from all 16 of the cameras into one seamless 360° degree 3D video. They are a very advanced and cost effective way to shoot for today's headsets.

However, they posed some serious problems for our shoot. To start with, these cameras require the subject to be 6-10 feet away from the sensor. Our premise was an intimate interview. 6-10 feet away felt more like a restraining order.

Not only that, these cameras had nowhere near the fidelity we wanted. When you take 16 GoPro's and stitch the video together you end up with a huge amount of pixels -- 4K or even 8K per eye. Those numbers are massive by streaming video standards, but when those pixels are spread out over 360 degrees it becomes fairly unimpressive 22 PPD. Fine for today's headsets (which only support 8-12 PPD), but nowhere near our 75 PPD target for Stan. 

Aside from that, it didn't "feel right" to use GoPros. Maybe if we were filming Stan on a half-pipe, but I can't imagine a single Director of Photography who, when presented with an opportunity to capture an intimate interview with Stan Lee would say, "Let's shoot it on a GoPro!"

We wanted 75PPD, 60 FPS, high dynamic range, and real glass lenses. So despite the fact that almost every single VR studio we spoke to pushed us to the Odyssey or Jaunt, we made the decision to follow the path less traveled.

FULL FRONTAL NODALTY

At this point we need to confess that the whole premise of this story is a lie. We didn't realize it when we started, but we also didn't make significant progress until we realized the truth. This wasn't a VR shoot. This was a movie shoot.

That's because, while the VR experience is all around you, all the "action" is happening right before your eyes. This is an intimate conversation, not a tennis match. If we put you at a normal talking distance from Stan and Kevin--as if you were standing together at a crowded cocktail party--you'd need to turn your head ever so slightly as you focused your eyes from one to the other. That meant everything would happen in about a 60 degree field of view. The remaining 300 degrees are just window dressing.

To be sure, the rest of the room is important for immersing you in the moment. We want you to look around, take-in the space, and relish in the fact that you are in Stan Lee's home.  But once you've done that, your focus will be on Stan. This is best summed up in a quote from a VR porn producer: "If you are turning around to look behind you, we're doing something wrong!"

To capture Stan and Kevin in all their glory we needed a 60 degree horizontal field of view. Vertical field of view would have been problem (people are a lot taller than they are wide), but adding a strategically placed cocktail table to mask Stan and Kevin's legs reduced it to about 45 degrees. That put us in the range of the highest end 3D movie cameras. We could capture all the live action with Kevin and Stan in-frame, then, after the live shoot was done, we could rotate the camera and capture the rest of the room. Since nothing was moving in the rest of the room, the viewer would never know the difference!

CHOOSE YOUR WEAPON!

Finding the right camera lens combination could be an article in and of itself, but I wouldn't be the one to write it. It would be filled with discussions of lens distortion, convergence, and chromic aberration. The Cliffs Notes version would be: Nobody had ever released a VR interview shot at such close range; Nobody had ever released a VR interview shot with such high fidelity; the challenges of 3D nodal are non-trivial; and the closer your get, the harder the problems become. 

After days of painstaking testing, our partners at VR Playhouse found the magic recipe. Dual Red Weapon Heliums 8K -- the highest end camera of the Red line. And ARRI Ultra Prime lenses -- the largest rectilinear lens on the market. The cameras were so new they had only been on the market a couple months. We doubled their hours during the camera tests.

It's difficult to understand just how big 8K is. To put it in perspective--when you go to a movie theater, they are generally projecting in 2K. Maybe 4K. The cameras we were using are 8K. There aren't even movie screens large enough to view it!  And we had two of them--one for each eye! Our fully assembled 360 degree video has over 50K worth of horizontal resolution and requires over 1TB of data every 30 minutes. If there's a spec for Li-Fi, that's it...

"OLD PEOPLE DON'T LOOK GOOD IN VR"

While it's easy to drown in the technical details of VR, the most important details are human. The value of this interview isn't the number of pixels--it's their ability to tell a story. 

We dealt with very human details when deciding where we shot. We wanted to shoot at Stan's house. We felt it would make him most comfortable, and most likely to open up. It also meant that his wife Joan could join us, which we felt added a whole new perspective on the interview.

A lot of people told us that was a mistake, and that we should film in a studio where we could control the environment. "Stan's house will never live up to people's expectations", they said, "but in a studio you can create whatever you want".  People also told us "old people don't look good in VR".

That sentiment isn't wrong. Stan is, after all, 94 years old. And though some consider him a God, he has the body of a mortal man. His Beverly Hills home is more evocative of the Valley than Valhalla.

But isn't that the point? Does virtual reality necessarily equate to artificial reality? If you could travel back to spend time with William Shakespeare would you rather hang out at his home or a 17th century set?

With the advancement of AI and 3D digitization techniques, we have every reason to expect an artificial virtual Stan will be available before long. This Stan will be completely digital, unencumbered by the inconveniences of being human. Talented artists will smooth his wrinkles, erase his blemishes, and place him in whatever environment the VFX team can dream up. But it won't be real--nothing more than a Stan Lee puppet manipulated by a digital hand.

When that time comes, the footage we captured will still exist. Thanks to the choices we made it should look as crisp and clean as anything else you can watch on your headset of tomorrow. As to which version of Stan people prefer--the real versus the artificial--your guess is as good as ours. But by the time virtual reality comes into it's own, we're banking that a well-preserved piece of actual reality might just by the killer app.