At the Sundance Film Festival, I unintentionally slayed a rat in a child’s bedroom, summoning experimental pop star Poppy to foment a rodent uprising. I faced an ultimatum to either sacrifice myself or kill a boy held in a cage behind a hidden panel in the wall. Like a live child actor blinking at me from behind bars.
Poppy, the boy and rats were a product of digital magic waved in front of my eyes courtesy of the Magic Leap augmented reality glasses. But that didn’t mean the plight didn’t feel real. My brain knew this was fake, but the mother in me screamed: “Nooooooo kill me oh my god noooooo!”
After years of skepticism that Magic Leap might be building the world’s best funded vaporware, its headsets were out in force in Park City, Utah. Three projects used Magic Leap as their central technology. Two were presented by Magic Leap itself, both polished to perfection. Its “digital human” Mica looked me in the eye with a self-assured gaze, and a Shakespearean actor’s miniature hologram recited a monologue from As You Like It to me from a moss-covered knoll.
But the third, the disquieting rat experiment A Jester’s Tale, was made outside Magic Leap’s auspices. It was subversive, glitchy and designed for me to fail. Guess which one left the most lasting impression.
“Generally a lot of the work that comes out in AR and VR … is really clean and nice and pretty and smooth and meant for everyone to enjoy,” Asad J. Malik, the creator of A Jester’s Tale, said in an interview during the festival last month. “I don’t want to do that. We’re trying to go rogue with AR.”
Make no mistake: Mass appeal is crucial for augmented and mixed reality, which cast digital creations in the midst of your physical surroundings, and virtual reality, which immerses you in a totally digital world. Magic Leap is wise to go after cutesy Star Wars porgs that toddle around your room and first-person-shooter games battling robots.
But just as the creators of YouTube couldn’t have predicted the rise of Let’s Play, ASMR videos or any of the scores of bizarre genres there, new technologies like Magic Leap need to put tech in the hands of outside creators to uncover new kinds of experiences. A Jester’s Tale is one of the first peeks at what outsider influence on Magic Leap looks like.
Magic Leap will need to tap into all of the genres it can get into. YouTube draws creators with its massive audience, but AR and VR haven’t proven to be a path for making bank on content yet. Magic Leap headsets themselves cost $2,295.
To its credit, Magic Leap has already tipped its hat to the importance of outside creators and developers. Its first headset available for sale, Magic Leap One, is also termed a “Creator Edition” meant more for creative types than for everyday consumers.
The company released Magic Leap One at its nose-bleed price point last year in only a handful of creative and technology hubs — Chicago, Los Angeles, Miami, New York, the San Francisco Bay Area and Seattle. It’s delivered and set up in your home by a white-glove installation service. The idea is to get Magic Leap first into the hands of people already skilled at conjuring up the kind of experiences and content that might eventually lure in the masses.
A Jester’s Tale is a glimpse of what Magic Leap can be like when an outsider pushes its limits. And Malik really pushed.
A Jester’s Tale uses Magic Leap’s mixed reality capabilities to beam two different types of character into a room with you. There’s a floating cloud of dots that represents a talking artificial intelligence that tests your humanity like a sci-fi CAPTCHA. Then there’s what’s known as volumetric assets: three-dimensional videos of real humans that you can walk around like they’re holograms.
There’s a hologram of a boy and a hologram of his mother. There are computer generated holograms of rats and their fur-coated queen played by Poppy. Malik shot the human actors at a green-screen stage surrounded by more than 100 cameras, creating the raw material to stitch together these holographic videos. They’re already data-heavy files, and A Jester’s Tale mercilessly punches 30 volumetric clips through Magic Leap’s technology one after another.
The app for A Jester’s Tale, as a result, ballooned to 30 to 40 gigabytes, far outstripping the limit Magic Leap sets. Magic Leap gave A Jester’s Tale access to sideload the holographic assets onto the headset to make it work. And sometimes it didn’t work — the experience was down for most of the day that members of the press were allowed to preview New Frontier projects before Sundance officially opened.
“We’re definitely pushing it to the 99-percent-limit of what the performance it can do,” Malik said.
Magic Leap itself presented two experiences at Sundance, Mica and Seven Ages of Man, and they landed with the perfection of an Olympic gymnast.
Mica is Magic Leap’s humanoid artificial intelligence. (Coincidentally, she looks a little bit like Poppy with a pixie cut and a warmer demeanor.) During my experience with Mica, she sat across from me at a table, holding my gaze. She’d lean forward to peer at me a little closer while gently smiling in ways that would crinkle her nose or wrinkle the skin around her eyes. She gestured to a wall for me to hang a picture, which revealed Magritte’s surrealist pipe painting. In a nod to how mixed reality breaks down presumptions about what’s real or what’s not, Mica pulled the pipe out of the picture so she was holding it in her hands.
Mica isn’t being designed as a virtual assistant to do humans’ bidding but as a “digital human” whose AI is geared toward “equalizing comfort,” according to Alice Wroe and John Monos, two Magic Leap employees working on the project. One goal with Mica is develop something so lifelike, it forces both her developers and her users to think about AI in human terms, they said.
Seven Ages of Man is the performance of a Shakespearean monologue by a pseudo-hologram of a Royal Shakespeare Company actor. The miniature thespian stands in front of a computer-animated tree as he recites the “all the world’s a stage” speech from As You Like It. As the actor recounts the life cycle of a man with Shakespeare’s words, the tree behind him blooms and withers to match.
My lasting impressions from Seven Ages of Man were to reflect on how Shakespeare can pack the most punch when delivered by a seasoned pro and to ponder how interesting it could be if the Royal Shakespeare Company performed whole plays live as holograms in living rooms around the world.
But experiencing A Jester’s Tale didn’t leave me wondering what the future of AR could be like. It plopped me right into a twisted pseudo-reality that was right in front of me.
“These ideas about one day storytelling will be characters in your room, or they will live with you… This is the closest I know to that at the moment,” Malik said. “That’s what I wanted to reach for.”
Culture: Your hub for everything from film and television to music, comics, toys and sports.
Movie Magic: The secrets behind the scenes of your favorite films and filmmakers.