Ryuichi Sakamoto is hunched over a grand piano, wearing a black suit and his customary tortoiseshell eyeglasses. The room around him is drenched in darkness, with no other textures visible but the freckles on his face and the glossy shine of the Yamaha. And then he lifts his hands and lets them fall back on the keys. They start to sing under his touch.
The performance lasts about 45 minutes. The Oscar-winning composer plays 10 of his own compositions, and at the end of every piece, people clap. Not me at first, but I make up for it with tears, which first build up in my throat, then stream down my cheeks. When it’s over, Sakamoto doesn’t bow and walk off the stage; he passed away in March this year after a years-long battle with cancer. He just fades into black as I take off the headset I am wearing.
Kagami, as the music performance is called, is a mixed-reality show on view through July 9 at The Shed in Manhattan and at Factory International in Manchester, with runs at the Sydney Opera House and the Big Ears Festival in Knoxville, Tennessee, up next. The show begins with 80 people sitting in a circle around absolutely nothing. After each concertgoer has slid on a Magic Leap 2 headset, a virtual Sakamoto appears in the center of the circle. The musician then performs as you mosey “around” him, being ever so careful not to bump into a piano that doesn’t exist—or your fellow attendees, who do.
Kagami—Japanese for “mirror”—was designed by the production studio Tin Drum, and fuses a 3D model of Sakamoto with the real world, resulting in an experience that feels material and ethereal at once. It is part of an ever-growing cohort of musical experimentations with AR and VR. In the thick of the pandemic in 2020, Billie Eilish performed a livestreamed concert jazzed up with extended reality effects that turned the stage into a gloomy ocean floor and a star-studded sky. In 2022, Gorillaz took over New York’s Times Square, then London’s Piccadilly Circus, as AR technology brought the band’s characters to life on giant screens. The list goes on to include Ariana Grande, Justin Bieber, and Travis Scott, most of whom started to dabble with mixed-reality during the pandemic.
With Kagami, the story began then, too. As Tin Drum founder Todd Eckert remembers it, he and Sakamoto were sitting in Eckert’s apartment in early 2020, kicking around the idea of a mixed-reality experiment. At the time, the musician didn’t know the cancer he had recovered from years prior had come back. “Maybe he said yes because he had missed two years [of performing] with treatment for his cancer before,” says Eckert. Or maybe he said yes because of his fascination with technology.
Fast forward to November 2020, and both men flew from New York City, where they were based, to locked-down Tokyo, one of only two cities in the world with the setup necessary to capture Sakamoto in 3D and generate a virtual model of him using volumetric and motion capture technology. The session lasted three days, during which Sakamoto played his compositions under the scrutiny of 48 cameras, various microphones, and lightboxes.
