The Technology Behind Lady Gaga's Tribute To David Bowie

Lady Gaga's breathtaking tribute to David Bowie at this year's Grammy Awards counted on Intel's technology. Vanity Fair published an article giving us more details on this partnership. 

"Holograms may seem like a bold new frontier for the entertainment industry. But for Lady Gaga, the envelope-pushing performer in all things audio-visual, holograms are already passé. Hence her mind-bending Grammys homage to David Bowie on Monday night, which paid tribute to various phases of the icon’s five-decade career, courtesy of Intel computer graphics, interactive video, and robotics. Given that Bowie passed away only a month ago, though, how the hell did Gaga pull together such a feat in a matter of weeks?

“Lady Gaga had been speaking to Intel as early as September, brainstorming ways in which she could use technology to express herself in a way that had never been done before,” explained Paul Tapp, Intel’s director of technology, by phone last week. “She’s an amazing dancer and she really pushes boundaries with fashion. She said, ‘Help me to go beyond these standard constraints.’” So, this past fall, a team of engineers began working on the technology that Gaga envisioned using. But when Bowie passed in early January, Gaga began re-imagining her multi-media performance as a tribute to the icon.

“Bowie had tons of iconic looks throughout his very rich career,” Tapp said, “so Lady Gaga was curious how we could pay an homage to that. We introduced ‘living canvas’ technology to her, which allows her to basically have what we call digital skin—which has been used in tech art installations, but never before for a live performance.” The digital skin is projected onto Gaga as she performs so that it looks like her “makeup” is changing instantaneously throughout her six-song performance.

To create the makeup animation—essentially C.G.I. done live—Intel’s team created a 3-D scan of Gaga’s face by capturing her in about 12 different poses with complex scanning technology. The team then created 3-D motion graphics based on Gaga’s facial measurements and the Bowie looks she wanted to replicate.

During the performance, those graphics—essentially “replacement skin”—are projected onto her face live, like a mask, thanks to real-time face-tracking. (Five infrared-sensing cameras pick up on infrared markers placed discreetly on Gaga’s face.)

“Once we’ve got the tracking in place, those cameras and markers are telling the computer exactly what angle and rotation and axis her face is and her facial expressions,” explained Tapp. “Are her eyes opened or closed? Is her mouth happy or sad? Are her cheeks up or down? We have to track all of that with very intricate detail and then in real time we have to adjust that digital makeup that we’ve computer generated, get it sorted according to how her facial position is, and project it exactly to the right place at the right time.”

Viewers have probably seen this technology before,” added Teresa Herd, Intel’s V.P. of global marketing, “but it’s all been a special effect done after a film is shot. This is done in real time—it’s literally changing and morphing right in front of your eyes.”

But the face-mapping projection is only one technological component Gaga incorporated into her Grammys segment. “She wanted to do a lot with space,” said Tapp. “She loves the cosmos. There’s obviously a lot of inherent beauty in the heavens, but it was more than that. She loves the kind of gravity and motion aspects of space as well. With computer graphics, you’re able to get some very rich, very photo-realistic, amazing looks, but we wanted to take that one step further and make those graphics respond to her. It’s not necessarily pre-rendered content. It’s content that actually reacts to how she’s moving onstage. Further more, we were able to project that content on to some very special materials, which make it appear like she’s hovering in space.”

Engineers also created a special accessory to give (or appear to give) her magical powers onstage. A bespoke ring tricked out with Intel Curie technology and holographic materials, Gaga’s ring allowed her to have real-time control of the stage, an L.E.D. wall, and the environment around her including a three-dimensional hologram of Bowie himself.

“The Curie ring gives her powers beyond her body and you’ll see that incorporated into this space, like a theme where stars and black holes all kind of come together under her control. We expect it to look very dramatic.”

The technology created for Gaga’s performance was so impressive and intricate that Tapp and Herd did not even reveal that robotic arms would appear onstage to move a grand piano until 35 minutes into a 40-minute conversation, almost as an afterthought.

“We’ve really tried to integrate robotics in a beautiful way creatively, and these are a rose gold,” said Tapp. It’s almost like an animate creature to some degree. We capitalized on that to do some unexpected motion effects with it live.”

In a statement issued seconds after Gaga’s performance, the pop star spoke about bridging music and technology at the Grammys. “I like to do things that integrate technology and art with powerful experiences,” she said. “I think that this collaboration with Intel has been very different than anything I have done before. They have really given me so much amazing technology to play with.”

Intel has a two-year contract with the Grammys, and will return with another technology-heavy performance next year."