At Monday night’s Grammy Awards, Lady Gaga gave the performance of a lifetime – not necessarily hers, but that of recently deceased rock icon David Bowie, whom Gaga feted in a seven-minute stage extravaganza that used robotics, advanced processors, audio and video technology – all courtesy of Intel – to virtually change her clothes, create a holograph of Bowie, and instantly turn her into a female version of the singer’s Ziggy Stardust character.
That latter trick was pulled off thanks to Israeli technology, an Intel spokesperson said. Using Intel’s RealSense 3D camera technology, digital artists created a 3D image of Gaga’s head and used it to project the makeup seen on the singer’s face at the beginning of her performance.
“The ‘makeup’ was all digital,” the spokesperson said. “You can actually see this at the beginning of the video – there is a second or two where she moves her face too quickly for the projection to keep up. That projection was done using a RealSense 3D camera, which, based on the 3D model which captured the contours of her face, ensured that the projectionists were able to precisely follow her moves and keep the ‘makeup’ where it belonged on her face.”
In the performance, Gaga starts out with Bowie’s classic “Space Oddity,” with a digital spread of red paint pouring down on her face to form Ziggy’s iconic lightning bolt face tattoo. A spider emerges from her eye and crawling across her face before dissolving into nothingness.
RealSense technology, developed in Haifa and now a part of Intel’s newest generation of 3D cameras, sees the distance between objects, separating objects from the background layers behind them. This gives much better object, facial and gesture recognition than a traditional camera, according to the company.
The visual data creates a touch-free interface that understands and responds to hand, arm, and head motions as well as facial expressions.
3D vision technology – in which 3D cameras can be used to interact with computers, TVs, and gaming consoles – has been around for a few years, with much of the technology developed in Israel. 3D vision tech from an Israeli firm called PrimeSense, for example, enabled Microsoft to to build its 3D interface for Kinect consoles.
But until now, 3D interactivity has been restricted to large devices, like gaming consoles. While RealSense could be used for those purposes as well, with a RealSense-based 3D camera for PCs already on the market, Intel has much more in mind for its 3D tech.
“Our objective is to provide the 3D tech that can be integrated into hundreds of devices, and provide a software development kit that will allow developers to build applications for those devices to interact with the environment and with users,” said Igal Iancu, a senior manager on Intel’s RealSense 3D vision tech team, based in Haifa.
Among the things Intel plans for RealSense, said Iancu, is advanced photography editing. “You could use the camera to create an automatic green screen, to mute the background and to segment the actor from it.” The current background could be replaced with any other one, and the viewer would see it as ‘real life,’ in 3D, on their screen – and that, Intel said, is how Gaga became a real life Ziggy.
“I like to do things that integrate technology and art with powerful experiences,” said Gaga. “I think that this collaboration with Intel has been very different than anything I have done before. They have really given me so much amazing technology to play with.”
“Intel is on a journey to create and share the amazing experiences that our technology enables,” said Steve Fund, senior vice president and chief marketing officer at Intel. “This performance has set a new precedent for live music experiences and is a major milestone for Intel and the Grammys.”