Moove over touch: Control your iPhone via head movements

Israeli start-up uMoove released on Tuesday the first and so far only, head and face tracking-based app for iPhones and iPads, opening up a new world of interaction with handheld devices, said CEO Yitzi Kempinski. “uMoove’s ‘Flying Experience’ is the only app for iOS that uses the camera and reads head movements to execute commands in a game,” he told the Times of Israel in an exclusive interview on the eve of the game’s official introduction in the App Store.

But games – and there will be many more to come after Flying Experience, said Kempinski – is just the beginning. “Eventually our technology will be used in many ways, such as tracking interest when a person is watching a movie, or seeing how engaged they are when looking at an ad. We even foresee medical treatment via eye tracking, in which a user will look at the device’s camera, and a doctor on the other end of a call will be able to provide a diagnosis via iridology, by looking at a patient’s eye and eye movements.”

All of uMoove’s features are software-based; there is no need to add any hardware, as the platform is designed to run with iPad and iPhone cameras. Utilizing a very complicated set of algorithms, Flying Experience, a free download, takes control of the device’s camera in order to “read” head movements. The game entails a user having to catch little bottles as they fall from the sky. Accompanied by a dreamy and vaguely Arabic-style soundtrack, the game is set in a desert; you control a pair of hands flying through the sky towards a village and try to collect as many bottles as you can. You approach a falling bottle by jutting your head forward, and move back by pulling slightly away from the screen. Looking up will send you skyward, while looking down will get you down to the ground.

The game would be a cinch if it were a standard touch game, but it turns out that navigating the screen by moving your head around — that’s the only way to play Flying Experience — takes some getting used to. And that’s understandable, said Kempinski. “When touch was first introduced people used to jab the screen to make the game or app work faster, because that’s what you did in a mouse and keyboard world. As more head and face tracking-based apps come out, people will learn how much or how little movement is needed to control it.” In the case of Flying Experience, said Kempinski, less gross head movement will yield you more results (ie. captured jars) on screen.

Yitzi Kempinski (Photo credit: Courtesy)

As the first and so far only head ad face gesture platform for iOS, uMoove has an opportunity to set some standards – and it’s opening up its software development kit for free use by any start-up or app developer with a good idea for head or face tracking-based apps or game.

There is a catch, though. “We have to approve an app before we allow them to use our SDK,” said Kempinski. “We want good apps or games that are going to truly take advantage of the platform, not gimmicks that won’t provide users with enriching experiences.” A first-person shooter game, or a map navigation app (in which you move your head to move around the map) or a motorcycle racing app (where you move your head to steer the bike) would be examples of the latter, he said.

Almost as important as the development of uMoove’s platform is that Flying Experience, or the company’s gesture technology in general, does not eat up a device’s battery. “It took us a long time to get this right, but the platform functions at a low CPU of less than 5% on contemporary devices,” said Kempinski. “The biggest power burden is in the camera, which we have no control over, but our platform code is extremely efficient, so there is no energy wasted at all.” Playing Flying Experience will be no more of a burden on the system, and probably much less, than the vast majority of games on a user’s device, he said. Those efficient algorithms are, of course, patented — one of the 15 patents uMoove has taken out for its platform.

With uMoove, Apple can proudly claim to have one-upped its chief rival, Samsung – despite the Korean company’s claims of having gotten head and eye-tracking on its devices first. Samsung has been marketing its Galaxy S4 device as supporting gestures, but according to Kempinski, Samsung is leagues behind what uMoove has done for Apple. “Samsung’s just isn’t very good,” Kempinski politely opined. “Their system only understands up and down, like a person nodding yes. It does not support other head gestures, and certainly does not support eye-tracking, as we do.”

That latter feature isn’t available in Flying Experience, nor will it be part of the open SDK – but uMoove is working with some large companies to implement it, said Kempinski. “Eye tracking is much more complicated than head tracking, because the eyes move involuntarily. But we have gotten over most of those issues, and currently we have a beta product for eye-tracking.” The industry is very excited about the possibilities of eye-tracking, said Kempinski. “We have gotten calls from gaming companies that ask us to implement it in various games, to allow users to see features when they look in the direction of the screen.”

But eye-tracking – along with gestures, both hand and head – will be used for a lot more than games in the coming years. “The possibilities are endless,” said Kempinski. Besides the medical app example mentioned earlier, gesture could change the way people interact with computers, TVs, advertisements, their vehicles, and much more.

“Users could change their device’s screen or a TV channel with the right flick of a head or eye movement, said Kempinski. “E-commerce apps could allow users to switch between products without requiring touch or keyboard input, making it easier for customers to navigate on-line stores. You could go on a virtual tour of the world, just by gazing east or west, or get a 360 degree view of a tourist site or a hotel room just by moving your head or eyes. Flying Experience is just the first of many more useful and fun apps to come, powered by tracking.”