Independence for the paralyzed via a life-changing cellphone
Sesame Phones, designed by Israeli software engineer Oded Ben-Dov, enables sufferers of ALS, Parkinson’s, and MS to use their smartphones without assistance
In just a few months, individuals who cannot use their limbs due to conditions like ALS, paralysis, Parkinsons, multiple sclerosis and spinal cord injuries will be able to – some for the first time in many years – make phone calls, surf the internet, read and send email, even play games, all on their cellphones.
The Sesame Phone, a product of Israeli computer vision and gesture technology research, will enable those suffering from near-paralysis to interact with their smartphones like anyone else, said the Sesame Phone’s inventor, Oded Ben-Dov.
“We’ve developed a completely software-driven system that could be used with any Android phone,” Ben-Dov told The Times of Israel. “It’s not a software patch, but a complete reengineering of the Android OS to allow users to control devices using just voice and head gestures.”
The system basically takes control of all of the device’s sensors and interaction components (cameras, accelerometer, GPS chip, etc.) and puts them all at the service of the revised OS, allowing full use of the device using just voice commands or head gestures.
While the system is suitable for any device running the Android operating system (an iOS version is still under development), as a practical matter – because people prefer a complete solution instead of an platform they have to install – Ben-Dov and his team will install the systems on Google Nexus 5 devices and sell them as a full package.
The device’s voice commands are used to open up applications, make calls, etc. For apps that usually require touch for interaction, the system uses head gestures, with users moving their head in various directions (up and down or left and right for navigation, a slight nod forward for selection, etc.),
“Our slogan is ‘touch is overrated,’ and with our device, touch is not even necessary,” said Ben-Dov. “For those who don’t have use of their limbs, this is so far the only solution that lets them use a standard smartphone, with all that means today.”
The fact that the Open Sesame solution is software-based is key to its abilities, said Ben-Dov. While there are other enabling technologies that allow the severely disabled to use smartphones, they are either hardware-based and cost many thousands of dollars, or limited in scope, able to control only one or two apps. By hacking the smartphone’s operating system, Ben-Dov can provide touch and gesture services throughout the device – for a lot less money that other solutions.
“For the phone, technology, training, and stand for the device (it will need to be mounted on a pole opposite the user’s head), we expect to charge about $1,000,” said Ben-Dov.
The device has been under development for several years, although the basic system was ready at least a year ago. “The Sesame Phone started a few years ago after I developed a game that used gestures, when I got a call from Giora Livne, a disabled IDF veteran who had been a quadriplegic for the last nine years. He asked me if I could design a phone that he could control with head gestures. At first I thought the call was a prank, but as I got to know Giora, I saw that he was serious – and the more I investigated, the more I saw it could be done.”
One of the challenges in designing the Sesame system was ensuring that the device’s camera could keep up with the very fine, incremental gestures that the users of the device would be making to control it. Gesture is nothing new, and there are several Israeli companies focused on that area, with PrimeSense the most well-known for its development of the tech used in Microsoft’s Kinect system.
“But Kinect and most gesture systems on the market use gross gestures, with users standing in front of the camera and waving their arms or moving their bodies to control the screen,” said Ben-Dov; engineering a gesture interface for fine movements is a greater challenge.
Another challenge was deciding just how fine to make the gesture sensitivity, said Ben-Dov. “Some people, like quadriplegics, have good control of their heads, while others, like ALS and Parkinson’s sufferers, are likely to be able to move their head just a little bit. We had to design the system in a manner that enables users with both good and poor head motor skills to use the device.”
The device is ready for market, and manufacturing – or rather, re-engineering of the Nexus phones – is going on right now. Shipping is scheduled for March 2015, and the company this week kicked off an IndieGogo campaign – which, after less than a week, has nearly reached its $30,000 goal. “We have funding from several angels who are very interested in the device, and we have been in touch with CEOs and top executives of many of the largest tech companies in the world, who have expressed support, both verbal and material,” said Ben-Dov.
It should be noted that while the Sesame phones are Google Nexus devices, the company does not have a partnership with Google. “We are using the Nexus 5 devices because they are good quality and easy to work with. We don’t have a formal relationship with Google, but if they’re interested, we certainly can have a conversation.”
Sesame is also developing an SDK that will be available to all developers, “so if the makers of the Angry Birds game wanted to, they could equip their app with head gesture controls,” said Ben-Dov. The system, obviously, has a million and one uses, from enabling hands-free reading of news sites to interacting with a cooking app while both hands are busy kneading the dough. But for a number of reasons – both moral and monetary – Ben-Dov is concentrating specifically on the disabled market, at least for now.
“According to the medical experts, there are about 5 million Americans with some form of paralysis, and we estimate that 1 or 2 million would directly benefit from this device, while in the rest of the world we estimate that there is a market of at least 10 million potential customers.”
Those numbers are enough to provide a business case for serving the disabled community, but for Ben-Dov, it’s about a lot more. “It’s just the right thing to do. The enabling technologies on the market today basically allow enabling for the disabled, but not independence.
“This is one of the first products – and the first smartphone – that provides that independence,” said Ben-Dov. “Giora told me that his greatest wish was to be able to pick up the phone and order flowers for his wife – such a simple thing, yet for him, it was like moving a mountain. Now, with our device, it is once again as simple a thing for him as it is for anyone else.”
