Israeli startup lets users check vital signs by looking at their smartphones
Binah.ai turns mobile devices, laptops into medical monitoring tools that can measure heartrate, oxygen saturation, stress levels
Luke Tress is a JTA reporter and a former editor and reporter in New York for The Times of Israel.
Israeli startup Binah.ai says it has developed technology that turns smartphones into health monitoring devices that can check vital signs including heartrate, oxygen saturation and respiratory rate.
The new technology comes as medical care worldwide has been stretched thin by the pandemic and other, longer-term trends, spurring demand for telemedicine and cheaper, more convenient health monitoring solutions.
Binah.ai, founded in 2016, uses cameras on smartphones, tablets and computers to scan a user’s skin to infer health information. The user just needs to look into the camera to let the company’s system measure their vital signs.
Our skin is constantly undergoing rapid changes in color, too subtle for us to notice, that reflect our body’s physical state and functioning. Binah.ai’s technology picks on these changes to draw conclusions about our vital signs, said company co-founder and CEO David Maman.
“Basically we’re following around the tiny color changes that are happening to the skin and the tiny color changes indicate the blood flow that is happening below the skin surface,” Maman said.
Someone using the company’s system looks at their phone’s camera, which detects exposed skin on the face, for up to 45 seconds. The cameras’ video functions can record video at a rate of 30-120 frames per second, meaning the phone has a lot of information to work with. The phone analyzes the video frames to process the differences in red, green and blue (RGB) light reflected by the skin while the user is looking at the camera.
The platform makes use of rPPG (remote photoplethysmographic) imaging technology more commonly known from Apple Watches and Fitbit devices. The system does not record video, but processes it in real-time, which lessens privacy risks.
Binah.ai’s technology does not need a powerful camera with high resolution, because color changes are relatively easy for cameras to detect, but needs a lot of processing power in the device itself, Maman said. In addition to analyzing the images for color changes, the technology needs to carry out camera calibration, face detection, face tracking, skin region selection, motion compensation and illumination normalization. With Apple products, for example, only the iPhone 8 or more recent models are compatible.
The only other limitations are caused by light — if it’s too dark, or if a powerful light source, like the sun, is behind a user, the technology will not work because the camera cannot pick up the changes in color tones. In low-light conditions, Binah.ai users can scan their vital signs using the flashlight on the back of most phones to extract indications from a user’s finger instead.
The technology uses artificial intelligence to detect a user’s face, and find the exposed skin, so skin tone, facial structure and obstructions such as glasses, hair or a beard are not an issue.
Machine learning mechanisms process the images differently for each vital sign, with some of the signs extractable through pure signal processing. Heartrate, for example, can be determined by detecting the pulse through peak blood flow in the skin.
“The pulse is actually the peak that is happening as part of the blood flow. You can actually see the peak in RGB colors that you extract from the person’s face,” Maman said. “It’s such a tiny change that our eye cannot even track it, but the RGB signals that are extracted from the face are sensitive to those tiny changes.”
Other vital signs, such as blood pressure, are more difficult to ascertain. The company had to build a massive data set, with the help of hospitals, that included video footage of hundreds of people while they were connected to medical devices. The video was processed, and multiple indicators drawn out from the subjects’ faces to establish a model, Maman said.
Binah.ai’s blood pressure function is in its final stage of validation at Tel Hashomer hospital.
The Times of Israel tested the technology on a recent model iPhone. Binah.ai’s system accurately determined heartrate, compared to readings from an Apple Watch and a manual count.
Binah.ai does not offer the public an app, but is a business-to-business firm that provides a software development kit (SDK), or code that customers can integrate with their own apps. It is compatible with iOS, Android and Windows operating systems.
Some of the firm’s current customers include the health insurance companies Momentum, in South Africa; Generali, in Europe; and Sompo, in Japan. One health care provider in Israel is awaiting approval to use the technology, Maman said.
The technology has not yet been approved for medical use, meaning it cannot be used to make medical decisions, but it can be used for health monitoring. So, for example, a physician cannot prescribe medicine based on Binah.ai’s readings, but an insurance company can suggest someone speak with a doctor, or send them a nurse.
The pandemic spurred the need for technology that can support remote patient monitoring, as health systems were overwhelmed and some people became afraid to visit medical clinics. Binah.ai’s amount of customers tripled during the pandemic, Maman said.
COVID-19 also made more people aware of the need to provide medical services remotely. Today, for example, many patients are sent home from hospitals with devices to monitor their health as they continue to recover, which is not always practical.
“People understand that the future is not to send you dozens of different devices that you will own and have available for yourself. It’s actually that any smartphone will be able to provide those kinds of services,” Maman said. “There are places that don’t even have a sewer system but everyone has a smartphone.”
For the company’s next steps, it is seeking medical approval in several regions and working on adding new vital sign measurements, such as body temperature and blood alcohol levels.
Maman envisions the technology being employed in an array of use cases, including for cosmetics treatment, elderly care, remote nursing and smart home appliances.
A smart refrigerator could read a person’s physical indications and make nutrition recommendations, or a shower mirror could perform a daily checkup in the morning while someone brushes their teeth and notify them of potential problems, for example. Or pharmaceutical companies could increase medical adherence — a major problem in which patients do not take prescriptions, or do not use them correctly — to issue suggestions or reminders about when to take a pill.
The company is based in Ramat Gan, outside Tel Aviv, and has 67 employees, mostly in Israel. Investors include iAngels, Maverick Ventures and Sompo International.
