Algorithms to look at video chat to detect heart rate and pressure levels

Israeli-based start-up has developed a new artificial intelligence (AI) algorithm that detects key vital signs such as heart rate and stress levels by observing changes in face color in video chat, and could lead to ubiquitous telemedicine. is one of the companies that uses AI to measure critical vital signs by video, a technology that transfers healthcare from hospital to home through telemedicine. can now test heart rate, stress levels, oxygen saturation, breathing, and heart rate variability. The company also plans to eventually add blood pressure tests to its list of services.

So how does the algorithm detect all these vital signs by video? has developed a technique called volume tracing, which uses cameras to detect a person’s facial color to detect slight changes in the pulse. declined to explain how to detect vital signs because of patent filings, but external studies on volume tracing provide more evidence that minor changes in facial color can also show breathing and oxygen levels.

Algorithms to look at video chat to detect heart rate and pressure levels

For more than a decade, researchers around the world have been studying volume tracing, and it has been used in an emerging field called “emotional computing”, which uses AI to understand a person’s emotional state as a diagnostic tool or to allow computers to react in a more emotionally intelligent way.

There has been no publicly available study to prove the accuracy of’s technology, although the company claims that potential customers have tested it with existing pulse-measuring medical devices, saying it is very accurate. Past studies have shown that controlled lighting and clear faces (i.e. no beards, glasses, or other blurred face decorations) produce the best results.

So far,’s technology has been embedded in an app by Sompo, a Japanese insurance agency, that allows drivers to monitor their stress levels while driving. But says its main goal is to introduce the technology into traditional medicine.

Mona Popian-Yona, a spokeswoman for the company, said’s software allows them to get basic and important information before meeting with a nurse or doctor face-to-face, reducing hospital admissions. Another way she expects the technology to be used is through video telemedicine. has submitted its technology to the U.S. Food and Drug Administration (FDA) for review and is awaiting approval. At the same time, the company said several medical institutions were testing its algorithms. also plans to launch a consumer-oriented app in March that allows anyone to test their signs and stress.

Mr Popilian-Jona said the company was still working on a pricing model, but could be sure it would charge for the service. The app will allow customers not to store any personal data. If the user does want to keep their data, the application will only save their vital signs and not store any images or videos.

Algorithms to look at video chat to detect heart rate and pressure levels

Other companies plan to roll out similar technology this year. Neurodata Lab, a Moscow-based start-up, launched heart rate detection video technology online last year as a demonstration. The company plans to publicly use the technology in the coming months in an app where its partners focus on healthcare.

A spokesman for Neurodata Lab says the company’s pulse detection technology works just as well as the current pulse detection wristband on the market. Pulse and other key vital signs data can show whether someone is stressed, anxious, or experiencing other emotional states, she said.’s technology has also raised concerns outside the medical community, which law enforcement believes could be useful during interrogations. At present, there are no rules on the use of this particular technique. Like a lie detector, the police may not be able to force a suspect to take part in a facial scan.

However, the collection of biological data outside medical facilities under U.S. health care law is still unclear, and there is no uniform standard for how that data should be handled and protected. For this reason, companies like and Neurodata Lab have largely left the responsibility for their algorithms to collect data to customers.

Popilian-Jona insists that her company’s greatest interest is in providing doctors with diagnostic tools rather than interrogation tools for the police. “We want to put this technology in the hands of healthcare providers,” she said. “