Scientists Mimic a Camera with Ear-Based Sonar to Image a Human Face

Traditionally, camera-based technology has been used to track facial movements in real-time. But now, researchers at Cornell University have developed a wearable earphone device — or “earable” — that bounces sound off of a human’s cheeks and transforms the echoes into an avatar of that person’s entire moving face.

The device performs as well as camera-based face tracking technology but uses less power, offers more privacy, and is less vulnerable to hackers, according to the researchers.

Devices that track facial movements using a camera are “large, heavy and energy-hungry, which is a big issue for wearables,” lead researcher Cheng Zhang tells the Cornell Chronicle .

“They capture a lot private information,” Zhang says.

Zhang says that facial tracking through acoustic technology — rather than camera technology — can offer better privacy, affordability, comfort and battery life.

The system, called EarIO, transmits facial movements to a smartphone in real-time and is compatible with commercially available headsets for hands-free video conferencing. EarIO functions like a ship that sends out sonar pulses. A speaker on each side of the earphone sends acoustic signals to each side of the face, using a microphone to pick up echoes. As wearers talk, smile or raise their eyebrows, the skin moves and stretches, changing the echo profiles. A deep learning algorithm developed by the researchers uses artificial intelligence (AI) to continually process the data and translate the shifting echoes into complete facial expressions.

The “earable”, which uses sound rather than data-intensive photos, can communicate via Bluetooth with smartphones. This keeps a user’s personal information safe. The device will need to be connected to Wi-Fi networks to send images back and forth to cloud. This could make it susceptible to hackers.

Using acoustic signals also takes less energy than recording images, and the EarIO uses 1/25 of the energy of other camera-based systems. Currently, the earable lasts around three hours on a wireless earphone battery, but future research will focus on extending the use time.

One limitation of the technology is that before the first use, the EarIO must collect 32 minutes of facial data to train the algorithm but Zhang says they hope to “make this device plug and play” in the future.


Image credits: Header photo by Cornell University.

Loading...