The earphone uses two RGB cameras that are positioned below each ear. They can record changes in cheek contours when the wearer’s facial muscles move.
Once the images have been reconstructed using computer vision and a deep learning model, a convolutional neural network analyzes the 2D images. The tech can translate those to 42 facial feature points representing the position and shape of the wearer’s mouth, eyes and eyebrows.
C-Face can translate those expressions into eight emoji, including ones representing neutral or angry faces. The system can also use facial cues to control playback options on a music app. Other possible uses include having avatars in games or other virtual settings express a person’s actual emotions. Teachers might be able to track how engaged their students are during remote classes too.
Due to the impact of COVID-19, the researchers could only test C-Face with nine participants. Still, the emoji recognition was more than 88 percent accurate and the facial cues more than 85 percent accurate. The researchers found the earphones’ battery capacity limited the system. As such, they plan to develop less power-intensive sensing tech.