A team at the University of Washington has unveiled a proof-of-concept device that uses artificial intelligence to let headphone wearers isolate and hear a single speaker in a noisy crowd simply by looking at them for a few seconds. The system, called Target Speech Hearing (TSH), was presented May 14 at the ACM CHI Conference in Honolulu.

To enroll a speaker, the user taps a button while facing the desired talker. Microphones on the headphones capture the audio, and an onboard embedded computer runs machine learning software that learns the speaker’s vocal patterns. The system then cancels all other environmental sounds and plays back only the enrolled voice in real time, even as the listener moves and no longer faces the speaker. The team tested the system on 21 subjects, who rated the clarity of the enrolled voice nearly twice as high as unfiltered audio.

The system currently can enroll only one speaker at a time, and struggles if another loud voice comes from the same direction. The researchers, led by professor Shyam Gollakota, are working to expand to earbuds and hearing aids. The code is available for others to build on.