Can we draw a picture of what you look like from a voice recording?

June 13

A few weeks ago we talked about how Baidu were able to make a fake recording of you saying anything from a 3.7 second recording of your voice (see the May 16 newsletter). What else can we infer from your voice?

When we listen to the radio or a podcast we usually develop a strong mental image of what the person speaking looks like. There is a strong connection between speech and appearance, part of which is a direct result of the mechanics of speech production, age, gender, the shape of our mouth, facial bone structure and lip shape and thickness. There are also voice-appearance correlations that stem from the way that we talk; language, accent, speed and pronunciation. These properties are often shared among nationalities and cultures.

Can we draw a picture of what you look like from a voice recording?

MIT have tried to do just that. They designed a deep neural network to perform the task of creating a visual image of the face of a speaker by only using a voice recording. The used millions of natural Internet/YouTube videos of people speaking to train the network. During training the model learned voice-face correlations that allow it to produce images that capture physical attributes of the speaker such as age, gender and ethnicity.

Here is a visual representation of the process that they followed.

How did they do? The goal of the study was to discover the extent that we can infer what a person looks like from the way they talk. The results did not show a one to one matching between faces and voices, however they did capture the dominant facial features. Here are some sample results.

This is very early research in this space. Further training of the model with vastly more data and a wider range of data will improve the ability of the model to develop more accurate outcomes. As the output becomes more accurate it becomes more useful. Commercial applications may still be some time away however applications will one day arise.

Paying it Forward

If you have a start-up or know of a start-up that has a product ready for market please let me know. I would be happy to have a look and give the start-up a shout out to my readers if it is something that I think they could use. If you have any questions or comments please email me via my website craigcarlyon.com

Till next week.