Robotic pest control and sporting spectators, helping the deaf to communicate

April 9

Welcome to another week completely dominated by Covid-19. This week I have two stories caused by Covid not about Covid. They really are stories about robots. I did promise a CFZ (Covid Free Zone). I will also look at ways that we can help the deaf communicate, an automated pest control robot for agriculture, a different type of sports fan and a novel graduation ceremony in Japan.

Helping the Deaf Communicate

Text to Speech Systems

Hearling has developed a system that makes it easy to create high quality audio from text. Simply type some text into their system and a voice will read out the text.

Try it out on their web page. The system is very good but I found it does struggle with words that are pronounced differently in different contexts or tenses. For example: read. The system is available in 28 languages (it is not a translation service only a reading service), with a range of different voices and dialects for each language. The system allows you to choose a male or female voice.

The system uses the Google Cloud text to speech platform. Google uses a Wave-net model for speech creation. It places more human-like emphasis and inflection on syllables, phonemes and words. Google’s research shows a preference for wave-net artificial speech over other forms of artificially generated language. Try some samples here (sorry I am not able to embed the audio into this email). The samples are at the bottom of the page.

Sign Language Gloves

Surprisingly, attempts to assist deaf people communicate automatically or electronically with others that don’t understand sign language date back to the 1980’s. In 1983 a Bell Labs engineer invented a glove for data entry using the 26 manual gestures from the American Manual Alphabet. The first talking glove was invented in 1998 by Stanford University researchers (cost at the time was $3,500 for the system plus the cost of the gloves). In 2001 a High School student in Colorado fitted a leather golf glove with 10 sensors that monitored finger position which was relayed to a computer for text entry. A group of Ukrainians won the 2012 Microsoft Imagine Cup with for their glove project and in 2014 Cornell University students developed a glove that interpreted the user’s signs and translated them into spoken english. In 2016 two University of Washington students developed SignAloud, a pair of gloves that recognized rudimentary ASL (American Sign Language) and turned them into speech. This clip shows the gloves in action.

These attempts at development of automatic communication are not without controversy, particularly within the deaf community. The deaf use far more than hand gestures to communicate. Facial expressions, eyebrow movement, the orientation of the torso and movement of the mouth all play an important part in communication. The gloves currently have no access to these other parts of communication. ASL consists of thousands of signs presented in sophisticated way that have, so far, confounded reliable speech recognition. There are limited data sets of people signing that can be used to train machine-learning algorithms. There is also the complaint that these types of systems don’t represent the deaf community appropriately.

Despite the decades of research there appears to be some way to go to develop a fully functioning and culturally acceptable system. Involvement from the target community will also be critical to success.

LadyBird

Ladybird is a custom designed robot built at the University of Sydney, for the vegetable industry to help farmers to control weeds, bugs and other pests. It is a 4 wheel, omnidirectional system that is powered by solar panels. A single sunny day will allow the robot to work for days on end without recharging. The solar panels also provide cover for the range of sensors and allow for better imaging than is possible in direct sunlight.

The robot uses infrared, optical and laser scanning, 3-D imaging and GPS tracking to aid in pest and weed control. A general purpose manipulator robot arm is in early testing for weed removal or spray. The goal is to reduce the amount of chemicals used, spraying can be very specific thus reducing the chemicals. Eventually, manual removal by the robot arm may eliminate the use of chemicals entirely. In the future there may also be applications where microwaves could also be used to eliminate weeds.

Robot Baseball fans

On the 11th of April, Taiwan will relaunch its’ baseball league. However due to the restriction that a total of 500 people, including players, officials, team support, stadium workers and spectators that could be at the venue, the Rakuten Monkeys enlisted the support of an army of robot spectators to try and bring some atmosphere to the game.

I am not sure how robotic these spectators will be, they look more like mannequins holding up signs. If so, it won’t take long for an enterprising engineer to develop automated chants and cheering in addition to waving flags and signs to inspire the home team.

Maybe we are seeing the future of sport. Whilst we watch at home on our 200inch screens, stadiums full of robotic spectators cheer on our team on for us. An audio link via the robot would let us shout encouragement to the players at the stadium. We might even be able to watch the game from our personal robots point of view. Our mates’ robots sitting beside us, instant replays of things we missed with no queue for beer or the toilets, it just might be our Covid future.

Japanese University Graduation Ceremonies

Like every public gathering in the past month, Graduation Ceremonies for Japanese University Students have been cancelled. This did not deter Business Breakthrough University (that is the real name of the University) from holding a virtual ceremony. A handful of academics greeted a line of robots that had an iPad in place of a head with each student streamed from home. A novel way of graduating.

Some Unique Music

When I said that this week the newsletter would be a CFZ, well, I lied. Only because this music is really, really interesting. Scientists are trying to rapidly understand the Covid-19 virus. MIT has turned to a novel way of looking at the virus. They have used an AI to transform the virus into a classical melody.

The virus is made up of thousands of amino acids. Each amino acid has been given a unique note in the musical scale. Following this the team created an algorithm that converted the data into a melody that shows how the proteins are arranged. Have a listen to the music here.

Mark Buehler the project lead explained that “our brains are great at processing sound. In one sweep our ears pick up all of a piece of music’s hierarchical features; pitch, timbre, volume, melody, rhythm and cords”. A high powered microscope would be needed to see the image in equivalent detail and we could never see it all at once.

I have been listening to the music whilst writing this newsletter. It is actually a beautiful, peaceful piece of music however there are several sections which jar the senses. Maybe this is where researches can hear that the protein patterns have mutated.

Researchers could also seek out an antibody that matches the melody and rhythm of a spike protein on the virus (Corona is the latin word for Crown, the virus has lots of spikes that make it look like a crown, these spikes are made of proteins). The antibody by binding to a spike protein, may interfere with the virus’ ability to infect.

Paying it Forward

If you have a start-up or know of a start-up that has a product ready for market please let me know. I would be happy to have a look and feature the startup in this newsletter. Also if any startups need introductions please get in touch and I will help where I can.

If you have any questions or comments please email me via my website craigcarlyon.com or comment below. 

I would also appreciate it if you could forward this newsletter to anyone that you think might be interested.