AI Powered Headphones, Whole Body Teleoperation and Your Cells can Hear
May 22
This week we examine a new AI driven headphone that can translate speakers in crowded spaces and follow the speaker as the user moves through the crowd. We investigate a new system of teleoperation which allows humanoid robots to copy the full body movements of a remote operator. We discover that our cells can hear sound and consider the potential applications. Finally we encounter a robotic centipede that has applications in agriculture.
AI Powered Headphones
A team at the University of Washington has developed headphone system that translates speakers in real time. The system allows you to hear a translation in a crowded room with a 2 to 4 second delay. Other current systems require silence around the speaker for the translation to be somewhat accurate.
The team built the Spatial Speech Translation system with off the shelf noise cancelling headphones fitted with microphones. They developed an algorithm that can separate out different speakers in a space and follow the speaker as it moves. The system maintains the acoustic features of the speakers voice when speaking in the translated language.
The system initially identifies how many speakers that there are in an indoor or outdoor space. Somewhat like a radar, the algorithm scans the 360 degree space determining and updating the number of speakers. The system then translates the speech while maintaining the expressive qualities and volume of each speaker. The system runs on mobile devices with an Apple M2 chip or higher (laptops or VisionPro). When a speaker moves their head, the system continues to track the direction and qualities of their voice as they move.
The system currently only works well on commonplace speech, not specialized language or technical jargon. The system can be trained to translate around 100 languages.
Whole Body Teleoperation
Teleoperation is the ability to remotely control robots in real time. A team at Stanford University (California) and Simon Fraser University (Canada) have developed a system that allows humanoid robots to closely imitate the whole body motions of human users in real time.
The system leverages motion capture data, reinforcement learning and imitation learning. If a human picks up a package with two hands, opens the door with the side of their body, uses elbows or knees to push it further open we want humanoid robots to be capable of similar movements in order for them to operate in human environments.
The system is powerful enough to control humanoid robots arms, legs, feet, waist etc.. The humanoid can move as a human does. This enables the robot to learn tasks in a human environment. These new foundation models for robotics can then be developed for robots to learn autonomous skills.
Your Cells can Hear
If you have ever stood in front of a speaker stack at a concert the sound is all encompassing and you seem to be able to feel the sound. This is your ears, brain and cells all experiencing the sound.
Sound is a series of compressed mechanical waves transmitted through substances. A team at Kyoto University (Japan) have investigated the cellular responses to sound transmission. The team developed a system to bathe cultured cells in acoustic waves.
The team exposed cells in a culture dish to acoustic pressure within the range of physiological sound before analyzing the effect of the sound using RNA sequencing and other methods. The results showed cell level responses.
The team found that the sound suppressed adipocyte differentiation, the process by which preadipotcytes transform into fat cells. This may mean that acoustics can be used to control cell and tissue states. Sound is noninvasive therefore it may lead to tools that provide safe and immediate benefits.
Robotic Caterpillars
Atlanta Georgia based startup, Ground Control Robotics have developed a giant robotic arthropod to assist with agricultural crop management. There are many long skinny wiggly animals that thrive in agricultural environments. The team studied these animals and developed a robot that could use those insights.
The robot has a head with sensors. It is connected via cable to several identical units that each have motors and legs for movement. Centipede robots are like snake robots i.e. they are swimmers. Adding the legs allows them to move through different kinds of environments. The correct pattern of lifting and lowering legs provides a fluid like thrust force that helps the robot motion more consistent and reliable.
The robot is suited to perennial crops where the plant lives for many years and not always on flat ground in neat rows. For example wine grapes can grow on steep rocky surfaces. The caterpillar robot allows automated weeding and other functions. Estimates are that weed control in strawberry fields are up to US$1,000 per acre. Humans don’t want to do this type of work any more, this is where the robots come in.
Initially the robots will be used for scouting fields however in the future lasers or some other weed destruction method could be added. The robots will work 24/7 for a lower cost than current options.
Paying it Forward
If you have a start-up or know of a start-up that has a product ready for market please let me know. I would be happy to have a look and feature the startup in this newsletter. Also if any startups need introductions please get in touch and I will help where I can.
If you have any questions or comments please comment below.
I would also appreciate it if you could forward this newsletter to anyone that you think might be interested or provide a recommendation on Substack.