Robot foundation model

With older robots, a person had to program every motion for every job. A robot foundation model first learns from huge numbers of videos and records of robots moving. So it can understand a scene seen through a camera and what a person says, and produce right away how to move the arms and hands. Because it sees (vision), understands (language) and moves (action), it is also called a VLA model.

The goal is to handle many kinds of robots and many kinds of jobs with one model. But unlike text or pictures, records of robots moving are not piled up on the internet, so they are hard to collect. That is why these models are still clumsy in a house they have never visited or in front of an unfamiliar object.

It is like a new hire who had to memorize a separate manual for every job, and after working in many workplaces, moves on their own just from hearing "clear that cup away."

For humanoids to be useful in real workplaces, beyond fixed demonstrations, this brain has to get better. News that a robot company and an AI company have teamed up usually means they are building this kind of model together.