Berkeley Talks: Jitendra Malik on the sensorimotor road to artificial intelligence
For a robot to have the ability to navigate specific terrain, like stepping stones or stairs, it needs some kind of vision system
March 24, 2023
Follow Berkeley Talks, a Berkeley News podcast that features lectures and conversations at UC Berkeley.
Jitendra Malik, a professor of electrical engineering and computer sciences at UC Berkeley, gave a lecture on March 20 called, “The sensorimotor road to artificial intelligence.” (Screenshot from video by Berkeley Audio Visual)
In Berkeley Talks episode 164, Jitendra Malik, a professor of electrical engineering and computer sciences at UC Berkeley, gives the 2023 Martin Meyerson Berkeley Faculty Research Lecture called, “The sensorimotor road to artificial intelligence.”
“It’s my pleasure to talk on this very, very hot topic today,” Malik begins. “But I’m going to talk about natural intelligence first because we can’t talk about artificial intelligence without knowing something about the natural variety.
“We could talk about intelligence as having started about 550 million years ago in the Cambrian era, when we had our first multicellular animals that could move about,” he continues. “So, these were the first animals that could move, and that gave them an advantage because they could find food in different places. But if you want to move and find food in different places, you need to perceive, you need to know where to go to, which means that you need to have some kind of a vision system or a perception system. And that’s why we have this slogan, which is from Gibson, “We see in order to move and we move in order to see.”
For a robot to have the ability to navigate specific terrain, like stepping stones or stairs, Malik says, it needs a vision system.
“But how do we train the vision system?” he asks. “We wanted it to learn in the wild. So, here was our intuition: If you think of a robot on stairs, its proprioception, its senses, its joint angles can let it compute the depth of its left leg and right leg and so on. It has that geometry from its joint angles, from its internal state. So, can we use it for training? The idea was the proprioception predicts the depth of every leg and the vision system gets an image. What we asked the vision system to do is to predict what the depth will be 1.5 seconds later.
“That was the idea — that you just shift what signal it will know 1.5 seconds later and use that to do this advanced prediction. So, we have this robot, which is learning day by day. In the first day, it’s clumsy. The second day, it goes up further. And then, finally, on the third day, you will see that it … makes it all the way.”
Malik’s lecture, which took place on March 20, was the first in a series of public lectures at Berkeley this spring by the world’s leading experts on artificial intelligence. Other speakers in the series will include Berkeley Ph.D. recipient John Schulman, a co-founder of OpenAI and the primary architect of ChatGPT; a professor emeritus at MIT and a leading expert in robotics, and four other leading Berkeley AI faculty members who will discuss recent advances in the fields of computer vision, machine learning and robotics.
Watch a video of Malik’s lecture below.
Listen to other episodes of Berkeley Talks: