Using AI, these robots learn complicated skills with startling accuracy

At UC Berkeley, researchers in Sergey Levine’s Robotic AI and Learning Lab eyed a table where a tower of 39 Jenga blocks stood perfectly stacked. Then a white-and-black robot, its single limb doubled over like a hunched-over giraffe, zoomed toward the tower, brandishing a black leather whip. Through what might have seemed to a casual viewer like a miracle of physics, the whip struck in precisely the right spot to send a single block flying out from the stack while the rest of the tower remained structurally sound.

This task, known as “Jenga whipping,” is a hobby pursued by people with the dexterity and reflexes to pull it off. Now, it’s been mastered by robots, thanks to a novel, AI-powered training method. By learning from human demonstrations and feedback, as well as its own real-world attempts, this training protocol teaches robots how to perform complicated tasks like Jenga whipping with a 100% success rate. What’s more, the robots are taught at an impressive speed, enabling them to learn within one to two hours how to perfectly assemble a computer motherboard, build a shelf and more.

Fueled by AI, the robot learning field has sought to crack the challenge of how to teach machines activities that are unpredictable or complicated, as opposed to a single action, like repeatedly picking up an object from a particular place on a conveyor belt. To solve this quandary, Levine’s lab has zeroed in on what’s called “reinforcement learning.” Postdoctoral researcher Jianlan Luo explained that in reinforcement learning, a robot attempts a task in the real world and, using feedback from cameras, learns from its mistakes to eventually master that skill. When the team first announced a new software suite using this approach in early 2024, Luo said they were heartened that others could quickly replicate their success using the open-source software on their own.

This fall, the research team of Levine, Luo, Charles Xu, Zheyuan Hu and Jeffrey Wu released a technical report about its most recent system, the one that aced the Jenga whipping. This new-and-improved version added in human intervention. With a special mouse that controls the robot, a human can correct the robot’s course, and those corrections can be incorporated into the robot’s proverbial memory bank. Using an AI method called reinforcement learning, the robot analyzes the sum of all its attempts  — assisted and unassisted, successful and unsuccessful — to better perform its task. Luo said a human needed to intervene less and less as the robot learned from experience. “I needed to babysit the robot for maybe the first 30% or something, and then gradually I could actually pay less attention,” he said.

The lab put its robotic system through a gauntlet of complicated tasks beyond Jenga whipping. The robot flipped an egg in a pan; passed an object from one arm to another; and assembled a motherboard, car dashboard and timing belt. The researchers selected these challenges because they were varied and, in Luo’s words, represented “all sorts of uncertainty when performing robotic tasks in the complex real world.” 

The timing belt task stood out in terms of difficulty. Every time the robot interacted with the timing belt — imagine trying to manipulate a floppy necklace chain over two pegs — it needed to anticipate and react to that change.

Jenga whipping constitutes a different kind of challenge. It involves physics that are difficult to model, so it’s less efficient to train a robot using simulations alone; real-world experience was critical. 

The researchers also tested the robots’ adaptability by staging mishaps. They’d force a gripper to open so it dropped an object or move a motherboard as the robot tried to install a microchip, training it to react to a shifting situation it might encounter outside a lab environment.

By the end of training, the robot could execute these tasks correctly 100% of the time. The researchers compared their results to a common “copy my behavior” method known as behavioral cloning that was trained on the same amount of demonstration data; their new system made the robots faster and more accurate. These metrics are crucial, Luo said, because the bar for robot competency is very high. Regular consumers and industrialists alike don’t want to buy an inconsistent robot. Luo emphasized that, in particular, “made-to-order” manufacturing processes like those often used for electronics, automobiles and aerospace parts could benefit from robots that can reliably and adaptably learn a range of tasks.

The first time the robot conquered the Jenga whipping challenge, “that really shocked me,” Luo said. “The Jenga task is very difficult for most humans. I tried it with a whip in my hand; I had a 0% success rate.” And even when stacked up against an adept human Jenga whipper, he added, the robot will likely outperform the human because it doesn’t have muscles that will eventually tire.

The Levine lab’s new learning system is part of a broader trend in robotics innovation. Over the past two years, the larger field has moved in leaps and bounds, propelled by industry investment and AI, which gives engineers turbocharged tools to analyze performance data or image input that a robot might be observing. Berkeley professors and researchers are part of this upswell in innovation; various cutting-edge robotics companies that have received substantial venture funding or even gone public have campus ties.

Levine co-founded the robotics company Physical Intelligence (PI), which is currently valued at $2 billion for its progress toward creating software that can work for a variety of robots. In its latest funding round, PI raised $400 million from investors, including Jeff Bezos and OpenAI. In 2018, Professor Ken Goldberg and other Berkeley researchers formed Ambi Robotics, which has raised some $67 million; the company creates robots trained via AI simulations that grasp and sort parcels into different containers, making them indispensable to e-commerce businesses. Pieter Abbeel, a director of the Berkeley Artificial Intelligence Research Lab, co-created the AI robotics startup Covariant, whose models — and brain trust — were enlisted by Amazon last year. And Homayoon Kazerooni, professor of mechanical engineering, founded the publicly traded company Ekso Bionics, which makes robotic “exoskeletons” for use by people with limited mobility. 

As for Luo’s research, he’s excited to see where his team and other researchers can push it. One next step, he said, would be to pre-train the system with basic object manipulation capabilities, eliminating the need to learn those from scratch and instead progressing straight to acquiring more complex skills. The lab also chose to make its research open source so that other researchers could use and build on it. 

“A key goal of this project is to make the technology as accessible and user-friendly as an iPhone,” Luo said. “I firmly believe that the more people who can use it, the greater impact we can make.”