Berkeley Talks: ChatGPT developer John Schulman on making AI more truthful
UC Berkeley alumnus John Schulman, the lead developer of ChatGPT, talks about how AI language models sometimes make things up — often convincingly — and offers solutions on how to fix this problem
April 24, 2023
Follow Berkeley Talks, a Berkeley News podcast that features lectures and conversations at UC Berkeley.
Berkeley alumnus John Schulman co-founded OpenAI, where he led the reinforcement learning team that developed ChatGPT. (UC Berkeley photo by Jim Block)
In Berkeley Talks episode 166, UC Berkeley alumnus John Schulman, the lead developer of ChatGPT, talks about how AI language models sometimes make things up — often convincingly — and offers solutions on how to fix this problem.
Language models, Schulman says, have pattern-completion behavior — they’re trained to maximize likelihood of a response. And part of the reason for generating untruthful content, which he calls “hallucinations,” is because the model doesn’t know that it’s allowed to say “I don’t know” or express uncertainty. If you tell a chatbot that it’s allowed to do that, he says, that partially fixes the problem.
“Sometimes it’s like the model is reluctant to challenge a premise … and sometimes it gets caught in a lie. Like, if it makes a mistake, it thinks it should continue, it should produce a coherent response. And that means continuing with the lie. So, I’d say there’s a class of issues that is covered there.
“And then, another set of hallucinations, you could say, is that it’s just guessing wrong. There’s always going to be something that’s a little bit fuzzy, like you’re not sure of this fact, you maybe saw it once, but you don’t fully remember it. And you’re going to have to guess a little bit and sometimes you’re going to guess wrong.”
“So, is it even possible to fix this problem?” Schulman asks.
He thinks that it is — and he says that reinforcement learning is part of the solution.
Schulman’s talk, which took place on April 19, was part of a series of public lectures at Berkeley this spring by the world’s leading experts on artificial intelligence.
Read a Berkeley News Q&A with Schulman in which he discusses why he chose Berkeley for graduate school, the allure of towel-folding robots and what he sees for the future of artificial general intelligence.
Watch a video of his lecture below.