Berkeley Voices: How seeing a new color stretches the limits of human perception

Anne Brice (narration): This is Berkeley Voices, a UC Berkeley News podcast. I’m Anne Brice.

(Music: “Trois Gnossiennes” by Blue Dot Sessions)

Last month, UC Berkeley researchers published a study about how they tricked the eye into seeing a new color. It was a highly saturated teal, a peacock green, the greenest of all greens. 

The scientists produced this color, which they named “olo,” by shining a laser into the eye and stimulating one type of color-sensitive photoreceptor cells called cones. 

The world was — and still is — very excited about this new discovery. Some who’ve heard about it have claimed that they, too, have seen the color. One person is even selling it as an acrylic paint color for $10,000 (with a steep coupon code, I have to add) with the name Yolo. But unless they were in this lab at Berkeley, the only lab where it has happened, it’s not actually possible. That’s because the cones the researchers stimulated — the middle cones, or M cones, which are sensitive to green light — can’t be stimulated by themselves in a natural setting. 

Austin Roorda: There’s no light in nature that can only stimulate the M-cones. 

(Music fades out)

Anne Brice (narration): Austin Roorda is a professor of optometry and vision science at the School of Optometry at Berkeley.

Austin Roorda: So we have long, middle and short wavelength-sensitive cones. And they’re called L, M and S. And the M cones are kind of sandwiched between the L and the S cones. They’re not sensitive to a single wavelength, but a range of wavelengths. So as a result, the sensitivities kind of all overlap. So whenever you tickle the M cone, you’re always tickling the L and the S a little bit.

Anne Brice (narration): Since 2018, Roorda has been developing the optical imaging platform the scientists used in this project. It’s called Oz, after the story The Wonderful Wizard of Oz.

Austin Roorda: And so, of course, Ozvision is really directly tied to the book and to the movie where the Emerald City is this unearthly green color. And, of course, in the movie, it was a major development in cinema, not the first color movie, but among the first, and they made this transition from the black-and-white world to the color world of Oz. 

(Music: “Coffeetop” by Blue Dot Sessions)

The intent and the aspiration was to elicit that same kind of response by going from a natural-colored world to a supernatural-colored world by a direct stimulation of these cones.

Anne Brice (narration): The study involved a team of researchers, including Ren Ng, a professor of electrical engineering and computer sciences at Berkeley. Their collaboration began years ago when Ng asked Roorda a question. 

Austin Roorda: Ren asked, “What would happen if we delivered light to thousands of M cones only?” He said, “Would it be the greenest green you’ve ever seen?” 

Anne Brice (narration): Turns out, the answer was “yes.”

Ng and Roorda were two of five people who saw olo as part of the study. More have seen it since then, but time and logistics limited the number of subjects who could see it. 

(Music fades out)

Roorda describes sitting perfectly still, gripping a bite plate between his teeth to keep his head from moving, staring into the Oz machine. The patch of color was about the size of his fingernail when viewed at arms’ length. When compared to the most saturated natural color of green next to it, Roorda says the natural green “paled in comparison.” 

Austin Roorda: It was exciting for me to perceive it, but it wasn’t like an otherwordly experience. You know, I tell people that, we have people who want to experience olo, and they’ll say, “Oh, I’ll do whatever it takes.” And I’d say, “Prepare to be underwhelmed,” because it’s just a small spot. (Laughs)

If we could deliver it over a large area, it would be really quite a profound experience, I’m sure. But until then, I think it’s most profound for the scientists and the people in the lab that are doing these comparisons.

Anne Brice (narration): It was profound, he says, not only because their project, years in the making, had worked, but because the brain was able to recognize and make sense of something it had never been exposed to before.  

Austin Roorda: It’s really about the capacity of the human brain to develop new perceptions to attribute to new sensory inputs. 

This could apply to any sensory inputs. It just turns out that we have a platform where we can directly manipulate the sensory inputs into the brain through the visual system in an unprecedented way. 

(Music: “Waltz and Fury” by Blue Dot Sessions)

Anne Brice (narration): It has enormous potential to transform how we understand and treat eye diseases, Roorda says, and to expand the way we see the world around us.

(Music fades out)

Anne Brice (narration): It’s easy, as humans, to assume everyone is seeing the world as you are. And to be fair, most of us do, at least visually. The majority of people are trichromats, which means that we distinguish colors using three different types of photoreceptor cells in our retinas.

Austin Roorda: The human color vision system is really quite incredible, that with just three types of cone photoreceptor — the L and the M and the S — we can compare the relative excitations of those three types to immediately and effortlessly appreciate up to 10 million different hues. Like, we can distinguish all the subtle differences in hues in the world and perceive them. 

This Oz platform not only allows us to elicit color sensations that natural light would not, but we can use this as a tool to try to understand the basic processing of colors that humans perform when we’re looking at the world.

Anne Brice (narration): Perception, Roorda says, can be shaped and learned and developed through exposure. Children’s brains, for example, are especially open to taking in and making sense of new stimuli. 

Austin Roorda: They’re bombarded with sensory inputs. From the day they’re born, their brain is working really hard to try to make sense of it. And then certain perceptual qualities start to evolve over time. They start to see in stereo. They start to see in color. They refine their sensation of space. And so, when you’re older, there may be a little less plasticity to do that, but we believe it’s there.

(Music: “Thumbscrew” by Blue Dot Sessions)

Anne Brice (narration): Even though our brains and how they process what we’re seeing gets a little more rigid as we get older, what could, say, seeing a new color do to the brain? Would it make it more flexible, more pliable, more receptive?

Kara Manke, my colleague at UC Berkeley News who co-produced this episode, asked Roorda. 

Kara Manke: I’m just wondering if our understanding of olo and our ability to see olo can give us any understanding of how to change or expand our perceptions of sensory phenomena, in general. Kind of a philosophical question. 

(Music fades down)

Austin Roorda: Well, that is one of the prevailing big questions in our lab. And there are a couple of theories. 

One theory is that you have, the way Ren describes it, is you’re born with a box of crayons. And those are the colors that you have at your disposal, and given the sensory inputs, you describe which crayon would be most appropriate. In that sense, there’s no opportunity for perceptual expansion. You would just be limited to that box. If something was new, you’d just have to reassign the box of crayons to attribute to those inputs. 

On the other hand, maybe there’s no limit to the percepts that we have. There’s no end to the number of crayons that you can have. We like to think that.

(Music comes up, then fades out)

Anne Brice (narration): Although most people are trichromats, there are also dichromats, who have two cones in their retinas, and tetrachromats, who have four. Most often, dichromats have difficulty telling red and green apart, and tetrachromats have another type of L cone that varies only slightly from the first L cone. 

Scientists at Cambridge University have studied tetrachromats to find out whether they could see more hues of color than a trichromat. While most tetrachromats couldn’t, there was one person who could clearly leverage her fourth cone type to appreciate greater differences in hues. 

(Music: “Spark” by Blue Dot Sessions)

And then, the number of cone types in different animal species’ eyes varies wildly. 

Austin Roorda: There’s mantis shrimp, for example, who have over 15 different types of cone. Some people say, “Well, mantis shrimp, because they have 15 types of cone, they might be able to see way, way, way more colors and appreciate way more different spectra than a human could.” 

But on the other hand, maybe the mantis shrimp brain does not know how to deal with that. And maybe they can only see 15 colors. So there’s two parts of the story, right? 

The one part is, how many cone types do you have? And the other part is, how does the brain leverage those cone types? And so a human leverages the three cone types in a really amazing way, like I said, to identify millions of different hues in the environment, to differentiate between them. 

Hummingbirds, for example, have an extra cone that resides kind of in the UV part of the spectrum. And studies have shown that hummingbirds can do the same. 

(Music fades out)

Anne Brice (narration): So if the Berkeley researchers’ theory is true — that the human brain, this “wonderful machine,” as Roorda calls it, is powerful enough to decode colors it had never seen before — it could potentially open up a new dimension of experience. 

And research suggests that the human brain could, in fact, leverage this new information quite well.

One of the Oz project’s chief collaborators, Ph.D. student Atsu Kotani, is running simulations where a computer can easily figure out inputs from four cone types and then generate a new dimension of vision. 

Austin Roorda: If a machine can do that, why couldn’t the human brain do that? 

Anne Brice (narration): There was also a 2009 study at the University of Washington that involved squirrel monkeys, which are dichromatic. Using gene therapy, the scientists gave the monkeys a third cone type. Afterward, the monkeys could differentiate between reddish and greenish tones, when they couldn’t before. 

So their brains were able to make sense of the new colors they were seeing and incorporate this new visual information into their lives. 

This type of gene therapy, although not available to humans right now, could be possible in the future, Roorda says. And Oz is one important tool to help us get there. 

(Music: “Coulis Coulis” by Blue Dot Sessions )

Anne Brice (narration): There are many eye diseases where people’s cones are damaged or lose functionality over time, resulting in vision loss. 

Hannah Doyle, a fourth-year Ph.D. student in the Department of Electrical Engineering, ran the Oz experiment. She says there are many potential therapeutic applications of Oz. 

Hannah Doyle: There are diseases that cause cones to die or to be lost in the retina. And people have been interested in how those diseases affect visual function. And typically, in order to study something like that, you would have to find patients who have the disease to figure out what stage of the disease they’re at, do some functional tests on their vision, that kind of thing. 

But now that we have the Oz system, we can very easily emulate the conditions of these kinds of diseases. 

(Music fades out)

Anne Brice (narration): In the same way that the researchers stimulated only the M cones for the eye to see olo, they can now use the imaging platform to stimulate a certain percentage of cones, mimicking the symptoms of cone loss of a given degenerative eye disease. 

This would allow doctors to better understand the experience of people with specific eye diseases and then treat them accordingly. 

Hannah Doyle: So you could wonder, how would you do looking at an eye chart if you’ve lost 70% of your cones? Can you still read the letters? What’s the smallest letter you could read? So I’m answering questions like that. 

It turns out that you can lose a lot of cones and still perform almost completely normally on an eye chart. You could lose 50% of your cones and have about the same visual acuity, is what I’m finding. 

Another cool thing that we’ve been noticing qualitatively is that you would imagine that if a bunch of the cones in your retina are dead, you might think that it would look very speckly like to you. Like, you would see dark spots where those cones are missing. 

And what people have been saying when I run these experiments is that maybe at first, when they’re looking at an image or something with this emulated cone loss, it looks speckly to them. But then there’s some sort of perceptual filling-in process that happens, and they start to just see things quite normally over time. 

So we’re also interested in how the visual system is seeming to compensate for that cone loss and generate a uniform percept. 

Austin Roorda: Just to add to that, if Hannah finds that you can still see reasonably well, maybe good enough to pass a driver’s test, the vision test, if you can do that with just 70% of your cones missing, then people that are developing treatments for eye disease are encouraged to know that, if they do a gene therapy or a stem cell therapy, If they could restore their cone densities to 30% of normal, that patient will have a high quality of life. 

So these are numbers that are very important in the area where there are treatments for any retinal disease coming online, like stem cells or gene therapies.

(Music: “10c Coffee” by Blue Dot Sessions)

Anne Brice (narration): There are huge personal costs of vision loss, says Roorda, and the benefits of preventing it are enormous. 

Austin Roorda: I might be a little biased, but it’s our most precious sense. And it’s the last one I would want to give up, for sure.

If you can give someone a better quality of life by maintaining their vision, whether it’s for reading or doing your hobbies or for driving, this is really important for the individual, but also for the health benefit of the world. 

Anne Brice (narration): Like it did for Dorothy, Oz could impact the way we see our surroundings in dramatic ways that we can’t even imagine yet. 

I’m Anne Brice, and this is Berkeley Voices, a UC Berkeley News podcast from the Office of Communications and Public Affairs. This episode was produced by Kara Manke and me. Script editing by Tyler Trykowski and Gretchen Kell. Music by Blue Dot Sessions. 

You can find Berkeley Voices wherever you listen to podcasts, including YouTube @BerkeleyNews. 

This is the final episode of our Berkeley Voices series on transformation. In eight episodes, we looked at how transformation — of ideas, of research, of perspective — shows up in the work that happens every day at UC Berkeley. We’ll be back with a new series in the fall.

We also have another show, Berkeley Talks, that features lectures and conversations at Berkeley. You can find all of our podcast episodes on UC Berkeley News at news.berkeley.edu/podcasts.

(Music comes up, then fades out)