Retraining the brain’s vision center to take action

Neuroscientists have demonstrated the astounding flexibility of the brain by training neurons that normally process input from the eyes to develop new skills, in this case, to control a computer-generated tone.

The researchers from the University of California, Berkeley, Columbia University and the Champalimaud Centre for the Unknown in Lisbon, Portugal, could just as easily have wired these neurons, located in the visual cortex, to operate a robotic arm or any other brain-machine interface, co-opting sensory neurons to do the work of motor neurons.

Areas of the brain that get input from the eyes can be hijacked to control something outside the brain, such as a computer that generates a tone. Yellow arrows represent feedback between the rat’s visual cortex and striatum, which is key to learning new tasks. Ryan Neely image.

“We arbitrarily hijacked small groups of neurons in the visual cortex and virtually re-routed their output to make them control a brain-machine interface, or BMI,” said Jose Carmena, senior author of a paper about the development that will appear March 1 in the journal Neuron. “To gain a reward, the rats learned to produce arbitrary patterns of neural activity unrelated to visual input in order to control a BMI, highlighting the power of neuroplasticity and the flexibility of the brain.”

Carmena and co-first author Ryan Neely also found that the connections from the cortex to the underlying striatum were key to this learning, representing a feedback loop that may be the building block for learning and memory throughout the brain.

“These findings suggest that the striatum has a broader role in shaping cortical activity based on ongoing experience and behavioral outcomes than previously acknowledged, and have wide implications for the neuroscience of thought and action and brain-machine interfaces,” said Carmena, who is a professor in UC Berkeley’s Department of Electrical Engineering and Computer Sciences and the Helen Wills Neuroscience Institute.

Are there limits to brain’s plasticity?

The plasticity of the brain is well known; people with damage to one area of the brain can retrain neighboring areas to take over the lost function. Carmena, who studies how the brain learns new skills and controls movement, including the control of objects outside the body, such as prosthetic limbs, has seen how easily the brain learns to control BMIs.

“When I first came to Berkeley,” he said, “my students and postdocs saw this plastic organization occurring in all the motor areas we have been using for BMI, and they asked: Which are the best areas to control BMI? What is the limit of this plasticity? Is there any area that will not work?”

He and Neely, who completed his Ph.D. with Carmena in 2017, tested these limits by trying to train visual neurons, which normally interpret input from the eyes but don’t control any physical action, to act like motor neurons that control muscles.

Neely implanted 16 electrodes into the visual cortex (V1) of 12 rats. He arbitrarily routed the electrical activity recorded by the electrodes – representing the spiking of several individual neurons – to a device that raised or lowered the pitch of an audible tone depending on which neurons were active.

The rats were then trained over several sessions to raise or lower the pitch, only one of which would reward them with a sweet drink. All rats learned to produce spike activity in the correct set of neurons, while silencing other neurons, in order to obtain the reward. They were successful in the dark – which the researchers thought would minimize interference from light shining in the rats’ eyes – but also performed equally well with the lights on. Electrodes in the dorsomedial striatum, part of the basal ganglia, recorded changes consistent with learning in the visual cortex.

Co-first author Aaron Koralek and the paper’s other senior author, Rui Costa, neuroscientists at the Champalimaud Centre for the Unknown, conducted similar experiments on mice, but also inserted optogenetic switches into cells of the dorsomedial striatum so that they could be turned off with the flash of a laser threaded into the brain. They employed a relatively recent optogenetic tool called Jaws, developed at the Massachusetts Institute of Technology four years ago, to silence the striatal neurons.

“These experiments further demonstrated that the striatum helps the organism learn to control patterns of activity in other areas of the brain, even if those patterns of activity are located in in primary sensory area, such as the visual cortex,” said Costa, who recently established a laboratory at Columbia University’s Mortimer B. Zuckerman Mind Brain Behavior Institute. “These findings suggest that the striatum may be important for regulating an organism’s ability for actively sensing, or perception.”

These mice also learned to control the frequency of the tone, but were unable to do so when the cells of the striatum were turned off. In 2012, Carmena, Costa and colleagues were the first to show that in the motor area of the brain, neurons could not learn a virtual task without plasticity in the connections to cells in the striatum.

“When these cells were inactivated, the animal would not be able to learn, which suggests that what we observed six years ago in motor areas with respect to the striatum seems to be common, like a building block for learning in the brain,” Carmena said.

Whole-cortex BMIs

Carmena suspects that another area of the brain is also part of the learning loop involving the cortex and striatum to provide feedback from the striatum to the cortex, since the feedback exists but no striatal cells connect directly back to the visual cortex. Ongoing studies are designed to locate the other areas that are part of these processes, and shed light on how the brain is able to retrain just a handful of cells in the cortex out of millions to operate an object outside the body.

“There are companies now, such as Neuralink started by Elon Musk and Kernel started by Bryan Johnson, which are working to create a whole-cortex interface in humans for future brain-machine direct communication” said Neely, who now works at iota Biosciences, a startup co-founded by Carmena and Berkeley colleague Michel Maharbiz. “An important question for this quest is to identify the supporting circuits that facilitate learning abstract skills, like communicating directly with machines. Our studies suggest that the cortex and the striatum are key elements of such a circuit.”

To Carmena, all regions of the cortex produce some “action,” whether that action is to interpret – as cells in the visual cortex interpret input from the eyes – or trigger a physical response, as do neurons in the motor cortex. From that perspective, the current study supports the idea that the circuit loop between the cortex and striatum is the same throughout the brain and capable of the same type of learning no matter what function it currently has.

“This study demonstrates the flexibility of the central nervous system and the brain for being able to connect something very arbitrary and abstract for an animal – something they don’t do in their ecological niche, such as tune a tone to a high pitch to get a reward – to neurons in the cortex,” Carmena said. “You can leverage all these built-in mechanisms for learning to learn these new modalities and change the visual or motor cortex into a BMI cortex. The brain isn’t saying, ‘Hey, what is this?’ The circuit is talking to them and they work.”

Berkeley graduate student Vivek Athalye is also a co-author. Carmena was supported by the U.S. Office of Naval Research, while Neely was supported by a National Defense Science and Engineering fellowship.

RELATED INFORMATION

• Volitional modulation of primary visual cortex activity requires the basal ganglia (Neuron)
• Carmena Lab
• Costa Lab