Amazing stuff!
"In one of the first single-neuron studies of imagination in the human brain, researchers found a striking match between how the brain sees objects and how it imagines them. The findings make important inroads into two previously indecipherable aspects of neuroscience: memory and the origins of creativity.
The research focused on a region of the brain called the ventral temporal cortex, which is responsible for object recognition. Measuring neurons in that region, researchers ... found that many of the same neurons fire in response to a specific object, whether it is seen or imagined. In other words, the brain draws on overlapping groups of cells to perceive an apple and to conjure a mental picture of one. The finding suggests that the brain implements a ‘generative model’: a model that can take a high-level concept like ‘flower’ and synthesize a detailed picture of one, complete with colors, shading, shape and textures. ...
The results ... are an important first step toward understanding how creative processes — like making art and music — arise in the brain and how memories are retrieved. ..."
From the editor's summary and abstract:
"Editor’s summary
The ventral temporal cortex (VTC) is a brain area involved in identifying and categorizing visual stimuli. Wadia et al. performed single-neuron recording in the VTC of patients with epilepsy while the subjects were presented with real visual stimuli or were asked to imagine them. Deep network analysis showed that visually responsive neurons were tuned on specific axes. While imagining the objects, around 40% of the visually responsive VTC neurons were also robustly activated. Thus, mental imagery reactivates the same sensory codes used during visual stimuli, suggesting the existence of a generative model capable of synthesizing detailed sensory contents from an abstract, semantic representation. ...
Structured Abstract
INTRODUCTION
Mental imagery refers to our brains’ capacity to generate percepts, emotions, and thoughts in the absence of external stimuli. ... Despite its importance in our lives, little is known about the single-neuron mechanisms of mental imagery. Neuroimaging results support a long-standing theory that imagery of a given sense is subserved by the reactivation of that specific sensory cortex. However, these studies lack the resolution to discern whether it is the same neurons or separate circuitry roughly located in the same regions that reactivates.
RATIONALE
We investigated the single-neuron mechanisms of visual imagery by recording single neurons in human patients implanted with electrodes to localize their focal epilepsy as they viewed and subsequently imagined objects. We focused our investigations on the ventral temporal cortex (VTC), a part of the temporal lobe dedicated to representing visual objects. We first determined the code for visual objects. We found that as in macaques, neurons in human VTC represent objects by using a distributed axis code. This code emphasizes the geometric picture that neurons project incoming stimuli—formatted as points in feature space—onto specific preferred axes and respond proportionally to the projection value. We then examined whether this code is reactivated during imagery.
RESULTS
We recorded 714 neurons in the human VTC across 16 patients as they viewed visual objects. A majority of neurons (456 of 714) were visually selective for one of the five object categories used (faces, plants, text, animals, and objects). To represent general objects with arbitrary features, we built a low-dimensional object space using the unit activations of deep networks trained to perform object classification. Nearly ~80% (367 of 456) of all visually responsive single neurons were significantly axis tuned. We used this axis code to reconstruct objects and generate maximally effective synthetic stimuli. Last, we recorded the responses of the same neurons in a subset of patients (6 of 16) as they imagined the same objects. Mean responses to perceived and imagined objects were comparable, with some neurons active only during perception, some only during imagery, and some during both. In particular, ~40% (43 of 107) of axis-tuned VTC neurons recorded during the imagery task reactivated, and the responses during imagery of individual neurons were proportional to the projection value of those objects onto the neurons’ viewing axes. We used this observation to reconstruct imagined objects while still easily distinguishing whether those objects were viewed or imagined.
CONCLUSION
We leveraged the opportunity to record from the same population of VTC neurons in humans as they viewed and imagined objects. Neurons use an axis code to represent visual objects, and neural activity during imagination reactivates this code. These findings provide single-neuron evidence for a generative model in the human brain."
Imagine That: Brain Uses Neurons from Vision System When Forming Mental Imagery (another original news release) "Creative endeavors, like making art, writing music, or penning a poem, require the recall of memories to fuel imagination. Many other human behaviors, including problem solving, also rely on mental imagery to complete tasks, but little was known about how imagery works at the level of single neurons in the brain—until now."
A shared code for perceiving and imagining objects in human ventral temporal cortex (no public access)
A shared code for perceiving and imagining objects in human ventral temporal cortex (preprint, open access)
A shared code for vision and imagination.
