Ripples of activity in the hippocampus prepare the brain for surprising visual events

Amazing stuff! However, it took almost 3 years for this research from preprint to be published as a journal article!

Ethical questions: The study apparently involved 17 patients with epilepsy, but that was not mentioned in the abstract.

"... Researchers ... recently carried out a study aimed at better understanding how the brain prepares for unexpected visual experiences. Their paper, published in Nature Neuroscience, suggests that fast, large-amplitude, brief bursts of oscillatory activity in the hippocampus (i.e., hippocampal ripples) influence how the visual cortex, the part of the brain responsible for processing visual information, responds to surprising stimuli after they appear. ...

This experiment involved 17 patients with epilepsy who had electrodes implanted in their brains as part of their treatment. ...

When participants were less certain about what image would next appear on screen, the team observed a greater frequency of specific patterns of activity in the hippocampus, called hippocampal ripples. Interestingly, these ripples of activity occurred more often and lasted longer before an unexpected stimulus appeared than before a familiar one. ..."

From the abstract:

"To encode information efficiently, our perceptual system should detect when situations are unpredictable (that is, informative) and modulate brain dynamics to prepare for encoding. Under uncertainty, there is an increased need to generate predictions about upcoming information, a process that has been proposed to require coordinated activity between the hippocampus and neocortex.

Here we show, with direct recordings from the human hippocampus and visual cortex, that after exposure to unpredictable visual stimulus streams, hippocampal ripple activity increases in frequency and duration before stimulus presentation.

Prestimulus hippocampal ripples suppress changes in visual cortex gamma activity associated with uncertainty and modulate poststimulus prediction error gamma responses in higher-level visual cortex to surprising stimuli.

We reveal a function of hippocampal ripples in facilitating the propagation of visual stimuli based on the expected information gain.

These results, therefore, link hippocampal ripples with predictive coding accounts of neuronal message passing and precision-weighted prediction errors, revealing a mechanism relevant for perceptual synthesis and subsequent memory encoding."

Ripples of activity in the hippocampus prepare the brain for surprising events

Human hippocampal ripples tune cortical responses based on predicted uncertainty (open access)

Fig. 2: The frequency and duration of hippocampal ripples increase with uncertainty or expected information gain.

Fig. 5: Surprise-driven responses.

First 3D Structure of Inactivated Cone Opsins Revealed in three different studies

Amazing stuff! I limit myself here to one of the three papers.

"... In a new study, ... two researchers ... have succeeded for the first time in determining the three-dimensional structure of human cone opsins in their dark state and showing how their molecular architecture enables their rapid activation by light. This provides important new insights into human vision and its evolution and may offer new starting points for the study of eye diseases that currently lack effective treatment. ..."

From the editor's summary and abstract:

"Editor’s summary

Human daytime vision relies on a trio of visual receptors called opsins, which are found in the cone cells in and around the central region of the retina.

The three opsins are tuned to long, medium, or short wavelengths of light, roughly corresponding to red, green, and blue, and mutations or other defects in cone cell function can lead to vision deficits.

Although the cell biology and biochemistry of color vision have been well studied, up to now, the molecular explanation for cone opsin spectral tuning and signaling kinetics has been limited by a lack of experimental structures.

Three papers in this issue now resolve this deficit.

Schmidt et al. determined structures of the dark state of the green and blue human cone opsins, which revealed important details of these receptors and provide a basis for a femtosecond-resolution spectroscopy study.

Ohashi et al. performed complementary structural, spectroscopic, and computational results with dark-state red and green cone opsins from macaques, which have color vision similar to humans.

Finally, Peng et al. studied all three human cone opsins in the presumed active state bound to a G protein and all-trans retinal.

The three papers together provide a clear picture of the features of these visual receptors that lead to different spectral properties, activation and inactivation kinetics, and recycling. ...

Structured Abstract

INTRODUCTION

High-acuity daylight vision relies on cone photoreceptors, specialized class A G protein–coupled receptors (GPCRs). Like other light-sensitive GPCRs, the three human cone opsins covalently bind vitamin A derivative 11-cis-retinal through a protonated Schiff base. Despite sharing the same chromophore, they detect distinct wavelengths of light and generate swift signaling responses at high repetition rates. Although cone opsins are central to human vision, and in contrast to the well-studied rod photoreceptor rhodopsin, the detailed molecular basis of these functional specializations remains elusive.

RATIONALE

To obtain structure-function relationships that extend the kinetic and mechanistic understanding of photopic vision, we solved cryo–electron microscopy (cryo-EM) structures of the two most evolutionarily and functionally divergent human cone opsins, short-wavelength-sensitive OPN1SW and medium-wavelength-sensitive OPN1MW, in their initial 11-cis-coupled state. We combined the structural data with multiple functional assays, hybrid quantum mechanics/molecular mechanics simulations, time-resolved spectroscopy, and multitaxon opsin sequence analysis.

RESULTS

Cryo-EM structures of cone opsins revealed receptor-specific activation mechanisms and distinct strategies for stabilizing the retinal Schiff base.

OPN1SW, representing the phylogenetically older vertebrate opsins, has a more constrained polar chromophore environment, which contributes to its blue-shifted maximum absorption wavelength (λmax), yet its stabilization is weaker than that of rhodopsin. The architecture of OPN1SW shows substantial divergences in the canonical GPCR microswitch networks, including the replacement of the toggle switch with Y6.48, a disrupted PIF triad, and the absence of a highly conserved sodium- or water-coordination site.

Collectively, these alterations favor a preactive conformation, also captured by cryo-EM. OPN1SW further uses W185ECL2 as a steric switch to transmit the retinal isomerization event across several helices through an extended aromatic network.

In contrast, OPN1MW contains a chloride ion within the chromophore-binding pocket that modulates wavelength sensitivity and influences the amplitude of G protein signaling. This chloride-binding site coevolved with a structural pathway on helix 2 that couples chromophore chemistry to canonical GPCR microswitches.

Both receptors have accessible binding pockets that allow rapid ligand hydrolysis and, consequently, fast retinal turnover.

Femtosecond transient-absorption spectroscopy resolved the photoisomerization cascade, supporting a model in which deprotonation and subsequent hydrolysis limit signal duration in cone opsins.

CONCLUSION

Our structural and mechanistic insights describe how distinctive chromophore environments and GPCR microswitch adaptations tune spectral sensitivity and signaling-state lifetimes in cone opsins. Conservation of central residues across short-wavelength-sensitive and medium-to-long-wavelength-sensitive opsins suggests shared mechanistic principles that shaped the evolution of daylight vision. Similar motifs in other GPCRs, including sensory receptors, inform the strategies for modulating receptor activation kinetics and signal duration."

First 3D Structure of Inactivated Cone Opsins Revealed | The Scientist "Scientists resolved the three-dimensional structures of light-sensitive cone opsins, offering new avenues for treating age-related vision loss."

New insights into human vision (original news release)

Experiment performed at ELI Enables New Insights into Human Vision (original news release) "The retina of the human eye contains six to seven million cone cells. These cells contain light-sensitive proteins known as cone opsins. They enable us to perceive our surroundings in detail in daylight. In a new study, researchers ... have now, for the first time, determined the three-dimensional molecular structure of human cone opsins in their dark state, that is, before they are activated by light."

Illuminating the molecular basis of human daylight vision (no public access)

Cone opsins use distinct chromophore-stabilization strategies to tune spectral sensitivity, activation kinetics, and retinal regeneration.

Seeing and Imagining Are Handled by the Same Brain Cells based on single neuron recording

Amazing stuff!

"In one of the first single-neuron studies of imagination in the human brain, researchers found a striking match between how the brain sees objects and how it imagines them. The findings make important inroads into two previously indecipherable aspects of neuroscience: memory and the origins of creativity.

The research focused on a region of the brain called the ventral temporal cortex, which is responsible for object recognition. Measuring neurons in that region, researchers ... found that many of the same neurons fire in response to a specific object, whether it is seen or imagined. In other words, the brain draws on overlapping groups of cells to perceive an apple and to conjure a mental picture of one. The finding suggests that the brain implements a ‘generative model’: a model that can take a high-level concept like ‘flower’ and synthesize a detailed picture of one, complete with colors, shading, shape and textures. ...

The results ... are an important first step toward understanding how creative processes — like making art and music — arise in the brain and how memories are retrieved. ..."

From the editor's summary and abstract:

"Editor’s summary

The ventral temporal cortex (VTC) is a brain area involved in identifying and categorizing visual stimuli. Wadia et al. performed single-neuron recording in the VTC of patients with epilepsy while the subjects were presented with real visual stimuli or were asked to imagine them. Deep network analysis showed that visually responsive neurons were tuned on specific axes. While imagining the objects, around 40% of the visually responsive VTC neurons were also robustly activated. Thus, mental imagery reactivates the same sensory codes used during visual stimuli, suggesting the existence of a generative model capable of synthesizing detailed sensory contents from an abstract, semantic representation.  ...

Structured Abstract

INTRODUCTION

Mental imagery refers to our brains’ capacity to generate percepts, emotions, and thoughts in the absence of external stimuli. ... Despite its importance in our lives, little is known about the single-neuron mechanisms of mental imagery. Neuroimaging results support a long-standing theory that imagery of a given sense is subserved by the reactivation of that specific sensory cortex. However, these studies lack the resolution to discern whether it is the same neurons or separate circuitry roughly located in the same regions that reactivates.

RATIONALE

We investigated the single-neuron mechanisms of visual imagery by recording single neurons in human patients implanted with electrodes to localize their focal epilepsy as they viewed and subsequently imagined objects. We focused our investigations on the ventral temporal cortex (VTC), a part of the temporal lobe dedicated to representing visual objects. We first determined the code for visual objects. We found that as in macaques, neurons in human VTC represent objects by using a distributed axis code. This code emphasizes the geometric picture that neurons project incoming stimuli—formatted as points in feature space—onto specific preferred axes and respond proportionally to the projection value. We then examined whether this code is reactivated during imagery.

RESULTS

We recorded 714 neurons in the human VTC across 16 patients as they viewed visual objects. A majority of neurons (456 of 714) were visually selective for one of the five object categories used (faces, plants, text, animals, and objects). To represent general objects with arbitrary features, we built a low-dimensional object space using the unit activations of deep networks trained to perform object classification. Nearly ~80% (367 of 456) of all visually responsive single neurons were significantly axis tuned. We used this axis code to reconstruct objects and generate maximally effective synthetic stimuli. Last, we recorded the responses of the same neurons in a subset of patients (6 of 16) as they imagined the same objects. Mean responses to perceived and imagined objects were comparable, with some neurons active only during perception, some only during imagery, and some during both. In particular, ~40% (43 of 107) of axis-tuned VTC neurons recorded during the imagery task reactivated, and the responses during imagery of individual neurons were proportional to the projection value of those objects onto the neurons’ viewing axes. We used this observation to reconstruct imagined objects while still easily distinguishing whether those objects were viewed or imagined.

CONCLUSION

We leveraged the opportunity to record from the same population of VTC neurons in humans as they viewed and imagined objects. Neurons use an axis code to represent visual objects, and neural activity during imagination reactivates this code. These findings provide single-neuron evidence for a generative model in the human brain."

Seeing and Imagining Are Handled by the Same Brain Cells "Researchers make advances in understanding how human brains are wired by pinning down how the same neurons are used for both imagining and seeing objects."

Imagine That: Brain Uses Neurons from Vision System When Forming Mental Imagery (another original news release) "Creative endeavors, like making art, writing music, or penning a poem, require the recall of memories to fuel imagination. Many other human behaviors, including problem solving, also rely on mental imagery to complete tasks, but little was known about how imagery works at the level of single neurons in the brain—until now."

A shared code for perceiving and imagining objects in human ventral temporal cortex (no public access)

A shared code for perceiving and imagining objects in human ventral temporal cortex (preprint, open access)

A shared code for vision and imagination.

How the brain retrieves previously viewed objects on images, perceiving and retrieving activates many of the same neurons

Amazing stuff!

"... A new study ... suggests ... a real neurological process, as the brain reactivates the same neural areas during recall that were used to see that image in the first place.

To measure this, researchers recorded the activity of over 700 neurons in the visual temporal cortex (VTC), a region involved in recognizing complex features and categorizing images, across 16 patients with previously implanted electrodes for epilepsy monitoring. The team tracked how individual brain cells responded as participants viewed a series of images of plants, animals, objects, and faces, and then later reconstructed them from memory.

Many of the same neurons were activated in both cases, with recall engaging 40% of the same visually responsive neurons active when an object was initially perceived. Based on this preserved structure, researchers were able to reconstruct which images patients were recalling from patterns of neural activity. In some cases, the reconstruction captured enough detail to distinguish general categories, like animals or faces. Still, the two processes were not identical, with recall producing less precise representations.

Exactly how these signals are generated and coordinated is still unclear, with authors noting that although the same regions of the brain are involved in both perception and recall, it’s not yet clear how the brain reactivates these patterns; current methods cannot fully distinguish which neurons are involved in each process."

From the editor's summary and abstract:

"Editor’s summary

The ventral temporal cortex (VTC) is a brain area involved in identifying and categorizing visual stimuli. Wadia et al. performed single-neuron recording in the VTC of patients with epilepsy while the subjects were presented with real visual stimuli or were asked to imagine them. Deep network analysis showed that visually responsive neurons were tuned on specific axes. While imagining the objects, around 40% of the visually responsive VTC neurons were also robustly activated. Thus, mental imagery reactivates the same sensory codes used during visual stimuli, suggesting the existence of a generative model capable of synthesizing detailed sensory contents from an abstract, semantic representation.  ...

Structured Abstract

INTRODUCTION

Mental imagery refers to our brains’ capacity to generate percepts, emotions, and thoughts in the absence of external stimuli. This ability allows us to generate art, simulate actions and outcomes, remember previous experiences, and imagine new ones.

Uncontrolled mental imagery can contribute to psychological disorders, including anxiety, schizophrenia, and posttraumatic stress disorder.

Despite its importance in our lives, little is known about the single-neuron mechanisms of mental imagery. Neuroimaging results support a long-standing theory that imagery of a given sense ...

RATIONALE

We investigated the single-neuron mechanisms of visual imagery by recording single neurons in human patients implanted with electrodes to localize their focal epilepsy as they viewed and subsequently imagined objects. We focused our investigations on the ventral temporal cortex (VTC), a part of the temporal lobe dedicated to representing visual objects.

We first determined the code for visual objects. We found that as in macaques, neurons in human VTC represent objects by using a distributed axis code. This code emphasizes the geometric picture that neurons project incoming stimuli—formatted as points in feature space—onto specific preferred axes and respond proportionally to the projection value. We then examined whether this code is reactivated during imagery.

RESULTS

We recorded 714 neurons in the human VTC across 16 patients as they viewed visual objects.

A majority of neurons (456 of 714) were visually selective for one of the five object categories used (faces, plants, text, animals, and objects). To represent general objects with arbitrary features, we built a low-dimensional object space using the unit activations of deep networks trained to perform object classification. Nearly ~80% (367 of 456) of all visually responsive single neurons were significantly axis tuned.

We used this axis code to reconstruct objects and generate maximally effective synthetic stimuli. Last, we recorded the responses of the same neurons in a subset of patients (6 of 16) as they imagined the same objects. Mean responses to perceived and imagined objects were comparable, with some neurons active only during perception, some only during imagery, and some during both.

In particular, ~40% (43 of 107) of axis-tuned VTC neurons recorded during the imagery task reactivated, and the responses during imagery of individual neurons were proportional to the projection value of those objects onto the neurons’ viewing axes. We used this observation to reconstruct imagined objects while still easily distinguishing whether those objects were viewed or imagined.

CONCLUSION

We leveraged the opportunity to record from the same population of VTC neurons in humans as they viewed and imagined objects. Neurons use an axis code to represent visual objects, and neural activity during imagination reactivates this code. These findings provide single-neuron evidence for a generative model in the human brain."

ScienceAdviser

A shared code for perceiving and imagining objects in human ventral temporal cortex (no public access)

A shared code for perceiving and imagining objects in human ventral temporal cortex (preprint, open access)

Fig. 1 Selective visual responses in human VTC.

Fig. 2 Axis tuning in human VTC neurons.

New electronic films could enable lightweight night-vision glasses

Amazing stuff! Will we soon be offered smartglasses with this feature? 😊

"As a demonstration, the team fabricated a thin membrane of pyroelectric material — a class of heat-sensing material that produces an electric current in response to changes in temperature. The thinner the pyroelectric material, the better it is at sensing subtle thermal variations.

With their new method, the team fabricated the thinnest pyroelectric membrane yet, measuring 10 nanometers thick, and demonstrated that the film is highly sensitive to heat and radiation across the far-infrared spectrum.

The newly developed film could enable lighter, more portable, and highly accurate far-infrared (IR) sensing devices, with potential applications for night-vision eyewear and autonomous driving in foggy conditions.

Current state-of-the-art far-IR sensors require bulky cooling elements. In contrast, the new pyroelectric thin film requires no cooling and is sensitive to much smaller changes in temperature. The researchers are exploring ways to incorporate the film into lighter, higher-precision night-vision glasses. ...

“remote epitaxy” — a technique where semiconducting materials are grown on a single-crystalline substrate, with an ultrathin layer of graphene in between. The substrate’s crystal structure serves as a scaffold along which the new material can grow. The graphene acts as a nonstick layer, similar to Teflon, making it easy for researchers to peel off the new film and transfer it onto flexible and stacked electronic devices. After peeling off the new film, the underlying substrate can be reused to make additional thin films. ...

discovered that the key to the material’s easy-peel property was lead. As part of its chemical structure ... discovered that the pyroelectric film contains an orderly arrangement of lead atoms that have a large “electron affinity,” meaning that lead attracts electrons and prevents the charge carriers from traveling and connecting to another materials such as an underlying substrate. The lead acts as tiny nonstick units, allowing the material as a whole to peel away, perfectly intact.

The team ran with the realization and fabricated multiple ultrathin films of PMN-PT [Lead Magnesium Niobate-Lead Titanate], each about 10 nanometers thin. They peeled off pyroelectric films and transferred them onto a small chip to form an array of 100 ultrathin heat-sensing pixels, each about 60 square microns (about .006 square centimeters). They exposed the films to ever-slighter changes in temperature and found the pixels were highly sensitive to small changes across the far-infrared spectrum.

The sensitivity of the pyroelectric array is comparable to that of state-of-the-art night-vision devices. These devices are currently based on photodetector materials, in which a change in temperature induces the material’s electrons to jump in energy and briefly cross an energy “band gap,” before settling back into their ground state. This electron jump serves as an electrical signal of the temperature change. However, this signal can be affected by noise in the environment, and to prevent such effects, photodetectors have to also include cooling devices that bring the instruments down to liquid nitrogen temperatures. ...

The researchers also found that the films were sensitive beyond the range of current night-vision devices and could respond to wavelengths across the entire infrared spectrum. ..."

From the abstract:

"Recent breakthroughs in ultrathin, single-crystalline, freestanding complex oxide systems have sparked industry interest in their potential for next-generation commercial devices. However, the mass production of these ultrathin complex oxide membranes has been hindered by the challenging requirement of inserting an artificial release layer between the epilayers and substrates.

Here we introduce a technique that achieves atomic precision lift-off of ultrathin membranes without artificial release layers to facilitate the high-throughput production of scalable, ultrathin, freestanding perovskite systems.

Leveraging both theoretical insights and empirical evidence, we have identified the pivotal role of lead in weakening the interface.

This insight has led to the creation of a universal exfoliation strategy that enables the production of diverse ultrathin perovskite membranes less than 10 nm. Our pyroelectric membranes demonstrate a record-high pyroelectric coefficient of 1.76 × 10−2 C m−2 K−1, attributed to their exceptionally low thickness and freestanding nature.

Moreover, this method offers an approach to manufacturing cooling-free detectors that can cover the full far-infrared spectrum, marking a notable advancement in detector technology."

New electronic “skin” could enable lightweight night-vision glasses | MIT News | Massachusetts Institute of Technology "MIT engineers developed ultrathin electronic films that sense heat and other signals, and could reduce the bulk of conventional goggles and scopes."

Atomic lift-off of epitaxial membranes for cooling-free infrared detection (no public access)

The wizardry of Oz shows people colors beyond normal human perception

Amazing stuff!

"But it turns out that some patterns of activation are impossible—the wavelengths that activate green-sensing M cones completely overlap with those of blue S cones and red L cones, so it’s impossible to just see pure, true greens… or it was. Now, thanks to a new system called Oz, people can see totally new colors , and scientists can learn more about how perception works.

Oz uses teeny lasers to light up individual cone cells on the retina, directly controlling how much they are activated. Theoretically, Oz can show people “any (L, M, and S) color coordinate,” ... And in their study, they succeeded in showing five participants a new, highly saturated blue-green, a theoretical color the team dubbed “olo” because of its color space coordinates—0 activation of S and L, and full activation of M, or [0,1,0]. ..."

"Ground Summary

• The Discovery: ... researchers used a laser system called 'Oz' to create 'olo', a never-before-seen blue-green color, by selectively stimulating M cone cells in five participants' retinas. The breakthrough occurred yesterday at the University of California.
• Why It Matters: This breakthrough could revolutionize treatment for color blindness and expand human color perception. The technology may eventually simulate tetrachromat vision, potentially allowing humans to see 100 times more colors than currently possible.

"

From the abstract:

"We introduce a principle, Oz, for displaying color imagery: directly controlling the human eye’s photoreceptor activity via cell-by-cell light delivery. Theoretically, novel colors are possible through bypassing the constraints set by the cone spectral sensitivities and activating M cone cells exclusively.

In practice, we confirm a partial expansion of colorspace toward that theoretical ideal. Attempting to activate M cones exclusively is shown to elicit a color beyond the natural human gamut, formally measured with color matching by human subjects.

They describe the color as blue-green of unprecedented saturation. Further experiments show that subjects perceive Oz colors in image and video form. The prototype targets laser microdoses to thousands of spectrally classified cones under fixational eye motion. These results are proof-of-principle for programmable control over individual photoreceptors at population scale."

ScienceAdviser

UC Berkeley Researchers Reveal 'Olo,' a Color Beyond Human Vision (original news release)

Novel color via stimulation of individual photoreceptors at population scale (open access)

By directly stimulating certain light-sensing cells (s, m, and l) in the retina, researchers can show people colors outside the natural range of perception—like “olo” (green circle).

Fig. 1. Overview of principle and prototype system.

Night-vision lenses so thin and light that we can all see in the dark

Good news for night owls!

"... Researchers from the ARC Centre of Excellence for Transformative Meta-Optical Systems (TMOS) in Australia have been on a quest to make night vision accessible and wearable, doing away with bulky and expensive headsets and lens attachments. ...

This new tech also captures the visible and non-visible (or infrared) light in one image as you look through the 'lens.' Traditionally, night-vision systems capture side-by-side views from each spectrum, so they can't produce identical images. ...

“This is the first demonstration of high resolution up-conversion imaging from 1550-nm infrared to visible 550-nm light in a non-local metasurface," ... "We choose these wavelengths because 1,550 nm, an infrared light, is commonly used for telecommunications, and 550 nm is visible light to which human eyes are highly sensitive. Future research will include expanding the range of wavelengths the device is sensitive to, aiming to obtain broadband IR imaging, as well as exploring image processing, including edge detection.” ..."

From the abstract:

"The ability to detect and image short-wave infrared light has important applications in surveillance, autonomous navigation, and biological imaging. However, the current infrared imaging technologies often pose challenges due to large footprint, large thermal noise and inability to augment infrared and visible imaging. Here, infrared imaging is demonstrated by nonlinear up-conversion to the visible in an ultra-compact, high-quality-factor lithium niobate resonant metasurface. Images with high conversion efficiency and resolution quality are obtained despite the strong nonlocality of the metasurface. The possibility of edge-detection image processing augmented with direct up-conversion imaging for advanced night vision applications is further shown."

Night-vision lenses so thin and light that we can all see in the dark

New all-optical approach to revolutionise night vision technology (original news release)

Enhanced Infrared Vision by Nonlinear Up-Conversion in Nonlocal Metasurfaces (open access)

Figure 1 Infrared (IR) to visible (VIS) up-conversion for vision applications.

Scientists can now tell where you're looking by listening to your ears

Amazing stuff!

"Back in 2018, scientists at Duke University discovered that each time our eyes move, our ears make an imperceptible squeaking noise. Now, the researchers have developed a method of telling where a person is looking by analyzing those sounds. ...

Lead scientist ... believes that they may occur as eye movements trigger the brain to contract either the middle ear muscles or the hair cells – doing the former helps dampen loud noises, while doing the latter amplifies quiet ones.

This arrangement could help us make sense of our surroundings, by automatically adjusting the sensitivity of our hearing based on what we're looking at. ..."

"... The exact purpose of these ear squeaks is unclear, but [researcher]’s initial hunch is that it might help sharpen people’s perception. ...
Just as the eye’s pupils constrict or dilate like a camera’s aperture to adjust how much light gets in, the ears too have their own way to regulate hearing. ..."

From the significance and abstract:

"Significance

... Here, we show that the small sounds generated within the ear by the brain contain accurate information about contemporaneous eye movements in the spatial domain: The direction and amplitude of the eye movements could be inferred from these small sounds. The underlying mechanism(s) likely involve(s) the ear’s various motor structures and could facilitate the translation of incoming auditory signals into a frame of reference anchored to the direction of the eyes and hence the visual scene.

Abstract

Eye movements alter the relationship between the visual and auditory spatial scenes. Signals related to eye movements affect neural pathways from the ear through auditory cortex and beyond, but how these signals contribute to computing the locations of sounds with respect to the visual scene is poorly understood. Here, we evaluated the information contained in eye movement-related eardrum oscillations (EMREOs), pressure changes recorded in the ear canal that occur in conjunction with simultaneous eye movements. We show that EMREOs contain parametric information about horizontal and vertical eye displacement as well as initial/final eye position with respect to the head. The parametric information in the horizontal and vertical directions can be modeled as combining linearly, allowing accurate prediction of the EMREOs associated with oblique (diagonal) eye movements. Target location can also be inferred from the EMREO signals recorded during eye movements to those targets. We hypothesize that the (currently unknown) mechanism underlying EMREOs could impose a two-dimensional eye-movement-related transfer function on any incoming sound, permitting subsequent processing stages to compute the positions of sounds in relation to the visual scene."

Scientists can now tell where you're looking by listening to your ears

Your Eyes Talk to Your Ears. Scientists Know What They’re Saying. A new finding that eye movements can be decoded by the sounds they generate in the ear reveals that hearing may be affected by vision.

Parametric information about eye movements is sent to the ears (open access)

Are you able to see faces in everyday objects?

Recommendable! Some faces are just optical illusions!

A New Research On Unsupervised Deep Learning Shows That The Brain Disentangles Faces Into Semantically Meaningful Factors, Like Age At The Single Neuron Level

Amazing stuff!

"With the emergence of deep self-supervised generative models that learn to “disentangle” high-dimensional sensory data into meaningful variables of variation, recent breakthroughs in machine learning have provided an implementational blueprint for this theory. The beta-variational autoencoder (β-VAE) is one such model that learns to faithfully reconstruct sensory data from a low-dimensional embedding while also being regularised in a way that encourages individual network units to code for semantically meaningful variables like object color, face gender, and scene arrangement. ...
The findings support previous evidence that the monkey IT’s face identification code is low-dimensional, with single neurons encoding independent axes of variance. Unlike earlier research, however, our findings show that such a code can be meaningfully interpreted at the level of a single neuron. The study shows that single IT neurons’ axes of variation align with single “disentangled” latent units that appear to be semantically meaningful and are discovered by the β-VAE ..."

From the abstract:

"... we model neural responses to faces in the macaque inferotemporal (IT) cortex with a deep self-supervised generative model, β-VAE, which disentangles sensory data into interpretable latent factors, such as gender or age. Our results demonstrate a strong correspondence between the generative factors discovered by β-VAE and those coded by single IT neurons, beyond that found for the baselines, including the handcrafted state-of-the-art model of face perception, the Active Appearance Model, and deep classifiers. Moreover, β-VAE is able to reconstruct novel face images using signals from just a handful of cells. Together our results imply that optimising the disentangling objective leads to representations that closely resemble those in the IT at the single unit level. This points at disentangling as a plausible learning objective for the visual brain."

A New Research On Unsupervised Deep Learning Shows That The Brain Disentangles Faces Into Semantically Meaningful Factors, Like Age At The Single Neuron Level - MarkTechPost

Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons (open access; the paper was previously published as a preprint in 2020: https://arxiv.org/abs/2006.14304; this paper is a collaboration between Google, Howard Hughes Medical Institute, and Chinese Academy of Sciences)