Observations

[:en]<div style="line-height: 1.5;">
On an ordinary evening, as you wrap up a day of work and sit down to start a game, the loading bar finishes, and a virtual world appears on the screen.

Immersed in the game, you suddenly notice an ant crawling slowly on your monitor. In that instant, an image from the novel The Three-Body Problem comes to mind. The ant feels like a subtle reminder of something profound, sending a chill down your spine. A startling thought strikes you like lightning—what if each of our lives, or even this vast world, is nothing more than a simulation crafted by an advanced civilization?

At first, the idea may seem absurd. But consider this: the foundation of our existence, much like many major discoveries in human history, has been overturned repeatedly. Five hundred years ago, geocentrism was an unshakable belief. Two hundred years ago, proposing that humans evolved from apes could have branded someone a heretic. Humanity's understanding of nature is a journey of realizing that we are not as exceptional as we once believed. This concept is embodied in the Copernican Principle, which asserts that no observer should regard themselves as uniquely special.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/simulation-1.png" alt="" width="433" height="399" class="aligncenter size-full wp-image-2697" />
▷ The diagram “M” (Latin for Mundus) from Johannes Kepler’s Epitome of Copernican Astronomy illustrates this point: Earth is just one of countless stars, a stark reminder that humans, whether on Earth or within the solar system, are not privileged observers of the universe.

1. Ancestor Simulation

If you are familiar with the history of video games, you might recall how gaming has evolved dramatically over just a few decades—from the simple, pixelated graphics of ping-pong games to the lifelike environments of modern massively multiplayer online games. Looking to the future, with the rapid advancement of virtual reality, the scenarios depicted in the sci-fi series Black Mirror—such as uploading and downloading consciousness—seem almost within reach.

In this context, the notion that "we are virtual beings living in a simulation" begins to feel less far-fetched. After all, on a cosmic timescale, a few decades are barely a blink of an eye. And as the Copernican Principle reminds us, it is statistically improbable that humanity represents the pinnacle of civilization in the universe.

So, how likely is it that we are living in a simulation? Once you begin to think seriously about this question, you'll realize you are not alone. Many physicists and philosophers have explored this possibility, sharing their thoughts through publications and public debates. In 2016, the American Museum of Natural History in New York hosted a panel where four physicists and a philosopher spent two hours discussing whether reality as we know it could be a simulation. Their estimates of the probability ranged widely, from as low as 1% to as high as 42%. That same year, Elon Musk famously declared in an interview that "we are most likely living in a simulation."

However, following the crowd has never been your style. Determined to explore this question independently, you start gathering evidence. Could we, in fact, be living in a simulation? And if so, is there a way to break free from it?

The type of "simulation" discussed here is not a simple game like The Sims. Instead, it is a comprehensive model of the observable universe—what scholars refer to as an Ancestor Simulation. [1] This hypothesis proposes that a civilization with sufficiently advanced technology might create a simulation on an immense scale. While we cannot predict the exact capabilities of such future technologies, we can attempt to estimate their upper limits based on the physical laws of our observable universe.

If running a simulation of this magnitude would require energy and computational power beyond the constraints of physics, we could argue that such a civilization could not exist. In that case, the Ancestor Simulation hypothesis would be disproven—or, at the very least, humanity’s existence within such a simulation would become highly improbable.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/simulation-cover-1024x768.jpg" alt="" width="1024" height="768" class="aligncenter size-large wp-image-2698" />
▷ Source: Claire Merchlinsky

2. The Energy Required for Simulation

High-fidelity simulations necessitate immense information processing, which, with current technological capabilities, translates into extraordinarily high computational demands. In such scenarios, the state of every fundamental particle would need to be recorded and updated in real time, requiring computational resources on an astronomical scale. Furthermore, the energy needed to power these highly complex simulations would entail staggering demands. According to thermodynamic principles, extracting energy from cosmic background radiation to sustain computations of this magnitude would not only require overcoming theoretical physical limitations but also addressing significant practical challenges in technology.

These arguments suggest that, within our current technological and theoretical framework, energy constraints make it improbable that we are living in a simulated universe. However, such constraints may not represent an insurmountable barrier. For instance, a super-advanced civilization might bypass energy limitations by creating less precise simulations. Quantum mechanics, with its uncertainty principle—which prevents precise knowledge of an electron’s exact position—and the non-local effects of quantum entanglement, could serve as efficient methods to conserve computational resources during simulation construction.

If we are indeed part of a simulation, and its creators used resource-saving measures, we might uncover evidence of this illusion. One potential approach is to continuously increase the resolution of our observations to detect whether we encounter discrete, pixelated structures. This is similar to video games, where increasing the resolution can expose hidden inconsistencies or graphical artifacts. Interestingly, quantum mechanics already reveals that the world appears discontinuous at extremely small scales, lending some plausibility to the simulation hypothesis.

Moreover, in video games, wandering to the edges of the map often leads to graphical glitches or invisible "walls" that limit further exploration. By analogy, searching for similar anomalies could help determine whether we exist within a simplified ancestor simulation. However, in reality, humanity's farthest-reaching probe, Voyager 1, has traversed the edge of the solar system without encountering any discontinuities or anomalies. Likewise, the Hubble Space Telescope has captured images of galaxies over 10 billion light-years away without uncovering any apparent anomalies. While these observations cannot definitively rule out the possibility that we are living in a simulation, they do significantly reduce its likelihood.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/simulation-3.png" alt="" width="838" height="599" class="aligncenter size-full wp-image-2696" />
▷ Source: Staudinger + Franke

3. A Simulated Universe Does Not Require Full Brain Simulation

Beyond the vastness of the starry sky, the human mind is equally complex. The number of neural connections in the human brain is estimated to rival the number of stars in the observable universe. A 2024 study published in Science [2] conducted a nanoscale simulation of just 1 cubic millimeter of brain tissue, encompassing 57,000 cells and 150 million synapses. The resulting mapping data totaled 1.4 petabytes (Pb). When extrapolated to the full volume of the human brain, creating a static map of the entire brain would require 1.76 zettabytes (1 Pb = 1,024 terabytes [Tb], 1 Zb = 1,024 Pb). This vastly exceeds the parameter count of current open-source models, such as Llama 3.1, which had 405 gigabytes (GB) of parameters as of October 2024.

To create an ancestor simulation, it would be necessary to simulate not only every star in the sky but also the mental activities of every human who has ever lived. Attempting to replicate all neural activity from ancient times to the present on a supercomputer would demand energy that exceeds the physical limits of our universe, challenging the notion that we are living in a simulation [3].

However, one might argue that simulating mental activities does not always require extensive computation. A super-advanced civilization might only simulate minimal portions of the brain for most people, reserving significant computational resources for individuals engaged in deep thought. This raises an intriguing testable hypothesis: if billions of people simultaneously engage in reflective thinking, requiring full brain simulations, the computational load might overwhelm the simulation, causing delays reminiscent of frame rate drops in video games. Such anomalies could provide evidence supporting the simulation hypothesis.

Additionally, there are two counterarguments to the assertion that simulating human minds consumes excessive energy:

(1) On-Demand Rendering: In constructing an ancestor simulation, a super-advanced civilization might only render areas being actively observed. This is analogous to the "fog of war" in video games, where unobserved areas remain hidden, significantly reducing computational demands.

(2) Selective Simulation: The creators might simulate only the neural activities of "you," ensuring your subjective experience feels authentic while generating others' responses through simplified models. Current large-scale AI models can already simulate human-like interactions; for a civilization capable of ancestor simulations, this task would be trivial, drastically reducing energy consumption.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/simulation-4-1024x576.png" alt="" width="1024" height="576" class="aligncenter size-large wp-image-2695" />
▷ Source: Mart Biemans

Beyond energy considerations, another way to evaluate the simulation hypothesis is to detect inevitable computational errors. If we live in a simulation, modeling inaccuracies would eventually accumulate, creating observable inconsistencies—similar to how The Matrix required regular reboots. In video games, prolonged gameplay often reveals modeling errors that necessitate developer patches to maintain consistency. By analogy, if we live in a simulation, changes in fundamental constants, such as the speed of light, could serve as evidence. However, the brevity of human history and our understanding of physics limits our ability to detect such long-term changes. It is possible that such patches exist but operate on timescales of tens of thousands or even millions of years. Therefore, even though we have not observed changes in physical constants, we cannot completely rule out the possibility that we are living in a simulation.

An additional counterargument is that a super-advanced civilization might prohibit ancestor simulations for ethical reasons. While less robust, this argument reduces the likelihood of such simulations existing. Similar to how the Drake Equation explores the probability of extraterrestrial civilizations' existence—calculated by multiplying the number of stars in the universe by the probability of life arising on them and the probability that intelligent life is willing to communicate—thereby estimating the number of extraterrestrial civilizations capable of establishing communication.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/simulation-5-1024x511.png" alt="" width="1024" height="511" class="aligncenter size-large wp-image-2694" />
▷ Source: University of Rochester
The Drake Equation, originally used to estimate the number of extraterrestrial civilizations, now includes:
N: The number of civilizations capable of creating simulations. R*: The rate of star formation in the Milky Way. fp: The fraction of stars with planetary systems. ne: The average number of habitable planets per system. fe: The fraction of habitable planets where life arises. fi: The fraction of life-bearing planets where intelligent life evolves. fc: The fraction of intelligent life willing to communicate or simulate. L: The time such civilizations remain active.

By adding two new factors to the Drake Equation—first, the probability that intelligent life advances to the point of being able to simulate a world; second, the probability that a super-advanced civilization would prohibit ancestor simulations for ethical reasons—we can use the modified equation, based on current technological and cultural development, to estimate the probabilities of these new factors. This allows us to assess the number of planets that might be willing to simulate worlds and, consequently, infer the probability that we are living in a simulation.

4. Three Possibilities for a Simulated World

In his 2003 paper Are You Living in a Computer Simulation?, philosopher Nick Bostrom redefined the debate by proposing three possibilities that cover all potential scenarios:

First, humanity or other intelligent beings go extinct before achieving simulation technology.
Second, super-advanced civilizations capable of creating ancestor simulations either lack the will or face prohibitions against using such technology.
Third, We are currently living in an ancestor simulation.

To evaluate which of the first two possibilities is more likely, two critical questions arise.The first question examines the probability of life successfully reaching the singularity. For example, before humanity develops general artificial intelligence, could it face extinction due to catastrophic events like runaway climate change or nuclear war? On this issue, the scientific community provides some predictions, though the outcomes remain uncertain.

The second question delves into the societal structures and moral frameworks of super-advanced civilizations—a nearly impossible task. Just as a gorilla cannot grasp the utility humans derive from advanced technology, we cannot envision the motivations or priorities of beings capable of ancestor simulations. Perhaps such civilizations lack the drive to undertake the creation of simulations that, once initiated, require no further involvement. Alternatively, ethical considerations or cultural values might prohibit such endeavors altogether. However, our complete lack of insight into their reasoning makes assessing this possibility extremely challenging.

At its core, if we ourselves do not believe we are living in a simulation, there is little reason to assume that future post-human civilizations would conduct large-scale ancestor simulations. Conversely, if we think humanity could one day simulate an entire world, why not accept the likelihood that we are already part of such a simulation?

One way to approach this paradox is through Occam’s Razor, which suggests that the simplest explanation is often more likely than a complex one. Believing we live in a simulation assumes the existence of a super-advanced civilization, thereby adding unnecessary complexity to the model. The maxim “entities should not be multiplied without necessity” thus provides a compelling framework for addressing the simulation hypothesis. However, Occam’s Razor is not absolute; in some cases, scientific consistency, predictive power, or internal logic may require adopting a more complex model.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/simulation-6-1024x510.png" alt="" width="1024" height="510" class="aligncenter size-large wp-image-2693" />
▷ Source: Claudio Araos Marincovic

5. Why Is Speculative Thinking Meaningful?

Readers who have reached this point might wonder: “Like Zhuangzi dreaming of a butterfly or the butterfly dreaming of Zhuangzi, why should we concern ourselves with such abstract questions when life is already fraught with its own challenges?” This line of inquiry may seem akin to the philosophical stance of agnosticism—ultimately irresolvable. However, our concerns and curiosities drive exploration and foster awe for the universe’s diversity and mysteries. Throughout the history of science, many revolutionary discoveries have emerged from profound questions about the essence of existence.

When we explore the possibility of a simulated universe, we are not only questioning the nature of reality but also sparking deeper discussions about technology, ethics, and existence itself. For example, this inquiry intersects with advancing technologies such as virtual reality (VR) and mixed reality (MR), raising questions about how we treat virtual beings powered by AI-driven models. As we approach the advent of general artificial intelligence (AGI), such discussions also highlight concerns about losing control over intelligent agents and whether these agents might attempt to escape the simulated environments we create for them. Speculating on whether we are living in a simulation inspires research into AI alignment while also deepening our understanding of existence.

In the future, as technology advances and humanity’s understanding of the universe grows, we may approach these questions with entirely new perspectives. More detailed observations, refined data analyses, and novel technological methods could help us test these hypotheses. Yet, in the pursuit of truth, maintaining curiosity and humility toward the unknown remains among the most enduring values in scientific inquiry.
<div/>[:]

How Can We Determine If the Universe Is Not a Simulation?

[:en]How Can We Determine If the Universe Is Not a Simulation?[:]

<div style="line-height: 1.5;">
Introduction: Using advanced neuroimaging technologies such as fMRI and EEG, scientists have identified subtle signs of consciousness in patients previously diagnosed with a &#8220;vegetative state&#8221; (VS). This breakthrough not only challenges the traditional understanding of the vegetative state but also highlights the concept of &#8220;covert consciousness&#8221; or &#8220;cognitive-motor dissociation.&#8221;
In Edward Yang’s film Yi Yi, an elderly woman falls into a coma following an accidental fall. Her caregiver advises the family to talk to her daily to aid recovery. But does speaking to a comatose individual truly make a difference? Can such patients actually hear, or even retain any degree of consciousness?
1. Conscious Responses in the “Vegetative State”
The question of whether comatose individuals retain consciousness has sparked debate since the 1990s.
Modern medicine traditionally defines the vegetative state as a condition characterized by a lack of conscious awareness and unresponsiveness. Yet, research indicates that 13–19% of patients diagnosed with VS or coma exhibit conscious responses to external stimuli.
<img loading="lazy" decoding="async" src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-1.png" alt="" width="970" height="274" class="aligncenter size-full wp-image-2688" srcset="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-1.png 970w, https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-1-300x85.png 300w, https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-1-768x217.png 768w" sizes="auto, (max-width: 970px) 100vw, 970px" /> 
▷ Owen, Adrian M., et al. &#8220;Detecting awareness in the vegetative state.&#8221; science 313.5792 (2006): 1402-1402.
A groundbreaking 2006 study published in Science (Owen et al., 2006) reported a compelling case [1]: a 23-year-old female patient in a vegetative state following a traffic accident demonstrated neural activity indistinguishable from that of healthy individuals when exposed to linguistic stimuli. Strikingly, when presented with sentences containing ambiguity, her left ventrolateral prefrontal cortex displayed neural responses typical of language comprehension.
Even more remarkably, the patient exhibited evidence of &#8220;imaginative behavior.&#8221; When instructed to imagine specific actions such as playing tennis or walking (motor imagery), her neural activity was indistinguishable from that of healthy controls.
<img loading="lazy" decoding="async" src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-2-1024x763.png" alt="" width="1024" height="763" class="aligncenter size-large wp-image-2687" srcset="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-2-1024x763.png 1024w, https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-2-300x223.png 300w, https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-2-768x572.png 768w, https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-2.png 1046w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /> 
▷ The study monitored the activity of the supplementary motor area (SMA) in both the patient and a control group of 12 healthy volunteers during the process of imagining playing tennis. When imagining moving around her home, the researchers detected neural activity in the parahippocampal gyrus (PPA), posterior parietal cortex (PPC), and premotor cortex (PMC).
Despite meeting the clinical criteria for a vegetative state, her ability to comprehend and follow instructions suggested a preserved awareness of herself and her environment. This study underscored that neural activity in response to stimuli can serve as a marker of consciousness, while its absence does not necessarily confirm unconsciousness.
Previously, it was widely believed that comatose patients either progressed to brain death or transitioned into a vegetative state—both thought to lack consciousness entirely. However, Adrian Owen’s findings introduced the possibility of intermediary states, such as the &#8220;minimally conscious state&#8221; (MCS) or &#8220;covert consciousness.&#8221;
This raises a profound question: how can we accurately determine whether a person, especially one unable to communicate verbally or behaviorally, retains consciousness?
2. Determining the Presence of Consciousness
To assess the presence of consciousness in individuals with severe neurological conditions, it is crucial to first understand the classifications of death. Death is generally divided into two main categories: legal death (the cessation of legal personhood) and natural death, which itself encompasses three progressive stages: clinical death, brain death, and biological death [2].
<img loading="lazy" decoding="async" src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-3.png" alt="" width="709" height="415" class="aligncenter size-full wp-image-2686" srcset="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-3.png 709w, https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-3-300x176.png 300w" sizes="auto, (max-width: 709px) 100vw, 709px" /> 
The vegetative state (VS) is a condition that resembles these stages but retains certain physiological functions. According to the Merck Manual of Diagnosis and Therapy [3]:
The vegetative state is a chronic condition in which patients can sustain blood pressure, respiration, and cardiac functions but lack higher cognitive abilities. While the hypothalamus and medullary brainstem functions remain intact to support cardiopulmonary and autonomic systems, the cortex suffers severe damage, eliminating cognitive function. Despite this, the reticular activating system (RAS) remains functional, allowing wakefulness. Midbrain or pontine reflexes may or may not be present. Typically, such patients lack self-awareness and interact with their environment only via reflexive responses.
However, findings from Owen’s team demonstrate that patients diagnosed as being in a vegetative state may indeed possess self-awareness and can even interact with their environment autonomously through neural activity. For example, patients have been shown to imagine actions, such as playing tennis, in response to instructions from clinicians. This neural activity, which can be detected using fMRI or EEG, indicates the presence of consciousness despite the absence of observable physical behavior. This phenomenon is referred to as &#8220;covert consciousness.&#8221;
3. Cognitive-Motor Dissociation: A New Understanding of Consciousness
Building on Adrian Owen&#8217;s groundbreaking research, subsequent studies have confirmed the phenomenon of &#8220;covert consciousness&#8221; in patients previously classified as being in a vegetative state. These findings, often based on motor imagery techniques, have identified a condition termed Cognitive-Motor Dissociation (CMD).
CMD describes a state in which patients demonstrate measurable neural responses to external stimuli, including motor imagery comparable to that of healthy individuals. However, due to motor system impairments, they are unable to execute corresponding physical actions.
<img loading="lazy" decoding="async" src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-4.png" alt="" width="856" height="310" class="aligncenter size-full wp-image-2685" srcset="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-4.png 856w, https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-4-300x109.png 300w, https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-4-768x278.png 768w" sizes="auto, (max-width: 856px) 100vw, 856px" /> 
▷ Schiff, Nicholas D. &#8220;Cognitive motor dissociation following severe brain injuries.&#8221; JAMA neurology 72.12 (2015): 1413-1415.
In 2015, Nicholas D. Schiff’s team used advanced imaging techniques such as Dynamic Causal Modeling (DCM) and Diffusion Tensor Imaging (DTI) to assess the functional integrity of brain networks supporting motor imagery in these patients. Their research aimed to uncover the neural mechanisms underlying Cognitive-Motor Dissociation [4].
The study revealed that in healthy individuals, excitatory coupling between the thalamus and motor cortex is critical for executing motor behaviors. However, in patients with covert consciousness, analyses using DCM and DTI demonstrated a selective disruption of this coupling. This disruption explains the dissociation between preserved motor imagery and absent skeletal movement. It is akin to operating the mechanical arm of an excavator when the circuit between the control panel and the arm is severed—you can still issue commands, but the arm cannot carry out the movements.
*DCM: A neuroimaging technique that examines causal relationships and connectivity between brain regions, providing insights into their functional interactions. 
DTI: An MRI-based method for mapping the direction and integrity of white matter tracts, offering a detailed view of brain structural connectivity and its changes.
4. Research on the Prevalence of Cognitive-Motor Dissociation
Patients with Cognitive-Motor Dissociation (CMD) are often misdiagnosed as being in a vegetative state. However, there is currently no standardized method for accurately identifying CMD. The mechanism identified by Schiff&#8217;s team, discussed earlier, may not apply universally to all CMD cases. Moreover, previous studies on CMD have generally been small-scale, fragmented, and lacking consistency, with limited sample sizes. Under these conditions, developing a globally applicable standard for patient assessment is challenging, let alone designing targeted treatment approaches for CMD.
To address this issue, the paper titled &#8220;Cognitive Motor Dissociation in Disorders of Consciousness&#8221;, [5] supported by the Tianqiao Chen Institute-MGH Research Scholar Award, offers a potential solution. This study, published in The New England Journal of Medicine in August 2024, features Chen Scholar Brian L. Edlow as a key participant.
<img loading="lazy" decoding="async" src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-5.png" alt="" width="744" height="352" class="aligncenter size-full wp-image-2684" srcset="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-5.png 744w, https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-5-300x142.png 300w" sizes="auto, (max-width: 744px) 100vw, 744px" /> 
▷ Bodien, Yelena G., et al. &#8220;Cognitive motor dissociation in disorders of consciousness.&#8221; New England Journal of Medicine 391.7 (2024): 598-608.
This study recruited patients from diverse settings, including intensive care units, rehabilitation centers, nursing homes, and community populations. A unified inclusion criterion was applied: participants had no prior history of neurological or psychiatric disorders and were physiologically capable of undergoing brain imaging techniques, including MRI and EEG.
The research analyzed 353 adult patients diagnosed with disorders of consciousness using motor imagery tasks assessed via fMRI and EEG. Among these, 241 patients were classified as being in a coma, a vegetative state, or a minimally conscious state (MCS) based on the internationally recognized Coma Recovery Scale-Revised (CRS-R). Notably, 25% (60 patients) of this group exhibited CMD, characterized by task-based neural responses despite the absence of behavioral responses. Additionally, 112 patients were diagnosed as being in an MCS+ state or as having recovered from MCS. Among this subgroup, 43 patients (38%) exhibited observable task-based responses to commands detected via fMRI, EEG, or both.
*Minimally conscious states are further categorized as MCS− and MCS+, based on language processing abilities: MCS+ patients demonstrate simple command-following, intelligible verbal expression, or conscious but non-functional communication. MCS− patients exhibit more limited behaviors, such as visual pursuit, localization to noxious stimuli, or basic spontaneous activities like grasping bedsheets.
This study reported a higher prevalence of CMD (25%) than previous research, which estimated that 10–20% of patients with disorders of consciousness exhibit CMD symptoms. While traditional behavioral metrics like CRS-R offer valuable insights into assessing patient consciousness and responsiveness, task-based neuroimaging technologies significantly enhance diagnostic accuracy. Notably, the combined use of EEG and fMRI further improves measurement precision. These findings suggest that the prevalence of CMD may have been underestimated in prior studies.
5. The Significance of Measuring Consciousness in the Vegetative State
Imagine being aware of everything around you—the hum of medical equipment by your bedside, the whispers of family and friends—yet being unable to speak, move, or respond in any way. This state of isolation and helplessness could lead one to abandon even the faintest will to live. Although the phenomenon of &#8220;living dead&#8221; occurs in only a minority of vegetative state patients, every individual, no matter how rare their condition, represents a unique and precious life.
Whether driven by technological advancement or humanistic ethics, we need a measurement method that is more accessible, precise, and cost-effective to determine the consciousness state of comatose patients. The research conducted by Y.G. Bodien and colleagues stands out by integrating patient data from multiple locations and unifying measurement methodologies through brain imaging techniques. This unprecedented approach has maximized the accuracy of identifying Cognitive-Motor Dissociation (CMD) patients and pointed the way forward for future medical technologies and treatments. Most importantly, it offers a renewed sense of hope to those trapped in a silent world.
<div/>

[:en]<div style="line-height: 1.5;">
Introduction: Using advanced neuroimaging technologies such as fMRI and EEG, scientists have identified subtle signs of consciousness in patients previously diagnosed with a "vegetative state" (VS). This breakthrough not only challenges the traditional understanding of the vegetative state but also highlights the concept of "covert consciousness" or "cognitive-motor dissociation."

In Edward Yang’s film Yi Yi, an elderly woman falls into a coma following an accidental fall. Her caregiver advises the family to talk to her daily to aid recovery. But does speaking to a comatose individual truly make a difference? Can such patients actually hear, or even retain any degree of consciousness?

1. Conscious Responses in the “Vegetative State”

The question of whether comatose individuals retain consciousness has sparked debate since the 1990s.

Modern medicine traditionally defines the vegetative state as a condition characterized by a lack of conscious awareness and unresponsiveness. Yet, research indicates that 13–19% of patients diagnosed with VS or coma exhibit conscious responses to external stimuli.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-1.png" alt="" width="970" height="274" class="aligncenter size-full wp-image-2688" />
▷ Owen, Adrian M., et al. "Detecting awareness in the vegetative state." science 313.5792 (2006): 1402-1402.

A groundbreaking 2006 study published in Science (Owen et al., 2006) reported a compelling case [1]: a 23-year-old female patient in a vegetative state following a traffic accident demonstrated neural activity indistinguishable from that of healthy individuals when exposed to linguistic stimuli. Strikingly, when presented with sentences containing ambiguity, her left ventrolateral prefrontal cortex displayed neural responses typical of language comprehension.

Even more remarkably, the patient exhibited evidence of "imaginative behavior." When instructed to imagine specific actions such as playing tennis or walking (motor imagery), her neural activity was indistinguishable from that of healthy controls.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-2-1024x763.png" alt="" width="1024" height="763" class="aligncenter size-large wp-image-2687" />
▷ The study monitored the activity of the supplementary motor area (SMA) in both the patient and a control group of 12 healthy volunteers during the process of imagining playing tennis. When imagining moving around her home, the researchers detected neural activity in the parahippocampal gyrus (PPA), posterior parietal cortex (PPC), and premotor cortex (PMC).

Despite meeting the clinical criteria for a vegetative state, her ability to comprehend and follow instructions suggested a preserved awareness of herself and her environment. This study underscored that neural activity in response to stimuli can serve as a marker of consciousness, while its absence does not necessarily confirm unconsciousness.

Previously, it was widely believed that comatose patients either progressed to brain death or transitioned into a vegetative state—both thought to lack consciousness entirely. However, Adrian Owen’s findings introduced the possibility of intermediary states, such as the "minimally conscious state" (MCS) or "covert consciousness."

This raises a profound question: how can we accurately determine whether a person, especially one unable to communicate verbally or behaviorally, retains consciousness?

2. Determining the Presence of Consciousness

To assess the presence of consciousness in individuals with severe neurological conditions, it is crucial to first understand the classifications of death. Death is generally divided into two main categories: legal death (the cessation of legal personhood) and natural death, which itself encompasses three progressive stages: clinical death, brain death, and biological death [2].

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-3.png" alt="" width="709" height="415" class="aligncenter size-full wp-image-2686" />
The vegetative state (VS) is a condition that resembles these stages but retains certain physiological functions. According to the Merck Manual of Diagnosis and Therapy [3]:

The vegetative state is a chronic condition in which patients can sustain blood pressure, respiration, and cardiac functions but lack higher cognitive abilities. While the hypothalamus and medullary brainstem functions remain intact to support cardiopulmonary and autonomic systems, the cortex suffers severe damage, eliminating cognitive function. Despite this, the reticular activating system (RAS) remains functional, allowing wakefulness. Midbrain or pontine reflexes may or may not be present. Typically, such patients lack self-awareness and interact with their environment only via reflexive responses.

However, findings from Owen’s team demonstrate that patients diagnosed as being in a vegetative state may indeed possess self-awareness and can even interact with their environment autonomously through neural activity. For example, patients have been shown to imagine actions, such as playing tennis, in response to instructions from clinicians. This neural activity, which can be detected using fMRI or EEG, indicates the presence of consciousness despite the absence of observable physical behavior. This phenomenon is referred to as "covert consciousness."

3. Cognitive-Motor Dissociation: A New Understanding of Consciousness

Building on Adrian Owen's groundbreaking research, subsequent studies have confirmed the phenomenon of "covert consciousness" in patients previously classified as being in a vegetative state. These findings, often based on motor imagery techniques, have identified a condition termed Cognitive-Motor Dissociation (CMD).

CMD describes a state in which patients demonstrate measurable neural responses to external stimuli, including motor imagery comparable to that of healthy individuals. However, due to motor system impairments, they are unable to execute corresponding physical actions.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-4.png" alt="" width="856" height="310" class="aligncenter size-full wp-image-2685" />
▷ Schiff, Nicholas D. "Cognitive motor dissociation following severe brain injuries." JAMA neurology 72.12 (2015): 1413-1415.

In 2015, Nicholas D. Schiff’s team used advanced imaging techniques such as Dynamic Causal Modeling (DCM) and Diffusion Tensor Imaging (DTI) to assess the functional integrity of brain networks supporting motor imagery in these patients. Their research aimed to uncover the neural mechanisms underlying Cognitive-Motor Dissociation [4].

The study revealed that in healthy individuals, excitatory coupling between the thalamus and motor cortex is critical for executing motor behaviors. However, in patients with covert consciousness, analyses using DCM and DTI demonstrated a selective disruption of this coupling. This disruption explains the dissociation between preserved motor imagery and absent skeletal movement. It is akin to operating the mechanical arm of an excavator when the circuit between the control panel and the arm is severed—you can still issue commands, but the arm cannot carry out the movements.

*DCM: A neuroimaging technique that examines causal relationships and connectivity between brain regions, providing insights into their functional interactions.
DTI: An MRI-based method for mapping the direction and integrity of white matter tracts, offering a detailed view of brain structural connectivity and its changes.

4. Research on the Prevalence of Cognitive-Motor Dissociation

Patients with Cognitive-Motor Dissociation (CMD) are often misdiagnosed as being in a vegetative state. However, there is currently no standardized method for accurately identifying CMD. The mechanism identified by Schiff's team, discussed earlier, may not apply universally to all CMD cases. Moreover, previous studies on CMD have generally been small-scale, fragmented, and lacking consistency, with limited sample sizes. Under these conditions, developing a globally applicable standard for patient assessment is challenging, let alone designing targeted treatment approaches for CMD.

To address this issue, the paper titled "Cognitive Motor Dissociation in Disorders of Consciousness", [5] supported by the Tianqiao Chen Institute-MGH Research Scholar Award, offers a potential solution. This study, published in The New England Journal of Medicine in August 2024, features Chen Scholar Brian L. Edlow as a key participant.

<img src="https://admin.next-question.com/wp-content/uploads/2025/01/vegetativee-5.png" alt="" width="744" height="352" class="aligncenter size-full wp-image-2684" />
▷ Bodien, Yelena G., et al. "Cognitive motor dissociation in disorders of consciousness." New England Journal of Medicine 391.7 (2024): 598-608.

This study recruited patients from diverse settings, including intensive care units, rehabilitation centers, nursing homes, and community populations. A unified inclusion criterion was applied: participants had no prior history of neurological or psychiatric disorders and were physiologically capable of undergoing brain imaging techniques, including MRI and EEG.

The research analyzed 353 adult patients diagnosed with disorders of consciousness using motor imagery tasks assessed via fMRI and EEG. Among these, 241 patients were classified as being in a coma, a vegetative state, or a minimally conscious state (MCS) based on the internationally recognized Coma Recovery Scale-Revised (CRS-R). Notably, 25% (60 patients) of this group exhibited CMD, characterized by task-based neural responses despite the absence of behavioral responses. Additionally, 112 patients were diagnosed as being in an MCS+ state or as having recovered from MCS. Among this subgroup, 43 patients (38%) exhibited observable task-based responses to commands detected via fMRI, EEG, or both.

*Minimally conscious states are further categorized as MCS− and MCS+, based on language processing abilities: MCS+ patients demonstrate simple command-following, intelligible verbal expression, or conscious but non-functional communication. MCS− patients exhibit more limited behaviors, such as visual pursuit, localization to noxious stimuli, or basic spontaneous activities like grasping bedsheets.

This study reported a higher prevalence of CMD (25%) than previous research, which estimated that 10–20% of patients with disorders of consciousness exhibit CMD symptoms. While traditional behavioral metrics like CRS-R offer valuable insights into assessing patient consciousness and responsiveness, task-based neuroimaging technologies significantly enhance diagnostic accuracy. Notably, the combined use of EEG and fMRI further improves measurement precision. These findings suggest that the prevalence of CMD may have been underestimated in prior studies.

5. The Significance of Measuring Consciousness in the Vegetative State

Imagine being aware of everything around you—the hum of medical equipment by your bedside, the whispers of family and friends—yet being unable to speak, move, or respond in any way. This state of isolation and helplessness could lead one to abandon even the faintest will to live. Although the phenomenon of "living dead" occurs in only a minority of vegetative state patients, every individual, no matter how rare their condition, represents a unique and precious life.

Whether driven by technological advancement or humanistic ethics, we need a measurement method that is more accessible, precise, and cost-effective to determine the consciousness state of comatose patients. The research conducted by Y.G. Bodien and colleagues stands out by integrating patient data from multiple locations and unifying measurement methodologies through brain imaging techniques. This unprecedented approach has maximized the accuracy of identifying Cognitive-Motor Dissociation (CMD) patients and pointed the way forward for future medical technologies and treatments. Most importantly, it offers a renewed sense of hope to those trapped in a silent world.
<div/>
[:]

Hearing the Silent Soul: Consciousness in Vegetative State Patients

[:en]Hearing the Silent Soul: Consciousness in Vegetative State Patients[:]

From The Journals

Why Can’t Musk’s “Blind Vision” Surpass Natural Sight?

[:en]Why Can’t Musk’s “Blind Vision” Surpass Natural Sight?[:]

“Fantasizing” May Not Be Able to Produce “Truth”

[:en]“Fantasizing” May Not Be Able to Produce “Truth”[:]

How to Unify the Measurement of Consciousness in All Things

[:en]How to Unify the Measurement of Consciousness in All Things[:]

Latest News

Interviews With Scientists

Interview with Prof. Moritz Helmstaedter: The decade-long debate has concluded, Has the European Brain Project failed?

[:en]Interview with Prof. Moritz Helmstaedter: The decade-long debate has concluded, Has the European Brain Project failed?[:zh]十年争议落幕，欧洲脑计划失败了吗？｜追问专访·Moritz Helmstaedter[:]

Interview with Professor Si Wu: Unlocking the "Door of Wisdom" in Artificial Intelligence

[:en]Interview with Professor Si Wu: Unlocking the "Door of Wisdom" in Artificial Intelligence[:zh]吴思：打开人工智能的“智慧之门”[:]

Interview with Prof. Wang Xiaojing: Fostering interdisciplinary talents for computational neuroscience

[:en]Interview with Prof. Wang Xiaojing: Fostering interdisciplinary talents for computational neuroscience[:zh] 汪小京 · 为计算神经科学培养跨学科人才[:]

Deep within the human brain, there&#8217;s a fascinating phenomenon that has captivated us for centuries—an intricate interplay between music and the brain. From soothing lullabies to exhilarating symphonies, music possesses boundless power to evoke emotions, ignite creativity, and offer us diverse sensory experiences.
&nbsp;
Scientists are currently dedicated to unraveling the mystery of how music influences human neural circuits, shedding light on its impact on various regions and functions of the brain. We are honored to have invited Professor Du Yi from Institute of Psychology, Chinese Academy of Sciences, to explore the effects of music on the brain from an anti-aging perspective.
<img loading="lazy" decoding="async" class="aligncenter wp-image-787" src="https://admin.next-question.com/wp-content/uploads/2023/11/1699193665366.png" alt="" width="600" height="274" srcset="https://admin.next-question.com/wp-content/uploads/2023/11/1699193665366.png 717w, https://admin.next-question.com/wp-content/uploads/2023/11/1699193665366-300x137.png 300w" sizes="auto, (max-width: 600px) 100vw, 600px" />
&nbsp;
<h4>Question: Please briefly introduce your research background.</h4>
&nbsp;
Professor Du: I took my undergraduate courses in medicine, but when it came to choosing my doctoral program, I developed a strong interest in psychology, especially in unraveling the mysteries of human brains. Consequently, I switched to a Ph.D. program in psychology. During my doctoral studies, my research primarily focused on auditory studies using rats as subjects, encompassing both physiological and behavioral research. However, in the later stages of my Ph.D. program and during my postdoctoral research, I became increasingly fascinated by the human brain and its processing of advanced auditory signals, such as language and music. As a result, after establishing my own laboratory, I have been dedicated to researching how the human brain comprehends language, appreciates music, and the neural mechanisms underlying these processes.
&nbsp;
As people age, the structure and function of the brain will degenerate. One significant change that impacts the quality of life for the elderly is the deterioration of hearing. In noisy environments like streets or restaurants, it becomes increasingly challenging for the elderly to discern what others are saying. This not only affects their social interactions but also their emotional well-being and even cause premature dementia. Therefore, we are deeply interested in understanding how the elderly manage speech comprehension in such noisy settings and whether they employ additional cognitive mechanisms for this purpose.
&nbsp;
In a paper we published in Nature Communications in 2016, we discovered that despite the decline in perceptual processing abilities among the elderly, they tend to engage additional brain regions, particularly those associated with speech motor control, to actively predict what others will say. This proactive, top-down prediction mechanism helps the elderly better compensate for their declining perceptual abilities, enabling them to comprehend language more effectively, especially in complex environments.
&nbsp;
Building upon this background, in our study published in PNAS in 2017, we found that young individuals, particularly young musicians, excel at speech comprehension in complex environments. We were interested in uncovering the underlying neural mechanisms, and our results revealed that young musicians possess stronger perceptual-motor integration capabilities. They rely not only on auditory input but also on predicting and compensating for speech motor movements and integrating cross-modal information to aid their speech comprehension in intricate environments.
&nbsp;
These two studies suggest a potential connection between perception and motor. It appears that the elderly may rely more on advanced, top-down perceptual-motor predictions and integration, while young musicians, attributable to their musical training experience, have advatanges in utilizing perceptual-motor integration for speech comprehension. Naturally, this leads us to inquire whether the elderly, through systematic musical training, can also enhance their cross-modal information integration abilities. Will such training potentially aid them in improving their speech comprehension within complex environments?
&nbsp;
Based on this premise, we embarked on a series of subsequent studies and found that compared to elder non-musicians, elder musicians are better at speech perception in noisy environments. In 2021, we published related behavioral research in Ear and Hearing, and followed up with new studies whose most recent findings have been published in Science Advances.
&nbsp;
<h4>Question: You mentioned that you recently published a study in Science Advances. Could you please introduce the new findings from this research?</h4>
&nbsp;
Professor Du: The recently published study in Science Advances is a featured cover article. We recruited three groups of subjects: one consisted of young non-musicians as a control group, but our primary focus was on the other two groups—elder non-musicians and elder musicians.
&nbsp;
We had them perform a visual-auditory speech syllable discrimination task in noisy environment, while monitoring their brain activity using fMRI. They would hear different syllables while simultaneously viewing matching lip movements on a screen. Their task was to determine what exactly they have heard against background noise of varying intensities.
&nbsp;
We found that behaviorally, the performance of elder musicians was closer to that of young individuals, with no significant differences in their performance, even in moderate and light noise backgrounds. In other words, musical training experience can assist the elderly in achieving performance similar to that of young individuals. This finding leads to the question: what are the underlying mechanisms?
&nbsp;
Firstly, compared to elder non-musicians, elder musicians exhibited neural processing patterns (also known as representation patterns) that were more similar to those of young individuals in both the left and right perceptual-motor regions when processing syllables in noisy environment. Sometimes, there’s even no significant differences between the two groups. In fact, their performance was significantly better than that of elder non-musicians who displayed a noticeable decline in distinguishing different speech sounds compared to young individuals. Their brains struggled to distinguish between, for example, “ba” and “da”. In contrast, elder musicians maintained strong neural differentiation abilities, similar to young individuals. This indicates that musical training experience can enhance and rejuvenate brain function which we refer to as functional preservation.
&nbsp;
Furthermore, we also discovered that elder musicians, in contrast to elder non-musicians, activated a broader range of brain regions within the frontoparietal network. These brain regions are associated with working memory and cognitive control. This suggests that elder musicians are capable of engaging more advanced brain regions for behavioral compensation.
&nbsp;
Simultaneously, elder musicians exhibited greater suppression of a brain region known as the angular gyrus compared to elder non-musicians. As a core component of the default brain network, angular gyrus is associated with autobiographical memory and long-term memory. Therefore, it tends to be activated when we are not engaged in specific tasks and relatively suppressed when we perform cognitive tasks. When we engage in cognitive tasks that require our attention to external stimuli, it becomes necessary to suppress brain regions in default network to prevent them from interfering.
&nbsp;
We found that elder musicians excel at suppressing this brain region, indicating their ability to focus more attentively on external stimuli and suppress irrelevant information. This enhanced suppression also assists them in achieving better behavioral performance. Elder musicians achieve improved performance by employing additional activation or inhibition of certain brain regions, a mechanism referred to as functional compensation.
&nbsp;
This mechanism is related to the first one mentioned earlier &#8212; functional preservation. We found that these two mechanisms are interconnected and mutually influence each other. When an elderly possess stronger abilities to activate the frontoparietal network and to inhibit the angular gyrus, his/her neural representation in the perceptual-motor brain regions also improves, rejuvenated to the level of young individuals. This suggests that functional compensation can support enhanced functional preservation. Based on these two mechanisms, musical training experience can assist elder individuals in speech perception in noisy environments and improve their behavioral performance.
<img loading="lazy" decoding="async" class="aligncenter wp-image-784" src="https://admin.next-question.com/wp-content/uploads/2023/11/81.png" alt="" width="600" height="761" srcset="https://admin.next-question.com/wp-content/uploads/2023/11/81.png 1080w, https://admin.next-question.com/wp-content/uploads/2023/11/81-237x300.png 237w, https://admin.next-question.com/wp-content/uploads/2023/11/81-808x1024.png 808w, https://admin.next-question.com/wp-content/uploads/2023/11/81-768x974.png 768w" sizes="auto, (max-width: 600px) 100vw, 600px" />
<h4>Question: Based on your research, how often does one need to play a musical instrument to potentially help preserve cognitive function and counteract the negative effects of aging?</h4>
&nbsp;
Professor Du: In this study, we asked the elderly to maintain a minimum of one hour of training per week within the last three years. Although they self-reported a substantial training frequency over the preceding three to four decades, it was challenging to track their training durations during the 1960s and 1970s due to historical reasons. Therefore, our focus was solely on the training frequency in the most recent three years.
&nbsp;
<h4>Question: Is it possible to achieve cognitive protection effects through temporary practice if one had no prior experience in playing a musical instrument?</h4>
&nbsp;
Professor Du: That&#8217;s a great question. What we&#8217;re currently conducting is a cross-sectional study where we directly compare the elderly with long-term musical training to those without such experience to observe differences, and we have found some. Additionally, there are numerous other longitudinal studies, both long-term and short-term, indicating that brief periods of musical training lasting from three to six months can also enhance language processing abilities for the elderly.
&nbsp;
<h4>Question: Can we achieve the same effects by listening to music?</h4>
&nbsp;
Professor Du: We all hope that passive music listening can yield similar positive effects, but current findings do not allow us to draw that conclusion definitively. Our study focuses on the effects of actively engaging in musical training, which involves multiple systems including visual, auditory, motor, emotional, and attentional processes. This multi-system engagement may have a more pronounced impact. However, listening to music also has its own benefits, such as mood regulation, stress relief, and potential improvement of focus and attention.
&nbsp;
<h4>Question: The participants in your study were engaged in a visual-auditory syllable discrimination task. Have you considered conducting other types of tasks?</h4>
&nbsp;
Professor Du: Both musicians and non-musicians in our study have completed other types of tasks. One of these tasks involved auditory working memory, where participants were presented with sequences of numbers of varying lengths and asked to repeat them in the original order or reversed order. Additionally, we assessed their executive control abilities, including the classic Stroop effect (the interference of literal meaning in font color).
&nbsp;
We found that elder musicians exhibited superior auditory working memory compared to elder non-musicians. While we didn&#8217;t observe better performance in cognitive control tasks like the Stroop effect in our study, it&#8217;s noteworthy that other research has indicated that musical experience can enhance working memory and executive control capabilities.
&nbsp;
The reason why we conducted the visual-auditory syllable discrimination task in our study was that speech processing involves real-time processing for everyone, and identifying consonants, vowels, and tones is crucial for recognition. Therefore, syllable discrimination tasks serve as a fundamental unit for speech processing.
&nbsp;
<h4>Question: Compared to non-musicians, musicians exhibit superior speech perception abilities in noisy environments. Do you believe this is solely attributable to their musical training, or could there be other factors at play?</h4>
&nbsp;
Professor Du: We chose musical training for our study because it represents a multi-system training approach that is relatively enjoyable. Moreover, it has been associated with improvements in various cognitive functions, such as working memory, selective attention, and cognitive control, among others. This includes findings from our research: stronger inhibition of the default brain network and enhanced modulation of the frontoparietal attention network among musicians. Therefore, we believe it is likely associated with musical training.
&nbsp;
However, since our study is cross-sectional, it&#8217;s important to acknowledge that there may be other differences between musicians and non-musicians in various aspects, and we cannot definitively rule out whether these differences are caused by innate factors or acquired ones.
&nbsp;
It&#8217;s possible that some individuals become musicians because of better innate structural and functional foundations or other traits that guide them toward a musical path. Therefore, our current cross-sectional study cannot definitively confirm the involvement of other factors, especially personal traits that might contribute to superior speech processing abilities. To address this question, conducting a longitudinal study would be ideal to establish whether these effects are indeed the results of musical training.
&nbsp;
<h4>Question: Are there any other studies examining similar topics? If so, are there any similarities and discrepancies between those studies and yours?</h4>
&nbsp;
Professor Du: Many researchers have indeed investigated whether musical training can assist the elderly in improving their speech perception, and they have found similar results at the behavioral level. What sets our new study apart from theirs is that we were able to identify cognitive neural mechanisms supporting both functional preservation and compensation in the brain. This allowed us to provide better explanation in terms of mechanisms to improved behavioral performance.
&nbsp;
<h4>Question: Has this study shed new light on anti-aging and brain health? Are there potential clinical application value of the findings, such as developing targeted interventions or treatment methods to enhance speech perception or other cognitive abilities among the elderly?</h4>
&nbsp;
Professor Du: We believe that musical training can have a neuroprotective effect on the brain by bringing it back to youth.
&nbsp;
While our study focused on the effects of long-term musical training, in the future, we are interested in investigating the minimum durations and frequencies of training that might also yield similar outcomes.
&nbsp;
This also reminds us that regardless of age, if you can start learning a musical instrument or even singing as early as possible, it may potentially slow down the aging process of the brain. Therefore, we currently advocate for the general public to consider learning music if plausible and to start engaging with music.
&nbsp;
In terms of clinical value, we are interested in exploring whether music therapy could be employed in the context of certain cognitive degenerative disorders among the elderly, such as Alzheimer&#8217;s disease, to improve their memory and language abilities.
&nbsp;
<h4>Question: Could music-based games potentially have similar effects?</h4>
&nbsp;
Professor Du: That&#8217;s a great approacg. Games like Rhythm Master can be effective training methods. We also refer to this as digital therapy. Using music as a form of digital therapy holds great promise for the future because gamified training like this is undoubtedly more intriguing and appealing for the elderly get fully engaged.
&nbsp;
Interviewed &amp; Edited by: Ashley Wang 
Proofread by: Xu Yunke

Interview with Prof. Du Yi: Can music rejuvenate the brain

[:en]Interview with Prof. Du Yi: Can music rejuvenate the brain[:zh]杜忆 · 音乐让大脑“返老还童”？[:]

Interview with Prof. Wu Mengyue: Do machines understand sounds better than humans?

[:en]Interview with Prof. Wu Mengyue: Do machines understand sounds better than humans?[:zh]吴梦玥 · 机器比人类更会理解声音吗？[:]

rectangle-68

Rectangle 68

NextQuestion Interviews

rectangle-63-1

Rectangle 63 (1)

NextQuestion Spotlight

nq_social_banner_en_112823

NQ_Social_Banner_EN_112823

How Does the Large Language Model Proclaim the Death of Psychology?

[:en]How Does the Large Language Model Proclaim the Death of Psychology?[:zh]万字长文，大语言模型如何宣告心理学的死亡?[:]

Ultrasound Telepathy: A New Breakthrough! Utilizing Intent to Control Direction, Achieving 'Precognition without Motion'

[:en]Ultrasound Telepathy: A New Breakthrough! Utilizing Intent to Control Direction, Achieving 'Precognition without Motion'[:zh]“超声读心”新突破！用意念控制方向，实现“未动先知”[:]

It has taken decades for computer scientists to finally materialize automatic speech recognition AI models that are closer to the human level. Such purely engineered AI models have completely abandoned earlier modeling frameworks based on linguistic theory, and entirely adopted data-driven, end-to-end, large-scale pre-trained deep neural networks. So how much does such a model resemble the auditory pathways of the human brain?
&nbsp;
To address this issue, Professor Yuanning Li&#8217;s team at the School of Biomedical Engineering, ShanghaiTech University, in collaboration with Professor Edward Chang at the University of California, San Francisco, and Professor Jinsong Wu/Junfeng Lu&#8217;s team at Fudan University, integrated a variety of technological approaches such as self-supervised pre-trained in-depth speech modeling, high-density intracranial electroencephalograms, and single-neuron simulation modeling, and thoroughly investigated the computational and representational similarities between AI speech models and the auditory pathways of the human brain utilizing a cross-language (English and Chinese) control experiment paradigm.
&nbsp;
Researchers can extract features at different levels &#8211; from purely acoustic spectrogram features to phonetic vowels and consonants and articulations, to relative pitches containing contextual information, etc. &#8211; and then use sliding time windows to predict neural responses. If a particular type of feature accurately predicts neural activity in a particular region, it is usually assumed that the neural activity encoded in that region expresses that type of feature.
&nbsp;
Over the past decade or so, using intracranial electrophysiological recording experiments as well as neural coding models, researchers have identified several important features of neural coding, e.g., the activity of different neural populations in the secondary auditory cortex of the superior temporal gyrus encodes features ranging from envelopes of voice to specific vowel and consonant phonemes, and so on.
&nbsp;
In this study, in addition to the secondary auditory cortex, which is closely related to speech, using intracranial high-density EEG recording technology and high-precision single-neuron biophysical simulation modeling, the researchers obtained neural responses covering the entire auditory pathway from the auditory nerves to the brainstem to the auditory cortex.
&nbsp;
Given that AI models and brain auditory circuits are capable of receiving the same speech input and performing similar cognitive functions, are there computational and representational similarities between the two? This is the key question on which this study focuses.
&nbsp;
To address this problem, the researchers have constructed a new deep neural coding model. This is a purely data-driven model that extracts feature representations from deep neural networks pre-trained with speeches, applies these data-driven features to construct a new linear coding model, and performs correlation analysis with authentic brain auditory response signals to investigate the similarity between the intrinsic feature representations of deep neural networks and the activities of different neural populations within the brain auditory pathway.
&nbsp;
By comparing the performance of neural activity prediction at different nodes of the auditory pathway of the neural coding models built based on these models, it was found that there is indeed a great similarity between the hierarchical structure of the end-to-end speech pre-training network, and that of the auditory circuit.
&nbsp;
First, for the entire auditory pathway, the encoding prediction model based on deep neural network features is comprehensively better than the traditional linear feature model based on linguistic theory. This indicates that the entire auditory pathway has strong nonlinear features. Second, models with different levels of complexity correspond to different regions in the auditory pathway. In addition, it was found that for the same self-supervised speech model, its overall hierarchical structure corresponds to the AN-IC-STG hierarchical structure of the auditory pathway.
&nbsp;
Having established the representational similarities between the deep speech model and the auditory pathway, the researchers further explored the computational mechanisms driving these representational similarities, focusing on the best-performing HuBERT model.
&nbsp;
The results show that as the network deepens, the attentional weights aligned to long-range contextual structures become progressively larger. It is worth emphasizing that the HuBERT model used here is fully self-supervised, and the training process does not include any explicit information about the context structure as well as the speech content information. This result suggests that a self-supervised trained speech model can learn key context structure information related to language and semantics in natural speech.
&nbsp;
In the secondary auditory cortex of the superior temporal gyrus, which is closely related to speech processing, the higher the alignment of the self-attentional weights with the contextual structures in speech, the better the neural network&#8217;s prediction of brain activity, while conversely, in the primary auditory cortex as well as in these regions of the auditory nerve and brainstem, the greater the expression of localized transient information in the temporal domain, and the greater the resemblance between neural networks and brain signals.
&nbsp;
Last but not least, the study further analyzed whether the self-supervised model was able to learn more advanced contextual information. The results show that the self-supervised model is able to learn higher-level contextual information related to linguistic specificity and that this specificity information is significantly correlated with computation and representation in the phonetic cortex of the brain.
&nbsp;
This study provides a new biological perspective on unlocking the “black box” of deep neural networks, especially the self-attention model called Transformer.
&nbsp;
<a href="https://www.nature.com/articles/s41593-023-01468-4">Read the paper on Nature Neuroscience</a>

How similar are AI speech models to the auditory pathways of humans?

[:en]How similar are AI speech models to the auditory pathways of humans?[:zh]AI语音模型与人的听觉有多相似？上科大/UCSF/复旦联合团队解析深度语音模型与人脑听觉通路的表征与计算相似性[:]

Because the Mountain is There - A Decade of Human Brain Projects

[:en]Because the Mountain is There - A Decade of Human Brain Projects[:zh]因为山就在那里——人类脑计划的十年[:]

Embodied Intelligence: The Pathway to the Future of Human Wisdom

[:en]Embodied Intelligence: The Pathway to the Future of Human Wisdom[:zh]具身智能：通向人类智慧的未来之路[:]

Why does the brain play a single song on a loop? 
Can listening to more music improve children's cognition?