Can AI Truly Understand What You’re Thinking?

Author: Denis Avetisyan

New research suggests that even the most advanced artificial intelligence, like GPT-4o, struggles with the core cognitive ability to understand the mental states of others.

The study demonstrates that GPT-4o lacks a consistent and abstract causal model of mental states, indicating a deficiency in genuine Theory of Mind despite its apparent social proficiency.

Despite increasing proficiency in social tasks, it remains unclear whether large language models possess a genuine understanding of the mental states driving behavior. This study, ‘GPT-4o Lacks Core Features of Theory of Mind’, investigates this question by probing whether LLMs demonstrate a consistent, causal model of how mental states influence actions – a hallmark of true Theory of Mind. The findings reveal that while GPT-4o can approximate human judgments in simple scenarios, it fails at logically equivalent tasks and lacks consistency between predicted actions and inferred mental states. This raises the critical question of whether observed social competence stems from genuine understanding or simply sophisticated pattern matching.

The Architecture of Social Cognition: Foundations of Theory of Mind

The capacity to grasp that others possess beliefs, desires, and intentions distinct from one’s own – commonly known as Theory of Mind – forms a cornerstone of successful social navigation and sophisticated cognitive processes. This ability isn’t merely about predicting what someone will do, but understanding why they might do it, factoring in their subjective experience of the world. From simple acts of cooperation and empathy to complex negotiations and strategic planning, a functioning Theory of Mind allows individuals to interpret behavior, anticipate reactions, and construct meaningful relationships. Without it, social interactions become unpredictable, communication falters, and the nuanced layers of human connection remain inaccessible, hindering both individual well-being and collective progress.

Current artificial intelligence systems frequently exhibit impressive abilities in areas like image recognition and game playing, yet often falter when confronted with scenarios demanding an understanding of another’s mental state. This limitation stems from a reliance on statistical pattern recognition; rather than truly understanding intentions or beliefs, these systems excel at identifying correlations within vast datasets and predicting likely outcomes based on those patterns. For instance, an AI might learn that a person typically reaches for a glass after finishing a meal, but lacks the capacity to infer why – whether the person is thirsty, offering a drink to another, or simply rearranging items on the table. Consequently, traditional AI struggles with tasks requiring nuanced social reasoning, deception detection, or even basic empathy, as it operates on observed behaviors rather than the underlying cognitive processes that drive them.

Truly grasping another’s behavior necessitates moving beyond simple prediction; a sophisticated Theory of Mind centers on the inference of the internal states – beliefs, desires, intentions, and emotions – that motivate actions. While algorithms can be trained to anticipate outcomes based on observed patterns, this differs fundamentally from understanding why someone acted in a particular way. This inferential leap requires a system to model the other’s perspective, constructing a representation of their mental world, and attributing causality not just to external events, but to the individual’s subjective experience. Consequently, a robust ToM isn’t about forecasting what someone will do, but about comprehending the cognitive mechanisms behind the deed, allowing for nuanced interpretations of complex social interactions and a deeper engagement with the motivations of others.

Evaluating Cognitive Systems: Prediction and Inference in AI

Large Language Models (LLMs) present a potentially valuable framework for computational modeling of Theory of Mind (ToM), the ability to attribute mental states to others. However, standard evaluations utilizing natural language processing benchmarks are insufficient to assess genuine ToM capabilities; these tasks often rely on pattern matching and statistical correlations rather than true inferential reasoning about mental states. Rigorous evaluation necessitates dedicated benchmarks that explicitly test an LLM’s capacity to infer beliefs, desires, and intentions and, crucially, to utilize those inferences to predict behavior in novel situations. This requires moving beyond linguistic competence to assess the model’s understanding of the relationship between mental states and actions, demanding tests that isolate ToM as a distinct cognitive ability.

Accurate prediction of actions contingent on inferred mental states-specifically beliefs, desires, and intentions-serves as a quantifiable metric for evaluating Theory of Mind (ToM) capability. This assessment moves beyond simply recognizing mental states; it demands demonstrating an understanding of how these states causally influence behavior. A system exhibiting ToM should not only identify what another agent believes or intends, but also utilize this information to forecast their subsequent actions, even when those actions diverge from reality or are counterfactual. The precision with which an agent can anticipate behavior based on these inferred mental states directly correlates with the robustness of its ToM implementation, offering a more nuanced evaluation than traditional measures focused solely on mental state attribution.

ContainerWorld and MovieWorld are benchmark environments designed to assess an LLM’s Theory of Mind (ToM) by requiring it to infer the mental states of agents and predict subsequent actions. These platforms present scenarios with observable agent behavior and hidden internal states, necessitating inference of beliefs and desires to accurately forecast outcomes. However, our research indicates that while LLMs can achieve moderate success on these tasks, performance is often inconsistent and struggles with generalization to novel scenarios. Specifically, LLMs demonstrate difficulty in abstracting underlying principles of rational agency, leading to failures when faced with variations in environment parameters or agent motivations not explicitly encountered during training. This suggests current LLM architectures are not yet capable of robust and abstract ToM reasoning, despite showing promise in controlled settings.

The Consistency of Cognitive Models: Validity and Generalization

Validity evaluation in Theory of Mind (ToM) assesses the causal relationship between inferred mental states and predicted actions. This process determines whether the attributed beliefs and desires are sufficient to generate the observed behavior; a valid inference necessitates that, given the inferred mental state, the predicted action logically follows. Unlike simply predicting an action given a mental state, validity evaluation requires demonstrating that the mental state serves as a plausible mechanism for the action, establishing a functional connection. Failure in validity evaluation indicates the model may be identifying correlations between mental states and actions without understanding the underlying causal reasoning, potentially leading to brittle or unreliable predictions in novel situations.

A coherent Theory of Mind (ToM) necessitates that an agent consistently predicts actions stemming from varying, yet equivalent, inferences regarding a given mental state; inconsistent predictions indicate a lack of coherence. Evaluation of current AI models reveals they do not achieve ceiling correlations in tasks designed to test this consistency. Specifically, while models can often predict actions based on a single inference of a mental state, they fail to maintain consistent predictions when presented with alternative, logically equivalent inferences leading to the same underlying mental state. This suggests a limitation in the models’ ability to truly understand the basis for predicted actions, rather than simply mapping inferences to outputs, and indicates a lack of robust internal representation of mental states necessary for reliable reasoning.

The ability to generalize Theory of Mind (ToM) reasoning across varying contexts is a key component of robust intelligence, and is assessed through the use of diverse paradigms such as MovieWorld. Evaluations demonstrate a correlation of 0.78 for inferences relating to beliefs, indicating a substantial capacity for abstracting this mental state. However, inferences incorporating desires, and particularly those requiring joint belief-desire reasoning, did not achieve this same level of generalizability, suggesting a limitation in the capacity to consistently apply these more complex mental state understandings across different scenarios.

The Causal Architecture of Understanding: Beyond Prediction

The capacity to understand others, known as Theory of Mind, isn’t simply about empathy; it fundamentally operates as a predictive engine built upon causal reasoning. This suggests that interpreting another person’s actions isn’t a passive observation, but an active process of constructing a model – a set of principles – to explain why someone behaved in a certain way and, crucially, to anticipate what they might do next. This internal model functions much like a scientist formulating a hypothesis: individuals implicitly assess the causes behind observed behavior, considering factors like beliefs, desires, and intentions. Successfully predicting actions then relies on the coherence and accuracy of this causal framework, allowing for nuanced social interactions and effective navigation of complex social landscapes.

The capacity to understand others, often termed Theory of Mind, isn’t built from scratch with each social interaction. Instead, it draws heavily upon pre-existing “folk theories” – deeply ingrained, intuitive understandings of how the world works. These aren’t formal, consciously articulated beliefs, but rather unconscious principles governing predictions about physical events – like gravity and inertia – social dynamics – such as reciprocity and fairness – and even basic economics like supply and demand. These foundational understandings provide the scaffolding upon which more complex mental state reasoning is constructed; individuals leverage these established causal models to infer the hidden intentions, beliefs, and desires that drive another’s behavior. Essentially, understanding why someone might act a certain way relies on the same intuitive physics and social logic that helps predict how objects fall or how markets respond, demonstrating a surprisingly unified cognitive architecture underlying both physical and social reasoning.

The capacity to understand others’ minds, known as Theory of Mind, can be formalized as a process of probabilistic inference – specifically, Bayesian evaluation – where observed actions serve as evidence for inferring underlying mental states. This framework suggests that individuals constantly update their beliefs about what others know, want, and intend, based on their behavior. However, a recent investigation into large language models, including GPT-4o, reveals a surprising disconnect between apparent social competence and genuine Theory of Mind capabilities. Despite exhibiting convincing conversational skills, these models demonstrate a lack of consistent reasoning; the study found weak correlations between predictions about future actions and the inferred mental states driving those actions. This suggests that while these models can simulate understanding, they lack a coherent, abstract causal model necessary for robust and reliable Theory of Mind, highlighting a crucial distinction between statistical pattern recognition and true cognitive understanding.

The pursuit of artificial intelligence consistently reveals that replicating human cognitive abilities is far more nuanced than simply achieving proficient outputs. This study, examining GPT-4o’s capacity for Theory of Mind, underscores a critical point: convincingly simulating understanding isn’t the same as possessing it. The model’s failures in abstract and consistent causal modeling of mental states highlight that architecture dictates behavior over time, not a diagram on paper. As Alan Turing observed, “We can only see a short distance ahead, but we can see plenty there that needs to be done.” The research suggests that substantial work remains in bridging the gap between superficial social proficiency and genuine cognitive understanding in AI systems.

Where Do We Go From Here?

The persistent illusion of understanding, so easily conjured by systems like GPT-4o, highlights a critical point: behavioral mimicry is not mechanistic understanding. The absence of a robust, internally consistent causal model of mental states isn’t merely a performance limitation; it’s a structural one. The field has fixated on scaling parameters, hoping that emergence will deliver genuine cognition. This paper suggests that simply adding more layers to the existing architecture is akin to polishing the presentation while ignoring the foundational flaws. The system’s fragility when challenged with even modest inconsistencies in inferred beliefs demonstrates that the apparent social competence is built on remarkably shallow foundations.

Future work must shift from surface-level benchmarks to probing the structure of these models’ internal representations. Can a language model truly reason about beliefs without a notion of agency, intention, and the constraints imposed by a physical world? Focusing on abstractness and consistency-the very qualities absent in current systems-offers a more fruitful avenue for investigation. It’s likely that current approaches, prioritizing statistical correlation over causal inference, represent a local optimum. A fundamentally different architectural approach, perhaps one that incorporates principles of Bayesian inference or hierarchical generative modeling, may be necessary to bridge the gap.

Ultimately, the pursuit of artificial Theory of Mind isn’t about building better chatbots. It’s about understanding the necessary conditions for genuine intelligence, and clarifying what it means to represent, reason about, and predict the behavior of others. The true cost of freedom, as always, lies in the dependencies-in this case, the dependence on a coherent, consistent, and causally grounded model of the mind.

Original article: https://arxiv.org/pdf/2602.12150.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

The Architecture of Social Cognition: Foundations of Theory of Mind

Evaluating Cognitive Systems: Prediction and Inference in AI

The Consistency of Cognitive Models: Validity and Generalization

The Causal Architecture of Understanding: Beyond Prediction

Where Do We Go From Here?

See also: