Modeling the Mind: A New Framework for Simulating Human Decision-Making

by

Author: Denis Avetisyan

Researchers have developed a novel hybrid approach combining artificial intelligence and computational psychiatry to create more realistic and interpretable simulations of how people make choices.

The BioLLMAgent framework integrates an Internal Reinforcement Learning Engine-which generates utilities based on Expected Value, Expected Frequency, and Perseveration-with an External Large Language Model Shell simulating complete trials, and a Decision Fusion mechanism balances these approaches via parameter ω, effectively converting probabilistic outputs into static utility-scale priors [latex]\Pi_{\text{util}}[/latex].
The BioLLMAgent framework integrates an Internal Reinforcement Learning Engine-which generates utilities based on Expected Value, Expected Frequency, and Perseveration-with an External Large Language Model Shell simulating complete trials, and a Decision Fusion mechanism balances these approaches via parameter ω, effectively converting probabilistic outputs into static utility-scale priors [latex]\Pi_{\text{util}}[/latex].

BioLLMAgent integrates reinforcement learning and large language models to offer enhanced structural interpretability in simulating decision-making processes, exemplified through the Iowa Gambling Task.

Computational psychiatry faces a persistent challenge: achieving both behavioral realism and mechanistic interpretability in models of decision-making. This is addressed in ‘BioLLMAgent: A Hybrid Framework with Enhanced Structural Interpretability for Simulating Human Decision-Making in Computational Psychiatry’, which introduces a novel framework integrating validated reinforcement learning with the generative capabilities of large language models. BioLLMAgent accurately reproduces human behavior-demonstrated on the Iowa Gambling Task and other paradigms-while maintaining parameter identifiability and enabling simulation of therapeutic interventions like cognitive behavioral therapy. Could this structurally interpretable “computational sandbox” unlock new insights into the underlying mechanisms of psychiatric disorders and accelerate the development of personalized treatment strategies?

Reframing Computational Psychiatry: Beyond Simplification

Historically, computational psychiatry has frequently employed streamlined models of the human mind, often prioritizing mathematical tractability over biological or psychological realism. These approaches, while useful for initial explorations, tend to abstract away the intricate details of cognitive processes, treating the brain as a relatively simple input-output system. Consequently, they frequently overlook the substantial influence of factors such as prior beliefs, emotional states, and contextual awareness – all critical components of human decision-making. This simplification, though sometimes necessary, limits the capacity of these models to accurately reflect the complexities of mental illness or to predict individual behavioral responses with sufficient fidelity, creating a need for more nuanced and integrated frameworks that better capture the richness of human cognition.

The field of computational psychiatry frequently encounters limitations when translating the intricacies of human decision-making into workable models. A significant challenge lies in accurately representing not just what choices are made, but why – a process deeply intertwined with an individual’s cognitive beliefs and their perception of the surrounding context. Current computational frameworks often prioritize quantifiable variables, overlooking the nuanced influence of subjective interpretations and situational awareness. This disconnect hinders the ability to predict behavior in dynamic, real-world scenarios, as choices are rarely based solely on calculated values; rather, they are a product of complex interactions between learned preferences, internal beliefs about the world, and a constant assessment of the immediate environment. Bridging this gap requires models that move beyond simple reinforcement learning and incorporate representations of belief states, contextual understanding, and the ability to reason about potential outcomes, ultimately offering a more holistic and accurate portrayal of the human mind.

The limitations of current computational psychiatry models stem from a fundamental difficulty in representing how learned experiences – encoded as values for different actions – interact with the complex, often abstract, reasoning that guides human behavior. Existing simulations frequently treat value-based learning and cognitive processes as separate entities, failing to account for how higher-level beliefs and situational understanding can modulate, or even override, predictions based solely on past rewards. This disconnect results in behavioral predictions that often lack the flexibility and nuance observed in real-world decision-making; a model might accurately predict choices in simple, well-defined environments, but falter when confronted with novel situations demanding inference, planning, or consideration of social context. Accurately capturing this interplay is crucial for developing simulations capable of mirroring the full spectrum of human behavioral responses and, ultimately, for gaining deeper insights into the mechanisms underlying mental health.

Across six datasets of human choices, a Cognitive Behavioral Therapy (CBT) intervention [latex] ext{(orange)}[/latex] demonstrably promotes more advantageous learning trajectories compared to an ORL model [latex] ext{(red)}[/latex], a neutral prior [latex] ext{(green)}[/latex], and observed human choices [latex] ext{(black)}[/latex], with the strongest effects observed in clinical populations.
Across six datasets of human choices, a Cognitive Behavioral Therapy (CBT) intervention [latex] ext{(orange)}[/latex] demonstrably promotes more advantageous learning trajectories compared to an ORL model [latex] ext{(red)}[/latex], a neutral prior [latex] ext{(green)}[/latex], and observed human choices [latex] ext{(black)}[/latex], with the strongest effects observed in clinical populations.

BioLLMAgent: A Hybrid Architecture for Behavioral Realism

BioLLMAgent leverages a hybrid architecture integrating reinforcement learning (RL) and large language models (LLMs) to achieve comprehensive behavioral simulation. The RL component focuses on modeling experience-driven value learning, where agents learn optimal actions through trial and error and the association of actions with rewards. Simultaneously, the LLM component provides high-level cognitive representation, enabling the agent to process contextual information and articulate beliefs about the environment. This combination allows BioLLMAgent to move beyond simple stimulus-response mechanisms, incorporating both learned valuations and symbolic reasoning into its decision-making process. The LLM doesn’t replace the RL engine, but rather augments it by providing a richer, more nuanced understanding of the situation, improving the agent’s ability to generalize and adapt to complex scenarios.

The BioLLMAgent framework’s Internal Reinforcement Learning (RL) Engine utilizes established computational models of valuation and learning. Specifically, it incorporates Outcome-Representation Learning, which posits that value is assigned based on learned representations of future states, and Prospect Valence Learning, a model that emphasizes the encoding of potential outcomes as gains and losses relative to a reference point. These models, validated through behavioral and neuroscientific studies, provide the computational basis for simulating experience-driven value judgments and informing decision-making processes within the agent. The engine’s implementation allows for quantifiable and replicable simulations of learned preferences and aversions, grounding the agent’s behavior in established principles of reward processing.

The External LLM Shell within BioLLMAgent functions as a contextual layer by processing environmental inputs and maintaining a representation of the agent’s beliefs about the situation. This is achieved through prompting the LLM with sensory data and internal state information, enabling it to infer relevant contextual factors and potential outcomes. The LLM’s output, consisting of reasoned beliefs and situational assessments, is then provided to the Decision Fusion Mechanism. This contextual information is critical because it allows the agent to move beyond purely reward-based decision-making and incorporate higher-level cognitive considerations into its behavioral responses, improving the realism and nuance of the simulation.

The Decision Fusion Mechanism within BioLLMAgent employs a weighted averaging approach to combine outputs from the Internal RL Engine and the External LLM Shell. This mechanism assigns a dynamically adjusted weight to each engine’s output based on contextual relevance and task demands, determined through a predefined set of heuristics and parameters. The RL Engine contributes value-based assessments of potential actions, while the LLM Shell provides probabilistic reasoning regarding situational factors and anticipated outcomes. The weighted average generates a final decision score, facilitating a more nuanced simulation of behavior than either engine could achieve independently. Importantly, the weighting parameters and individual engine outputs are logged, enabling full traceability and interpretability of the simulated agent’s decision-making process.

Using the DeepSeek backend consistently yields trajectory tracking fidelity comparable to GPT-4o, confirming the generalizability of the hybrid approach across different large language models.
Using the DeepSeek backend consistently yields trajectory tracking fidelity comparable to GPT-4o, confirming the generalizability of the hybrid approach across different large language models.

Validating the Framework: The Iowa Gambling Task and Network Dynamics

The Iowa Gambling Task (IGT) is utilized as a standardized assessment of experience-based learning and risk-taking behavior, providing a quantifiable metric for evaluating the performance of the Internal Reinforcement Learning (RL) Engine. The IGT presents participants with a simulated gambling scenario involving four decks of cards with varying reward and penalty structures; successful performance requires learning to favor decks with lower immediate rewards but higher long-term gains, demonstrating an ability to adapt to changing reward contingencies. By measuring the agent’s evolving deck selection strategy and cumulative reward, researchers can assess the RL Engine’s capacity for acquiring and utilizing learned information to optimize decision-making in uncertain environments, and compare its performance against established human behavioral baselines documented in neuropsychological literature.

BioLLMAgent utilizes a Markov Decision Process (MDP) to represent the Iowa Gambling Task as a sequential decision-making problem. In this implementation, the agent exists within a discrete state space defined by the available decks of cards, and actions consist of selecting a deck to draw from. The MDP incorporates a reward structure where card selections result in either positive or negative monetary gains, reflecting the inherent risk associated with each deck. The transition probabilities between states are deterministic, governed by the card draw itself, while rewards are stochastic, reflecting the probabilistic nature of the gambling task. This formulation allows for the application of reinforcement learning algorithms to model and optimize the agent’s decision-making strategy over time, explicitly capturing the sequential dependencies and reward feedback critical to the task.

BioLLMAgent utilizes network modeling to define the interactions and connectivity between agents, drawing from established graph theory concepts. Watts-Strogatz Small-World Networks introduce localized clustering with long-range connections, potentially facilitating rapid information dissemination. Barabási-Albert Scale-Free Networks, characterized by power-law degree distributions, model networks where a few highly connected nodes (“hubs”) dominate, reflecting potential influence dynamics. Conversely, Erdős-Rényi Random Networks provide a baseline for comparison, establishing connectivity through random edge creation. These network structures directly impact the agent-to-agent communication pathways and, consequently, the collective decision-making process within the simulations.

The Omega parameter within the BioLLMAgent framework functions as a weighting factor for external priors, specifically controlling the degree to which the Large Language Model (LLM) component influences decision-making. Simulation results from agent-based modeling of decision-making scenarios indicate that community-wide educational interventions yield the most significant positive impact, achieving an average health score of 0.950. This outcome surpasses the performance of both targeted interventions, focused on specific individuals, and randomly applied interventions, demonstrating the effectiveness of broad-based educational strategies within the modeled system.

GPT-4o accurately predicts choice patterns across diverse behavioral datasets, including those from individuals with amphetamine or heroin use disorder and healthy controls.
GPT-4o accurately predicts choice patterns across diverse behavioral datasets, including those from individuals with amphetamine or heroin use disorder and healthy controls.

Implications for Understanding and Modeling Mental Health

BioLLMAgent presents a novel computational framework designed to dissect the intricate cognitive processes that contribute to mental health disorders, with a specific focus on conditions characterized by impulsivity and compromised decision-making abilities. By integrating large language models with reinforcement learning, the system simulates human thought processes, allowing researchers to observe how individuals might evaluate options, weigh risks, and ultimately make choices. This simulation capability offers a unique window into the ‘black box’ of the mind, enabling the investigation of cognitive biases and impairments that often underlie mental illness. The framework doesn’t simply offer a descriptive account; it allows for the manipulation of variables and the testing of hypotheses regarding the neural mechanisms driving these behaviors, providing a powerful tool for both understanding and potentially treating conditions like addiction, ADHD, and certain anxiety disorders.

BioLLMAgent uniquely simulates the cognitive processes of Delay Discounting and Hyperbolic Discounting, offering a computational window into how the brain assigns value to rewards over time. These models aren’t merely abstract representations; they capture the observed human tendency to prioritize immediate gratification, even if it means sacrificing larger, later rewards. By manipulating parameters within the framework – such as the rate at which future rewards are devalued – researchers can explore how alterations in neural systems related to reward processing might manifest as behavioral changes. This approach allows for the investigation of conditions where reward valuation is disrupted, like addiction or impulsivity, offering potential explanations for why individuals might make choices that appear irrational from a purely logical standpoint. Ultimately, the framework provides a platform to connect computational models of behavior to underlying neural mechanisms, furthering understanding of the complex interplay between brain function and decision-making.

BioLLMAgent’s architecture allows for the direct implementation of Cognitive Behavioral Therapy (CBT) principles within its simulated environment, offering a novel approach to modeling therapeutic efficacy. By translating core CBT techniques – such as exposure therapy, cognitive restructuring, and behavioral activation – into computational algorithms, researchers can observe how these interventions impact the agent’s decision-making processes and impulsive behaviors. This capability extends beyond simple simulation; the framework allows for the testing of personalized treatment strategies, tailoring interventions to the agent’s unique cognitive profile and observed responses. Consequently, BioLLMAgent facilitates a virtual testing ground for refining therapeutic approaches and predicting individual patient outcomes, potentially accelerating the development of more effective and targeted mental healthcare interventions.

A novel pathway for both constructing and rigorously testing computational models of mental illness is now available, potentially revolutionizing diagnostic and treatment strategies. This framework doesn’t simply simulate illness; it allows for the accurate recovery of key cognitive parameters-exceeding a correlation of 0.67 for core elements within the reinforcement learning engine-demonstrating a strong ability to pinpoint underlying mechanisms. Moreover, the model exhibits a high degree of accuracy-a correlation above 0.84-in identifying patterns related to frequency preference and perseveration, behaviors often associated with a range of mental health conditions. This level of identifiability suggests the potential for personalized interventions tailored to an individual’s specific cognitive profile, and offers a robust platform for validating the efficacy of new therapeutic approaches before clinical implementation.

The framework generalizes to a delay discounting task, revealing that clinical groups exhibit lower baseline choice rates but steeper ω-response curves similar to those observed in the Iowa Gambling Task, and benefit most from cognitive behavioral therapy guidance at ω = 0.25, validating the cross-task generalization of the underlying fusion mechanism.
The framework generalizes to a delay discounting task, revealing that clinical groups exhibit lower baseline choice rates but steeper ω-response curves similar to those observed in the Iowa Gambling Task, and benefit most from cognitive behavioral therapy guidance at ω = 0.25, validating the cross-task generalization of the underlying fusion mechanism.

The development of BioLLMAgent exemplifies a crucial principle: systems break along invisible boundaries – if one cannot see them, pain is coming. This framework, integrating reinforcement learning and large language models, doesn’t simply model decision-making; it strives for structural interpretability. By explicitly linking behavioral outputs to underlying mechanisms – a core aim of computational psychiatry – BioLLMAgent attempts to make those previously invisible boundaries visible. As Bertrand Russell observed, “The point of the system is to make simple things simple, not complex things simple.” BioLLMAgent attempts this by clarifying the interactions within the decision-making process, offering a more transparent and robust simulation than purely ‘black box’ approaches.

Looking Ahead

The introduction of BioLLMAgent represents a cautious step toward bridging the explanatory gap between algorithmic performance and the phenomenological experience of decision-making. The framework’s strength lies in its attempt to constrain the inherent opacity of large language models with the structural demands of reinforcement learning-a necessary, if imperfect, marriage. However, the current iteration remains, fundamentally, a simulation of structure, not a revelation through it. Future work must move beyond behavioral mimicry and grapple with the question of what constitutes meaningful internal representation within such a hybrid system.

A critical limitation resides in the reliance on task-specific proxies for internal states. The Iowa Gambling Task, while useful, provides a narrow window onto the complexities of human choice. Expanding the framework’s scope to incorporate more ecologically valid scenarios, and developing methods for validating the emergent ‘reasoning’ of the agent-not just its outcomes-will be essential. The temptation to over-interpret behavioral convergence must be resisted; correlation is not causation, especially when dealing with artificial cognitive architectures.

Ultimately, the true test of such models will not be their ability to replicate existing data, but their capacity to generate novel, testable predictions about the underlying mechanisms of mental illness. Good architecture is invisible until it breaks, and only then is the true cost of decisions visible.

Original article: https://arxiv.org/pdf/2603.05016.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2026-03-07 05:49