Robots with Memory: Learning from the Way We Do

by

Author: Denis Avetisyan

Researchers have developed a new robotic control system inspired by the human memory process, enabling improved long-term task performance and adaptability.

Systems designed to navigate complex tasks often falter when faced with repeated scenarios, their performance limited by an inability to retain crucial historical information-a deficiency addressed by MemoAct, a mechanism inspired by the Atkinson-Shiffrin model of human memory that achieves both precise task state tracking and robust long-horizon retention, demonstrably outperforming existing approaches on benchmarks like MemoryRTBench and RMBench, as well as in real-world applications.
Systems designed to navigate complex tasks often falter when faced with repeated scenarios, their performance limited by an inability to retain crucial historical information-a deficiency addressed by MemoAct, a mechanism inspired by the Atkinson-Shiffrin model of human memory that achieves both precise task state tracking and robust long-horizon retention, demonstrably outperforming existing approaches on benchmarks like MemoryRTBench and RMBench, as well as in real-world applications.

MemoAct leverages a hierarchical memory architecture, inspired by the Atkinson-Shiffrin model, to enhance task state tracking and long-horizon retention in robotic manipulation.

Effective robotic manipulation often demands retaining information over extended horizons, a capability challenging for policies reliant on limited observational histories. This limitation motivates the development of memory-augmented approaches, as explored in ‘MemoAct: Atkinson-Shiffrin-Inspired Memory-Augmented Visuomotor Policy for Robotic Manipulation’, which introduces a hierarchical memory framework inspired by the human cognitive architecture. Specifically, MemoAct leverages distinct short- and long-term memory tiers to achieve both precise task state tracking and robust long-horizon retention-capabilities validated through extensive experiments and a new benchmark, MemoryRTBench. Could this biologically-inspired approach unlock more adaptable and reliable robotic systems capable of truly complex, memory-dependent tasks?

The Erosion of Present-State Reliance: Beyond the Markovian Ideal

Conventional robotic control systems are frequently built upon the Markov Assumption, a principle stating that a system’s future state is entirely determined by its present state, disregarding past events. While simplifying computational demands, this approach introduces limitations when robots encounter real-world complexity. The assumption effectively creates a form of short-term memory loss, hindering performance in tasks requiring an understanding of history – such as assembling objects from a disordered pile or navigating previously explored environments. Consequently, robots operating under this framework often struggle with tasks that demand sequential reasoning or the ability to differentiate between perceptually similar situations based on prior experience, ultimately limiting their adaptability and robustness in dynamic, unpredictable settings.

Effective robotic manipulation in real-world environments often demands more than just reacting to the present; it requires a robot to remember and utilize past interactions. Consider a scenario where a robot is tasked with assembling objects from a cluttered table – successfully grasping a component might depend on recalling which objects were previously moved, or the order in which they were manipulated. This is because perceptual information can be ambiguous, and relying solely on the current sensor data is insufficient to accurately infer the state of the environment and plan future actions. Such tasks are fundamentally memory-dependent, meaning that optimal policies must incorporate a history of states to disambiguate perceptions, infer hidden information, and execute long-horizon plans – a capability that traditional robotic systems, built on the simplifying Markov Assumption, frequently lack.

Robotic systems built upon the Markov Assumption frequently encounter difficulties when faced with real-world complexity. Because these policies prioritize the present state, they struggle to resolve perceptual ambiguity – situations where current sensor data is insufficient to uniquely identify objects or their configurations. This limitation is particularly pronounced in long-horizon planning, where a sequence of actions must be considered over extended periods. Without the ability to retain and utilize information about past states, the robot’s predictive capacity diminishes, leading to suboptimal decisions and an inability to effectively navigate uncertain or partially observable environments. Consequently, even seemingly simple tasks requiring memory of previous interactions, such as assembling parts in a cluttered workspace, can prove challenging for robots reliant on this simplifying assumption.

The model's ability to track task state and retain long-horizon information is evaluated across a sequential series of simulation tasks (A through C from MemoryRTBench) and real-world scenarios, with identical observations highlighted in red or blue to indicate shared information.
The model’s ability to track task state and retain long-horizon information is evaluated across a sequential series of simulation tasks (A through C from MemoryRTBench) and real-world scenarios, with identical observations highlighted in red or blue to indicate shared information.

Echoes of Cognition: Modeling Memory with MemoAct

MemoAct’s memory architecture is fundamentally based on the Atkinson-Shiffrin Memory Model, a cognitive psychology framework that posits three sequential memory stores. Sensory memory, the initial stage, briefly holds perceptual information. This information is then transferred to short-term memory (STM), which has a limited capacity and duration. Through rehearsal and encoding, information can be consolidated into long-term memory (LTM), a relatively permanent and limitless store. MemoAct mirrors this structure by employing dedicated modules for processing and storing information across these three temporal stages, enabling the system to manage and retain information with varying degrees of persistence and accessibility.

The MemoAct architecture utilizes a Long Short-Term Memory (LSTM) Module to manage both short-term and long-term data storage. The LSTM maintains a lossless Short-Term Memory Bank (STMB) for immediate task-relevant information, allowing precise tracking of current state. Simultaneously, a Compressed Long-Term Memory Bank (LTMB) stores information in a compressed format, enabling retention of data over extended periods without significant computational overhead. This dual-bank system allows MemoAct to leverage the strengths of both immediate, accurate recall and durable, efficient storage, crucial for complex, extended tasks.

The Sensory Distillation Module within MemoAct utilizes the DINOv2 self-supervised vision transformer to process raw sensory inputs. DINOv2 is employed to generate a compact, information-rich representation of the observed environment, effectively reducing dimensionality while preserving key features. This distilled sensory information, consisting of a learned embedding vector, serves as the initial input to the memory system, replacing the need to process full-resolution images or other complex sensory data. By focusing on essential features, this module improves computational efficiency and facilitates long-term retention of relevant environmental context.

MemoAct’s modular design facilitates robust state tracking and extended information retention by segregating functional components. The Sensory Distillation Module provides a focused input stream, minimizing irrelevant data and improving the efficiency of subsequent memory storage. This distilled information is then processed by the Long Short-Term Memory Module, which independently manages both short-term and long-term memory buffers. The separation of these memory types – lossless STMB for immediate task data and compressed LTMB for persistent knowledge – allows MemoAct to maintain precise task state across extended operational periods, effectively addressing the limitations of traditional recurrent neural networks in long-horizon tasks.

MemoAct utilizes a sensory distillation module to encode current observations into a sensory memory, retrieves relevant historical context from a long short-term memory bank via a temporal transformer, and fuses this with current sensory input to guide an action decoder in generating history-aware action trajectories, with the memory bank continuously updated via a consolidation module.
MemoAct utilizes a sensory distillation module to encode current observations into a sensory memory, retrieves relevant historical context from a long short-term memory bank via a temporal transformer, and fuses this with current sensory input to guide an action decoder in generating history-aware action trajectories, with the memory bank continuously updated via a consolidation module.

The Architecture of Persistence: Compressing the Past

The Long-Term Memory Bank employs Causal Attention and Similarity-Based Merging to achieve efficient compression of historical data. Causal Attention restricts the attention mechanism to only previous tokens, minimizing computational requirements while preserving temporal dependencies. Similarity-Based Merging identifies and consolidates redundant or highly similar historical states into single representative states, further reducing the memory footprint. This process isn’t a simple truncation of history; instead, it intelligently reduces data volume by focusing on the most salient changes and relationships within the historical record, leading to both decreased computational cost and improved processing efficiency.

Temporal Positional Embeddings (TPE) within the Long Short-Term Memory Module serve to represent the sequential order of input tokens. Unlike methods that treat tokens as unordered sets, TPEs assign a unique vector to each token based on its position in the sequence. These vectors are then added to the token embeddings, allowing the model to differentiate between tokens with identical values but different positions. This positional information is critical for understanding temporal relationships and dependencies within the historical data stored in the Long-Term Memory Bank, as the meaning of a token can change based on its context within the sequence. The embeddings are learned during training, enabling the model to effectively encode and utilize temporal order for improved performance.

MemoAct’s compression strategy addresses the challenge of maintaining extensive historical data within a fixed computational budget. By employing Causal Attention and Similarity-Based Merging, the Long-Term Memory Bank reduces the overall data volume needing storage and processing. This is achieved by selectively retaining and consolidating relevant historical tokens, effectively distilling past experiences into a compact representation. The system prioritizes information retention based on both temporal relationships and semantic similarity, allowing it to preserve crucial contextual details while discarding redundant or irrelevant data points. Consequently, MemoAct can access and utilize a significantly larger historical record compared to systems with fixed-size memory buffers, without incurring a proportional increase in computational cost.

Unlike First-In, First-Out (FIFO) memory systems which discard older observations upon reaching a predetermined capacity, the architecture avoids fixed-size observation windows. These fixed windows inherently lose temporal context as they remove data points representing past interactions, limiting the system’s ability to understand long-range dependencies. By retaining a more comprehensive historical record through mechanisms like Causal Attention and Similarity-Based Merging, the architecture mitigates this data loss and preserves crucial information regarding the sequence of events, allowing for a more informed response to current inputs.

The Long Short-Term Memory Consolidation Module compresses and migrates short-term memory embeddings to a long-term memory bank by encoding and summarizing saturated short-term memory blocks with a temporal transformer, then merging similar entries in long-term memory to maintain storage efficiency.
The Long Short-Term Memory Consolidation Module compresses and migrates short-term memory embeddings to a long-term memory bank by encoding and summarizing saturated short-term memory blocks with a temporal transformer, then merging similar entries in long-term memory to maintain storage efficiency.

From Remembrance to Action: History-Informed Control

The core of MemoAct’s control strategy lies in its Action Decoder, a component designed to produce actions informed by a comprehensive understanding of past events. This decoder doesn’t simply react to the present state; it leverages memory-augmented tokens – data representations enriched with historical context – to anticipate the consequences of each action. By effectively ‘remembering’ previous steps and outcomes, the decoder can generate more robust and reliable behaviors, particularly in complex tasks demanding long-term planning. This history-aware approach allows the system to navigate challenges that would confound traditional policies, enabling consistent performance even as task parameters evolve or unexpected disturbances occur. The result is a system capable of adapting to dynamic environments and maintaining task success over extended periods.

The action generation process within MemoAct leverages a Denoising Diffusion Probabilistic Model (DDPM), a powerful generative technique known for creating diverse and high-quality outputs. However, unlike standard DDPM applications, this decoder isn’t operating in a vacuum; it’s fundamentally informed by a Long Short-Term Memory (LSTM) module. This LSTM serves as a historical context provider, enriching the diffusion process with a detailed understanding of past states and actions. By conditioning the DDPM on these memory-augmented tokens, the system moves beyond purely reactive control, gaining the ability to anticipate future needs and generate actions that are not only appropriate for the present but also strategically aligned with long-term task goals. This synergistic combination allows for nuanced and robust behavior, particularly in complex environments demanding sustained reasoning and adaptation.

Rigorous evaluation of MemoAct against established policies-utilizing the MemoryRTBench and RMBench datasets-reveals substantial performance gains. These benchmarks, designed to assess robotic control in memory-intensive tasks, consistently demonstrate MemoAct’s superiority; the system achieves improvements of up to 24.5% on MemoryRTBench and 21.0% on RMBench. This quantitative data highlights MemoAct’s ability to effectively leverage historical information, enabling more robust and efficient action selection compared to traditional approaches like Diffusion Policy, and validating its potential for complex, long-horizon robotic applications.

Evaluations within realistic environments reveal MemoAct consistently achieves a 98% success rate, representing a substantial 22.5% performance gain over existing robotic control methodologies. This heightened reliability stems from the system’s refined capacity to meticulously monitor task progression and preserve crucial information across extended operational timelines. The ability to accurately track state-even in complex and dynamic scenarios-allows MemoAct to anticipate future requirements and execute actions with a level of precision previously unattainable, ultimately broadening the scope of tasks robots can autonomously undertake and signaling a significant advancement in the field of robotic control.

MemoAct’s performance demonstrates that long-term memory governs retention over extended horizons, while short-term memory is essential for precise task state tracking, with varying memory capacities impacting these capabilities.
MemoAct’s performance demonstrates that long-term memory governs retention over extended horizons, while short-term memory is essential for precise task state tracking, with varying memory capacities impacting these capabilities.

The presented MemoAct framework acknowledges the inherent transience of systems, mirroring the natural decay observed in all complex processes. Like the Atkinson-Shiffrin model it draws inspiration from, the policy architecture doesn’t strive for perfect, immutable retention, but rather for a graceful degradation through hierarchical memory. This approach-consolidating critical task states into long-term storage while maintaining a short-term buffer-implicitly understands that ‘the essence of mathematics lies in its elegance and logical simplicity.’ G. H. Hardy, a proponent of mathematical purity, might appreciate the MemoAct’s effort to distill complex visuomotor tasks into fundamental, storable components, accepting that complete recall is impossible, and focusing instead on preserving the core functionality over extended horizons.

What Lies Ahead?

The pursuit of robust robotic manipulation inevitably encounters the limitations of transient states. MemoAct, by attempting to model the architecture of human memory, acknowledges this inherent decay – but perhaps frames it as a challenge to be solved rather than simply managed. Systems learn to age gracefully, and the emphasis on long-horizon retention, while commendable, begs the question of what is truly lost in the consolidation process. Is perfect recall even desirable, or does the selective forgetting inherent in biological systems confer an advantage in adapting to novel circumstances?

Future work will undoubtedly focus on scaling these memory architectures, increasing their complexity, and refining the mechanisms of task state tracking. However, a more fruitful avenue might lie in exploring the boundaries of memory. Where does the system begin to misremember, and how does it respond? Can imperfection be leveraged to create more resilient and adaptable policies? Sometimes observing the process of degradation is better than trying to speed it up.

Ultimately, MemoAct represents a step toward acknowledging that robotic systems, like all others, exist within the current of time. The true test will not be whether they can defeat decay, but whether they can learn to navigate it with a semblance of elegance.

Original article: https://arxiv.org/pdf/2603.18494.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2026-03-22 16:05