Beyond Collaboration: Engineering Intelligence at the Human-AI Frontier

Author: Denis Avetisyan

A new framework formalizes decision-making competence as the core of intelligence, paving the way for truly integrated human-AI systems.

Human and artificial intelligence are synergistically linked through the integration of their respective decision matrices, facilitated by a Human-Machine Interface, effectively creating a unified cognitive system where decisions are informed by both human intuition and computational analysis.

This review proposes a Joint Hybrid Intelligence approach leveraging extended swarming and joint agent engineering for optimized human-swarm interaction and decentralized control.

Despite advances in artificial intelligence, effectively integrating human oversight and responsibility remains a critical challenge. This paper, ‘An Approach to Joint Hybrid Decision Making between Humans and Artificial Intelligence’, proposes a framework for Joint Hybrid Intelligence (JHI) that formalizes intelligence as decision-making competence and emphasizes the synergistic combination of human and artificial capabilities. Through joint agent engineering-integrating operator training, AI development, and interface design-the research outlines a design space exemplified by the ‘extended swarming’ concept for human-swarm interaction. How can such a framework pave the way for more robust and adaptable human-AI systems in complex, real-world scenarios?

Beyond Algorithmic Limitations: The Rise of Joint Intelligence

Traditional automation strategies, reliant on pre-programmed responses, frequently falter when confronted with the unpredictable nature of real-world scenarios. These systems, designed for static conditions, exhibit limited capacity to adjust to unforeseen events or nuanced variations within complex environments. Consequently, applications ranging from autonomous navigation in cluttered spaces to dynamic resource allocation in rapidly changing markets often reveal the shortcomings of purely algorithmic control. The demand for more resilient and versatile systems has therefore spurred research into adaptive architectures capable of learning, generalizing, and responding effectively to the inherent uncertainty present in dynamic operational landscapes. This necessitates a move beyond rigid programming toward systems exhibiting a degree of cognitive flexibility and situational awareness.

Traditional control systems, whether relying on fully centralized direction or independent decentralized agents, frequently falter when confronted with real-world complexity and unpredictability. Centralized approaches become bottlenecks, unable to process information and react quickly enough to dynamic changes, while purely decentralized systems often lack the necessary coordination and overarching strategic awareness. Consequently, a fundamental shift is occurring, recognizing the need to move beyond these limitations through integrated human-autonomy teaming. This emerging paradigm acknowledges that the most robust and adaptable systems will leverage the unique strengths of both humans – particularly in areas like nuanced judgment, ethical reasoning, and novel problem-solving – and artificial intelligence – excelling at data processing, pattern recognition, and rapid computation. The convergence of these capabilities promises solutions that surpass the effectiveness of either approach in isolation, paving the way for more resilient and intelligent systems capable of navigating increasingly complex environments.

Joint Hybrid Intelligence represents a move beyond traditional automation by explicitly designing systems that leverage the complementary strengths of both humans and artificial intelligence. This framework, detailed in the present work, moves past simply assisting human operators; instead, it proposes a synergistic partnership where human cognitive abilities – such as nuanced judgment, adaptability, and common sense reasoning – are integrated with the speed, precision, and data-processing capabilities of AI. The resulting joint agent isn’t merely the sum of its parts, but a cohesive entity capable of outperforming either human or AI acting alone, particularly in complex and unpredictable environments. By formalizing this integration, the study offers a pathway to optimize joint performance, improving decision-making and operational effectiveness across a range of applications, from robotics and aerospace to healthcare and cybersecurity.

This joint agent architecture enables humans to deploy AI for semi-autonomous action within limited, defined regions of a complex situation space, as demonstrated in previous work (Hasbach 2021b).

Extending Agency: The Logic of Swarm Interaction

Robot swarms present opportunities to augment human capabilities beyond the limitations of single robots or traditional teleoperation; however, realizing this potential requires control mechanisms distinct from those used with individual robots. Traditional interfaces, designed for direct manipulation, become impractical when managing numerous agents simultaneously. Effective interaction necessitates methods for high-level command specification, abstracting away low-level control of each robot. This includes developing interfaces that allow operators to define goals and constraints, rather than explicitly dictating individual robot movements. Furthermore, such systems must account for the swarm’s inherent complexity – including communication limitations, individual robot failures, and emergent behaviors – to ensure predictable and reliable performance as an extension of the operator’s agency.

Extended Swarming establishes a human-swarm interface where the robot swarm functions as a physical extension of the operator’s capabilities. This framework moves beyond traditional teleoperation by enabling the swarm to perform tasks at a scale or in environments inaccessible to a single human or robot. The operator maintains high-level control, specifying goals and constraints, while the swarm autonomously determines the optimal execution through collective behavior. This distributed approach effectively increases the operator’s spatial reach, manipulation capacity, and ability to operate in complex or dangerous environments, allowing for tasks such as large-area search, distributed sensing, and cooperative manipulation of objects exceeding individual robot capabilities.

Successful human-swarm interaction necessitates the development of a “Sense of Ownership” in the operator, defined as the feeling of control and agency over the swarm’s actions. This is not simply about issuing commands; it involves the operator perceiving the swarm as an extension of their own capabilities, fostering a direct mapping between intent and swarm behavior. Research indicates that a strong Sense of Ownership correlates with increased operator performance, reduced cognitive load, and improved trust in the swarm system. Achieving this requires transparent communication of swarm status, predictable responses to operator input, and mechanisms for intuitive control, allowing the operator to effectively internalize the swarm’s actions as their own.

A robot swarm utilizes a distributed fusion network to extend body postures towards sensed objects of interest (red arrows), presenting a collective movement gradient observable by a human operator.

Joint Agent Design: A Systems-Level Imperative

Effective joint agent engineering necessitates the concurrent consideration of multiple, traditionally separate design areas, including human-computer interaction, multi-agent systems, and artificial intelligence. This integration is crucial for establishing shared awareness – a common understanding of the environment, agent states, and goals – between human operators and automated agents. Focusing on shared awareness directly supports collaborative decision-making, where agents and humans can contribute complementary expertise and resources to achieve objectives. This requires defining clear communication protocols, shared data representations, and mechanisms for conflict resolution between agents with potentially differing objectives or interpretations of the environment. Successful joint agent systems are not simply about combining individual components; they are about creating a cohesive system where information flows freely and decision-making authority is appropriately distributed.

The application of Cognitive Systems Engineering (CSE) to joint agent design centers on modeling and incorporating principles of human cognition – including perception, memory, reasoning, and action – into the agent’s architecture and operational parameters. This approach moves beyond purely algorithmic solutions by explicitly addressing how agents process information, form beliefs, and make decisions under conditions of uncertainty, mirroring human cognitive limitations and strengths. Specifically, CSE provides frameworks for analyzing task demands, identifying cognitive constraints, and designing interfaces and interactions that optimize the distribution of cognitive workload between human operators and automated agents, ultimately enhancing overall system performance and situation awareness. The resulting designs prioritize usability, adaptability, and robustness by grounding agent behavior in empirically validated models of human cognitive processes.

Information fusion is a critical process in joint agent systems, combining data from multiple sources – including human input and swarm sensor data – to construct a unified environmental representation. This integration goes beyond simple data aggregation; it necessitates resolving inconsistencies, reducing uncertainty, and generating a shared understanding accessible to both human operators and the autonomous swarm. Within the framework presented in this paper, intelligence is formalized as decision-making competence, and information fusion serves as the foundational element for enabling that competence by providing the necessary contextual awareness for effective action selection. The quality of fused information directly impacts the swarm’s ability to operate effectively and the human’s ability to supervise and collaborate with the swarm, thus highlighting its centrality to the overall system performance.

Joint agent engineering integrates diverse design subspaces to facilitate a triad of hybrid intelligence, enabling analysis beyond those individual components.

From Cybernetics to Embodied Intelligence: A Foundational Shift

Agent theory, stemming from the insights of Cybernetics, posits that intelligent behavior arises not from internal complexity alone, but from the dynamic interplay between an agent and its environment. This perspective reframes intelligence as a process of effective action – an agent perceives its surroundings, formulates plans, and executes them to achieve goals, constantly adjusting based on feedback. Rooted in the study of control and communication in both machines and living organisms, this framework emphasizes the importance of feedback loops and information processing. Consequently, understanding intelligence requires analyzing how agents maintain stability and adapt to change within their environments, moving beyond a focus solely on internal cognitive processes to consider the reciprocal relationship between the agent and the world it inhabits. This foundational approach provides a crucial lens through which to examine and ultimately design intelligent systems, offering a powerful alternative to traditional, purely computational models.

Contemporary cognitive science increasingly views intelligence not as isolated computation within the brain, but as a dynamic process deeply shaped by the body and its interactions with the environment – a perspective encapsulated by the 4E Cognition framework. This approach posits that cognition is fundamentally embodied, meaning it’s inextricably linked to the physical form and sensorimotor capabilities of an agent; embedded, highlighting the crucial role of the surrounding environment in shaping cognitive processes; enactive, emphasizing that cognition arises through skillful action and interaction, rather than passive reception of information; and extended, suggesting that cognitive processes can extend beyond the brain and body to include external tools and resources. Consequently, understanding intelligence requires examining how an agent’s physical presence, environmental context, active engagement, and use of external aids collectively contribute to its ability to navigate and make sense of the world.

The development of genuinely integrated human-swarm systems necessitates a shift in how intelligence is conceptualized, moving beyond traditional agent-centric views. Recognizing intelligence as fundamentally situated – embodied, embedded, enactive, and extended – allows for the creation of systems where the distinction between the acting agent and its surrounding environment becomes increasingly porous. This perspective, formalized within this work’s framework of intelligence as decision-making competence, highlights that effective action isn’t simply a product of internal computation, but arises from the dynamic interplay between an agent and its context. Consequently, designing systems that leverage this fluidity requires prioritizing adaptive responsiveness and contextual awareness, enabling seamless collaboration and shared decision-making between humans and swarms – a departure from systems where the swarm merely executes pre-defined commands.

An agent interacts with its environment through sensors, decision-making, and physical effectors.

Decentralized Collaboration: Envisioning a Future of Augmented Intelligence

Decentralized control architectures represent a significant departure from traditional, centralized systems, fostering resilience and adaptability through distributed decision-making. Rather than relying on a single point of failure, these systems distribute control across multiple agents, allowing the overall network to continue functioning even if individual components fail. When coupled with principles of augmented cognition – technologies designed to enhance human cognitive capabilities – the potential for robust collaboration expands dramatically. This synergy allows human operators to oversee and guide swarms of autonomous agents, benefiting from the swarm’s collective processing power and rapid response while retaining critical oversight and ethical control. The result is a system capable of navigating complex, unpredictable environments and responding effectively to unforeseen challenges, exceeding the capabilities of either humans or machines acting independently.

The emergence of Joint Hybrid Intelligence represents a fundamental re-evaluation of human-machine interaction, extending beyond simple technological progress. This paradigm envisions a collaborative partnership where the strengths of both agents – human cognitive flexibility and machine processing power – are synergistically combined to achieve outcomes exceeding individual capabilities. The framework detailed in this study moves beyond treating humans as supervisors of automated systems, instead proposing a system for optimizing performance through a continuous exchange of information and adaptive task allocation between human and machine partners. This isn’t simply about automating tasks; it’s about redefining the roles within a collaborative effort, allowing each agent to specialize in areas where it excels, and dynamically adjusting to changing circumstances with enhanced resilience and efficiency. Ultimately, this approach promises a future where intelligent systems don’t just assist humans, but genuinely collaborate with them, fostering a new era of joint problem-solving.

The convergence of decentralized control and augmented cognition unlocks a remarkable spectrum of practical applications, extending far beyond theoretical possibilities. Consider the implications for search and rescue operations, where swarms of drones, guided by human operators with enhanced situational awareness, could navigate complex terrains and locate individuals in distress with unprecedented speed and accuracy. Similarly, environmental monitoring stands to be revolutionized; distributed sensor networks, functioning as intelligent swarms, could provide real-time data on pollution levels, deforestation patterns, or wildlife migration, enabling proactive conservation efforts. This isn’t simply about automating tasks, but fostering a synergistic partnership where human expertise and collective intelligence combine to address challenges previously considered insurmountable, ultimately promising a future defined by seamless human-swarm collaboration across diverse domains.

This intelligent architecture integrates fast, reactive low-level loops with slower, higher-level cognitive processes for adaptable behavior, as illustrated by the work of Rockbach.

The pursuit of Joint Hybrid Intelligence, as detailed in the paper, necessitates a rigorous formalism of intelligence itself. It’s not merely about achieving a functional outcome, but establishing a provable competence in decision-making. This echoes John von Neumann’s sentiment: “The sciences do not try to explain why we exist, but how we exist.” The paper’s emphasis on formalizing intelligence as ‘decision-making competence’-and the subsequent design of joint agent systems through pattern-based approaches like extended swarming-is akin to defining how intelligence functions within a combined human-AI framework, rather than speculating on its origins. The focus remains on the logical completeness and non-contradiction of the system’s operational principles.

What’s Next?

The formalization of intelligence as decision-making competence, while logically sound, merely shifts the burden of proof. One is left to rigorously define – and subsequently verify – ‘competence’ itself. The presented framework, predicated on joint agent engineering and extended swarming, appears promising, yet fundamentally relies on the assumption that human cognitive processes can be effectively mapped onto, and integrated with, decentralized control algorithms. This remains an assertion, not a demonstrated truth.

Future work must confront the inherent limitations of pattern-based design. While identifying recurring patterns in complex systems is valuable, extrapolating from these patterns to predict future behavior-particularly in the presence of novel stimuli-introduces an irreducible element of uncertainty. The elegance of a mathematical model is inversely proportional to its dependence on empirical observation. Therefore, the true challenge lies not in building more sophisticated swarms, but in establishing a formal, provable relationship between information, computation, and genuine understanding.

The persistent question remains: can a system, however cleverly engineered, truly decide? Or does it merely execute a pre-determined sequence of operations, masked by the illusion of autonomy? The pursuit of Joint Hybrid Intelligence may ultimately reveal not the limits of artificial intelligence, but the profound and perhaps insurmountable complexities of intelligence itself.

Original article: https://arxiv.org/pdf/2512.00420.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/