Building Smart Worlds: Visual Programming for the Web of Things

Author: Denis Avetisyan

A new approach empowers non-programmers to design and deploy complex multi-agent systems for connected devices and applications.

A computational agent, constructed within a block-based programming environment, demonstrates the fundamental principle that even simple systems can exhibit complex, interactive behavior through iterative signaling-a “ping”-and response.

This review explores agent-oriented visual programming techniques for end-user development in hypermedia-driven Web of Things environments.

While increasingly complex systems permeate daily life, translating domain expertise into functional software remains a significant challenge for non-programmers. This paper introduces an approach to address this gap, detailing ‘Agent-Oriented Visual Programming for the Web of Things’, a visual programming system enabling the creation and reconfiguration of multi-agent systems using intuitive, blocks-based interfaces. By leveraging agent abstractions aligned with human reasoning, and integrating with the Web of Things, the system empowers end-users to design autonomous behaviours for interconnected devices. Could this approach unlock a new era of user-driven automation and truly democratize access to intelligent systems development?

The Illusion of Access: Why We Build Barriers into Software

For decades, the creation of software has been largely confined to those possessing specialized training in formal programming languages and complex development methodologies. This reliance on expert knowledge presents a significant barrier to entry, effectively limiting participation in the digital landscape to a relatively small segment of the population. The steep learning curves associated with languages like Python, Java, or C++ demand substantial time and resources, precluding many individuals – those with valuable domain expertise but lacking technical skills – from directly translating their ideas into functional applications. Consequently, a vast potential for innovation remains untapped, as the ability to shape technology is restricted to a select few, hindering a more inclusive and democratized approach to software development.

End-User Programming, or EUP, represents a significant shift in how software is created, moving beyond the traditional reliance on skilled programmers. This approach deliberately empowers individuals with minimal technical training to independently construct applications and automate tasks tailored to their specific needs. Rather than mastering complex coding languages, users engage with visual interfaces, intuitive scripting tools, or natural language processing to define desired behaviors. The goal isn’t to replace professional developers, but to democratize software creation, enabling a broader range of people to solve problems and express creativity through technology – from crafting personalized data analyses to building simple mobile applications, EUP unlocks digital agency for those previously excluded from the development process.

Despite the promise of democratizing software creation, current end-user programming (EUP) tools frequently struggle with complexity. While effective for automating simple tasks or configuring existing applications, these approaches often prove inadequate when users attempt to build more sophisticated solutions. This limitation stems from a trade-off: EUP prioritizes ease of use through simplified interfaces and restricted functionality, which inherently limits expressiveness. Consequently, users often encounter roadblocks when trying to implement nuanced logic, manage intricate data structures, or integrate with external systems, forcing them to revert to traditional programming methods or abandon their projects altogether. This lack of scalability and power remains a significant barrier to widespread adoption of EUP, hindering its potential to truly empower non-programmers.

Current limitations in end-user programming necessitate innovative strategies that reconcile ease of use with functional depth. While existing tools empower individuals without formal training to automate tasks or personalize software, these approaches frequently struggle with the complexity required for truly sophisticated applications. Researchers are actively exploring hybrid models – systems that blend visual, block-based interfaces with access to more expressive underlying code, or that intelligently translate simplified user actions into robust programming instructions. This fusion aims to unlock the creative potential of a wider audience, allowing non-programmers to build powerful, customized solutions without being constrained by the limitations of overly simplistic tools, ultimately democratizing software creation and fostering a more inclusive technological landscape.

Orchestrating Collective Intelligence: A New Paradigm for Agents

Agent-Oriented Visual Programming (AOVPr) represents an evolution of established visual programming (VP) methodologies by specifically addressing the complexities inherent in multi-agent systems. Traditional VP focuses on single-program logic; AOVPr extends these principles to encompass the design, coordination, and interaction of multiple autonomous agents. This involves visually representing agent behaviors, communication protocols, and environmental interactions, enabling developers to model decentralized systems without relying on traditional text-based coding. The core distinction lies in the shift from programming a single entity to orchestrating a collective of interacting agents, demanding visual tools capable of representing agent autonomy, reactivity, and proactivity within a shared environment.

Agent-oriented visual programming facilitates the creation and deployment of autonomous agents by individuals with domain expertise, but limited programming experience. This is achieved through the use of graphical user interfaces where agents and their behaviors are represented as visual components, eliminating the need for traditional code writing. Domain experts can define agent actions, interactions, and environments through manipulation of these visual elements – connecting blocks, adjusting parameters, and configuring relationships. The resulting visual representations are then compiled or interpreted by the system to generate executable agent programs, thereby lowering the barrier to entry for developing complex multi-agent systems and enabling rapid prototyping and iteration.

Block-Based Visual Programming (BBVP) constructs programs by manipulating graphical blocks representing code elements, rather than requiring textual code input. These blocks, typically interconnected via drag-and-drop interfaces, define program logic and data flow. The modular nature of BBVP facilitates the creation of complex behaviors from simpler components, and the visual representation minimizes syntax errors common in text-based coding. This approach lowers the barrier to entry for non-programmers, allowing domain experts to directly implement and test agent behaviors. Furthermore, BBVP environments often provide immediate visual feedback, accelerating the development and debugging cycles, and simplifying the process of creating and modifying agent programs.

Agent-Oriented Visual Programming intentionally leverages the well-established principles of Visual Programming (VP) to minimize the learning curve for developers and domain experts. VP, characterized by the use of graphical elements to represent program logic, has a history of improving accessibility in software development. This approach ensures that users familiar with existing VP environments can readily transfer their skills to designing multi-agent systems. By retaining core VP concepts such as drag-and-drop functionality, node-based connections, and visual feedback, Agent-Oriented Visual Programming aims to provide an intuitive and easily navigable interface, reducing the need for specialized coding expertise and fostering broader participation in the development of autonomous agent applications.

The Web of Things: A Fragile Consensus

The Web of Things (WoT) is an architectural approach that leverages established web standards – including HTTP, REST, and JSON – to integrate devices and services into a unified, interconnected system. This differs from traditional Internet of Things (IoT) architectures by abstracting away device-specific protocols and data formats, thereby promoting interoperability and simplifying development. Specifically, WoT utilizes a hypermedia-driven approach, allowing devices to advertise their capabilities and available actions through standardized metadata known as Thing Descriptions (TDs). These TDs, typically expressed in JSON-LD, provide a machine-readable description of a device’s interfaces, interactions, and security profiles, enabling dynamic discovery and seamless integration with web applications and other services. The resulting infrastructure allows for a scalable and flexible ecosystem where devices can be easily connected, controlled, and monitored using familiar web technologies.

Multi-Agent Systems (MAS) offer a distributed problem-solving approach well-aligned with the decentralized nature of the Web of Things (WoT). Each agent within a MAS can represent a device, service, or user, possessing autonomy to perform specific tasks and communicate with other agents to achieve complex goals. This architecture facilitates intelligent automation by enabling negotiation, coordination, and reactive behavior in response to dynamic conditions within the WoT. Agents can leverage information from WoT resources, represented in Thing Descriptions, to make informed decisions and execute actions without requiring centralized control, thereby increasing system robustness and scalability. The inherent flexibility of MAS allows for the seamless integration of new devices and services into existing WoT ecosystems, promoting adaptability and innovation.

Yggdrasil is a software platform designed to enable the deployment of Multi-Agent Systems (MAS) within the Web of Things (WoT) ecosystem. It functions as an intermediary, translating between the abstract logic of agent behavior and the concrete interactions required by WoT devices and services. Specifically, Yggdrasil supports hypermedia-driven MAS, meaning agents discover and interact with resources through links embedded in Web of Things Thing Descriptions (TDs). This approach allows for dynamic adaptation and composition of agent-based solutions without requiring pre-defined, rigid integrations. The platform provides tools for defining agent behaviors, managing communication, and orchestrating interactions with WoT resources, effectively bridging the gap between intelligent software and connected devices.

Interoperability within integrated Multi-Agent Systems (MAS) and the Web of Things (WoT) is achieved through the utilization of standardized machine-readable descriptions and hypermedia-driven interactions. WoT Thing Descriptions (TDs) provide a consistent format for defining a device’s capabilities, properties, actions, and associated metadata, enabling agents to discover and understand available resources. Furthermore, adherence to the principles of Hypermedia as the Engine of Application State (HATEOAS) ensures that agents navigate and interact with WoT resources dynamically, guided by links provided in resource representations rather than relying on pre-configured knowledge. This approach decouples agents from specific URI patterns and allows for flexible evolution of the WoT infrastructure without requiring changes to the agents themselves.

The Illusion of Rationality: Modeling Intent in Agents

The BDI (Belief-Desire-Intention) agent architecture is a widely used framework for designing autonomous agents that exhibit rational behavior. An agent’s Beliefs represent its knowledge about the environment and its own internal state. Desires define the agent’s goals – what it wants to achieve. However, not all desires are immediately acted upon; Intentions represent the desires that the agent has committed to pursuing. This distinction is crucial, as intentions drive the agent’s actions and are maintained until either fulfilled, abandoned, or superseded by new intentions. The architecture facilitates a deliberate and goal-directed approach to problem-solving by allowing agents to reason about their beliefs, evaluate potential desires, and commit to specific courses of action based on these considerations.

AgentSpeak is a declarative programming language tailored for the implementation of BDI agents, diverging from traditional imperative paradigms. It employs a rule-based approach where agent behavior is defined by a set of beliefs, desires, and intentions, expressed as event-condition-action rules. These rules specify under what conditions an agent should act, and what actions to perform based on its current beliefs and desires. The language supports features like prioritized rule selection, handling of belief updates, and the management of goals and sub-goals, facilitating the creation of agents capable of rational decision-making and goal-directed behavior. AgentSpeak’s syntax is designed to directly reflect the BDI concepts, enhancing readability and simplifying the development process for complex agent systems.

The JaCaMo platform is a Java-based framework designed to facilitate the development and deployment of multi-agent systems utilizing the BDI (Belief-Desire-Intention) agent architecture. It provides a comprehensive set of tools and libraries, including an agent kernel, a communication infrastructure based on message passing, and support for various agent interaction protocols. JaCaMo enables developers to define agents with beliefs, desires, and intentions, and to program their behavior using AgentSpeak, a dedicated agent programming language. The platform supports both simulation and real-world deployment, offering features for agent monitoring, debugging, and integration with external systems. Furthermore, JaCaMo provides a flexible architecture allowing for the dynamic creation and modification of agents and their interactions, making it suitable for complex and evolving multi-agent applications.

Programming by Example (PbE) and Programming by Demonstration (PbD) represent extensions to the BDI agent architecture that enable agents to acquire new behaviors through observation. In PbE, the agent learns a general function mapping observations to actions from a set of provided examples, effectively inducing a policy. PbD focuses on direct imitation of demonstrated actions, often employing techniques like sequence learning or reinforcement learning to generalize from the demonstrations. Both approaches bypass the need for explicit, hand-coded rules for certain tasks; instead, the agent infers its behavior from provided data, increasing adaptability and reducing development time. Successful implementation requires methods for effectively representing demonstrated behaviors and generalizing them to novel situations, often involving machine learning algorithms to handle variations in input and context.

The Limits of Usability: Acknowledging Inevitable Failure

Rigorous usability studies are paramount when designing programming tools intended for individuals without formal training in computer science. Evaluating accessibility isn’t merely about ensuring the interface is visually clear; it demands quantifying how effectively non-programmers can translate their intentions into functional agent behaviors. Tools like the System Usability Scale (SUS) provide a standardized metric, allowing researchers to move beyond subjective impressions and establish objective benchmarks for ease of use. Such evaluations pinpoint areas where the visual programming interface excels, and, crucially, identify specific obstacles that prevent novice users from successfully creating and deploying agent-based systems. Without this iterative process of testing and refinement, even the most intuitively designed interface risks remaining inaccessible to those it aims to empower, hindering the broader adoption of agent-oriented programming beyond the realm of experienced developers.

The effectiveness of the agent-oriented visual programming approach was rigorously assessed through usability testing, yielding an average System Usability Scale (SUS) score of 73.3 out of 100. This score indicates a demonstrably good level of usability, suggesting that individuals without prior programming experience can effectively interact with and utilize the interface. The SUS, a widely recognized questionnaire for gauging perceived usability, provides a quantifiable metric for evaluating the system’s learnability, efficiency, and overall satisfaction. A score exceeding 68 is generally considered above average, highlighting the potential of this visual programming method to empower non-programmers to create complex agent-based systems with relative ease.

The study’s evaluation revealed a 60% task completion rate among participants, indicating a moderate level of success with the Agent-Oriented Visual Programming interface. Specifically, 12 out of the 20 participants were able to successfully complete at least three out of the five assigned tasks, suggesting a core group demonstrated proficiency. While not a perfect score, this result offers a valuable baseline for understanding how readily non-programmers can utilize the system to achieve desired outcomes, and highlights areas where further refinement of the interface could improve overall task success and broaden accessibility for a wider range of users.

Further development of this agent-oriented visual programming system centers on broadening the scope of its capabilities. Currently, the interface allows for the creation of relatively simple agent behaviors; future iterations will prioritize enhanced expressiveness, enabling users to design more complex and nuanced interactions. This expansion will be coupled with the integration of advanced reasoning capabilities, allowing agents to not simply react to stimuli, but to proactively plan, anticipate, and adapt to changing circumstances. Such enhancements promise to move beyond basic automation towards genuinely intelligent systems that can tackle more challenging and open-ended problems, ultimately making this visual programming approach accessible to an even wider range of users and applications.

The pursuit of accessible systems development, as detailed in this exploration of agent-oriented visual programming, feels less like construction and more like tending a garden. One plants seeds – abstractions, visual elements – but the resulting growth is rarely what was precisely intended. As Ken Thompson observed, “Software is a complex medium – it’s not about creating something perfect, but about creating something that works.” This sentiment resonates deeply; the paper’s emphasis on empowering non-programmers to shape multi-agent systems acknowledges the inherent unpredictability. The goal isn’t to eliminate compromise – that’s impossible – but to distribute the capacity for adaptation, allowing systems to evolve with the needs of those who inhabit them. Architecture isn’t structure-it’s a compromise frozen in time, and this work seeks to thaw that freeze, bit by bit.

What Lies Ahead?

This work proposes a bridge, yet every span casts a shadow. The ambition – to place the power of multi-agent systems directly into the hands of those who understand the domain, not merely the syntax – is laudable. However, the garden will inevitably grow weeds. Visual programming, while lowering the initial barrier, does not eliminate the complexities of interaction, concurrency, and the subtle pathologies that emerge from distributed systems. The true challenge isn’t building the tools, but cultivating a literacy – a way of thinking about systems as forgiving ecosystems rather than brittle machines.

The focus on the Web of Things suggests a natural habitat for these agents, but the scale of the internet presents a particular kind of entropy. Maintaining coherence, ensuring security, and diagnosing failures within a web of user-created agents will require more than just visual interfaces. It demands a shift towards decentralized governance, towards mechanisms that allow agents to self-organize, to repair, and even to negotiate their own evolution. Resilience lies not in isolation, but in forgiveness between components.

The next iteration won’t be about more visual nodes or more elegant connections. It will be about the scaffolding – the meta-tools that allow users to understand the shape of their systems, to anticipate unintended consequences, and to cultivate a healthy, evolving garden rather than simply planting seeds and hoping for the best. A system isn’t a machine, it’s a garden – neglect it, and you’ll grow technical debt.

Original article: https://arxiv.org/pdf/2511.13158.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/