Seeing is Understanding: Figures for the Age of AI

by

Author: Denis Avetisyan

A new approach reimagines scientific figures not just as static images, but as interactive, data-rich interfaces for both humans and artificial intelligence.

The framework facilitates a non-linear, iterative exploration of data through an architecture that interlinks user input with AI-generated visual and analytical representations-[latex]VV[/latex] and [latex]CC[/latex] & [latex]DD[/latex]-supported by an underlying large language model and databases, enabling progressive in-depth analysis through both refinement of existing figures and generation of new, coordinated visuals from user-selected data points.
The framework facilitates a non-linear, iterative exploration of data through an architecture that interlinks user input with AI-generated visual and analytical representations-[latex]VV[/latex] and [latex]CC[/latex] & [latex]DD[/latex]-supported by an underlying large language model and databases, enabling progressive in-depth analysis through both refinement of existing figures and generation of new, coordinated visuals from user-selected data points.

This review proposes a framework for constructing ‘LLM-native’ figures that combine human legibility with machine interpretability to enhance scientific reasoning and data exploration.

Despite advances in large language models (LLMs) for scientific workflows, figures-critical outputs for conveying data and insights-remain largely static images requiring re-interpretation by both humans and machines. This limitation motivates the work presented in ‘Figures as Interfaces: Toward LLM-Native Artifacts for Scientific Discovery’, which introduces a paradigm shift toward data-driven artifacts embedding complete provenance, analytical operations, and visualization specifications. These “LLM-native figures” enable LLMs to not only view but understand visualizations, facilitating interactive exploration, code generation, and transparent reasoning. Could this approach redefine the scientific figure as an active interface for discovery, rather than a final product?

The Erosion of Traditional Inquiry: Navigating Data’s Complexity

The pursuit of scientific understanding now relies heavily on navigating increasingly complex datasets, yet conventional analytical approaches often prove inadequate for this task. Traditional workflows tend to be linear – a rigid sequence of hypothesis, experimentation, and analysis – which limits the ability to adapt to unexpected findings or refine investigative paths. This contrasts sharply with the iterative nature of true discovery, where initial observations frequently necessitate revisiting assumptions and exploring alternative avenues. Consequently, researchers may overlook crucial patterns or miss opportunities for innovation simply because existing tools and methods lack the flexibility to support a dynamic, responsive exploration of the data landscape. The limitations of these established approaches underscore the need for novel methodologies capable of mirroring the organic, evolving process of scientific inquiry.

Current data visualization techniques frequently present scientific findings as finished products, static images or charts that obscure the exploratory process itself. This emphasis on polished presentation often overshadows the crucial, iterative nature of discovery, limiting a researcher’s ability to identify subtle patterns or unexpected insights hidden within complex datasets. Unlike dynamic systems that allow for manipulation and re-evaluation of data from multiple angles, traditional tools encourage a linear interpretation, potentially leading to overlooked correlations or premature conclusions. The inability to readily revisit prior analytical steps and assess the impact of different parameters hinders a truly nuanced understanding, effectively creating a ‘black box’ around the science and reducing the potential for serendipitous breakthroughs.

Scientific inquiry increasingly generates datasets so vast and interconnected that traditional research methods often capture only a snapshot of understanding, losing the crucial context of discovery. Preserving the complete trajectory of exploration – detailing the questions asked, the paths investigated, and the reasoning behind each analytical step – is no longer simply good practice, but a fundamental necessity. This approach, often termed ‘exploratory process capture,’ allows for reproducibility beyond mere code and data, enabling researchers to revisit the why behind conclusions and identify potential biases or overlooked avenues. By documenting not just what was found, but how it was found, science can move beyond isolated results towards a more robust and transparent understanding of complex phenomena, fostering greater collaboration and accelerating the pace of innovation.

This framework facilitates dynamic data exploration by translating user interactions with visualizations and natural language into precise database queries, generating linked, interactive figures and persistently storing the exploration trajectory to enable revisitable and extensible analyses.
This framework facilitates dynamic data exploration by translating user interactions with visualizations and natural language into precise database queries, generating linked, interactive figures and persistently storing the exploration trajectory to enable revisitable and extensible analyses.

Nexus: A System for Reclaiming the Iterative Process

Nexus is a novel system leveraging ‘LLM-Native Figures’ and ‘Data-Driven Artifacts’ to integrate human exploratory data analysis with computational methods. LLM-Native Figures are visual representations directly generated and interpretable by large language models, enabling programmatic access to visual insights. Data-Driven Artifacts are composable visual elements that encapsulate data, analyses, and associated metadata. This architecture allows users to iteratively refine analyses and facilitates a bidirectional flow of information between intuitive visual exploration and rigorous computational processing, effectively bridging the gap between qualitative and quantitative approaches to data understanding.

Data-driven artifacts within Nexus are designed as modular, composable units that extend beyond static visual representations. Each artifact encapsulates a complete record of user interactions, including all applied filters, specific analytical operations performed, and associated parameters. This detailed logging creates a comprehensive provenance trail, allowing for precise reconstruction of the analytical process. The composable nature of these artifacts enables users to build complex analyses from simpler components, while the preserved provenance ensures reproducibility and facilitates detailed examination of the reasoning behind each step.

Nexus enhances research reproducibility and insight by directly integrating analytical lineage with visual data representations. Each visual element within Nexus isn’t merely a display of processed data, but a record of the complete computational steps used to generate it; this includes specific filters, transformations, and analytical functions applied. This embedded provenance allows users to precisely retrace the analytical pathway used to create a particular view, validating results and identifying potential sources of error. Furthermore, visualizing this lineage alongside the data itself provides a comprehensive understanding of the exploratory process, enabling researchers to identify patterns, refine hypotheses, and build upon previous findings with greater confidence and efficiency.

inNexus utilizes a hybrid user interface combining natural language and graphical interactions to access and manage scientific datasets and knowledge through a multi-agent LLM engine.
inNexus utilizes a hybrid user interface combining natural language and graphical interactions to access and manage scientific datasets and knowledge through a multi-agent LLM engine.

The Multi-Agent LLM Engine: Orchestrating Automated Reasoning

The Nexus Multi-Agent LLM Engine functions by distributing analytical tasks across three distinct agents: a Planner, an Executor, and an Evaluator. The Planner agent is responsible for decomposing complex queries into a series of actionable steps. The Executor agent then carries out these steps, utilizing external tools and data sources as needed. Finally, the Evaluator agent assesses the results of each step and provides feedback to refine the process, ensuring accuracy and completeness. This modular design allows for a more robust and adaptable system capable of handling multifaceted scientific reasoning challenges by breaking them down into manageable components.

Retrieval-Augmented Generation (RAG) is employed within the Multi-Agent LLM Engine to improve the reliability and accuracy of generated responses. This process involves supplementing the Large Language Model’s (LLM) parametric knowledge with information retrieved from external knowledge sources prior to response generation. Specifically, relevant documents or data points are identified and incorporated into the LLM’s context, effectively grounding the response in factual evidence. This mitigates the risk of hallucination and enables the system to address queries requiring information beyond its pre-training data, resulting in more informed and verifiable outputs.

The Multi-Agent LLM Engine demonstrates a measurable capability for automated scientific reasoning, achieving 92.7% End-to-End Accuracy in initial question generation and 79.8% accuracy for subsequent follow-up question answering. This performance indicates a functional bidirectional mapping, allowing the system to not only generate questions but also to correctly interpret and respond to related inquiries. Furthermore, the system achieved a 96.7% Execution Success Rate during the initial question generation phase, signifying a high degree of reliability in task completion.

The Multi-Agent LLM Engine incorporates both Python and SQL capabilities to directly access and manipulate data required for analytical tasks. This allows the system to move beyond text-based reasoning and perform computations and database queries as needed. The ‘Action Space’ defines the permissible operations available to the agents, encompassing a compositional set of functions that can be combined to formulate complex data interactions; these operations include, but are not limited to, data retrieval, filtering, aggregation, and basic statistical analysis, all executed through Python and SQL interpreters.

Beyond Static Visualization: Towards Machine-Interpretable Scientific Communication

Nexus introduces ‘LLM-Native Figures’ that redefine how scientists interact with data by establishing ‘Bidirectional Mapping’ between visual representations and the analytical processes that generated them. This innovative approach moves beyond static charts and graphs, allowing users to directly manipulate visual elements – such as highlighting a trend or zooming into a specific data point – and instantly see the corresponding changes reflected in the underlying analytical operations. Conversely, modifications to the analytical parameters are immediately visualized, fostering an iterative exploration process where insight emerges from the dynamic interplay between observation and computation. This seamless transition empowers researchers to not only view data, but to actively query and refine their analyses directly through the visual interface, unlocking a more intuitive and powerful method for scientific discovery.

Conventional scientific visualizations often serve as static summaries of data analysis, limiting interaction to panning, zooming, and basic filtering. However, a new paradigm is emerging where visualizations are no longer passive displays but dynamic, computational interfaces. This approach allows researchers to directly manipulate underlying data and analytical processes through the visual representation itself, fostering an iterative cycle of exploration and discovery. By embedding computational power within the visualization, complex operations – such as altering parameters, applying filters, or testing hypotheses – become intuitive and visually driven. This bidirectional mapping between visual elements and analytical functions transforms scientific communication, moving beyond simply showing results to actively enabling investigation and promoting a more profound understanding of the data’s origins and implications.

Nexus prioritizes the crucial elements of scientific rigor by meticulously preserving the analytical lineage of data transformations and computations. This isn’t simply about displaying results; the platform establishes a traceable record of how those results were obtained, facilitating independent verification and bolstering confidence in the findings. More than just transparency, Nexus actively enables computational interaction, allowing users to directly manipulate the underlying analytical processes within the visualization itself. This fosters a deeper, more intuitive understanding of the data and the methods used to analyze it, while simultaneously promoting reproducibility by making the entire workflow readily accessible and executable. The result is a shift from static reports to dynamic, interactive explorations that empower researchers to not only view data, but to interrogate, validate, and build upon it with greater confidence and efficiency.

This case study demonstrates an interactive data exploration workflow where figures are dynamically generated and refined-including adjustments to logarithmic scales and cross-figure filtering-to reveal insights about a university's innovation landscape and its research groups, ultimately culminating in a data-driven artifact.
This case study demonstrates an interactive data exploration workflow where figures are dynamically generated and refined-including adjustments to logarithmic scales and cross-figure filtering-to reveal insights about a university’s innovation landscape and its research groups, ultimately culminating in a data-driven artifact.

The pursuit of LLM-native figures, as detailed in the paper, acknowledges an inherent truth about all complex systems: they are not static entities, but rather evolve through interaction and refinement. This echoes Vinton Cerf’s observation that, “Any sufficiently advanced technology is indistinguishable from magic.” The creation of these data-driven artifacts-figures capable of both human comprehension and machine interpretation-is akin to building systems designed to age gracefully. The paper’s emphasis on provenance and interactive exploration suggests an acceptance that errors are not failures, but rather essential steps toward a more robust and mature understanding, transforming data into a medium for discovery rather than merely a metric of observation.

What Lies Ahead?

The construction of ‘LLM-native figures’ represents a subtle, perhaps inevitable, shift in how scientific knowledge is represented. These artifacts, simultaneously legible to humans and machines, do not so much solve the problem of scientific communication as they reframe it. The enduring challenge is not to eliminate ambiguity-ambiguity is inherent to complex systems-but to manage its propagation. These figures, like all interfaces, will accrue entropy. The precision with which data is embedded will degrade, not through error, but through the simple passage of time and the evolving standards of machine interpretation.

The current work highlights the potential for interactive exploration, but sidesteps the question of scale. A handful of demonstrative figures, however elegant, do not constitute a robust ecosystem. The true test will lie in the ability to construct and maintain vast networks of these artifacts, linked by provenance and capable of supporting genuinely emergent discovery. This is not merely a technical challenge; it’s a question of long-term stewardship.

One might even suggest that the pursuit of ‘machine-interpretability’ is a form of optimistic engineering. Stability is, after all, often just a temporary reprieve from inevitable decay. The figures will change, the models will evolve, and the meaning embedded within them will drift. The task, then, is not to create perfect representations, but to build systems capable of gracefully accommodating-and even leveraging-that inevitable transformation.

Original article: https://arxiv.org/pdf/2604.08491.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2026-04-10 17:40