AI Coders Rely Heavily on Existing Tools

by

Author: Denis Avetisyan

A new study reveals how automated coding agents utilize existing software libraries when generating code contributions.

This research presents the first empirical analysis of library usage patterns in pull requests authored by AI coding agents, demonstrating a preference for established dependencies and robust version control.

While increasingly capable of automating complex software engineering tasks, the library usage patterns of AI coding agents remain largely unexplored-a critical gap given the centrality of external dependencies in modern development. This paper, ‘A Study of Library Usage in Agent-Authored Pull Requests’, presents an empirical analysis of over 26,000 agent-authored pull requests, revealing that agents frequently import existing libraries but rarely introduce new ones, and demonstrate surprisingly robust versioning practices when doing so. This contrasts with prior work on direct LLM usage and suggests agents are learning to navigate software ecosystems more effectively-but how will these patterns evolve as agents become even more autonomous and sophisticated?

The Evolving Landscape of Automated Code Generation

Artificial intelligence is fundamentally reshaping software development through the emergence of rapidly evolving coding agents. These tools, designed to automate and accelerate various stages of the coding process, are no longer limited to simple code completion; they now generate substantial code blocks, propose entire function implementations, and even contribute to complex system architectures. This progression promises a significant boost in developer productivity, allowing engineers to focus on higher-level design challenges and innovation rather than repetitive coding tasks. The increasing sophistication of these agents stems from advancements in machine learning, particularly large language models trained on vast repositories of code, enabling them to understand and generate syntactically correct and functionally relevant code with increasing accuracy and efficiency. Early adoption indicates a potential shift towards a more collaborative development paradigm, where AI agents function as powerful assistants, augmenting human capabilities and accelerating the pace of software innovation.

Modern AI coding agents demonstrate a significant reliance on pre-existing code, integrating external libraries into nearly 30% of submitted pull requests. This practice highlights a core strategy in their functionality: leveraging established solutions rather than consistently generating entirely novel code. The agents effectively function as assemblers, skillfully incorporating components from a vast digital ecosystem to expedite development. This dependency on external libraries isn’t merely a shortcut; it underscores the complexity of modern software development, where few projects are built entirely from scratch. Consequently, understanding how these agents select and integrate libraries is crucial, not just for optimizing their performance, but also for assessing the long-term implications for software supply chains and potential security vulnerabilities.

Recent advancements in AI-driven code generation are reshaping software development, yet a comprehensive understanding of their impact on project dependencies remains crucial. A large-scale analysis of code contributions reveals that these agents consistently introduce new external libraries into projects, increasing the total number of dependencies by approximately 1.3% with each pull request. This seemingly small percentage aggregates over time, potentially leading to substantial increases in project complexity and maintenance overhead. Investigating how these AI agents select and integrate dependencies-whether prioritizing widely-used libraries or introducing novel, less-tested ones-is essential for assessing the long-term stability and security implications of increasingly automated coding practices. Such an analysis will not only illuminate current trends but also inform strategies for managing dependencies effectively in the age of AI-assisted development.

Observing Agent Behavior Through the Lens of Pull Requests

The AIDev Dataset distinguishes itself by offering direct observation of code modifications authored by AI agents through the analysis of Pull Requests. Unlike datasets focused solely on completed code, this collection captures the iterative process of AI development, providing insights into how changes are made, not just what the final code looks like. Each Pull Request represents a discrete unit of work, detailing specific code additions, deletions, and modifications enacted by the AI. This granular level of access allows researchers to examine the agent’s reasoning and behavior during code development, enabling analysis of factors such as dependency management, bug fixing, and feature implementation as they occur within a realistic software development workflow. The dataset comprises a substantial number of these AI-authored Pull Requests, creating a statistically significant corpus for behavioral analysis.

Analysis of Pull Requests within the AIDev Dataset demonstrates distinct patterns in AI agent library incorporation. Agents not only utilize pre-existing libraries – a total of 3,988 unique external libraries were imported across all changes – but also actively introduce new dependencies into projects, adding 1,120 unique libraries as new dependencies. This behavior indicates an ability to identify missing functionality and independently select and integrate appropriate external resources, going beyond simple code completion or modification of existing codebases. The data suggests a proactive approach to dependency management, rather than solely relying on pre-configured project structures.

The methodology for identifying library usage within the AIDev dataset relies on parsing the file differences contained within each Pull Request. Specifically, regular expressions are applied to the diffs to detect import statements and dependency declarations commonly used in software development. This process isolates lines of code that introduce or utilize external libraries, allowing for automated extraction of dependency information. The regex patterns are designed to accommodate various import syntaxes across different programming languages represented in the dataset, ensuring accurate identification of both direct and transitive dependencies introduced by the AI agents.

Analysis of the AIDev Dataset’s pull requests demonstrates that AI agents actively integrate external code into their projects through the import of 3,988 unique libraries. Beyond utilizing pre-existing resources, these agents also introduce 1,120 unique libraries as new dependencies within the analyzed projects. This indicates a capacity not merely to leverage established codebases, but to expand the project’s dependency landscape by incorporating previously unused external components, suggesting a level of autonomous project configuration.

Dependency Manifests: A Blueprint for External Code

Dependency manifests, such as package.json for Node.js projects or requirements.txt for Python, function as a complete listing of external code packages required for a software project to build and run. These manifests detail each dependency, including its name and a specified version or version range. By explicitly declaring these dependencies, the manifest ensures reproducibility and facilitates consistent behavior across different development environments. Furthermore, it enables automated dependency resolution and management tools to efficiently handle the acquisition and installation of required libraries, streamlining the development workflow and mitigating potential compatibility issues.

AI agents utilize dependency manifests to incorporate external libraries into a project; however, the specification of version constraints during this process is critical for maintaining code stability and security. When introducing a new dependency, the agent must define acceptable version ranges or specific versions to prevent compatibility issues or the introduction of vulnerabilities present in newer, unvetted releases. Without these constraints, the project may inadvertently pull in breaking changes or insecure code, leading to runtime errors or security exploits. Therefore, the agent’s ability to reliably and accurately define these version constraints is a key factor in the overall robustness of the resulting software.

Recent research indicates a strong correlation between the prompting of Large Language Models (LLMs) and the inclusion of version constraints when introducing new dependencies. Specifically, 75.0% of newly added dependencies included version specifications as a result of targeted prompting. This demonstrates that agent instruction can directly influence dependency security practices, ensuring that projects utilize predictable and tested library versions. The ability to consistently specify version constraints is a marked improvement in automated dependency management, enhancing code reliability and reducing the risk of introducing vulnerabilities through outdated or incompatible libraries.

Recent research indicates a substantial increase in the specification of version constraints when introducing new dependencies via Large Language Models (LLMs). Current findings show that approximately 75.0% of newly added dependencies include version constraints, a marked improvement over previous LLM-based dependency introduction studies which reported a rate of roughly ≈9%. This significant increase suggests a more reliable approach to maintaining dependency security and overall code reliability, mitigating potential issues arising from unexpected updates or incompatible library versions.

The Shifting Landscape of Software Development Practices

The increasing complexity of modern artificial intelligence systems is inextricably linked to a growing reliance on external dependencies, demanding a heightened focus on robust dependency management practices. Analyses of AI-driven software development reveal a pronounced pattern of incorporating numerous external libraries – often exceeding those found in traditional projects – to facilitate machine learning tasks and integrate pre-trained models. This expanded dependency graph introduces significant challenges, including potential version conflicts, security vulnerabilities, and unpredictable behavior stemming from updates to third-party code. Consequently, developers must prioritize tools and strategies for meticulously tracking, validating, and isolating these dependencies, ensuring the stability and reliability of AI-powered applications. Effective dependency management is no longer merely a best practice, but a critical necessity for successful AI-driven software engineering.

The integration of AI agents into software development introduces a unique dynamic concerning external library usage, differing substantially from conventional human coding practices. Unlike developers who consciously select and integrate libraries based on explicit requirements and thorough understanding, AI agents often exhibit a more exploratory approach, potentially incorporating dependencies with varying degrees of necessity or compatibility. This can lead to a proliferation of dependencies, increased complexity, and unforeseen conflicts. Consequently, established best practices centered around meticulous dependency management, rigorous testing, and comprehensive documentation require substantial re-evaluation. A shift towards automated dependency analysis, proactive vulnerability scanning, and AI-assisted code review becomes crucial to mitigate risks associated with this evolving development paradigm, ensuring the stability and security of AI-driven software systems.

The increasing integration of AI-driven code generation necessitates a heightened emphasis on robust testing methodologies. As AI agents introduce external dependencies and complex logic, the potential for unforeseen errors and vulnerabilities expands significantly. Consequently, the utilization of comprehensive test frameworks becomes paramount, moving beyond simple unit tests to encompass integration and system-level validation. These frameworks must be capable of not only verifying functional correctness but also assessing the reliability and security of AI-introduced components, ensuring that the overall software system remains stable and predictable despite the dynamic nature of AI-generated code. The automation afforded by these frameworks is no longer a convenience, but a critical safeguard against the inherent uncertainties of AI-assisted development.

The increasing prevalence of external library usage in AI-driven software development signals a notable shift away from reliance on built-in, standard libraries. This isn’t merely a matter of convenience; it reflects a changing approach to problem-solving where developers increasingly leverage pre-built, specialized modules to rapidly integrate complex functionalities – particularly those related to machine learning and data science. This paradigm favors composition over creation, prioritizing the efficient assembly of existing components rather than bespoke implementation. Consequently, software projects are becoming more dependent on a broader ecosystem of external code, demanding a heightened awareness of supply chain security, licensing, and long-term maintenance – a substantial departure from the more self-contained projects of the past.

The study of AI coding agents and their library usage reveals a fascinating pattern of stability. These agents, while prolific importers of existing libraries, demonstrate a marked reluctance to introduce novel dependencies. This behavior speaks to a systemic preference for known quantities, a conservative approach to expansion that prioritizes predictable function over exploration. As Donald Knuth observed, “Premature optimization is the root of all evil,” and it seems these agents, in their own way, avoid the ‘evil’ of introducing instability through untested additions. This aligns with the core idea that these systems age not because of errors, but because time is inevitable – the agents utilize what is rather than risk what could be, demonstrating a kind of calculated stagnation.

What Lies Ahead?

The study reveals a curious pattern: these agent-authored pull requests are, at present, largely conservative acts. Frequent library imports suggest fluency in existing tools, yet the scarcity of new dependencies implies a reluctance to extend the ecosystem-or perhaps an inability to reliably assess the value of doing so. Every commit is a record in the annals, and every version a chapter, yet this one feels more like meticulous curation than bold expansion. The system ages gracefully, for now.

A critical unresolved question concerns the longevity of this approach. Delaying fixes is a tax on ambition; can these agents maintain a functional codebase indefinitely by simply rearranging existing components? Future work must investigate the correlation between dependency novelty and code quality, and explore how agents navigate the inevitable friction between stability and innovation. The current findings offer a snapshot-a baseline-against which to measure the evolution of these systems as they encounter more complex problems.

Ultimately, the true test will not be the agents’ ability to use libraries, but their capacity to become libraries – to generate reusable components that enrich the software landscape. Time is not a metric; it’s the medium in which systems exist, and the slow accumulation of dependencies-or the absence thereof-will tell a definitive story of adaptation, or entropy.

Original article: https://arxiv.org/pdf/2512.11589.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2025-12-16 02:17