Can AI Really Help in the Lab?

Author: Denis Avetisyan

A new study investigates whether artificial intelligence can effectively assist novices performing complex biological experiments.

Research reveals modest gains in cell culture, but limited overall improvement in novice performance on complex laboratory procedures, raising questions about the reliability of in silico benchmarks for biosecurity assessments.

Despite strong performance of large language models (LLMs) on biological benchmarks, their translation to improved human performance in real-world laboratory settings remains unclear. This study, ‘Measuring Mid-2025 LLM-Assistance on Novice Performance in Biology’, assessed the impact of LLM assistance on novice actors completing a viral reverse genetics workflow via a randomized controlled trial. Results indicated that while mid-2025 LLMs did not substantially increase overall workflow completion, they were associated with modest performance benefits in specific tasks like cell culture. Does this gap between in silico capabilities and practical utility necessitate revised approaches to biosecurity assessments as AI and user expertise continue to evolve?

The Tacit Knowledge Bottleneck in Biological Research

Reverse genetics workflows, and many other complex biological tasks, aren’t simply a matter of following a protocol; they require a substantial degree of tacit knowledge – the kind of procedural skill developed through experience that isn’t easily written down. Researchers must intuitively understand nuances in cell behavior, anticipate potential failures, and adapt techniques based on subtle observations. For instance, successful cell lysis often depends on feeling the appropriate resistance when pipetting, or recognizing the correct viscosity of a solution – details absent from standard operating procedures. This expertise extends beyond technical proficiency, encompassing an understanding of potential experimental artifacts and the ability to troubleshoot unforeseen problems, ultimately impacting both the efficiency and reliability of biological research.

The execution of intricate biological procedures, such as those involved in reverse genetics, frequently presents substantial difficulties for researchers new to the field. These challenges aren’t simply about understanding the theoretical underpinnings; rather, they stem from a lack of deeply ingrained practical skills and an inability to anticipate subtle variations or potential pitfalls inherent in each step. Consequently, experimental timelines can be significantly extended as novices grapple with technique, troubleshoot unexpected results, and require increased oversight. More critically, this struggle directly impacts reproducibility; inconsistent execution due to insufficient expertise introduces variability that can obscure genuine biological effects, leading to unreliable data and hindering the progress of scientific inquiry. The reliance on tacit knowledge – the ‘how’ rather than the ‘what’ – creates a bottleneck, slowing down research and potentially compromising the validity of findings.

The subtle, unwritten rules governing successful biological experimentation – often termed ‘tacit knowledge’ – create a significant bottleneck in modern life science research. While protocols can be meticulously documented, the nuanced understanding required to troubleshoot experiments, interpret ambiguous results, and adapt procedures to specific biological contexts remains difficult to codify and transfer. This gap directly impedes the development of truly automated workflows, as machines currently lack the flexibility to navigate the unpredictable nature of biological systems without human intervention. Furthermore, it hinders effective knowledge transfer between researchers and across generations, potentially leading to duplicated effort, reduced reproducibility, and a slower pace of discovery. Closing this knowledge gap is therefore crucial not only for accelerating automation, but also for ensuring the long-term robustness and efficiency of biological research itself.

Evaluating LLM Assistance in Novice Reverse Genetics Workflows

A randomized controlled trial was conducted to quantify the effect of Large Language Model (LLM) assistance on the performance of individuals new to reverse genetics workflows. Participants were randomly assigned to either an experimental group receiving LLM support or a control group completing tasks independently. The reverse genetics workflow included standard biological procedures such as cell culture, molecular cloning, and viral vector production. Randomization ensured groups were comparable at baseline, minimizing bias in evaluating the LLM’s impact on task completion and efficiency. Data collected from both groups were subjected to statistical analysis to determine if observed differences in performance were attributable to the LLM intervention.

Participants in the study performed a series of standard reverse genetics procedures commonly utilized in biological research. These procedures included mammalian cell culture, requiring aseptic technique and maintenance of cell lines; molecular cloning, encompassing DNA manipulation, restriction enzyme digestion, ligation, and transformation; and viral production, specifically the generation of recombinant viral vectors through transfection and amplification in cell culture. Successful completion of each procedure was determined by predefined criteria relating to technique, yield, and quality control metrics, allowing for quantitative assessment of participant performance.

Statistical analysis of trial data utilized Bayesian modeling to determine the probability of task completion given LLM assistance versus the control group. This approach allowed for the quantification of uncertainty surrounding observed completion rates and facilitated the identification of specific experimental bottlenecks hindering novice performance. Bayesian methods were selected due to their capacity to incorporate prior knowledge and provide probabilistic estimates of performance differences, offering a more nuanced evaluation than frequentist approaches. The resulting posterior distributions enabled precise comparisons of task completion probabilities and the ranking of procedures based on their susceptibility to error, ultimately informing targeted improvements to the reverse genetics workflow.

A Nuanced Look at Performance Insights from the Trial

Analysis of task completion rates in the reverse genetics workflow demonstrated no statistically significant improvement with the implementation of Large Language Model (LLM) assistance. The LLM-assisted group achieved a 5.2% completion rate, compared to 6.6% for the group utilizing internet search; this difference was not statistically significant, as indicated by a p-value of 0.759. These findings suggest that, across the entire workflow, LLM assistance did not demonstrably alter the probability of successful task completion when contrasted with standard internet-based research methods.

Post-hoc analysis of the reverse genetics workflow revealed a statistically non-significant, but notable, improvement in cell culture success rates when utilizing LLM assistance. Specifically, the LLM-assisted group achieved a success rate of 68.8%, compared to 55.3% in the group utilizing internet searches (p = 0.059). This suggests that LLMs may provide a modest benefit for tasks heavily reliant on procedural knowledge, as opposed to broader research or problem-solving required in other stages of the workflow. Further investigation is warranted to determine the specific mechanisms driving this observed difference and to identify the types of cell culture procedures where LLM assistance is most effective.

The pooled analysis yielded a risk ratio of 1.42, indicating that researchers utilizing LLM assistance experienced a 42% higher chance of success compared to those relying on standard internet searches. However, the 95% Credible Interval (0.74-2.62) includes 1, meaning this observed effect is not statistically significant at the conventional alpha = 0.05 level. This interval suggests that the true risk ratio likely falls between 0.74 and 2.62, and therefore, the data do not provide conclusive evidence of a statistically significant benefit from LLM assistance, despite the observed trend.

The Limitations of Current LLMs and Their Impact on Scientific Discovery

Current large language models, despite demonstrating proficiency in protocol recall, exhibit a notable struggle with the complexities inherent in genuine scientific inquiry. Analyses reveal these models falter when confronted with the iterative nature of experimentation – specifically, in designing robust studies that account for potential failure points, identifying the root causes of unexpected outcomes, and adapting procedures in real-time based on observed data. This limitation isn’t simply a matter of insufficient training data; rather, it points to a fundamental difficulty in replicating the nuanced reasoning and contextual awareness that characterize expert scientists. The models often proceed logically from established protocols but lack the capacity to critically evaluate the validity of those protocols in situ, or to generate novel solutions when faced with anomalies-a critical component of successful scientific discovery.

Simply following a prescribed laboratory protocol, while a necessary component of scientific work, represents a limited form of expertise. True scientific skill lies not in rote execution, but in the capacity to navigate unexpected outcomes and adjust methodologies accordingly. A researcher’s ability to critically analyze discrepancies, troubleshoot malfunctioning equipment, or recognize the implications of anomalous data demands a level of reasoning that extends beyond strict adherence to instructions. It is this adaptive problem-solving, this capacity for contextual understanding and inventive modification, that ultimately distinguishes a proficient scientist from a mere technician, and represents a significant hurdle in the development of truly intelligent laboratory automation.

Future advancements in large language models for scientific application necessitate a shift beyond simple protocol execution towards systems capable of genuine biological reasoning. Current models often lack the ability to synthesize existing knowledge – the vast web of established biological principles and previously observed phenomena – with the specifics of a given experimental context. Consequently, development should prioritize architectures that can integrate external databases, interpret experimental goals, and offer assistance tailored not just to the procedure, but to the underlying scientific question. This targeted support could include suggesting alternative approaches when faced with unexpected results, identifying potential confounding factors, and even proposing novel experiments based on a comprehensive understanding of the biological system under investigation, ultimately moving beyond task completion towards collaborative scientific discovery.

Navigating the Biosecurity Implications of LLMs and Responsible Innovation

Large language models represent a double-edged sword for the life sciences, simultaneously offering unprecedented access to complex biological information and introducing novel biosecurity challenges. The very capabilities that allow these models to accelerate research – synthesizing data, predicting protein structures, and even designing novel genetic sequences – also create opportunities for misuse. Individuals with malicious intent could leverage LLMs to design harmful pathogens, circumvent safety protocols, or disseminate dangerous biological knowledge. Moreover, even without deliberate malice, the potential for accidental harm exists through the generation of inaccurate or misleading information, or the design of experiments with unintended consequences. This democratization of biological knowledge necessitates a proactive approach to risk mitigation, ensuring that the benefits of LLMs are realized without compromising global health security.

Mitigating the biosecurity risks presented by large language models necessitates a proactive and multifaceted approach to their development. Beyond simply preventing the generation of harmful biological sequences, responsible innovation demands robust safety protocols embedded throughout the LLM lifecycle – from data curation and model training to deployment and ongoing monitoring. Access controls are equally crucial; carefully managed permissions can limit the potential for malicious actors to exploit these powerful tools for nefarious purposes, while still enabling legitimate scientific inquiry. This includes establishing clear guidelines for acceptable use, implementing safeguards against prompt injection attacks designed to bypass safety filters, and fostering collaboration between AI developers, biosecurity experts, and policymakers to ensure these technologies benefit, rather than threaten, global health security.

Ongoing investigation centers on developing techniques to harmonize large language model outputs with existing biosecurity protocols, a crucial step in fostering responsible advancement within the life sciences. This involves not only refining algorithms to identify and flag potentially harmful information – such as instructions for synthesizing dangerous pathogens – but also proactively embedding ethical considerations into the very core of LLM design. Researchers are exploring methods like reinforcement learning from human feedback, specifically tailored to biosecurity concerns, and the creation of ‘guardrails’ that constrain LLM responses within safe and ethical boundaries. The ultimate goal is to move beyond simply detecting misuse and toward building systems that actively promote innovation aligned with established safety standards, thereby unlocking the potential of LLMs to accelerate beneficial discoveries while minimizing existential risks.

The study highlights a crucial point about systems – altering one component doesn’t guarantee improvement of the whole. While large language models demonstrated some benefit in isolated tasks, like cell culture, their overall impact on novice performance in complex biological procedures remained statistically insignificant. This echoes Blaise Pascal’s observation: “The eloquence of the tongue consists not in the words, but in the thought.” The models, proficient in ‘words’-the information they process-fell short in translating that to improved ‘thought’-successful execution of intricate laboratory work. It suggests that benchmarks focused on in silico performance may not accurately reflect real-world utility, particularly when evaluating biosecurity risks associated with synthetic biology.

Future Directions

The observed disconnect between in silico performance and practical laboratory outcomes suggests a fundamental miscalibration in how these systems are currently evaluated. Benchmarking, focused as it often is on isolated knowledge recall, fails to capture the embodied cognition required for even seemingly straightforward biological manipulation. The modest gains in cell culture-a task demanding tactile understanding and immediate feedback-hint at where future work might productively focus: not on increasing knowledge access, but on mediating the interaction between the model and the physical world.

A truly robust assessment of large language model assistance in this domain requires shifting the frame. The question is not whether the model ‘knows’ reverse genetics, but whether it can facilitate a novice’s ability to learn reverse genetics through iterative practice and error correction. This necessitates a move toward dynamic, embodied interfaces-systems that can interpret experimental failures, suggest refinements, and, crucially, acknowledge the limits of their own understanding.

The field must also grapple with the implications of this apparent asymmetry. If these models excel at simulating competence but falter when faced with genuine complexity, the biosecurity risks are not diminished, but subtly altered. The danger lies not in a sudden, catastrophic failure, but in a gradual erosion of critical thinking, as novices come to rely on plausible-sounding outputs rather than developing a nuanced understanding of the underlying biology.

Original article: https://arxiv.org/pdf/2602.16703.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/