Searching for Answers: A New Tool for FDA-Authorized AI in Medicine

Author: Denis Avetisyan

A new website leverages the power of semantic search to help clinicians and researchers quickly find and evaluate AI-enabled medical devices cleared by the FDA.

AI device intelligence is unlocked through a three-stage process: data is extracted from FDA databases, textual features are generated via Gemini-2.5-flash prompting, and semantic search capabilities are then enabled by embedding these features using MedEmbed.

This paper details the development of FDA AI Search, a platform using large language models and embedding techniques to improve information retrieval for authorized AI medical devices.

Despite the increasing prevalence of AI-enabled medical devices-over 1,200 of which have received FDA authorization-identifying devices suitable for specific clinical applications remains a significant challenge due to limitations in existing search capabilities. This paper introduces ‘FDA AI Search: Making FDA-Authorized AI Devices Searchable’, a novel website leveraging semantic search and large language models to enable efficient querying of these authorized devices. Our approach utilizes embedding-based retrieval, comparing user queries to features extracted from FDA authorization summaries to identify relevant matches, demonstrably outperforming keyword-based methods. As the landscape of AI in healthcare continues to expand, will tools like FDA AI Search become essential for both clinicians seeking appropriate technologies and developers innovating new applications?

Deconstructing the Device Discovery Dilemma

The proliferation of Food and Drug Administration-authorized artificial intelligence devices is rapidly reshaping healthcare, yet simultaneously creating a significant challenge for clinicians attempting to identify the most appropriate tools for specific patient needs. While the increasing availability of these technologies promises enhanced diagnostics, treatment planning, and patient monitoring, the sheer volume of options – spanning radiology, cardiology, dermatology, and beyond – demands more sophisticated methods of discovery than traditional approaches allow. Clinicians face the complex task of sifting through a growing landscape of devices, each with unique capabilities and limitations, to pinpoint those that genuinely address particular clinical scenarios and integrate seamlessly into existing workflows. This necessitates a shift from simply searching for AI devices to actively discerning which devices best solve specific medical problems.

The proliferation of AI-enabled medical devices, while promising, is increasingly challenging clinicians attempting to identify tools suited to specific needs. Traditional search methodologies, reliant on keyword matching, frequently prove inadequate when applied to these complex technologies. A device capable of detecting subtle anomalies in retinal scans, for example, might not be discovered by searching for “retinal scan analysis” if its underlying AI utilizes a novel, non-standard descriptor for that function. This disconnect arises because keywords fail to capture the semantic meaning of a device’s capabilities-the precise clinical task it performs and how it performs it. Consequently, valuable tools remain hidden behind a wall of technical jargon and imprecise labeling, hindering effective integration into healthcare workflows and potentially delaying access to beneficial technologies.

Current search methodologies often struggle with the intricacies of artificial intelligence-enabled medical devices, highlighting a critical need to move beyond simple keyword matching. These devices aren’t defined by what they are called, but by what they do – the specific clinical problems they address and how they improve patient outcomes. A search for “cardiac monitoring” yields numerous results, but fails to distinguish a device that passively records heart rhythms from one that actively predicts and prevents arrhythmias. Consequently, clinicians require search tools that can interpret the semantic meaning of device capabilities – understanding, for example, that a device “detects early sepsis” accomplishes more than simply “monitoring vital signs.” This shift toward meaning-based search is essential for effectively navigating the rapidly expanding landscape of AI in healthcare and ensuring clinicians can quickly identify the tools best suited to their specific needs.

Semantic search effectively identifies relevant devices even when a keyword search, such as for “Genitourinary”, yields no results.

Unlocking Device Intelligence: A Semantic Approach

Traditional keyword-based searches for medical devices rely on exact matches between a user’s query and the terms used in device descriptions, often resulting in incomplete or irrelevant results. FDA AI Search employs semantic search, a technique that prioritizes the meaning of the query and descriptions rather than literal keyword occurrences. This is accomplished by analyzing the contextual relationships between words to understand the user’s intent and the device’s functionality. Consequently, the system can identify devices that are conceptually related to the search, even if they do not share identical keywords, improving the precision and recall of device discovery.

Text embeddings are a core component of the FDA AI Search system, functioning as numerical representations of both medical device descriptions and user queries. These representations, or vectors, exist within a high-dimensional space – typically hundreds or thousands of dimensions – where the spatial relationship between vectors reflects the semantic similarity of the corresponding text. Each dimension captures a latent feature derived from the text’s meaning, allowing the system to quantify conceptual relatedness. A device and a query with similar meanings will be represented by vectors that are close to each other in this space, even if they do not share identical keywords. This allows for the identification of relevant devices based on their underlying concepts rather than strict textual matches.

The FDA AI Search system employs MedEmbed, a natural language processing (NLP) embedding model developed and trained specifically on a large corpus of medical text. This focused training differentiates MedEmbed from general-purpose language models and allows it to generate more accurate and clinically relevant vector representations of medical devices and user queries. By leveraging a model pre-trained on medical terminology, concepts, and relationships, the system improves its ability to understand the nuanced meaning of device descriptions and search terms, leading to more precise and meaningful search results. The use of MedEmbed directly addresses the challenges of polysemy and synonymy common in the medical field, ensuring that conceptually similar devices are identified even if they do not share identical keywords.

The FDA AI Search system determines conceptual matches between clinician queries and medical devices by calculating the similarity between their respective vector representations. These vectors, generated using the MedEmbed model, position devices and queries in a multi-dimensional space where proximity indicates semantic relatedness. The system employs cosine similarity-a measure of the angle between two vectors-to quantify this relatedness; a smaller angle, and thus a higher cosine similarity score, suggests a stronger conceptual connection. Devices are ranked based on these similarity scores, ensuring that results are not limited to those containing specific keywords but instead reflect a deeper understanding of the clinician’s intent.

The AI system incorrectly identified mammography devices generally, rather than limiting results to those compatible with GE machines, demonstrating a failure to adhere to specified compatibility criteria.

Deconstructing Documentation: From PDF to Device Representation

Feature Extraction utilizes optical character recognition (OCR) and natural language processing (NLP) techniques to analyze Food and Drug Administration (FDA) authorization summary documents in PDF format. This process systematically identifies and isolates specific device characteristics, including intended use, materials, operating parameters, and safety features. The extracted data is then structured and categorized to create a standardized representation of each device’s attributes, enabling consistent comparison and retrieval. This automated extraction minimizes manual review and ensures comprehensive coverage of the information contained within the regulatory documents.

Feature Extraction leverages Gemini-2.5-flash, a large language model developed by Google, to process unstructured text data contained within FDA authorization summary PDFs. Gemini-2.5-flash is specifically chosen for its capacity to interpret complex medical terminology and nuanced language common in regulatory documents. The model’s architecture enables it to effectively summarize lengthy texts, identify key information pertaining to device characteristics, and discern relationships between different features described within the source documents. This capability is crucial for accurately translating the PDF content into structured data suitable for subsequent analysis and search functionality.

Query Match Features are discrete, textual representations of characteristics a clinician would likely use when searching for medical devices. These features are derived from the FDA authorization summary PDFs through the Feature Extraction process and are designed to mimic realistic search queries. For example, a device described as “minimally invasive, for use in the femoral artery, with a 7 French catheter” might yield Query Match Features such as “minimally invasive femoral artery access”, “7 French catheter”, and “vascular access”. This approach moves beyond simple keyword matching, allowing the Semantic Search to identify devices based on conceptual similarity to a clinician’s expressed needs, even if the exact wording differs from the source documents.

Query Match Features, derived from FDA authorization summary analysis, are transformed into Text Embeddings using a vectorization process. These embeddings are numerical representations of the semantic meaning of each feature, enabling the Semantic Search functionality. Each feature is mapped to a high-dimensional vector space where features with similar meanings are located closer to each other. This allows the search engine to identify relevant devices not simply by keyword matching, but by understanding the intent behind a clinician’s query, even if the exact terms don’t appear in the device feature descriptions. The resulting vector database facilitates efficient similarity comparisons, allowing for rapid retrieval of devices based on semantic relevance.

The user interface allows keyword searches to efficiently locate desired content.

Harmonizing Approaches: A Hybrid Search Architecture

FDA AI Search utilizes a Hybrid Search approach to optimize both recall and precision in information retrieval. This method combines the capabilities of Semantic Search, which focuses on the meaning of queries, with traditional keyword-based methods. By integrating these two distinct approaches, the system aims to overcome the limitations of each individual technique; Semantic Search can sometimes struggle with nuanced or specific terminology, while keyword searches may miss conceptually relevant results. The hybrid architecture allows the system to leverage the strengths of both, identifying both exact matches and conceptually similar documents, ultimately providing a more comprehensive and accurate search experience.

The FDA AI Search system utilizes a hybrid approach to information retrieval by combining embedding similarity with BM25. Embedding similarity leverages vector representations of queries and documents to identify conceptually similar content, while BM25 is a statistically-based ranking function that assesses relevance based on keyword frequency and inverse document frequency. Integrating these two methods allows the system to benefit from both semantic understanding and precise keyword matching. BM25 provides a strong baseline for recall, while embedding similarity enhances the ability to identify relevant documents that may not share exact keywords but possess similar meaning, resulting in a more comprehensive and accurate search.

To optimize the balance between embedding similarity and BM25 ranking, the FDA AI Search system employs Bayesian Optimization. This probabilistic model-based method iteratively refines the weighting assigned to each component by evaluating performance across a defined parameter space. The process involves constructing a surrogate model to predict performance based on given weights, then selecting the next set of weights to evaluate using an acquisition function that balances exploration and exploitation. This allows the system to automatically determine the optimal weighting for the specific dataset, maximizing search performance metrics such as Hit Rate@K without manual tuning.

System performance was evaluated using Hit Rate@K and Inference Time metrics across a dataset of 22,552 queries. Results indicate a high degree of accuracy, with the tool achieving a Hit Rate@K=3 of over 95%. Efficiency is also demonstrated by a Mean Inference Time of 0.38 seconds, with a Standard Deviation of 0.11 seconds, indicating consistent response times across the evaluated query set.

A search for the string “zzzzzzzz” returns irrelevant results, highlighting the limitations of semantic search when confronted with meaningless queries.

Empowering Clinical Practice: The Future of Device Discovery

The Food and Drug Administration’s new AI Search tool represents a considerable leap forward in how clinicians locate appropriate, AI-enabled medical devices. Traditional search methods often rely on keyword matching, which can be imprecise and return irrelevant results, requiring significant time to sift through options. This novel system, however, utilizes natural language processing to understand the intended function of a device, going beyond simple keyword recognition. By analyzing the device’s capabilities – what it does rather than just what it is called – the AI Search delivers more accurate and pertinent results. This capability drastically reduces the time needed to identify suitable tools, allowing healthcare professionals to focus on patient care and potentially accelerating the adoption of innovative technologies within the clinical setting.

The challenge of locating appropriate AI-enabled medical devices often extends beyond simply matching keywords; clinicians require tools that grasp the functional meaning of a device’s capabilities. Traditional search methods frequently return lengthy lists requiring manual review, whereas this new approach prioritizes understanding what a device does, not just what it’s called. By analyzing device descriptions through the lens of clinical intent, the system can swiftly pinpoint tools tailored to specific patient needs or diagnostic challenges. This semantic understanding allows clinicians to move beyond broad searches and quickly identify devices offering precise functionalities, ultimately streamlining workflows and fostering more informed treatment decisions.

The enhanced accessibility of AI-enabled medical devices, facilitated by tools like the FDA AI Search, promises a cascade of benefits extending beyond immediate clinical application. By swiftly connecting clinicians with the most appropriate technologies, the system effectively lowers the barrier to adoption for innovative tools, fostering a more rapid cycle of experimentation and refinement. This acceleration isn’t merely about speed; it’s about expanding the scope of possible interventions and tailoring treatments with greater precision. Consequently, patients stand to gain from earlier diagnoses, more effective therapies, and ultimately, improved health outcomes as the entire medical field leverages the power of artificial intelligence with increased efficiency and confidence.

Ongoing development prioritizes a continually evolving system, with efforts centered on significantly broadening the scope of the AI’s medical device knowledge base. This expansion isn’t simply about adding more data; it involves refining the AI’s ability to understand nuanced device capabilities and clinical applications. Crucially, the system is being designed to actively learn from user interactions; clinician feedback on search results and device relevance will be directly incorporated to improve the accuracy and utility of future searches. This iterative process of expansion and refinement aims to create a self-improving tool that adapts to the ever-changing landscape of medical technology and consistently delivers increasingly precise and helpful device discovery.

The pursuit of accessible information regarding FDA-authorized AI devices, as detailed in this work, mirrors a fundamental tenet of understanding any complex system: dismantling established methods to reveal underlying structures. Andrey Kolmogorov observed, “The most important thing in science is not to be afraid of making mistakes.” This resonates deeply with the approach taken here; traditional metadata-based search proved inadequate, necessitating the exploration of semantic search and LLMs. The team didn’t simply refine existing methods; they challenged the core assumption of how information should be retrieved, embracing the potential for error as a pathway to a more effective solution. By reverse-engineering the limitations of current systems, they’ve built a tool to navigate the burgeoning landscape of AI-enabled medical devices.

Beyond the Search: A Landscape of Questions

The construction of FDA AI Search is, at its core, a focused demolition of prior search limitations. Yet, the very act of building a structured query system highlights the inherent messiness of categorization itself. The semantic space, while seemingly more fluid than rigid metadata, still demands boundaries-artificial constraints imposed upon the continuous spectrum of medical innovation. The true test won’t be whether the system finds devices, but what unanticipated connections emerge when one can ask questions previously unformulable.

Future iterations must grapple with the ephemeral nature of ‘authorization’. Algorithms evolve, datasets shift, and regulatory landscapes are rarely static. A searchable archive is only valuable if it reflects a living, breathing reality – a constant recalibration of what ‘authorized’ truly means. The system’s architecture should anticipate, even invite, controlled failures – a means of exposing the edges of its own knowledge and identifying blind spots in the regulatory framework.

One wonders if the ultimate search engine isn’t for devices at all, but for the unarticulated problems they might solve. To move beyond retrieval to genuine discovery, the system must learn to infer need, to anticipate questions before they are asked. This is, of course, a path fraught with the risk of imposing solutions where none are desired – a testament to the fact that even the most elegant tools are, at best, imperfect reflections of a chaotic world.

Original article: https://arxiv.org/pdf/2602.00006.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/