Enhancing Regulatory Communication and Decision-Making with an AI/ML FDA Copilot and SmartSearch+

This article explores how the integration of artificial intelligence (AI) and machine learning (ML) into pharmaceutical regulatory processes can revolutionize the preparation and review of regulatory applications, using Quality Overall Summaries (QOS) as an example. Utilizing a 2009 public case study that exemplifies the quality-by-design (QbD) approach and focusing on the large language module (LLM) aspects, the paper demonstrates the application of an AI/ML-driven FDA Copilot enhanced by SmartSearch+ and the innovative "Dueling Banjo" methodology. This methodology aims to seamlessly combine AI-generated insights with expert human validation to improve communication with regulators and ensure that submissions—such as New Drug Applications (NDAs) and Biologics License Applications (BLAs)—meet evolving regulatory standards. Ultimately, this approach streamlines regulatory analysis and promotes a globally harmonized standard in drug development, emphasizing the critical importance of effective human-AI collaboration to maintain scientific integrity and compliance in regulatory science.

Introduction

Pharmaceutical regulatory affairs are evolving amid increasing complexity, the demand for faster drug reviews, and streamlined communication with regulatory bodies. The Quality Overall Summary (QOS) is central to this transformation, providing a structured overview of quality-related data within NDAs, ANDAs, and BLAs. As the FDA explains, the QOS “puts the pieces of a puzzle together,” thereby enhancing transparency, expediting reviews, and reinforcing quality assurance throughout a drug’s lifecycle.

Building on previous discussions in Pharma Now, this paper examines how an AI/ML-driven FDA Copilot, combined with SmartSearch+, can support regulatory excellence. Using the “Dueling Banjo” approach—which blends AI-generated insights with expert human verification through comprehensive full-text searches of the US FDA.Gov Guidance PDFs and 21 CFRs issued by CDER (drugs), CBER (biologics) & CDRH (medical devices)—the article explores how this synergy can refine QOS preparation and regulatory review processes. The key question is: How can human experts collaborate with an AI/ML-driven system to enhance QOS development and enable efficient, evidence-based decision-making in drug approvals?

Background

In a 2022 report, “Is a Globally Harmonized Quality Overall Summary Possible?” industry experts discussed the challenges and benefits of establishing a standardized QOS format worldwide. A harmonized QOS could streamline reviews, reduce redundancy, and improve clarity by offering a unified narrative that links product attributes to manufacturing processes. Such a standard would also support lifecycle management by defining Established Conditions (ECs) to ensure consistent product quality and simplify post-approval modifications.

Expanding on this, “A Proposal for a Comprehensive Quality Overall Summary” (2023) critiques the limitations of the Common Technical Document (CTD) structure, particularly Module 2.3. This module often redundantly replicates data from Module 3 without delivering a cohesive narrative or regulatory justification for the control strategy. The authors advocated restructuring Module 2.3 to integrate key elements from the Target Product Profile (TPP) and the Quality Target Product Profile (QTPP), providing a straightforward developmental narrative and supporting smoother regulatory transitions.

The “Dueling Banjo” Methodology

The “Dueling Banjo” methodology is a human-led framework that leverages AI-generated insights to build a comprehensive narrative across fragmented data sources and disciplines. Much like two musicians engaging in an improvisational duet, this method interweaves computational analysis with human expert interpretation. AI/ML tools like the FDA Copilot hosted within a Microsoft OpenAI ChatGPT-4 platform, for instance, can generate initial draft responses, forming a solid foundation grounded with SmartSearch+ on credible public knowledge (e.g., the FDA.Gov website) and corporate confidential knowledge for further expert refinement in the context of a patient-centered future such as “Pharma 5.0.” Human oversight remains indispensable in this context, given that AI output may sometimes lack the nuanced reasoning required for regulatory precision, not to mention LLMs' inherent machine learning bias.

SmartSearch+ plays a crucial role by enabling humans to perform full-text searches of FDA regulations and guidance documents. This allows experts to cross-reference and validate AI-generated content, ensuring the final narrative is accurate, complete, and aligned with regulatory expectations. This methodology was implemented in the A-Mab Case Study (2009), designed initially to illustrate QbD principles in monoclonal antibody (mAb) development. Two former regulators (AH and AA) applied Retrieval-Augmented Generation (RAG) to enhance GPT 4.0 capabilities by integrating from external knowledge bases with informed prompting. At the same time, IT expert SmartSearch+ developer RB used SmartSearch+ to uncover additional regulatory insights, recognizing that the US FDA is the single source of regulatory truth.

Example Prompt for AI-Assisted Review:

"Act as an FDA Review Expert evaluating CMC requirements for monoclonal antibodies (mAbs). Integrate key regulatory frameworks (ICH Q8(R2), Q9, Q10, Q11) and relevant FDA guidance with retrievable best practices. Assess the adequacy and completeness of the mAb Development Report."

Observations and Insights

Integrating Retrieval-Augmented Generation (RAG) prompting with SmartSearch+ validation within a single app UI mitigates common AI limitations, such as contextual gaps and hallucinations, ensuring the factual consistency of regulatory responses. In the “A-Mab” Case Study, the initial AI/ML FDA Copilot’s response largely aligned with expert assessments, successfully identifying critical elements such as the evaluation of Critical Quality Attributes (CQAs), a robust Quality-by-Design (QbD) approach, and a risk-based control strategy incorporating design space considerations. However, as the adage goes, “the devil is in the details,”—underscoring the necessity of rigorous expert validation to refine AI-generated insights and maintain regulatory precision. For instance, when prompted to justify acceptance limits in the QOS, the AI-generated details—such as references to “in vitro–in vivo” correlation (IVIVC) studies—were absent in the original case study. It hallucinated! Further prompting led the AI to recognize its error, underscoring the necessity of disciplined human validation through the “Dueling Banjo” approach.

The Importance of a Disciplined “Dueling Banjo” Approach

Combining AI-generated insights with thorough human oversight and preventing “cherry-picking” regulatory principles and guidance via SmartSearch+ is essential for maintaining the integrity of regulatory submissions. This structured approach minimizes the risk of gross “hallucinations” and factual inaccuracies in the details and enhances the overall quality of the review process. By ensuring that every AI-generated insight is cross-referenced with authoritative regulatory sources, the collaborative methodology improves efficiency and upholds the precision required for regulatory compliance.

Outlook: The Future of AI in Regulatory Science

Advancements in chain-of-thought reasoning, context-aware prompting, and enhanced retrieval mechanisms are poised to revolutionize regulatory decision-making further. Experiments with AI models beyond GPT-4.0, including DeepSeek-R1, have shown exponential improvements, setting the stage for more refined applications in regulatory science, which we plan to explore in future Pharma Now contributions.  

Furthermore, integrating large language models (LLMs) with quantitative cause-and-effect relationship pattern recognition through machine learning (ML) is an area of significant interest and activity. This concept aligns with the idea of "computer-aided development," envisioned decades ago to leverage prior knowledge and adopt a product life-cycle approach to development. Other applications include overcoming the blind spots in the assurance of Therapeutic Equivalence, improving predictably of scale-up and reliability of process validation, continued process verification, effective management of out-of-specification incidents, corrective and preventive actions (CAPA), and ongoing improvement efforts. We will also explore these topics and share insights in future editions of Pharma Now.

The true strength of AI lies not merely in its computational power but in the quality of the questions asked and the rigor of the validation process. Expanding the “Question-Based Review” approach—long established in clinical pharmacology and CMC review—to all sections of the CTD could promote a “One Quality Voice” across regulatory communications. With tools like FDA Copilot and SmartSearch+, regulatory professionals can bridge fragmented data, synthesize diverse sources, and maintain a broad strategic overview and detailed precision in decision-making.

Conclusion

Integrating AI/ML-driven tools such as the FDA Copilot with SmartSearch+ presents a transformative opportunity for regulatory communication and decision-making in the pharmaceutical industry. The “Dueling Banjo” methodology combines AI efficiency with human expert validation within one cloud app to enhance regulatory communication, such as QOS preparation, reduce redundancy, and ensure alignment with evolving regulatory standards. As the regulatory landscape continues to evolve, disciplined human-AI collaboration will be essential for upholding scientific integrity, improving efficiency, and fostering patient-centered innovation in drug development.