CFR Parts 50, 56, 812, and 814.

The FDA defines software intended for medical purposes as SaMD under the Digital Health Innovation Action Plan. SaMD requires that manufacturers follow rigorous pathways for premarket and post-market oversight. For AI ML models, which often adapt based on real-world data, understanding how to manage changes systematically becomes increasingly crucial due to their inherent complexity.

Importantly, the FDA’s guidance document on **Clinical/Medical Device Software** outlines expectations regarding software validation, including how changes to software—including algorithm modifications—are to be controlled and reported. Moreover, the premise of a **predetermined change plan** allows manufacturers to define specific allowable changes to AI algorithms, thereby easing the regulatory burden while ensuring patient safety.

Key Regulations

• 21 CFR Part 50: Protection of Human Subjects – Ensures that all clinical investigations including AI-driven SaMD adhere to ethical standards.
• 21 CFR Part 56: Institutional Review Boards – Informs requirements for IRB oversight of clinical investigations.
• 21 CFR Part 812: Investigational Device Exemptions – Outlines the requirements for clinical trials of devices, applicable for those developing AI solutions.
• 21 CFR Part 814: Premarket Approval – Governs the approval process for high-risk medical devices.

Understanding these frameworks will help clarify the responsibilities of digital health entities in managing changes throughout the lifecycle of AI algorithms.

Configuring a Robust Software Configuration Management System

Effective software configuration management (SCM) serves as a foundational element in ensuring the integrity of AI ML SaMD. SCM encompasses the activities of identifying, organizing, and controlling changes to the software components throughout its lifecycle.

Step 1: Establish SCM Processes

Begin by developing systematic processes that define how software changes, including AI algorithms, will be managed. This should include, but is not limited to:

• Version control mechanisms to track modifications.
• Documented procedures for change requests.
• Audit trails detailing the decision-making processes behind changes.
• Backup and recovery procedures to ensure data integrity and availability.

Step 2: Define Roles and Responsibilities

Clearly outline the roles and responsibilities of teams involved in the lifecycle management of AI algorithms. Typically, these would include:

• Data scientists who design and develop algorithms.
• Regulatory affairs professionals who ensure compliance with FDA guidelines.
• Quality assurance teams who validate and verify the algorithms.

Step 3: Implement Technical Controls

Utilize technology solutions to aid in SCM. This may involve employing specialized software tools that provide SCM capabilities tailored for software deployment and modifications. Ensure technical controls include:

• Real-time monitoring of software performance.
• Automated rollback functionalities for unnecessary changes.
• Integrated risk management tools to assess the impact of algorithm changes.

These controls will enable organizations to manage the complexities associated with adaptive algorithms and minimize the risk of model drift.

Change Control Mechanisms for AI Algorithms

Change control mechanisms are critical to ensure that any adjustments to AI algorithms are documented, assessed, and managed in accordance with regulatory obligations. The FDA’s guidance on AI ML SaMD emphasizes the importance of maintaining control over algorithm modifications.

Step 4: Implement Formal Change Control Procedures

Create tailored change control procedures for AI algorithms. This should delineate the steps required to initiate, assess, and implement changes. Key elements to incorporate are:

• Change Request Submission: Define how team members can submit changes and what information must be included, such as the rationale and potential impact.
• Impact Assessment: Conduct a thorough analysis to evaluate how the proposed change could affect performance, safety, and efficacy of the SaMD. It is essential to assess whether the change falls within the parameter of a predetermined change plan.
• Approval Process: Establish multi-tiered approval processes where necessary, allowing input from cross-functional teams.
• Documentation and Training: Maintain robust documentation of all changes and ensure staff is trained on any revisions to processes or algorithms.

These steps are fundamental to maintain compliance while fostering an environment of continuous improvement.

Predetermined Change Plans and Locked Models

Predetermined change plans allow SaMD manufacturers to proactively define types of changes that will not require regulatory resubmission for premarket review, provided they remain within predefined boundaries. This is particularly important for software with adaptive algorithms that may experience continuous updates.

Step 5: Develop a Predetermined Change Plan

When developing a predetermined change plan, consider the following guidelines:

• Define Acceptable Changes: Specify what constitutes a minor versus a major change. Minor changes, such as occurrences that do not significantly alter the intended use or safety profile may be more easily integrated.
• Control Mechanisms: Describe the controls that will be maintained, such as continual performance monitoring and audit measures for any changes made.
• Documentation and Reporting: Record all changes executed under this plan and understand the documentation required to demonstrate compliance during post-market monitoring.

Step 6: Implement Locked Models

The concept of locked models is also integral to managing control in AI algorithms. A locked model does not alter its performance parameters post-deployment, which may be necessary for specific use cases where stability is critical. However, when re-evaluating locked models, any updates must respect the regulatory framework governing the design and validation of AI systems.

• Consider establishing criteria and processes for model lock-in points, allowing only approved updates and testing regularly to ensure output consistency.
• Document when a model is locked and delineate the procedure for any necessary un-locking in cases where immediate updates are required.

Post-Market Monitoring for Continuous Compliance

Post-market monitoring serves as the final step in maintaining compliance and safeguarding public health. The FDA requires a robust system to track algorithm performance and collect real-world evidence post-deployment.

Step 7: Establish a Reporting Mechanism

Develop an efficient reporting mechanism to analyze real-world performance data and capture adverse events. Key elements should include:

• Surveillance Protocols: Regularly gather data post-launch that assesses algorithm performance against predefined safety and effectiveness standards.
• Adverse Event Reporting: Report any incidents that arise during the use of your SaMD as per FDA reporting requirements, including necessary documentation.
• Feedback Mechanisms: Foster channels for user feedback to swiftly adapt to any discrepancies identified in algorithm behavior.

Step 8: Continuous Evaluation and Improvement

Lastly, implement a culture of continuous evaluation and improvement. Utilize real-world data and post-market findings to refine and enhance your AI algorithms over time. Regularly revisit your predetermined change plan and ensure that SCM processes evolve alongside technological advancements and regulatory changes to maintain compliance with the FDA, UK, and EU regulations.

This cyclical process ensures not only compliance with regulatory requirements but also bolsters trust with stakeholders and the general public, affirming the commitment to safety and efficacy in healthcare technology innovations.

Conclusion

Aligning software configuration management with the AI model lifecycle is a complex but essential undertaking for organizations involved in the SaMD. By implementing structured change control processes, establishing effective predetermined change plans, and committing to post-market monitoring, organizations can successfully navigate the FDA’s regulatory landscape while furthering innovation and enhancing patient outcomes. The evolving nature of AI technologies necessitates thoughtful and proactive management strategies to ensure compliance and overall success in the digital health arena.