Our Work: Assessing AI Solutions
Multinational Uses TRCG To Verify Effectivity of New Implementation of AI Post-Market Surveillance Solution
Background
A large multinational medical device manufacturer needed to enhance their post-market surveillance activities with a machine learning solution. Their current system processed a very high volume of incidents due to a large install base, and as a result, required a large team to support the execution of requirements.
This human-centric approach has an inherent potential for variability between case handlers, which could affect compliance. To manage this variation and improve consistency, they aimed to implement a new technology that would reduce case-to-case variation and ensure regulatory compliance—a critical aspect frequently scrutinized by regulatory bodies. To address this, the organization developed a Machine Learning solution to support the activity.
Client Challenge
The client, a company newly venturing into machine learning (ML) solutions, faced significant hurdles due to their limited experience with this advanced technology. Their internally developed ML algorithm was designed to manage compliance processes, but the company's leadership expressed concerns about its performance and overall effectiveness. Given the novelty of ML in their operations, they sought to ensure thoroughness in all aspects, aiming to meticulously cross every 'T' and dot every 'I'. Additionally, they were apprehensive about the robustness of their implementation under the scrutiny of an external audit. This led them to seek expert guidance to guarantee that their solution would stand up to rigorous regulatory standards and deliver reliable compliance management.
TRCG Approach
Engagement Objectives:
TRCG was engaged to perform an in-depth assessment focusing on three key areas:
Solution Effectivity: Assess the effectiveness of the machine learning solution in achieving its intended purpose in the complaint handling process.
Audit Readiness: Ensure that the new system and support team were well-prepared for regulatory inspections.
Project Effectivity: Evaluate the adequacy of the implementation project in maintaining compliance and driving effectiveness, and its likelihood of achieving project goals (compliance and efficiency) across the organization.
Assessment Process:
Solution Effectivity: TRCG associates began an analysis of quality and relevance of the training and test data sets, algorithm training and test methodology, performance data, and a comparison against the human process.
Project Effectivity: The overall project implementation was assessed to determine the likelihood of achieving project goals. The team assessed the rollout and change control plans and compared them with the realities of the organization and products. These included project plans, post-market surveillance procedures, and associated reference documents.
Audit Readiness: Audit readiness would be assessed through the execution of the assessments of both Solution and Project Effectivity.
Conducting the Audit
Solution Effectivity:
The audit team evaluated the algorithm model selected and compared it with the intended use case and clinically relevant factors therein. Our associated then used the algorithm training plan to evaluate the quality and robustness of the training dataset. This involved confirming the accuracy and diversity of the training data to ensure comprehensive coverage of potential product and use scenarios. Similarly, the test dataset assessment carefully verified appropriate segregation to maintain data integrity. The team reviewed performance data, comparing the algorithm's results against traditional human processes. To further test the algorithm's limitations and weaknesses, "dummy cases" were generated and analyzed, providing a thorough evaluation of the algorithm’s reliability and effectiveness.
Project Effectivity:
The project effectivity portion of the audit took a macro approach rather than focusing on the algorithm performance itself. The audit team catalogued all post-market surveillance guidance material and assessed whether the training plan adequately considered important factors from training. Leveraging their organizational experience, TRCG associates looked for opportunities for the system training to come out of sync with product and install base reality, assessed the capability and mandate of the governing organization, algorithm surveillance strategies, and algorithm and project maintenance plans.
Audit Readiness:
Throughout the Algorithm and Solution Effectivity assessment, special attention was paid to how different regulatory requirements, and Good Machine Learning Practices were followed; as well as how well SMEs could describe their processes. As a novel implementation, the team also reviewed the overall justification and risk-benefit for the project.
Results
Solution Effectivity:
TRCG's audit confirmed that the machine learning solution was limitedly effective in reducing case-to-case variation and improving consistency in post-market surveillance activities. While very effective for several product modalities; some major modalities suffered from invisible defects and would remain ineffective for the foreseeable future due to a misalignment of the algorithm model and the clinically relevant risk factor.
Project Effectivity:
After the macro review of the project, TRCG determined the sustainability plan was ineffective. Our associates identified the necessity for a more robust retraining plan to address and mitigate the limitations of the initial algorithm training set. This retraining plan was also crucial to avoid systemic defects driven by data bias, which posed high regulatory risks and could begin to manifest within as little as six months.
Audit Risk
The assessment of overall audit risk determined that while the initial risk was moderate, implementing small changes to the use case definition and parameters would be necessary to significantly lower the residual contextual audit risk.
Following TRCG’s assessment, the organization took remediation steps and was ultimately better prepared for regulatory scrutiny. The insight provided also mitigated the risk of further quality issues and helped drive practical boundaries for the implementation of the solution.
Timeline
TRCG understands that while compliance may be our business, it’s not yours. Keeping that top of mind, our teams seek to minimize disruption to business operations.
We were able to execute this entire engagement, from request and planning through reporting and advisement on remediations within three months — a single quarter. Keeping as many activities as possible remote. This swift and efficient timeline ensured that the client could quickly address and rectify the identified issues, and realize the benefits of their new solution with clear understanding of its limitations.
Why TRCG
TRCG's associates bring decades of Quality Management and Audit experience, working with both industry giants and agile startups. We have successfully defended quality systems with regulatory authorities and designed and maintained highly effective quality systems. Our comprehensive, focused approach provides not only compliance assurance but also strategies for building a value-added quality system, driving better product performance and customer satisfaction.
With over a decade of Artificial Intelligence and Machine Learning (AI/ML), Clinical/Care Decision Support Systems, and Software as a Medical Device (SaMD) experience, TRCG has the “Digital DNA” to unlock the insights necessary to help you address quality issues and prepare for regulatory inspections. Our expertise in assessing risk and evaluating quality system capabilities ensures that your organization can promptly identify and correct underlying problems, securing regulatory compliance and enhancing overall effectiveness for you Digital Health and MedTech solutions.