Good Machine Learning Practice (GMLP) for Medical Device Development: Guiding Principals

Explore the 10 guiding principles of Good Machine Learning Practice (GMLP) and the importance of ethics, data integrity, and model transparency.

Machine learning in healthcare presents unprecedented opportunities for diagnosis, treatment planning, and patient care. However, medical device development in particular demands rigorous compliance with regulatory standards to ensure safety, reliability, and effectiveness.

In October 2021, the U.S. Food and Drug Administration (FDA), Health Canada, and the United Kingdom’s Medicines and Healthcare products Regulatory Agency (MHRA) identified 10 guiding principles to help promote safe, effective, and high-quality medical devices that use artificial intelligence (AI) and machine learning.

The Vision for GMLP Guiding Principles

We already see AI and machine learning transforming healthcare. Brilliant insights are derived from an unprecedented data revolution in life sciences and software algorithms are trained on real-world use cases.

The 10 guiding principles for GMLP lay the foundation for medical device development. They also identify areas where the International Medical Device Regulators Forum (IMDRF), international standards organizations, and other collaborative bodies could advance GMLP.

GMLP guiding principles help medical device developers to:

• Adopt good practices that have been proven in other sectors
• Tailor practices from other sectors and apply them to medical technology and the healthcare sector
• Create new practices specific to medical technology and the healthcare sector

So what are the 10 guiding principles for GMLP? Let’s take a look.

1. Multi-disciplinary expertise is leveraged throughout the total product life cycle. Cultivate an in-depth understanding of the desired benefits and associated patient risks to help ensure that ML-enabled medical devices are safe and effective.
2. Good software engineering and security practices are implemented. Incorporate good software engineering practices, data quality assurance, data management, and robust cybersecurity practices into model design.
3. Clinical study participants and datasets are representative of the intended patient population. Ensure that relevant characteristics of the intended patient population, use, and measurement inputs are sufficiently represented in an adequate sample size so that results can be reasonably generalized to the population of interest.
4. Training datasets are independent of test sets. Consider and address all potential sources of dependence, including patient, data acquisition, and site factors.
5. Selected reference datasets are based upon best available methods. Ensure that clinically relevant and well characterized data are collected and that the limitations of the reference are understood.
6. Model design is tailored to the available data and reflects the intended use of the device. Understand the clinical benefits and risks related to the product to derive clinically meaningful performance goals for testing and to confirm that the product can safely and effectively achieve its intended use.
7. Focus is placed on the performance of the Human-AI team. Keep a human in the loop so that human interpretability of model outputs are addressed, rather than just the performance of the model in isolation.
8. Testing demonstrates device performance during clinically relevant conditions. Generate device performance information independently of the training dataset and consider the intended patient population, important subgroups, the Human-AI team, measurement inputs, and potential confounding factors.
9. Users are provided clear, essential information. Ensure that the information is appropriate for the intended audience, such as healthcare providers or patients. Make users aware of device modifications, updates from real-world performance monitoring, and how to communicate product concerns to the developer.
10. Deployed models are monitored for performance and re-training risks are managed. Monitor deployed models during real-world use (post-market surveillance) and, when models are trained after deployment, have appropriate controls in place to manage risks.

Three More Facets of Machine Learning for Medical Devices

While the 10 guiding principles lay the foundation for GMLP and medical device development, ethics, data integrity, and model transparency are the infrastructure for responsible and reliable medical devices.

Ethical Considerations

Ethics are the ultimate guide in developing and deploying machine learning in medical devices. Developers must prioritize patient safety and privacy; therefore, ethical machine learning practices require transparent data usage and patient data confidentiality. They also help to mitigate biases in machine learning models to prevent unequal treatment outcomes across various patient demographics.

Data Integrity

The quality, accuracy, and representativeness of the data used to train machine learning models influences its reliability and validity. Comprehensive and diverse datasets that accurately reflect a target population consist of data collected from a wide range of ages, genetic backgrounds, and health conditions. Data curation and preprocessing remove inaccuracies and ensure the data is relevant to the medical conditions the device addresses.

Model Transparency and Explainability

Understanding the decision-making process for medical device development is important for clinical acceptance. Model transparency and explainability ensure that healthcare professionals are able to interpret and trust the outputs of devices powered by machine learning. Techniques like model simplification help to demystify complex algorithms and make them more accessible to clinicians and patients.

How USDM Can Help

Data is the lifeblood of AI, but people are the weak link in responsible AI and machine learning. USDM Life Sciences provides the training and expertise and helps your life sciences organization establish a data governance framework for the integrity and security of your data.

To learn more about integrating advanced technologies into medical device development, contact us today. Our industry experts will help your organization realize the benefits of machine learning and its responsible use and develop an AI governance framework that will work for your organization.