Artificial intelligence in medicine is gaining tremendous attention. This new digital tool seems perfectly to fit with the need for making the enormous amount of health data accessible for diagnostics and therapeutic or even predictive purposes. Typical fields of application for artificial intelligence in medicine are neurology, cardiology, oncology and ophthalmology.
The technical possibilities of artificial intelligence in medicine are striking. But equally important is access to the highly regulated market for medical devices in order to make the technology accessible to patients at all. In 2018, Topol already counted 18 approvals of artificial intelligence products by the FDA compared to 2 the year before.
In this blog article we explain how market access for a medical device can be achieved in the US and the EU if it is equipped with artificial intelligence. Let us take a look at an example of an already approved medical device of this type
- Software as Medical Device
- Real-World Example for Artifical Intelligence in Medicine
- Approval of Artificial Intelligence in Medicine
- Addressing Regulatory Requirements for Artificial Intelligence in Medicine
- Future Regulation Approaches Artificial Intelligence in Medicine
- Conclusions and Outook
Software as Medical Device
The first question one may ask is, if there are any special characteristics for software and especially for artificial intelligence regarding regulation as a medical device. Indeed, medical software has some special features of which most are related to the different development compared to other medical devices:
- the faster and iterative design,
- the use of agile methods,
- the easier implementation of changes and new features,
- the difficulty of comprehensive testing, and
- the different type of validation.
Moreover, new software versions are quickly released. Finally, cybersecurity risks are associated with software products requiring special attention during the development process.
And What is Special for Artificial Intelligence Software?
Due to the broad range of the technology and the black box character, artificial intelligence seems to be difficult to grasp for regulators as well as clinicians.
Only if the data have a certain quality, artificial intelligence can play its strengths in medicine, especially in pattern recognition in large and complex data sets. This must also be taken into account in the regulatory processes of the manufacturer.
The ability to learn within the analyzed data (continuous adaption of parameters) leads to a continuous evolution of the software. This makes it difficult to predict the future quality of the results.
Note, however, that the existence of exceptions cannot be learned from data.
Finally, some artificial intelligence systems generate their results completely autonomously. This also makes it necessary to strengthen control mechanisms in the market phase.
Real-World Example for Artifical Intelligence in Medicine
As already mentioned, we look at an existing medical device as an example from practice. Of course, we have selected this product only because of the free accessibility of regulatory data and not for advertisement purposes.
Diabetic retinopathy occurs as a result of high blood sugar levels and is the most common cause of vision loss in diabetes patients. Many patients do not receive retinal examinations by eye-care professionals at a regular basis.
IDx-DR developed by IDx Technologies Inc. is a deep learning algorithm automatically detecting diabetic retinopathy in diabetes patients. Non-ophthalmic healthcare practitioners should be able to operate IDx-DR thereby allowing easier access to retinal examinations by patients. The following figure shows the core components of IDx-DR:
First, the professional user receives a 4-hour standardized training to operate the retinal camera. Second, the acquired images are then sent to a cloud running the IDx-DR software. After a quality check the professional user receives feedback whether more than one mild diabetic retinopathy has been diagnosed. If no retinopathy has been diagnosed, a new screening procedure will be performed in 12 months. In case of a positive retinopathy diagnosis, the software recommends immediate referral of the patient to an eye-care professional for further diagnostic assessment and treatment.
Note that the FDA released also warnings concerning use and limitations as well as precautions and contraindications for IDx-DR.
Approval of Artificial Intelligence in Medicine
U.S. Market Access via De Novo
On April 11, 2018 the U.S. Food and Drug Administration (FDA) announced the approval of IDx-DR.
Note that the FDA granted breakthrough device designation for IDx-DR leading to intensive interaction and guidance by the FDA. Likely, this was of advantage for the manufacturer in the subsequent approval process.
Since there existed no predicate device for IDx-DR the manufacturer applied for the De Novo re-classification. The FDA classified the software as class II device (for details refer to FDA De Novo Summary DEN180001). We have already described the De Novo route in detail in our blog post “Market Access for Medical Software in the United States“.
For the approval of a class II device via De Novo, general and special controls must be met. In the case of IDx-DR the following special controls were requested by the FDA:
- software verification and validation documentation, based on a comprehensive hazard analysis,
- clinical performance data supporting the indications for use,
- training program with instructions on how to acquire and process quality images,
- human factors validation testing,
- protocol describing level of change in device technical specifications that could significantly affect the safety or effectiveness of the device, and
- special labeling provisions.
The manufacturer submitted comprehensive documentation for this software “having a major level of concern”:
- software/firmware description,
- device hazard analysis,
- software requirement specifications,
- architecture design chart,
- software design specifications,
- software development environment description,
- revision level history,
- unresolved anomalies, and
European Market Access for Artificial Intelligence Software According to MDD
Market access under the Directive 93/42/EEC (EU MDD) regulatory framework was achieved as well. The software was assigned to medical device class IIa. For conformity assessment the Annex II EU MDD route was chosen. Subsequently, the respective Notified Body issued in 2013 a certificate for full quality assurance system approval.
On May, 25 2017 the European Medical Device Regulation (EU MDR) entered into force with a 3-years transition period. In our step-by-step guide we tell you in detail what new challenges are awaiting you. Many requirements are tightened with the EU MDR like risk classification, conformity assessment, general requirements, clinical evaluation and trials, technical documentation etc. For some obligations like post market surveillance and unique device identification the EU MDR even contains mostly new provisions.
What is happening with MDD devices like IDx-DR after EU MDR date of application? According to Art. 120 (3) EU MDR no significant changes in design and intended purpose are allowed for MDD devices put into service. After May 27, 2025 this software as all other MDD devices is not allowed to put into service not at all.
Addressing Regulatory Requirements for Artificial Intelligence in Medicine
Essential elements for access of software to the medical device markets in the U.S. and Europe are clinical evaluation, risk management as well as verification and validation. In the following paragraphs we explain how these requirements were addressed by the IDx-DR software.
The basis for the submission was a clinical trial with 900 subjects enrolled at 10 primary care sites. We summarize the major outcomes as follows:
The far majority (96%) of acquired images was of sufficient quality for analysis.
The software showed 87% sensitivity and 90% specificity. What does that mean? Sensitivity refers to the number of patients diagnosed for more than mild diabetic retinopathy (true positives) in relation to the total number of sick individuals in the population. Specificity refers to the number of patients diagnosed not for more than mild diabetic retinopathy (true negatives) in relation to the total number healthy individuals in the population.
Finally, the validation of the IDx-DR software included camera, camera operation, imaging protocol, standardized training, and operational materials. 96% of acquired images were of sufficient quality. Obviously, users had no problems to operate the camera and to transfer the images from the client computer to the cloud-based IDx-DR software. Another important finding underlined this conclusion: 99.6% agreement of IDx-DR outputs was found across repeats, operators and cameras.
In the FDA De Novo Summary you will find much clinical data showing how complex the collection of clinical data was.
Risk Management Issues
Let’s first have a look at the identified risk for IDx-DR as reported in the FDA De Novo Summary:
- False positive or negative results caused by failure of the diagnostic algorithm and/or the software. As a consequence this could lead to additional and unnecessary medical procedures or to delay of further evaluation or treatment, respectively.
- User failures in providing images of sufficient quality.
As shown in the following figure a number of risk controls have been applied.
Cybersecurity risks are part of the documentation. Regarding future risk management software manufacturers have to pay attention to safety and security impacting each other. For example, if the IDx-DR algorithm is updated online for better performance as safety risk control it is thereby also exposed to attackers. Consequently, this would have an impact on security risk analysis.
Overall, the FDA concluded that “a comprehensive risk analysis was provided for the software […]”.
Future Regulation Approaches Artificial Intelligence in Medicine
The ability of artificial intelligence to continuously learn from real data could raise problems with traditional regulatory approaches. Since artificial intelligence is regulated as a medical device in the US, certain modifications may require premarket submission. The questions arise which changes could trigger this process and how could future regulation approaches look like.
FDA Pre-Cert Program as a Basis
The currently tested FDA Software Pre-Certification (Pre-Cert) Program addresses software as medical device and, thus, also artificial intelligence software. To summarize, the Pre-Cert program applies the “total product lifecycle” (TPLC) approach allowing continuously improvement of the software accompanied by effective protective measures. With this new regulation approach the FDA is shifting from the product to the organization as a decision factor for approval. Get more insights on the Pre-Cert program in our blog article on market access in the USA.
On April 2, 2019 the FDA released the discussion paper “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning-Based Software as a Medical Device”. This framework serves for the further development of Pre-Cert for applications of artificial intelligence. Here, too, the TPLC approach forms the basis. The software is evaluated and monitored over the entire life cycle from development to market entry and also when used in the real world.
Real-Life Example for Future Market Access of Artificial Intelligence in Medicine
In the FDA discussion paper a number of examples are given how the new regulatory framework could be applied. One example is a mobile medical app providing physical characteristics of a skin lesion based on smartphone image to a dermatologist.
As a pre-requisite the manufacturer has identified potential app modifications and algorithm changes that were approved by the FDA.
As a pre-requisite the manufacturer has identified potential app modifications and algorithm changes that were approved by the FDA.
In a first scenario the assessment performance of the app was increased by real-world data. Both the app modification and the algorithm change were consistent with the initial approval by the FDA. Market access would be granted without additional FDA approval.
In contrast, in the second scenario, the app would direct the patient to a dermatologist based on the determination of the malignancy of a skin lesion. This direct feedback for the patient would introduce new risks not being anticipiated in documents for the initial approval by the FDA. Market access would require a premarket submission (FDA review) prior to market access.
Conclusions and Outook
Market access for artificial intelligence in medicine is already possible with the existing regulatory procedures in the USA and Europe.
Until now, the FDA’s market approval required artificial intelligence algorithms to be frozen in in a particular version. However, this is against the nature of artificial intelligence software. Usually, it changes its algorithm because of its continuous learning feature. Future regulatory strategies must adequately take into account these changes by defining clear rules on when a change leads either to new documentation or to pre-market reviews.
The FDA has started a corresponding discussion in the USA with the preposed new regulatoty framework and we can be curious to see how this will end.