Manufacturers must clinically evaluate a medical device before they can market it in Europe. To do so, the manufacturer has to examine clinical data in order to investigate whether the medical device is safe and performing. The clinical evaluation of a medical device extends over the entire product life cycle and is a complex process. It begins with initial explorative studies and product ideas and reaches all the way to the storage of regulatory documents. The manufacturer can use the results to assess whether the risks associated with the use of the medical device are appropriate to the expected benefit.The clinical evaluation of software is based on the same legal requirements as the clinical evaluation of all medical devices. In this article, we explain what you need to consider when clinically evaluating medical software.
- Legal Basis for Medical Devices in Europe
- Clinical Data
- What Does Equivalence to Marketed Devices Mean?
- Practical Implementation of Equivalence
- When Is a Clinical Trial Required?
- Guidance Documents
- The Clinical Evaluation Report
- Valid Clinical Association
- Analytical Validation
- Clinical Validation
- Our Recommendations
Legal Basis for Medical Devices in EuropeThe clinical evaluation of medical devices according to the EU MDR is a mandatory process for the manufacturer (Art. 61 and Annex XIV Part A). It aims to examine and evaluate clinical data to verify the clinical safety and performance of the medical device. Obviously, the kind and extent of the clinical evaluation depend on the intended purpose defined by the manufacturer. Furthermore, it includes a post-marketing clinical follow-up (PMCF, Art. 61 (11) and Annex XIV Part B).
The manufacturer must keep the clinical evaluation up to date throughout the entire life cycle. He must update the PMCF assessment report for Class III and implantable medical devices annually.
Clinical DataClinical data is any “information concerning safety or performance that is generated from the use of a device” (Art. 2 (48)). It can come from the following sources:
- clinical trial of the device concerned or of an equivalent device,
- clinical experience with the device concerned or an equivalent device, or
Manufacturers should link and keep up to date risk management, quality management and clinical evaluation procedures on a regular basis.Note that pre-clinical and clinical data must be part of the technical documentation (Annex II (6.1.)) including:
- test results,
- test design and protocols (in the case of software the verification and validation),
- clinical evaluation report / plan, and
- PMCF plan / evaluation report.
What Does Equivalence to Marketed Devices Mean?The manufacturer can demonstrate equivalence to already marketed medical devices. Annex XIV (3) of EU MDR specifies 3 characteristics, that manufacturers must consider when demonstrating equivalence:
- technical (e.g. conditions of use, properties and algorithms),
- biological (not applicable for software) and,
- clinical (e.g. clinical condition or purpose, population and performance).
Practical Implementation of EquivalenceObviously, the manufacturer has a considerable scope for interpretation as to how he can practically implement these requirements of the EU MDR. For instance, the manufacturer may refer to the clinical evaluation of equivalent devices from other manufacturers. What extent of data of medical devices from other manufacturers does he then need?Annex XIV (3) EU MDR, for instance, refers to non-implantable devices of risk classes I – IIb. It specifies, that manufacturers must provide clear evidence that they have sufficient access to the data of the device with which they claim equivalence. Art. 61 para. 5, in contrast, refers to implantable devices or devices in risk class III. It says that two manufacturers need a contract explicitly allowing the manufacturer of the second device full access to the technical documentation on an ongoing basis.How manufacturers can deal with the difference between “sufficient access” and “full access to technical documentation” in practice is not clear. However, the clinical evaluation of the equivalent device must comply with EU MDR requirements. The EU guidance document MEDDEV 2.7/1 Rev. 4 provides further information on demonstrating equivalence (see Appendix 1). However, this document refers to the “old” EU MDD and AIMDD.If the clinical evaluation reveals gaps, the manufacturer can conduct a clinical trial to generate the missing data. The clinical trial may be designed to demonstrate the comparability of the clinical safety and performance of the device with the device to be compared.
When Is a Clinical Trial Required?In any case, the manufacturer must conduct a clinical trial if he:
- launches completely new software with new features and functionality,
- he has modified an existing software in such a way that clinical safety and efficacy may be affected, or
- he uses an existing software for a new medical purpose.
Guidance DocumentsAt present the two guidance documents MEDDEV 2.7/1 Rev. 4 from the European Commission and N41 for the clinical evaluation of stand-alone software from the International Medical Device Regulators Forum (IMDRF) provide supporting information for manufacturers and notified bodies regarding clinical evaluation. Both guidance documents are legally not binding and the MEDDEV 2.7/1 Rev. 4 still refers to the EU MDD and AIMDD. Nevertheless, for the time being we recommend that manufacturers use these documents.
The Clinical Evaluation ReportAnnex 9 of MEDDEV 2.7/1 Rev. 4 proposes the content and structure of the clinical evaluation report (CER):
- scope of the clinical evaluation and description of the medical device,
- state-of-the-art in science and technology as well as clinical background, and
- route of clinical evaluation/device under evaluation.
Valid Clinical AssociationFirst, there must be a valid clinical association between the output of the software and the clinical condition, disease, injury, or disability. Therefore, the manufacturer must carry out a comprehensive literature study by searching the relevant databases (e. g. PubMed) for systematic and non-systematic reviews as well as meta-analyses. Moreover, he can use guidelines of medical societies as a source.Here the key questions are:
- Does the output of the medical software correspond to the current state of science and technology regarding its intended purpose?
- Is the functionality of the software of clinical relevance in medical practice?
Analytical ValidationSecond, the manufacturer must validate his device analytically or technically by carrying out validation and verification tests as well as comparative tests in the case of device equivalence. In addition, the device must comply with the relevant standards.Finally, the manufacturer has to demonstrate the risk-based approach in the clinical evaluation by addressing software risks, e. g. caused by incorrect data input, insufficient precision of output, inadequate upper/lower limits, technical failure of e.g. mobile devices or the software environment, missing operability, and compatibility.Here the key questions are:
- Is the software algorithm working correctly and reliably?
- Are all specifications fulfilled regarding the intended purpose?
Clinical ValidationThird, the manufacturer must measure the ability of the software to yield a clinically meaningful output. Clinical trials conducted with the own medical device or an equivalent one, validation and verification, post-market surveillance and usability studies are possible.Here, the key questions are:
- Does the desired (precise and reliable) output of the medical software meets the intended purpose in the target population?
- Does the output have the desired clinical significance for the corresponding patient population?
- Is the result of the algorithm relevant to the diagnosis, treatment or prevention of a disease in a patient?
DocumentationThe overall documentation of the clinical evaluation is getting more complex according to EU MDR. It requires the following documents:
- clinical evaluation plan (CEP, Annex XIV, Part A, 1.) describing the procedure for clinical evaluation to demonstrate the benefit-risk ratio based on the state of the art,
- clinical evaluation report (CER, Art 61 (12)),
- post-market surveillance plan (PMS, Art. 84, Annex III) covering proactive and passive activities to collect and analyze market-related experience including disclosure of methods and protocols to manage those post-market activities,
- post-market clinical follow-up (PMCF, Annex XIV, Part B) or a justification as to why a PMCF is not applicable (outlined in the PMS plan),
- post-market surveillance (PMS report for class I devices which summarizes the results and conclusions of the post-market surveillance data,
- periodic safety update report for classes IIa, IIb and class III devices (PSUR, Art. 86), and
- clinical evaluation assessment report (CEAR, Annex VII section 4.6; to be compiled by the notified body).
At present, there are no official templates for the documentation of the clinical evaluation of medical devices. Here it is worth looking at the corresponding documents of the pharmaceutical field.The clinical evaluation report (CER) compiles the conclusions of the clinical evaluation. In addition, the manufacturer must provide a publicly available “Summary of safety and clinical performance” (Art. 32) for certain high-risk devices. The notified body in turn documents the results of the clinical evaluation assessment in the clinical evaluation assessment report (CEAR). In the case of certain high-risk devices the expert panel, the competent authorities, the authority responsible for notified bodies and the EU Commission have these documents at their disposal.The manufacturer commits himself to the collection of post-market surveillance data in the PMS plan. The periodic safety update report (PSUR) summarizes this data and contain among others also the main findings of the post-market clinical follow-up (PMCF). Notably, the EU MDR does not say something about the frequency of clinical evaluation updates. MEDDEV 2.7/1 Rev. 4 includes such information.
- Monitor the implementing legislation of the EU commission and guidance documents released by the Medical Device Coordination Group (MDCG).
- Constantly watch the market for existing and new equivalent medical devices and have a respective process in place. Are new findings (clinical study and scientific results) published in professional journals? Are any warning published by competent authorities?