An Inherently Flawed System or a Valuable Pathway for Innovation?
When considering device pathways to FDA approval, it may be helpful to first review the pathway to FDA approval for drugs. It is estimated that the average length of time from concept to market for investigational new drugs is about 12 years, which has increased significantly from just under 8 years in the 1960s, with an estimated total cost per drug of $800 million.
The process can be divided into several stages: a research and development phase with preclinical testing (average 1–3 years), a clinical research and development period including phase I, II and III testing (average 5–10 years), and a new drug application FDA review with post-marketing surveillance (average 2 years). During this time period, drugs are tested in a sequential manner that incrementally increases patient risk while targeting a specific therapeutic goal.
Phase I studies are usually conducted in healthy volunteers to determine the side effect profile and relative safety of the medication as well as the route of metabolism. If phase I studies demonstrate an acceptable safety profile, phase II studies are undertaken that evaluate medication effectiveness. This phase aims to obtain preliminary data on whether the drug is effective against a target condition, usually through randomization of patients with the diagnosis of interest to varying drug doses, including a placebo group. Safety continues to be evaluate, and short-term side effects are studied. Patients’ responses to each dose are monitored and optimal dosing, based upon a risk-benefit ratio, is identified.
At the conclusion of phase II the FDA and drug development company meet to plan phase III studies. Phase III studies evaluate the new medication head-to-head with other standard treatments, with the goal of comparing the intrinsic effectiveness and safety of the new medication against alternative or standard-of care therapies. Phase III studies are frequently randomized controlled trials that compare the new medication with alternative therapies that are already accepted treatments for the given condition. Drugs that successfully navigate these three phases with satisfactory effectiveness and safety profiles are reviewed and approved for use. Following approval, post-marketing requirements and commitment studies (phase IV) are mandated in which the medication is monitored for safety, efficacy and alternative uses even after release onto the market.
In contrast, the regulatory process for medical devices is much shorter and, generally, less stringent and costly. It has been estimated that the time from concept to market for medical devices is 3–7 years, although no concrete data could be identified in the literature regarding time or cost. The Medical Device Amendments of 1976 to the Federal Food Drug and Cosmetic Act established the current FDA policies regarding medical device approval. Within this framework, many new regulated devices are catalogued as Class III, which is defined as a device that ‘supports or sustains human life or is of substantial importance in preventing impairment of human health or presents a potential, unreasonable risk of illness or injury’. Manufacturers may petition to have their device downgraded to Class I (low risk) or II (moderate risk) should the device harbor only minor differences from devices previously approved. All such devices placed into Class III are subject to premarket approval (PMA) requirements, while those that are classified as Class I or II are subject to less stringent requirements. Therefore, unlike the drug development pathway that mandates successful results in all three clinical phases to obtain new drug approval, the medical device pathway has separate fast-track routes of obtaining approval. These pathways are discussed in the sections that follow. Further information regarding medical device approval is available on the FDA website at http://www.FDA.gov/MedicalDevices.
Premarket approval (PMA) is the most stringent type of device marketing application required by the FDA and is required for new devices for which there is no existing equivalent or predicate (Class III devices). PMA approval is granted only if the FDA determines that the new device has sufficient scientific evidence demonstrating that the device is safe and effective for its intended use. Usually, Class I or Class II evidence (prospective data compared with historical controls or randomized clinical trials) are necessary to obtain PMA. In effect, a PMA acts as a license granted to the applicant for the sale and use of their product in the USA. PMA may be considered the ‘gold standard’ regulatory process through which devices are approved because these devices must have valid prospective scientific evidence supporting their benefit and safety. However, it should be emphasized that PMA approval can be achieved with Class II data (such as the Pipeline Embolization Device; ev3, Irvine, California, USA) without an active comparator. Evidence-based medicine specialists will point out that this raises significant potential limitations to the quality of the data regarding some PMA-approved devices. Additionally, there is no requirement for post-marketing surveillance studies to validate the pre-marketing experience.
A Premarketing Notification (510(k)) is a fast-track process wherein applicants must demonstrate that the device to be marketed (moderate risk or Class II) is ‘substantially equivalent’ to a pre-existing legally-marketed device (predicate) in terms of safety and effectiveness. The predicate must have been approved either via PMA or 510(k); devices currently under review are not acceptable predicates. The 510(k) application to the FDA is required at least 90 days before marketing.
This process is usually used when manufacturers develop small iterations upon a previously approved device that are thought to improve effectiveness without compromising safety, allowing for expedited approval without costly and lengthy scientific studies confirming safety and effectiveness. Although this process allows for quick turnover of cutting edge technology from bench to bedside, it also introduces an element of risk should the equivalence assumption be invalid (eg, the Depuy ASR). Furthermore, as devices may be approved based on equivalence to devices now on the market that had 510(k) approval, it is possible that a device could be found equivalent to one approved years ago and that the prior device was deemed equivalent to one three decades ago, and so on, without any recent scientific evidence supporting the technology’s use.
The ‘de novo’ 510(k) process was initiated as part of the 1997 FDA Modernization Act and may be used when no predicate exists but there are substantial data to suggest the device does not carry high-risk (Class III) status. Most devices without a predicate are automatically classified as Class III. This process involves the submission of a 510(k) application, even though a predicate does not exist, resulting in a letter of non-substantial equivalence from the FDA. The manufacturer may then petition the FDA (‘de novo’ petition) to have the device reclassified to Class I or II by providing ample evidence that Class III status is not necessary.
Humanitarian Device Exemption (HDE)
The third means of approval is via a HDE application. It is important to note that neurointerventional surgery, as a specialty, features a relatively high number of HDE-approved devices. A Humanitarian Use Device (HUD) is a medical device designed to treat or diagnose a condition that affects <4000 individuals in the USA annually. In addition, the use of a HUD requires local institutional review board (IRB) approval and supervision. The HDE application is similar to the PMA application; however, the HDE is exempt from the PMA requirement of valid prospective scientific evidence arguing its effectiveness.
The HDE carries this exemption because it could potentially take years just to enroll enough patients with a rare disease to obtain a power sufficient to demonstrate statistical effectiveness. However, data to support the HDE must demonstrate that there is a probable benefit to health from the use of the device and that the probable benefit outweighs the risk of injury or illness from the use of the device. Therefore, to allow for continued technological advancement and treatment of diseases of low prevalence, the HDE only requires demonstration of device safety with the assumption of device effectiveness. To keep manufacturers from profiting from devices that lack evidence supporting their effectiveness but allow for patients with rare disorders to receive continued treatment, the HDE mandates that manufacturers charge a price that covers manufacturing fees, research and development and other associated expenditures only.
If this value is more than $250, the HDE holder must provide the FDA with an independent certified accountant report or representative attestation indicating the reasons for the higher cost. An exception to this rule is an HUD designed to be used in pediatric populations and some devices used to treat both children and adults. Similar to the 510(k) process, this process helps to expedite approval for medical devices aimed at benefiting uncommon diseases, but also introduces an element of risk should the effectiveness assumption be invalid or become outdated with advancing alternative treatments.
Investigational Device Exemption (IDE)
An investigational device exemption (IDE) is required prior to evaluating investigational devices in a clinical study. Unlike other device pathways, an IDE requires local IRB approval, informed consent from all treated patients, labeling of the device for investigational use only and rigorous monitoring of the study. Investigational devices are dichotomized into two groups based on the potential for serious risk to health of subjects: significant risk devices and non-significant risk devices. Significant risk devices, given their inherent risk to patients, require both FDA and IRB approval before initiation of a clinical study, while non-significant risk devices require only IRB approval. The IDE process provides manufacturers of new devices a means to evaluate for device safety and effectiveness to support a PMA or 510(k) application.
Postmarket Device Reporting
All devices approved for market have mandatory manufacturer and facility reporting requirements. Most notably, manufacturers and the facility must report all device-related deaths, serious injuries and adverse events secondary to device failure or adverse events in which the device may have contributed. These include 30-day reports in which manufacturers have 30 days from time of event to report device-related deaths, serious injuries or malfunctions to the FDA (available athttp://www.FDA.gov/downloads/Safety/MedWatch/HowToReport/DownloadForms/UCM082728.pdf); 5-day reports which require manufacturers to report serious public health concerns stemming from device use to the FDA within 5 days of becoming aware of the concern; baseline reports which are for first-time adverse events; supplemental reports; and annual certifications. However, substantial criticism exists with this reporting process as no formal system is in place to ensure capture of all events. For the most part, the reporting process is dependent upon physicians reporting any events back to the company. It is probably fair to say that, currently, such reporting is sporadic at best.
The FDA may order manufacturers of certain Class II and Class III devices to establish tracking systems in which each individual device may be tracked to the patient in which it was used. This provision allows the FDA and manufacturer to locate and expeditiously remove those devices from the market that have been identified in postmarket reporting as potentially dangerous or defective (facilitating device recalls) or to notify treated patients of a potential health concern associated with device use. Generally, devices subject to such tracking provisions are those that are intended for implantation in the body for >1 year, those that may cause significant harm or death should the device malfunction, or those that are intended for use outside the treatment facility and are life-saving or life-sustaining.
The FDA may also order holders of a PMA or HDE to perform a 522 postmarket surveillance (522PMS) study to help assure continued safety and effectiveness after the device has been released on the market for a period of up to 36 months. The FDA has authority to require a 522PMS on any Class II or Class III device that meets one of the three tracking criteria (listed above) and/or is expected to have significant use in a pediatric population. The 522PMS is highly specific to the given device and may range from animal studies to randomized controlled trials. Frequently, a required 522PMS will involve active or enhanced surveillance studies where the incidence, distribution and trends of adverse events are actively or passively recorded and reported.
Growing Concerns About the Current FDA Approval Processes
There is a growing discussion in both the medical literature and in public commentary regarding potential faults in the FDA approval process for medical devices.
Most notably, the 510(k) and HDE processes have sparked considerable controversy due to the Depuy ASR and Wingspan System, as well as other devices, which were approved through ‘fast track’ routes lacking scientific evidence confirming device safety and effectiveness.
Safety concerns have been raised over the FDA’s 510(k) clearance process whereby devices demonstrated to be ‘substantially equivalent’ to previously approved devices, and therefore thought to share similar safety and benefit profiles, are approved for marketing without clinical trials. These concerns led the Institute of Medicine (IOM) to recommend eliminating the 510(k) process altogether in their July 2011 FDA-commissioned report.
Within this report the IOM argues that the 35-year-old 510(k) process cannot ensure device safety or effectiveness because it lacks any means to do so; it can merely determine equivalence to a predicate device. The IOM therefore argues that the 510(k) process should be disbanded and a new forward-thinking process developed. Furthermore, the report argues for enhanced post-marketing surveillance monitoring of devices, a feature that 510(k) approval currently lacks. Unfortunately, the report does not deliver new blueprints for overhauling the system; it merely identifies that the 35-year-old 510(k) process is antiquated and no longer appropriate. The report does list a number of attributes that would be ideal for any new FDA approval system for Class II devices including evidence-based, fair, clear, self-improving, risk-based, and others.This report has been met with praise but also criticism, particularly from the medical device industry which argued that changes to regulation would slow technological innovation, cost jobs and harm patients.[30, 31]
An additional concern of this process revolves around the financial incentive for manufacturers to develop new devices via the 510(k) clearance process with only minor improvements. As stated by Curfman and Redberg in a commentary published in the New England Journal of Medicine: “Since regulatory approval hinges on claims of similarity to previously approved devices, the process may encourage the development of devices that provide only small improvements at higher cost than their predecessors. The trade-offs between incremental improvement and the additional costs and technical complexity of the required procedure are poorly understood and seldom investigated rigorously.” As seen with the Cerecyte coils, the manufacturer was able to charge a premium due to purported superiority over traditional coils without any prospective evidence confirming superiority. When a prospective trial was performed, equivalence was confirmed but superiority was not, indicating that patients had been charged an increased cost for years for a device that was no more effective than its cheaper alternatives.
The HDE approval process may not adequately require substantial proof of efficacy. Nevertheless, the majority of these concerns are addressed at individual devices, not at the fundamental principles of the HDE approval process. Concerns have arisen due to the recent SAMMPRIS results regarding the Wingspan System[13–15] as well as devices for other treatments such as deep brain stimulation in obsessive-compulsive disorder.
Critics note the differing level of requirement for the introduction of drugs versus medical devices. Ironically, the FDA is being criticized at both ends of the spectrum while remaining substantially under-funded for the difficult tasks that it must oversee. On the one hand, there are concerns over the seemingly slow pace of introduction of new devices and drugs to the US medical market. Simultaneously, the FDA faces criticism for allowing drugs and devices to enter the market prematurely. Finally, some critics argue that the FDA and the Centers for Medicare and Medicaid Services (CMS), who are responsible for reimbursement for devices, play a critical role in the speed at which enrollment occurs in important randomized controlled trials based upon reimbursement patterns, and whether or not devices are reimbursed outside the context of a clinical trial. One example of this phenomenon is new acute ischemic stroke devices, which some argue are enrolling patients for randomized trials very slowly due to the fact that these devices are being reimbursed prematurely by CMS without Class 1 evidence supporting their use.[33, 34] In addition, poor coordination between the FDA and CMS with regard to physician reimbursement for FDA-approved devices (eg, foreign body retrievers with a stroke indication) further contributes to this issue. Finally, an inefficient FDA approval process is resulting in an increasing number of device manufacturers outsourcing their randomized trials to Europe or other countries in an effort to expedite accrual and trial completion. This fact may be a further indication that flaws inherent to the current FDA regulatory processes may be beginning to undermine the ability of the USA to remain competitive in the medical device industry.
The Role of Experience in Device Effectiveness and Safety
An important issue that has so far been left out of the FDA clearance debate is the role of operator experience in determining device safety and effectiveness. Most new medical technologies have a learning curve wherein clinicians receive initial training once the device is released for use and then subsequently improve with experience. Logically, practitioners using new devices are more likely to cause patient harm when first learning how to use the device than after proficiency has been obtained. The ‘learning curve’ effect plays a significant role in those devices that require new advanced skill sets and, through its effect on patient outcomes, may be a substantial contributor to the early results of mandated safety and effectiveness trials for devices. Recognition of this learning curve effect by the FDA has led to important FDA-manufacturer agreements regarding training for some new devices such as the Pipeline Embolization Device, wherein clinicians must undergo course training and then be supervised by a proctor for a designated number of cases before being able to use the device independently.
An excellent example of the learning curve effect comes from the Carotid Revascularization Endarterectomy versus Stenting Trial (CREST), a randomized trial comparing carotid endarterectomy to carotid angioplasty and stenting with the primary outcome of stroke, myocardial infarction or death. Although carotid angioplasty and stenting was first described in 1994, significant improvements in device technology, such as distal embolic protection devices, were not widely available until the beginning of the 21st century. The fundamental goal of CREST was to compare a new and exciting technology—angioplasty and stenting (with which most clinicians having only limited experience)—with a tried and tested technique—carotid endarterectomy, a commonly-performed procedure introduced in the 1950s. CREST began enrolling patients in 2000 and finished in 2008, with 50% of total enrollment reached in 2006. The final results of the trial demonstrated statistical equivalence of stenting with endarterectomy for the primary outcome, with a higher risk of stroke in the stenting group and higher risk of myocardial infarction in the surgical group. Interestingly, the risk of major stroke or death in the stenting group was 2.5% over the period 2000 to 2005 (n=361) and <1% from 2006 to 2008 (n=770). The substantial reduction in serious complications during this time period for the endovascular treatment arm is most likely secondary to improvements in technique from gained operator experience at the treatment centers, but may also be partly due to changes in enrollment criteria during the study period. The learning curve effect is well illustrated in this example because of the longevity of the trial, which provided ample time for operators to develop proficiency with the devices and techniques. Had the trial been halted before 2005, the results from stenting would have appeared worse than endarterectomy because operator experience was poor and complications were high. However, we now know that carotid stenting is a safe and effective option for patients with carotid stenosis because the trial allowed ample time for operators to become proficient and for associated technologies to be developed and widely implemented, with the lower complication rate towards the end of the trial nullifying the higher rate of poor outcomes at the beginning.
Expanding the concept of a learning curve effect to the approval process makes the situation even more complicated. Assuming this process occurs universally for most new technology, early trials evaluating the effectiveness of a new device are likely to overestimate complications and underestimate effectiveness because clinicians have limited experience with the device and are more prone to error. Consequently, early studies are likely to show no difference in outcomes (or potentially worse) compared with standard of care therapies. Studies performed years after approval of a device, after clinicians have gained experience with the technology and acquired proficiency, are more likely to report lower complication rates and a better safety profile. Extrapolating this argument further, one can predict how rigorous early testing of new technology, such as in a PMA, has a bias towards device rejection. Conversely, continued post-marketing monitoring of device safety and effectiveness in the years that follow approval is likely to show improved results as time progresses. Therefore, while early tests are crucial in detecting devices that are unsafe, post-marketing monitoring may be just as important in capturing the true risks and benefits of new technology.
There certainly exists a subset of newly-approved devices with inherent flaws that will continue to show inferior results regardless of advancements in proficiency (eg, the Depuy ASR). However, there are probably devices approved by the FDA with mediocre initial results that could show improvements in safety and effectiveness with time, and eventually become a standard of care therapy with profound benefits to patients with a particular condition.
Optimizing the FDA Approval Process
Recent and major device failures suggest a rationale for change within the FDA device approval framework. There has been surprisingly little argument from neurointerventional physicians regarding the PMA process and the need for rigorous testing demonstrating safety and effectiveness of new high-risk Class III devices. Furthermore, although the HDE process has limitations, the rarity of the diseases for which the devices are designed to treat makes obtaining effectiveness data impractical. In addition, it seems impractical to mandate that new devices with only small iterations upon previously-approved medical devices (ie, 510(k) approved devices) show robust effectiveness and safety data prior to approval. However, it is difficult to support the notion that merely being able to argue that a new device has ‘substantial equivalence’ to a predicate is an acceptable surrogate to actually having demonstrated it through clinical studies.
A potential solution to resolving the problems with the 510(k) and HDE processes does not necessarily lie in a complete overhaul of the system but, instead, lies in the realm of post-marketing monitoring and reporting. Mandatory post-marketing reporting of outcome, safety and complication data on new 510(k) or HDE devices by clinicians using newly-approved devices would provide an additional screening process by which 510(k) devices actually demonstrate equivalence and by which HDE devices actually demonstrate effectiveness. Essentially, this solution would make 522PMS mandatory for all newly-approved Class II or III devices, with most devices requiring active surveillance studies for recording and reporting of all adverse outcomes. As an example, it could be mandated that the first 1000 devices used (or implanted) after FDA approval are monitored closely for early and long-term outcomes. This would allow physicians to treat patients with new technology that they deem to be of benefit and allow manufacturers to continue to profit from their research and development by selling devices, while simultaneously providing a validation process for new technologies that can weed out those that are causing harm. This would not only allow patients to be treated with cutting edge technology but would continue to support technological innovation. If stringent post-marketing monitoring was performed for the Depuy ASR, it is certainly possible that an unacceptably high revision rate would have been detected much earlier and the device could have been removed from the market. Additionally, with mandated ongoing data accrual, it is not unreasonable to expect that the overall field would actually benefit as subsequent iterative advances would be based on valuable newly collected data rather than on anecdotal and marketing projections.
Limitations of mandated post-marketing monitoring for HDE and 510(k) devices largely appear to be related to the additional costs of data collection and data review. Unless strictly regulated, data provided by physicians or industry to the FDA would likely also contain an inherent bias. Strict guidelines for accurate, honest and clear reporting would be an essential element of any post-marketing amendment to the approval process. An adjudication procedure in which unbiased external experts review and evaluate clinical and outcome data in the post-marketing period may be necessary to ensure the quality of the post-marketing monitoring process.
The authors of this commentary are sensitive to the numerous burdens facing healthcare providers and medical device manufacturers. The Affordable Care Act produced legislative changes to healthcare greater than many US-based doctors have experienced in their professional lifetime. As part of the funding for the Affordable Care Act, device manufacturers have had a 2.3% tax imposed on the sale of their products. Physicians have increased demands on their time with diminishing reimbursements. Mandating post-marketing monitoring has the potential to be perceived as an unfunded mandate. We propose it in the absence of an alternative to potential changes of a more draconian nature as could occur by a sensitive or reactive FDA. Neurointerventionists, like other medical specialists with practices closely tied to the availability, safety and effectiveness of cutting edge medical devices, should be involved in designing and refining such processes to ensure that the proposed post-marketing monitoring remains efficient and effective in capturing credible information.
Recent challenges with medical devices suggest a need to reform the FDA medical device approval process. Disbandment of the 510(k) process, as is being suggested by the IOM, with mandatory completion of safety and effectiveness trials before device approval for all new devices is impractical and may harm technological innovation and, indirectly, patients. Instead, measured consideration of mandatory post-marketing surveillance for all newly-approved HDE or 510(k) devices, such that safety and effectiveness data may be demonstrated and suspect devices be identified and removed from the market expeditiously, may provide a better solution to this problem. Although this approach would certainly add cost, mandatory post-marketing surveillance will continue to promote technological innovation and device profitability while ensuring patient safety, and provide a more reasonable alternative to mandatory expansive comparator-controlled pre-marketing requirements.