Randomized controlled trials are the gold standard for efficacy so we might ask the question: What is the gold standard for safety? Unfortunately, the answer is not so easy since there is no one type of study that can provide us with all the evidence we need. We might think of the gold standard for safety as many years of use by millions of people accompanied by systematic monitoring for adverse events. Although this may be the ideal standard, it presents us with a catch-22. Safety cannot be established without first subjecting millions of people to potential harm.
To understand safety, we need to recognize that most interventions produce harms as well as benefits. The goal of measuring safety is to be sure that the harms are minimized and that the benefits are clearly greater than the harms. The generic term “acceptable harm” is increasingly being used instead of “safe” since the term safe may provide a false sense of security that no undesirable outcomes will occur. Safety also may imply that we understand the relationship between our interventions and any harms that occur after an intervention. The term adverse events is increasingly being used instead of side effects or adverse effects to avoid the implication of a clear-cut cause-and-effect relationship since cause-and-effect relationships are especially difficult to establish for harms.
Because of the complexity of the investigation of harm, we need to take a different approach than that used to understand the benefits of interventions. The investigation of harms requires the use of multiple types of studies. The emphasis is not on one type of investigation but on how a range of investigations fit together to provide evidence about potential harms. Thus, we need a multipart systematic approach to evaluating safety recognizing that it will not be perfect.
The U.S. Food and Drug Administration (FDA) has developed and recently expanded its approach to assessing adverse events associated with drugs. This framework is useful in understanding the process and in integrating the various types of research studies that contribute to our understanding of harms. Although developed and used for drug approval and monitoring, it has been extended to use for vaccines and increasingly for other types of interventions. The FDA has begun to utilize what is called a systems approach to drug safety. Learn More 6.1 introduces the concept of systems approaches and illustrates its successful use in airline safety.
Learn More 6.1: Systems Approaches—Airline Safety and Drug Safety
When examining the role of the FDA in preventing harm, it can be helpful to think about the harms from health interventions as parallel to the safety issues faced by the airline industry. What is known as a systems approach to commercial airline safety has been credited with reducing deaths despite the rapidly increasing number of passenger miles flown each year. A systems approach examines the multiple influences on a problem and the interactions of these influences, identifies bottlenecks and leverage points, and looks for changes over time.
Despite the potential harms of airline travel, a complex system of prevention, protection, and response is now in place. The government and the airline industry have developed a complex systematic approach to equipment testing, routine maintenance, training of personnel, protective equipment, and thorough investigation of not only crashes but near misses as well. This has been achieved by examining all the known influences on safety, looking for bottlenecks and leverage points in the process, and performing continuing monitoring to detect new risks and the return of previously known hazards.
Systems thinking or system approaches are based on an integrative approach to evidence. An integrative approach implies that evidence from a variety of sources is brought together to address decision-making and problem-solving issues. In an integrative approach, the interaction between factors is a major focus. This approach contrasts with reductionist approach more frequently used in health research. In a reductionist approach, the investigator focuses on one factor at a time.
Data are collected on other factors in large part to take these potential confounding factors into account, to control, or to adjust for them. Thus, most of the specific types of investigations that we have discussed take a reductionist approach. However, once we step back and attempt to put together the evidence, we often need to take an integrative or systems approach.
No comprehensive systems approach to safety is in place in health care. As we proceed to think through the issues raised by safety, the airline industry’s systems approach provides an illustrative model for what is possible.6.1
Our look at harms will focus on drugs and vaccines, but adverse events are possible with all interventions whether they are called surgery, medical devices, or dietary supplements. Surgery, medical devices, and, especially, dietary supplement generally undergo less investigation before their use in clinical practice than drugs and vaccines.
Herbals, vitamins, minerals, and other substances that are classified as dietary supplements do not require demonstrations of efficacy or acceptable harm before being marketed and sold. In fact, marketing and use of dietary supplements do not require FDA approval. Dietary supplements may not be advertised for the treatment or prevention of disease, but they can be and are advertised for use to alter anatomy and physiology from weight to blood pressure to energy to mood. In monitoring for adverse events associated with dietary supplements, the burden is on the FDA to demonstrate that important adverse events occur.
The system for evaluating drugs and vaccines, as we will see, is generally more tightly regulated than surgery, medical devices, or dietary supplements. However, the surveillance system for medical devices is the only one that requires institutions to report the occurrence of all potentially life-threatening failures. Expect to see changes in the regulation of medical devices in coming years.
The system for drug and vaccine monitoring for harms has evolved over the last 100 years reflecting the history of the FDA. Changes have often been instituted after highly publicized tragedies and epidemics caused by drugs or vaccines and therapeutic devices. Learn More 6.2 reviews key elements of this history.
Learn More 6.2: Safety and The Fda (1)
The FDA was established in 1906. Its authority over drugs was initially very limited. It had no authority to require testing for efficacy, and it had very limited authority over safety mostly confined to requiring disclosure of the ingredients. The authority of the FDA over drugs, vaccines, and medical devices has grown enormously since 1906. The following important events led to changes in the FDA law:
- In 1937, an outbreak of kidney failure in children occurred after exposed to a dangerous preparation of the first antibiotics, sulfa. The FDA law was amended to require premarket testing for safety.
- In the mid-1950, distribution of early live polio vaccine produced polio-like illness. The FDA law was amended to establish standards for vaccines.
- The early 1960s thalidomide case in which thousands of children were born especially in Europe with dramatically shortened limbs. Despite the fact that the United States was spared the tragedy of thalidomide, the FDA law was amended to greatly strengthen the standards for efficacy testing for the first time integrating randomized controlled trials into the process.
- The 1970s widespread use of the Dalcon Shield IUD produced an epidemic of tubal infection, infertility, and ectopic pregnancies. The FDA law was amended to provide authority for medical device regulation.
- In the first decade of the 21st century, the cardiac risks of Cox-II inhibitors were acknowledged only after they were used for years by many millions of people.
In response to this latest tragedy, Congress gave the FDA extensive new authority to take a systems approach, including a wide range of new tools for postmarket monitoring, researching, labeling, and controlling the use of drugs
6.1 Systems approaches are being used to address food safety. The 2010 U.S. federal food safety legislation is built on principles of systems thinking. Motor vehicle injuries have also been addressed using a systems approach.
FDA Phases ()
The FDA’s regulation of prescription drugs is organized into the following phases:
- Prehuman animal and laboratory testing
- Preapproval-Phase 1
- Preapproval-Phase 2
- Preapproval-Phase 3
- Postapproval-Phase 4
Let us take a look at the process and limitation of what takes place in each of these phases. Table 6.1 summarizes the key information related to each phase.6.2
TABLE 6.1.FDA Phase of Evaluation of Prescription Drugs
|Prehuman animal and laboratory testing||Evaluation for harms on at least two species at high doses before initial use on humans
New approaches being developed
|Assess carcinogenic, teratogenic, and fertility effects||High dose effects may not correlate with effects on humans
Species differences may result in missing effects that later appear in human testing or after widespread clinical use
|Phase 1||Initial testing of drug on humans may include healthy volunteers or terminally ill patients but not generally those on whom drug will be used in clinical practice
Small studies, often several studies including 5–10 persons each
|Designed to assess pharmacology including metabolism and excretion in effort to establish dose, timing, and route of administration
Evaluation for adverse events especially on vulnerable organs including liver, kidney, and bone marrow
|Small numbers mean many adverse events may be missed
When includes patients not representative of those on whom the drug will be used, may not help predict adverse events
|Phase 2||Initial small-scale controlled or uncontrolled trials of efficacy with secondary assessment of harms
Size designed to estimate efficacy often 20–200
|Designed to establish that there is enough evidence of efficacy to warrant phase 3 randomized controlled trials.
Focus not primarily on harms
|Primary intent is assessment of efficacy. Small numbers and less than comprehensive assessment of harms often limit ability to draw conclusion about potential harms|
|Phase 3||At least two independently conducted randomized controlled trials required unless not practical or ethical
Size designed to demonstrate efficacy usually ranging from 100 in each group to 3,000 in each group
|Designed to establish efficacy in the shortest possible time for one indication for a group defined by uniform inclusion and exclusion criteria. Study group compared with conventional treatment
Investigate short-term harms relative to conventional treatment
|Uniform characteristics of the participants may result in study and control groups with only one disease who are taking only one medication.
Randomized controlled trials often too small, too short, and too simple to detect rare but serious adverse events
|Phase 4||Postmarket surveillance
A system to monitor interventions such as drugs and vaccines after approval
Potentially uses multiple types of data and multiple types of follow-up investigations including surveillance systems, meta-analysis, retrospective cohort studies, and case–control studies
May also include limitations on authority to prescribe and/or required testing or monitoring before prescribing or refilling prescription
|Once a drug is used in practice, adverse events may be more frequent than seen before approval because of the use on patients with multiple or more severe disease and/or the use of multiple treatments
Spontaneous reports to FDA from clinicians and patients form the basis of current postmarket surveillance.
There is currently no required reporting of even serious adverse events or universal required follow-up of patients previously treated as part of a randomized controlled trial
Meta-analysis, case–control studies, and retrospective cohort studies from clinical practice are increasingly being used to evaluate potential harms
|Spontaneous reporting system results may not detect adverse events especially if they are not dramatic, do not occur soon after the intervention, require special testing to detect their presence, are clinically unexpected, and/or are similar to the effects of the disease being treated.|
It is useful to categorize these phases as a preapproval or premarket process followed by a postmarket process that occurs after FDA approval. The first four components through phase 3 can be viewed as premarket; that is, they occur before FDA approval to market the drug. Phase 4 is often called postmarket since it collects data on adverse events after the approval of the drug. Let us walk through these phases to better understand what is involved, how they relate to each other, and why there are still holes in this “safety net.” The process we will describe is a general or generic process that may be modified for specific new drugs or categories of drugs or vaccines.6.3
Prehuman animal and laboratory testing
Before studying a drug on humans, the FDA requires that animal and laboratory studies be performed. Animal testing is usually administered to two different species at levels well above the weight equivalent dosage expected to be used in humans. Studies are done primarily to detect cancer, birth defects, and effects on fertility. However, toxicity to drug-sensitive organs such as the liver, kidneys, and bone marrow is also investigated. Unfortunately, the effects on animals may fail to either detect subsequent effects in humans or demonstrate high dose effects that are difficult to interpret as illustrated in the following hypothetical example:
A new drug is studied in two animal species. In one species, there was a slightly greater frequency of thyroid cancer seen at doses several times higher than those expected to be used in humans; otherwise, the animal testing did not suggest the potential for harm. When the drug was studied in humans, tests revealed rare but dramatic birth defects but no evidence of cancer.
Humans may absorb, metabolize, and excrete drugs differently than animal species. High dose effects on one animal species alert us to the possibility of similar effects in humans but by no means guarantees their occurrence. In addition, as we will see repeatedly, humans have an enormous range of reactions to drugs even when given by the same route with the dosage adjusted for body weight. Thus, it should not be surprising that animal testing can easily miss the rare but serious adverse events experienced by humans.6.4
This phase of drug testing is undergoing revision and hopefully improvement. Advances in our understanding of the mechanism of drug action on a molecular basis may in the future allow us to do a better job of understanding what, where, and how a drug is acting, allowing researchers to focus in on the impacts on a molecular and cellular level. This may require a reexamination of the role of animal testing.
6.2 The conclusions of this process are generally not published in journals, but are summarized in the official FDA product descriptions that are widely available to clinicians through the Physicians’ Desk Reference.
6.3 The FDA treats prescription drugs and nonprescription or over-the-counter drugs as two different categories with different requirements and procedures for approval. The FDA’s expectations for approval of a drug for over-the-counter use go beyond the requirements discussed here for approval of a prescription drug. The FDA outlines the following five criteria for the approval of a drug for nonprescription use:  their benefits outweigh their risks,  the potential for misuse and abuse is low,  consumer can use them for self-diagnosed conditions,  they can be adequately labeled, and  health practitioners are not needed for the safe and effective use of the product.
6.4 Human testing generally awaits the results of initial animal testing, although long-term animal test may continue while initial human testing is underway. Animal testing may also include a measure known as LD50 or the dose that is required to kill 50% of the animals tested at that dose. This data provide information on the potential toxicity of overdoses of the medication