Chapter 8: Examining Evidence


The preceding chapters considered the processes by which diagnoses are made and described approaches to understanding the tradeoffs between the risks and benefits of tests and treatments. However, process is never sufficient to deal with clinical medicine, even the simplest cases. There is no substitute for marrying effective clinical decision making with hard facts about disease entities and the characteristics of diagnostic tests, therapeutic agents, medical devices, and surgical treatments.

In the past, there was a tendency to rely on a physician’s recall of facts from his or her intuition or experience and their application to the decision at hand, but in an effort to reduce the remarkable variation in clinical practice between one physician and another and between physicians in different parts of the country, a new standard of factual use has emerged, often called “evidence-based medicine.”

Because our goal in this book is to elaborate on the processes underlying diagnosis and clinical management, a detailed exposition on evidence-based medicine is not appropriate, but the student or resident beginning to understand these processes must also have a framework of how the facts and the processes intersect.

Evidence-Based Medicine

( Case 23, Case 48, Case 49– Case 50)

The intention of this discipline is to replace intuition, unsystematic clinical experience, and pathophysiologic rationale with solid, experimental-based evidence for clinical decision making.[ 62 ] Evidence-based medicine can be defined as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research.”[ 63] To practice evidence-based medicine involves formulating clinical questions explicitly and searching for evidence that might bear on the decision; summarizing and appraising the evidence; and applying the evidence for the patient’s benefit.

Asking Questions

The process starts with a patient and his or her clinical problem, leading to the formulation of a critically important question that affects clinical decision making. Questions such as “Should my patient receive long-term anticoagulation for a single episode of pulmonary embolism?” or “Should my patient have back surgery for severe pain thought to be caused by spinal stenosis?” or “Should I recommend that my patient undergo bronchoscopy for a lung lesion?” cannot be answered without detailed information about the risks and benefits of anticoagulation, the outcomes of spinal surgery, and the chance of having cancer and risks of bronchoscopy, respectively.

Evidence-based medicine makes a distinction between foreground and background questions. Background questions involve general knowledge about a condition, such as etiology, manifestations and treatment—for example, who develops hypertension, what are the symptoms of hypertension, and how does hypertension arise? Foreground questions require specific knowledge necessary for the treatment of individual patients; for example, in 60-year-olds with mild hypertension (systolic blood pressure 140 to 179 mm Hg), are angiotensin-converting-enzyme inhibitors more effective than beta-blockers in minimizing cardiovascular events (or adverse effects, e.g., developing diabetes)? If so, how much more effective? The formulation of the question involves four components, often represented by the acronym PICO:

Patient: Describe the patient or population.

Intervention: Describe what you want to do.

Comparison: Describe what are you comparing with the intervention.

Outcome: Describe the effect that you want to examine.

Common categories of clinical questions involve diagnosis, screening, etiology, prognosis, harms (potential risk or adverse effects), and economics.

Searching for Evidence

It is often surprising to beginning students that the evidence base for many common clinical problems has many deficiencies. In fact, studies are always underway to fill in the gaps in our knowledge base, as well to keep it current. Nonetheless, the strength of evidence varies considerably from field to field. To standardize the classification of evidence, a scheme has evolved to define the strength of medical evidence supporting a therapy or strategy as hierarchical, a fundamental principle of evidence-based medicine.[ 64] The highest level of evidence is the so-called “N-of-1 randomized, controlled trial,” in which a single unique patient is randomized to placebo or intervention, such as two identically appearing pills for blood pressure.[ 65]

Because randomized, controlled drug trials yield only an average observed effect, some individuals in the study population will have a substantial benefit and others will have no effect. The N-of-1 trial clarifies definitively whether a particular individual with his or her unique genetic makeup and dietary and lifestyle habits will indeed respond, for example, to a particular blood pressure medication. The next highest level of evidence is a systematic review of randomized trials of patients with a single condition, evaluating not only repeatability of the benefits and risks, but generalizability in different patient populations and clinical settings.

Next in the hierarchy is a single randomized trial. Subsequent levels of evidence include a systematic review of observational studies, followed by a single observational study. Of course, heed must be taken of the potential biases of observational studies. Physiologic studies and unsystematic clinical observations comprise the lowest levels of evidence.

Search efforts begin by seeking the highest possible level of evidence. In this information age, an abundance of potential sources exist, in particular on the Internet. With regard simply to free public Web sites that provide dedicated medical information, the National Library of Medicine and the National Institutes of Health–sponsored PubMed ( including), which has more than 17 million citations dating back to the 1950s, and the Agency for Healthcare and Quality Guideline Clearinghouse ( deserve mention. Numerous tutorials now exist online and in print to assist with literature searching.[ 66],[ 67] Increasingly, search engines in the public domain, such as Google and Google Scholar, are valuable sites for starting an evidence-based information search.

Summarizing and Appraising Evidence

Summarizing and appraising the evidence combines clinical expertise with knowledge in epidemiology, statistics, and clinical trial design. Summarizing the study involves determining the study type in the hierarchy of evidence, patient population, inclusion and exclusion criteria, statistical methods, and outcomes. The appraisal criteria typically consist of a series of structured questions specific to the type of study being evaluated. For example, when examining a randomized, controlled trial involving therapy, key questions to assess the validity of the study include the following: (1) Was treatment assignment randomized? (2)

Were all patients enrolled in the trial accounted for and their final outcome assigned? (3) Was follow-up sufficiently long and complete? (4) Were investigators blinded to treatment assignment? (5) Were the patient groups similar at the start? and (6) Were groups treated equally except for therapy? Many journal articles and books provide help in appraising the quality of any study.[ 64],[ 68], [ 69], [ 70]–[ 71]

Applying the Evidence

The medical literature provides evidence of varying quality, but because of the need for researchers to keep the number of variables to a minimum, the data in an individual study may or may not be representative of any particular patient. Compounding this application of evidence from one group of patients to individual patients is that analysis of subgroups of a study is more likely to yield spurious results in the absence of replication or statistical adjustment for multiple comparisons.[ 72 ] Moreover, even when results are considered “statistically significant,” the physician must assess whether any difference is clinically significant: Small significant differences between one treatment and another may not matter in decision making (see Chapter 7).

Finally, every patient is unique in his or her preferences for risk and the values that he or she places on medical outcomes consequent on any testing and therapeutic decisions under consideration. Thus, another fundamental principle for evidence-based medicine is that “Evidence is never enough for clinical decision making.”[ 64] The ultimate goal for clinical reasoning is to use evidence-based practice and integrate the best research with clinical expertise and patient values for optimum care.

Practice Guidelines

Clinical practice guidelines are compiled recommendations issued by professional organizations and are based largely on published data. They are generally intended as an aid to therapeutic decision making once a working diagnosis has been achieved. Because guidelines must be somewhat general, it may be difficult to extract a recommendation for an individual patient, and some extrapolation may be required. Guideline developers attempt to use systematic and explicit methods to encourage their transparency and acceptance. Most guidelines separate the strength of the recommendation regarding benefits versus harms from the certainty of the benefit or precision regarding the quality of the evidence supporting the recommendation.