Chapter 3: Studying a Study: M.A.A.R.I.E. Framework—Interpretation, Extrapolation

Intro

Interpretation

Interpretation asks us to address questions about the meaning of the investigation’s results for those who have participated in the investigation. There are three types of questions that can be addressed by interpretation.

  • Contributory cause or efficacy: Does the factor being investigated alter the probability that the disease will occur (contributory cause) or work to reduce the probability of an undesirable outcome (efficacy)?
  • Harms: Are adverse events that affect the meaning of the results identified?
  • Subgroups and interactions: Do the outcomes in subgroups differ and are there interactions between factors that affect outcome?

Questions of contributory cause or efficacy are the first questions that are addressed by interpretation, and at times may be the only questions. Questions of adverse outcomes and questions about subgroups may only be important when there is evidence for contributory cause or efficacy. Therefore, we will take a close look at the issues of contributory cause and efficacy and then outline key concepts for understanding adverse outcomes and subgroups.

Contributory Cause or Efficacy [(1],[2)]

In Chapter 1, we introduced a definition of cause and effect termed contributory cause. This same definition is used to establish efficacy. To definitively establish the existence of a contributory cause or efficacy, all three of the following criteria must be fulfilled:

  1. Association: Does the investigation establish a statistically significant and substantial association that provides convincing evidence that individuals with the “cause” also have an increased probability of experiencing the “effect”?
  2. Prior association: Does the investigation establish that the “cause” precedes the “effect”?
  3. Altering the cause alters the effect: Does the investigation establish that altering or modifying the frequency or severity of the “cause” alters the frequency or severity of the disease or other “effect”?

Figure 3.1 again shows the overall framework for using specific types of investigations to establish these criteria.

FIGURE 3.1. Using multiple types of studies to establish criteria for contributory cause and for efficacy.

Using multiple types of studies to establish criteria for contributory cause and for efficacy.

Association

Establishing the first criterion of contributory cause, association at the individual level requires that we examine the magnitude and the statistical significance of the relationship established in the analysis of results. To establish the existence of an impressive association, we expect a statistically significant and substantial relationship.

Remember, statistical significance testing is designed to help us assess the role of chance when we observe a difference or an association in any of the forms of investigation that we have examined. Thus, the evidence provided in the results section is the basis for determining that an association exists between those with the factor and those with the outcome under investigation.

“Cause” Precedes the “Effect”

To establish the second and third criteria, we must rely on more than statistical analysis. It may appear to be simple to establish that a cause precedes a disease, but let us look at two hypothetical studies in which the authors may have been fooled into believing that they had established that the cause preceding the effect:


Mini-Study 3.1

Two investigators conducted a case–control study to determine whether antacids were taken by patients with myocardial infarction (MI) the week preceding an MI. They were looking for causes of the condition. MI patients were compared with patients admitted for elective surgery. The authors found that the MI patients were 10 times as likely to have taken antacids as the controls were during the week preceding admission. The authors concluded that taking antacids is associated with subsequent MIs.


The authors believed that they established not only the first criterion of causation (an association at the individual level) but also the second criterion (that the cause precedes the effect).

But did they? If individuals have angina before MIs, they may misinterpret the pain and try to alleviate it by self-medicating with antacids. Therefore, the medication is taken to treat the disease and does not truly precede the disease. This study failed to establish that the cause precedes the effect because it did not clarify whether the disease led the patients to take the medication or whether the medication led to the disease. This example illustrates what is called reverse causality.

It illustrates the potential difficulty encountered in separating cause and effect in case–control studies. At times case–control studies, however, may be capable of providing convincing evidence that the cause precedes the effect. This occurs when there is good documentation of previous characteristics that are not affected by knowledge of occurrence of the disease. Alternatively, we may believe that the characteristic of an individual is not likely to change over time such as the presence of a gene.

Cohort studies often have an advantage in establishing that the possible cause occurs before the effect. The following example, however, illustrates that even in cohort studies we may encounter reverse causality:


Mini-Study 3.2

A group of 1,000 patients who had stopped smoking cigarettes within the last month were compared with 1,000 current cigarette smokers matched for total pack-years of smoking. The two groups were monitored for 6 months to determine with what frequency they developed lung cancer. The study showed that 5% of the study group who had stopped smoking cigarettes was diagnosed with lung cancer as opposed to only 0.1% of the currently smoking controls. The authors concluded that stopping cigarette smoking was associated with the subsequent development of lung cancer. Therefore, they advised current smokers to continue smoking.


The cessation of cigarette smoking appears to occur before the development of lung cancer, but what if smokers stop smoking because of symptoms produced by lung cancer? If this was true, then lung cancer stops smoking, and not vice versa. Thus, one must be careful in accepting that the hypothesized cause precedes the effect. The ability of cohort studies to establish that the cause precedes the effect is enhanced when the time lapse between cause and effect relative to the natural history of the disease is longer than in this example. Short time intervals still leave open the possibility that the presumed cause has been influenced by the presumed effect instead of the reverse.

Altering the “Cause” Alters the “Effect”

Even if one has firmly established that the possible cause precedes the effect, to completely fulfill the criteria for contributory cause, it is necessary to establish that altering the cause alters the probability of the effect. This criterion can be established by performing an intervention study in which the investigator alters the cause and determines whether this subsequently contributes to altering the probability of the effect. Ideally, this criterion is fulfilled by performing a randomized controlled trial. As we will discuss in Chapter 5, randomized controlled trials may not be ethical or practical, thus we need to examine other ways to establish the definitive criteria including altering the cause alters the effect.3.1

When contributory cause cannot be definitively established, we may need to make our best judgments about the existence of a cause-and-effect relationship. For this situation, a series what have been called ancillary, adjunct, or supportive criteria for contributory cause can be used. These include the following:

  1. Strength of association. A strong association between the risk factor and the disease as measured, for example, by a large relative risk.
  2. Consistency of association. Consistency is present when investigations performed in different settings on different types of patients produce similar results.
  3. Biological plausibility. Biological plausibility implies that a known biological mechanism is capable of explaining the relationship between the cause and the effect.3.2
  4. A dose-response relationship. A dose-response relationship implies that changes in levels of exposure to the risk factor are associated with changes in the frequency of disease in a consistent direction.

Data that support each of these four criteria help bolster the argument that a factor is actually a contributory cause. When these criteria are fulfilled, it reduces the likelihood that the observed association is due to chance or bias. These criteria, however, do not definitively establish the existence of a contributory cause.

In addition, none of these four criteria for contributory cause are essential. A risk factor with a modest but real association may in fact be one of a series of contributory causes for a disease. Consistency is not essential because it is possible for a risk factor to operate in one community but not in another. This may occur because of the existence in one community of other prerequisite conditions. Biological plausibility assumes that we understand the relevant biological processes. Finally, demonstrating a dose-hypersensitivity reactions may result from exposure to even a small amount of an agent.

Larger exposures may not result in greater reactions. Even when a dose-response relationship is present, it usually exists only over a limited range of values. For cigarettes and lung cancer, one or two cigarettes per day may not measurably increase the probability of lung cancer, and the difference between three and four packs per day may not be detectable. Dose-response relationships may be confusing, as illustrated in the next example:


Mini-Study 3.3

An investigator conducted a cohort study of the association between radiation and thyroid cancer. He found that low-dose radiation had a relative risk of five of being associated with thyroid cancer. He found that at moderate levels of radiation, the relative risk was 10, but at high levels, the relative risk was 1. The investigator concluded that radiation could not cause thyroid cancer because no dose–response relationship of more cancer with more radiation was demonstrated.


The relative risk of 10 is an impressive association between radiation and thyroid cancer. This should not be dismissed merely because the relative risk is diminished at higher doses. It is possible that low-dose and moderate-dose radiation contributes to thyroid cancer, whereas large doses of radiation actually kill cells and thus do not contribute to thyroid cancer.

For many biological relationships, a little exposure may have little measurable effect. At higher doses, the effect may increase rapidly with increases in dose. At still higher doses, there may be little increase in effect. Thus, the presence of a dose–response relationship may depend on which part of the curve is being studied.3.3

These ancillary, adjunct, or supportive criteria for judging contributory cause are just that: They do not in and of themselves settle the issue. If present, they may help support the argument for contributory cause. These criteria help in understanding issues raised in a controversy and the limitations of the data.

3.1 It is important to recognize that contributory cause is an empirical definition. It does not require an understanding of the intermediate mechanism by which the contributory cause triggers the effect. Historically, numerous instances have occurred in which actions based on a demonstration of contributory cause reduced disease despite the absence of a scientific understanding of how the result actually occurred. Puerperal fever was potentially controlled through hand washing before the bacterial agents were recognized. Malaria was controlled by swamp clearance before its mosquito transmission was recognized. Scurvy was prevented by citrus fruit before the British ever heard of vitamin C. Once we understand more about the direct mechanisms that produce disease, we are able to distinguish between indirect and direct contributory causes. What we call a direct cause of disease depends on the current state of knowledge and understanding of disease mechanism. Thus, over time, many direct causes may come to be regarded as indirect causes. In addition, it is important to distinguish these terms from the legal concept of proximal cause. Proximal cause refers to the timing of actions that could prevent a particular outcome and should not be confused with the definition of causation used here.

3.2 The biological plausibility of the relationship is evaluated on the basis of clinical or basic science principles and knowledge. For instance, hypertension is a biologically plausible contributory cause of strokes, coronary artery disease, and renal disease because the mechanism for damage is known and the type of damage is consistent with that mechanism. However, data suggesting a relationship between hypertension and cancer would not be biologically plausible, at least on the basis of current knowledge. Biological plausibility also implies that the timing and magnitude of the cause are compatible with the occurrence of the effect. For instance, we assume that severe, long-standing hypertension is more likely to be a contributory cause of congestive heart failure or renal disease than mild hypertension of short duration.

3.3 Other ancillary criteria have been developed and used. Although not universally accepted, their presence may increase the probability that a contributory cause is present. These criteria include specificity (i.e., one “cause” produces one “effect”) and analogy (i.e., there are other well-established examples of similar relationships). As with the other ancillary criteria, these criteria are not necessary to establish a contributory cause. [(1)]