Chapter 1: Studying a Study: M.A.A.R.I.E. Framework—Method, Assignment, Assessment

Intro

Introduction

Four basic types of investigations which compare groups of people are found in the health research literature ([1],[2]):

  • population comparisons or ecological studies
  • case-control studies or retrospective studies
  • cohort studies or prospective studies
  • randomized controlled trials or randomized clinical trials

Each type of investigation attempts to address a defined question or hypothesis by comparing one or more study groups with one or more control groups.1.1

An organizing framework can be used to evaluate each of these types of investigation. The framework is divided into six components:

  • Method
  • Assignment
  • Assessment
  • Results
  • Interpretation
  • Extrapolation

We call this the M.A.A.R.I.E. framework, an acronym using the first letter of each component: Method, Assignment, Assessment, Results, Interpretation, and Extrapolation. Figure 1.1 outlines the general application of the framework to a research study.

FIGURE 1.1. M.A.A.R.I.E. framework for studying a study.

M.A.A.R.I.E. framework for studying a study.

1.1 The investigations discussed in the “Studying a Study” section are sometimes called analytical studies. Analytical studies compare one or more study groups with one or more control groups. However, investigations do not always have control groups. Descriptive studies obtain data on a group of individuals without comparing them to another group. Sometimes descriptive studies may use data external to the investigation to compare a group in the investigation with other groups or to the same group at an earlier period. These comparison groups are sometimes called historical controls.

Method Questions

Method issues are common to all types of health research. They require the investigators to clarify exactly what they are attempting to achieve by defining what they will investigate, who they will investigate, and how many they will investigate. Each of the six components in the M.A.A.R.I.E. framework can be divided into three specific issues. For method, the issues and key questions are as follows:

  • Study hypothesis: What is the study question being investigated?
  • Study population: What population is being investigated including the inclusion and exclusion criteria for the participants in the investigation?
  • Sample size and statistical power: How many individuals are included in the study and in the control groups? Are the numbers adequate to demonstrate statistical significance if the study hypothesis is true?

Before investigators can decide which and how many individuals to include in an investigation, they need to define the study hypothesis. Then they can focus on the question of which individuals from which populations should be included in the investigation.

Health research is not generally conducted by including everyone in the population of interest. Rather, it is performed using only a smaller group, or sample, of all individuals who could in theory be included. For all types of health research, choosing whom to include and how many to include in an investigation are basic method issues. Thus, Method, the first component of the M.A.A.R.I.E. framework, defines the study question and sets the rules for obtaining the study and control samples.

In recent years, ethical issues have been of increasing concern. Much of the attention on research issues in health research has focused on the methods used in randomized controlled trials. However, a number of issues apply to all forms of health research. As part of the methods component it is important to ask the question: Who is the investigator(s) and who is the funder(s) of the research? Potential conflicts of interest are now handled using formal procedures that are based on the belief that full disclosure of potential conflicts is the appropriate strategy for addressing these issues. Thus in high quality journals expect to see extensive information on the potential conflicts of interest of the investigators, the roles that each investigator played in the research, and a prominently placed identification of the funder(s).

The M.A.A.R.I.E. framework continues with the following additional components:

Assignment: Allocation of participants to study and control groups

Assessment: Measurement of outcome(s) or end point(s) in the study and control groups

Results: Comparison of the outcome in the study and control groups

Interpretation: Meaning of the results for those included in the investigation

Extrapolation: Meaning of the results for those not included in the investigation

To illustrate the application of the M.A.A.R.I.E. framework to population comparisons, case-control studies, cohort, and randomized controlled trials, let us outline the essential features of each type of study. We will then see how we can apply each type of investigation to the question of the potential risk of stroke with birth control pill use. The implications of these components of the M.A.A.R.I.E. framework differ slightly according to the type of investigation, as we discuss later in this chapter.

We will discuss each type of investigation by assuming that there is one study group and one control group. However, in all types of studies, more than one study group and more than one control group can be included.

Applying The M.A.A.R.I.E. Framework

Population Comparisons

The unique feature of population comparisons or ecological studies is their ability to suggest relationship between risk factors and diseases or other outcomes without having information on any one individual. Population comparisons are designed to compare the rates of events in two or more populations or to investigate changes that have occurred in the same population over a period of time. Alternatively, population comparisons may examine differences between two or more populations at the same point in time (Fig. 1.2).

FIGURE 1.2. M.A.A.R.I.E. framework for a population comparison.

M.A.A.R.I.E. framework for a population comparison.

Population comparisons compare rates in two or more populations without having available information on particular individuals. Population comparisons typically observe the rates of a disease or other outcome and the rates of a risk factor or other characteristic. They often ask whether populations with higher rates of the risk factor also have higher rates of the disease.

To examine the relationship between the use of birth control pills and stroke in young women, the investigator using a population comparison might proceed as follows:

Assignment: Select a study population and measure its rate of strokes among young women and a similar comparison control population and measure its rate of strokes among young women

Assessment: Determine the rate of use of birth control pills among young women in the study population and in the control population

Results: Compare the rates of use of birth control pills with the rates of strokes among young women in the study population and in the control population

Interpretation: Draw conclusions about the meaning of birth control pill use for women included in the investigation

Extrapolation: Draw conclusions about the meaning of birth control pill use for women not like those included in the investigation, such as women who have the option to use newer low-dose birth control pills.

When populations of young women with high rates of strokes also have high rates of use of birth control pills compared with populations of young women with low rates of strokes we say that a group association exists. That is, even though we do not know whether the particular women who developed stroke actually used birth control pills, we can conclude that an association exists at the group or population level. Identifying group associations are often the first step in demonstrating a cause and effect relationship, but as we will see group associations often merely suggest hypotheses for further investigation.

Case–Control Study

The unique feature of case-control studies of disease is that they begin by identifying individuals who have developed or failed to develop the disease or condition being investigated. After identifying those with and without the disease, they look back in time to determine the characteristics of individuals before the onset of disease. In case-control studies, the cases are the individuals who have developed the disease, and the controls are the individuals who have not developed the disease. To use a case-control study to examine the relationship between birth control pill use and stroke in young women, an investigator might proceed as follows:

Assignment: Select a study group of young women who have had a stroke (cases) and a group of otherwise similar young women who have not had a stroke (controls). Because the development of the disease has occurred without the investigator’s intervention, this process can be called observed assignment.

Assessment: Determine whether each woman in the case or study group and also in the control group previously took birth control pills. The previous presence or absence of the use of birth control pills is the outcome in a case-control study.

Results: Calculate the chances that the group of women with a stroke had used birth control pills versus the chances that the group of women without stroke had used birth control pills.

Interpretation: Draw conclusions about the meaning of birth control pill use for women included in the investigation.

Extrapolation: Draw conclusions about the meaning of birth control pill use for categories of women not like those included in the investigation, such as women on newer low-dose birth control pills.

Figure 1.3 illustrates the application of the M.A.A.R.I.E. framework to this investigation. Notice that case-control studies unlike population comparisons identify individuals and ask whether there is an association at the individual level between young women with strokes and the use of birth control pills. Thus case-control studies are capable of establishing what we will call an individual association.

FIGURE 1.3. Application of the M.A.A.R.I.E. framework to a case–control study.

Application of the M.A.A.R.I.E. framework to a case–control study.

Cohort Study

Cohort studies of disease differ from case-control studies in that they begin by identifying individuals for study and control groups before the investigator is aware of whether they have developed the disease or other outcome. A cohort is a group of individuals who share a common experience. A cohort study begins by identifying a cohort that possesses the characteristics under study as well as a cohort that does not possess those characteristics. Then the frequency of developing the disease in each of the cohorts is obtained and compared. To use a cohort study to examine the relationship between birth control pill use and stroke, an investigator might proceed as follows:

Assignment: Select a study group of women who are using birth control pills and an otherwise similar control group of women who have never used birth control pills. Because the use of birth control pills is observed to occur without the investigator’s intervention, this process is also called observed assignment.

Assessment: Determine who in the study group and the control group develops strokes. As opposed to a case-control study, the outcome for a cohort study is the subsequent presence or absence of a stroke.

Results: Calculate the chances of developing a stroke for women using birth control pills versus women not using birth control pills.

Interpretation: Draw conclusions about the meaning of birth control pill use for women included in the study.

Extrapolation: Draw conclusions about the meaning of birth control pill use for women not included in the study, such as women on newer low-dose birth control pills.

Figure 1.4 illustrates the application of the M.A.A.R.I.E. framework to a cohort study.

FIGURE 1.4. Application of the M.A.A.R.I.E. framework to a cohort study.

Application of the M.A.A.R.I.E. framework to a cohort study.

Randomized Controlled Trial

Randomized controlled trials are also called randomized clinical trials. They are a form of experimental study. As in cohort studies, individuals are assigned to study and control groups before determining who develops the disease or other outcome. The unique feature of randomized controlled trials, however, is the process for assigning individuals to study and control groups. In a randomized controlled trial, participants are randomized either to a study group or to a control group.

Randomization means that chance is used to assign a person to either the study or the control group. This is done so that any one individual has a known, but not necessarily equal, probability of being assigned to the study group or the control group. Ideally, the study participants as well as the investigators are not aware of which participants are in which group. Double-blind assignment means that neither the participant nor the investigators know whether the participant has been assigned to the study group or the control group. Single blinding implies that the participants are unaware of their group assignments.

To use a randomized controlled trial to examine the relationship between birth control pill use and stroke, an investigator might proceed as follows:

Assignment: Using randomization, women are assigned in a double-blind fashion to a study group that will be prescribed birth control pills or to a control group that will not be prescribed birth control pills.

Assessment: Observe these women to determine who subsequently develops stroke. As in a cohort study, in a randomized controlled trial the outcome is the presence or absence of stroke.

Results: Calculate the chances that women using birth control pills will develop a stroke versus women not using birth control pills.

Interpretation: Draw conclusions about the meaning of birth control pill use for women included in the study.

Extrapolation: Draw conclusions about the meaning of birth control pill use for women not included in the study, such as women on new low-dose birth control pills.

Figure 1.5 illustrates the application of the M.A.A.R.I.E. framework to a randomized controlled trial.

FIGURE 1.5. Application of the M.A.A.R.I.E. framework to a randomized controlled trial.

Application of the M.A.A.R.I.E. framework to a randomized controlled trial.