Advertisement

A Critical Evaluation of Clinical Research Study Designs

      INTRODUCTION

      Prior to starting any clinical research, an investigator must determine the appropriate study design to answer the question at hand. Selecting the correct study type also depends on ethical considerations, disease of interest, and the resources available. A well-designed study will clearly identify an exposure and an outcome in an objective, quantifiable manner to answer a defined hypothesis. After thorough data analysis and discussion of the results, the study will ideally prompt further research on the subject. Understanding the various indications for different study designs is important not only for devising one’s own study but also for critically reviewing the literature. This article outlines frequently encountered study designs in clinical research and their respective strengths and drawbacks (Table 1).
      Table 1Comparing study designs in clinical research
      Table thumbnail gr3
      Figure thumbnail fx1

      OVERVIEW

      In clinical research, studies can be classified as either interventional (experimental) or noninterventional (observational) studies. The National Institutes of Health defines an interventional study as one in which “participants receive specific interventions according to the research plan created by the investigators. These interventions may be medical products, such as drugs or devices; procedures; or changes to participants’ behavior” (http://www.clinicaltrials.gov). Subcategories include randomized controlled trials (RCTs) and clinical trials. This article addresses RCTs.
      In observational studies, there is no intervention; that is, subjects are observed and evaluated for exposures encountered as part of the natural course of their lives. This article will address the following types of studies: case–control, cohort, cross-sectional, and case reports/series.

      RANDOMIZED CONTROLLED TRIALS

      RCTs are considered the “gold standard” for evaluating a given therapy and its causal impact on an outcome. In an RCT, study subjects are randomly assigned to one of two groups: the treatment arm, which receives the therapy, or the control arm, which receives a placebo or no treatment. Both study arms are subsequently followed in an identical manner and analyzed for differences in outcomes.
      Unlike other studies, the intrinsic design of an RCT allows investigators to assess causality of a variable of interest, rather than simply a correlation. These studies generally have stringent selection criteria to ensure that subjects are comparable in most respects, thereby reducing confounding and isolating the effect of the intervention. Randomization ensures that any confounding factors are equally divided between groups. Furthermore, blinding reduces the likelihood that behaviors of subjects or investigators could influence the results of the study. In a single-blinded study, subjects are unaware of their treatment status, whereas in a double-blinded study both the investigators and the subjects are unaware of which intervention the subjects receive (
      • Röhrig B.
      • du Prel J.-B.
      • Wachtlin D.
      • et al.
      Types of study in medical research: part 3 of a series on evaluation of scientific publications.
      ).

      Advantages

      RCT is considered the most reliable study design and a high-impact method when practicing evidence-based medicine because of its ability to minimize confounding factors through randomization. By reducing biases, causality of an intervention on a defined outcome can be most effectively determined (
      • Fletcher R.
      • Fletcher S.
      • Wagner E.
      Clinical Epidemiology: The Essentials.
      ).

      Disadvantages

      The main disadvantage of RCTs compared to other prospective studies is that they are typically more expensive and tedious to perform. In addition, the applicability of the study results to real-world situations may be limited by the study population characteristics, procedures implemented, or outcomes measured (
      • Fletcher R.
      • Fletcher S.
      • Wagner E.
      Clinical Epidemiology: The Essentials.
      ).

      COHORT STUDIES

      Unlike RCTs, there is no intervention in a cohort study. Cohort studies are useful when determining a correlation between cause and effect. First, investigators select a group of people who share a common characteristic or exposure, such as a medication or procedure, but do not have the disease or outcome of interest. A control group is often selected from the general population that has not previously encountered the exposure. In a prospective study, both groups are followed over time to determine whether they develop the outcome of interest. In a retrospective cohort study, cohorts are identified by the same process, but are instead followed via historical data. Analysis of risk factors aids in studying causal associations between exposures and disease, but does not identify true causality (
      • Mann C.
      Observational research methods. Research design II: cohort, cross sectional, and case–control studies.
      ).
      Several important epidemiological measures can be obtained from cohort studies. Investigators can determine the relative risk, or the likelihood that an exposed individual will develop a given disease when compared to a nonexposed individual. In addition, the incidence, or development of new cases of an outcome, can be determined based on their prospective nature (
      • Röhrig B.
      • du Prel J.-B.
      • Wachtlin D.
      • et al.
      Types of study in medical research: part 3 of a series on evaluation of scientific publications.
      ).
      Recently,
      • Chren M.-M.
      • Linos E.
      • Torres J.S.
      • et al.
      Tumor recurrence 5 years after treatment of cutaneous basal cell carcinoma and squamous cell carcinoma.
      conducted a prospective cohort study consisting of 1,253 patients with a history of nonmelanoma skin cancers (NMSCs) to determine the recurrence of skin cancer. The cohort of patients was grouped according to treatment of the primary NMSC by destruction, excision, or Mohs surgery (Figure 1).This study design allowed investigators to conclude there was no difference in recurrence rates for these treatments (
      • Chren M.-M.
      • Linos E.
      • Torres J.S.
      • et al.
      Tumor recurrence 5 years after treatment of cutaneous basal cell carcinoma and squamous cell carcinoma.
      ).
      Figure thumbnail gr1
      Figure 1Prospective cohort design. A cohort of patients with a history of nonmelanoma skin cancer (NMSC) was grouped based on treatment for primary skin cancer with destruction, excision, or Mohs surgery. Patients were followed for tumor recurrence. Reprinted with permission from
      • Chren M.-M.
      • Linos E.
      • Torres J.S.
      • et al.
      Tumor recurrence 5 years after treatment of cutaneous basal cell carcinoma and squamous cell carcinoma.
      .

      Advantages

      A main advantage of a cohort study is that multiple variables may be assessed concurrently and disease risk factors may be identified. The population evaluated in a cohort study is not as restrictive as that in an RCT. As a result, the findings of a cohort study may be more generalizable. Finally, cohort studies can prospectively study the relationship between certain outcomes and exposures that could not otherwise be ethically administered to subjects (
      • Mann C.
      Observational research methods. Research design II: cohort, cross sectional, and case–control studies.
      ).

      Disadvantages

      Similar to RCTs, cohort studies are costly and time-consuming, as well as vulnerable to subject attrition or loss to follow-up during the course of the study. They are not ideal for studying rare diseases because very few patients will develop the disease. Finally, because of the absence of randomization, cohort studies are more prone to bias and confounding than are RCTs (
      • Mann C.
      Observational research methods. Research design II: cohort, cross sectional, and case–control studies.
      ).

      CASE–CONTROL STUDIES

      In a case–control study, those affected by a disease (cases) are compared to disease-free controls from within the population. This type of study is frequently retrospective and aims to identify an association between a disease and potential risk factors. Figure 2 compares the designs of case–control and cohort studies. These studies may be performed through interviews or patient chart review. This allows investigators to determine prior exposure to a potential risk factor and the weight of its impact on disease development. An approximation of the relative risk, known as an odds ratio, can be calculated (
      • Mann C.
      Observational research methods. Research design II: cohort, cross sectional, and case–control studies.
      ).
      Figure thumbnail gr2
      Figure 2Comparison of cohort and case–control study designs. (a) In a prospective cohort study, healthy subjects are followed over time and assessed for exposure status and disease development. (b) In a retrospective case–control study, historical data are reviewed to compare exposure status in diseased (case) and nondiseased (control) individuals. Reprinted with permission from
      • Röhrig B.
      • du Prel J.-B.
      • Wachtlin D.
      • et al.
      Types of study in medical research: part 3 of a series on evaluation of scientific publications.
      .
      • Robinson S.N.
      • Zens M.S.
      • Perry A.E.
      • et al.
      Photosensitizing agents and the risk of non-melanoma skin cancer: a population-based case–control study.
      identified patients with NMSC (cases) matched by age and gender to a population of control subjects within the same geographic region. Subjects were assessed for use of photosensitizing medications and odds ratios were calculated. This allowed the authors to postulate that the risk of developing certain skin cancers was enhanced by use of these medications (
      • Robinson S.N.
      • Zens M.S.
      • Perry A.E.
      • et al.
      Photosensitizing agents and the risk of non-melanoma skin cancer: a population-based case–control study.
      ).

      Advantages

      Case–control studies do not rely on patient follow-up and therefore require less time and cost than the aforementioned studies. It is the ideal design for researching rare diseases because disease status is known from the beginning of the study. Although it is difficult to show causality, associations may be observed that can be studied further using a more rigorous method (
      • Fletcher R.
      • Fletcher S.
      • Wagner E.
      Clinical Epidemiology: The Essentials.
      ).

      Disadvantages

      Case–control studies are limited by the potential for confounding. There is the additional risk of recall bias, in which case subjects may have a skewed recollection of exposure to a potential risk factor as compared with controls (
      • Mann C.
      Observational research methods. Research design II: cohort, cross sectional, and case–control studies.
      ).

      CROSS-SECTIONAL STUDIES

      Whereas RCTs and cohort studies study subjects longitudinally, cross-sectional studies are a “snapshot” and assess disease and exposure status of a population at one particular time point. A survey is one prototypical example. As a descriptive study, the cross-sectional design evaluates the association between certain factors and an outcome of interest. The most important epidemiological contribution is the prevalence of disease, or the number of cases in a population (
      • Mann C.
      Observational research methods. Research design II: cohort, cross sectional, and case–control studies.
      ).

      Advantages

      Cross-sectional studies are cost-efficient and can be completed in a relatively short amount of time on a large scale. The initial associations observed in this type of study are ideal for prompting further research using study designs that yield a stronger level of evidence, such as case–control or cohort studies (
      • Noordzij M.
      • Dekker F.W.
      • Zoccali C.
      • et al.
      Study designs in clinical research.
      ).

      Disadvantages

      Causality cannot be determined with this study design. It is also prone to the forms of bias associated with other observational studies, such as recall and selection bias.

      CASE SERIES/REPORT

      A case report details a rare finding, such as a peculiar drug reaction or a new disease entity. Case series chronicle multiple patients with the same finding. Typically, these studies are retrospective and serve to garner attention for unique findings.

      Advantages

      Case reports can be an excellent way to rapidly and inexpensively disseminate information about a new finding to the medical community. They can help generate awareness of new disease entities and spark hypotheses about pathophysiology (
      • Noordzij M.
      • Dekker F.W.
      • Zoccali C.
      • et al.
      Study designs in clinical research.
      ).

      Disadvantages

      Given that there may be a limited number of cases and absence of a control group, no definitive conclusions can be drawn from case reports. They serve as a preliminary study.

      SUMMARY

      Interventional and observational studies have important roles in dermatology. The results from any study must be put in the context of the appropriate research design selection. RCTs are most useful in determining causality and effectiveness of treatment. Observational studies provide information about epidemiology and possible links between exposure and disease. Understanding the fundamentals underlying the various study designs is critical to both investigators and clinicians alike.
      Figure thumbnail fx2

      ACKNOWLEDGMENTS

      L.S. is supported by the German Cancer Aid, and J.G.Z. is supported by the DFG.

      CME ACCREDITATION

      This CME activity has been planned and implemented in accordance with the Essential Areas and Policies of the Accreditation Council for Continuing Medical Education through the Joint Sponsorship of ScientiaCME and Educational Review Systems. ScientiaCME is accredited by the ACCME to provide continuing medical education for physicians. ScientiaCME designates this educational activity for a maximum of one (1) AMA PRA Category 1 Credit. Physicians should claim only credit commensurate with the extent of their participation in the activity.
      To take the online quiz, follow the link below: http://www.classmarker.com/online-test/start/?quiz=nta52b8684581d68

      SUPPLEMENTARY MATERIAL

      A PowerPoint slide presentation appropriate for journal club or other teaching exercises is available at http://dx.doi.org/10.1038/jid.2013.545.

      REFERENCES

        • Chren M.-M.
        • Linos E.
        • Torres J.S.
        • et al.
        Tumor recurrence 5 years after treatment of cutaneous basal cell carcinoma and squamous cell carcinoma.
        J Invest Dermatol. 2013; 133: 1188-1196
        • Fletcher R.
        • Fletcher S.
        • Wagner E.
        Clinical Epidemiology: The Essentials.
        in: 3rd edn. Lippincott Williams & Wilkins, Baltimore1996
        • Mann C.
        Observational research methods. Research design II: cohort, cross sectional, and case–control studies.
        Emerg Med J. 2003; 20: 54-60
        • Noordzij M.
        • Dekker F.W.
        • Zoccali C.
        • et al.
        Study designs in clinical research.
        Nephron Clin Pract. 2009; 113: c218-21
        • Robinson S.N.
        • Zens M.S.
        • Perry A.E.
        • et al.
        Photosensitizing agents and the risk of non-melanoma skin cancer: a population-based case–control study.
        J Invest Dermatol. 2013; 133: 1950-1955
        • Röhrig B.
        • du Prel J.-B.
        • Wachtlin D.
        • et al.
        Types of study in medical research: part 3 of a series on evaluation of scientific publications.
        Dtsch Arztebl Int. 2009; 106: 262-268