Advertisement

Comparative Effectiveness Research

      WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH.... AND WHY DO IT?

      Comparative effectiveness research (CER) aids clinicians faced with medical decision making by identifying the best strategies among a variety of available preventive, diagnostic, and treatment options. Differing from early-phase clinical trials—in which an intervention is compared with a placebo and assessed for efficacy—the goal of CER is to discriminate among clinical interventions on the basis of clinical effectiveness, cost-effectiveness, adverse effects, or other distinguishing factors.
      As part of the American Recovery and Reinvestment Act of 2009, the US government allocated $1.1 billion for the funding of CER with two primary aims: “(1) to conduct, support, or synthesize research that compares the clinical outcomes, effectiveness, and appropriateness of items, services, and procedures that are used to prevent, diagnose, or treat diseases, disorders, and other health conditions; and (2) encourage the development and use of clinical registries, clinical data networks, and other forms of electronic health data that can be used to generate or obtain outcomes data” (Department of Health and Human Services, http://www.hhs.gov/recovery/programs/cer/index.html, accessed 15 September 2012). One motivation behind the funding of CER is stimulating the delivery of higher-quality health care in a more cost-effective manner. Through well-designed and executed studies, CER has the potential to greatly enhance the practice of evidence-based dermatology (
      • Williams H.C.
      Evidence-based dermatology: everyone's business.
      ). Common methodological approaches to conducting CER include randomized controlled trials and systematic reviews. This article will review recent examples of CER study designs in the dermatology literature as well as statistical analyses used to interpret such designs.

      METHODS OF CER

      CER may be conducted through a variety of study methods. One approach is to perform a systematic review of existing literature addressing one clinical question. Systematic reviews are detailed analyses and evaluations of all the published data on a specific topic to date. The aim is to draw conclusions from the large volume of data that are assessed across multiple published studies to answer the question at hand. These reviews offer the opportunity to conduct statistical analyses of aggregated data—a so-called meta-analysis—to gain broader insights that any one study would not have been large enough to assess. The use of patient registries built around specific clinical conditions facilitates such research by aggregating data for further study and analysis.
      Another approach to CER is to design a randomized controlled trial to answer a specific clinical question. Studies that randomize patients to receive one commonly used medication versus another constitute a fundamental exercise of CER. Under this method, participants are randomly assigned to two or more groups that differ only on the basis of exposure to the study variable addressing the clinical question (namely, the medications, procedures, or diagnostic tools being compared). The groups are followed for predetermined outcomes of interest to address the question at hand, and the results of the two groups are compared by statistical analyses. Patients may be randomized at the individual level, or whole groups of patients may be randomized to particular interventions in the “cluster randomized” approach. Although often considered the gold standard for clinical research, randomized controlled trials are expensive, labor intensive, and time-consuming and may be particularly difficult to conduct for studying rare diseases.
      Figure thumbnail fx1

      CER IN DERMATOLOGY: THERAPEUTICS

      Several common dermatologic conditions may be initially managed with a variety of medication classes. In a patient presenting with moderate acne, topical macrolide antibiotics, topical retinoids, topical benzoyl peroxide, and systemic antibiotics may all be considered part of the initial therapeutic regimen; similarly, for a patient presenting with mild to moderate atopic dermatitis of the face, topical corticosteroids or topical calcineurin inhibitors may be considered. Within each of these broad classes of medications, several treatment choices exist. Large randomized trials comparing multiple treatments head to head for a single condition—such as acne (
      • Ozolins M.
      • Eady E.A.
      • Avery A.J.
      • et al.
      Comparison of five antimicrobial regimens for treatment of mild to moderate inflammatory facial acne vulgaris in the community: randomised controlled trial.
      ) or head lice (
      • Chosidow O.
      • Giraudeau B.
      • Cottrell M.S.
      • et al.
      Oral ivermectin versus malathion lotion for difficult-to-treat head lice.
      )—offer important lessons for therapeutic agent selection by demonstrating significant differences in clinical effectiveness across treatments. The important point is that study participants must be randomly assigned to two or more treatments in the same study, using the same study design, double blind, with efficacy measured using the same disease severity indices. Surgical therapeutics may also be compared for effectiveness via randomized controlled trials, as has been done to assess surgical excision versus Mohs micrographic surgery for basal-cell carcinoma of the face (
      • Mosterd K.
      • Krekels G.A.
      • Nieman F.H.
      • et al.
      Surgical excision versus Mohs' micrographic surgery for primary and recurrent basal-cell carcinoma of the face: a prospective randomised controlled trial with 5-years' follow-up.
      ). “Real-world” studies in which participants are patients treated in private-practice as well as academic settings, using on-label medications to manage the same disease process, are also considered within the scope of CER. The study design is cross-sectional, in which patients with the same clinical condition are treated with a range of therapeutic interventions and assessed for clinical response in a nonrandomized manner, as has been done with a variety of psoriasis treatments (
      • Gelfand J.M.
      • Wan J.
      • Callis Duffin K.
      • et al.
      Comparative effectiveness of commonly used systemic treatments or phototherapy for moderate to severe plaque psoriasis in the clinical practice setting.
      ).
      A large comparative effectiveness study published in the Journal of Investigative Dermatology in 2009 assessed two treatment regimens of the same steroid, clobetasol propionate, for disease control and event-free survival in patients with bullous pemphigoid (
      • Joly P.
      • Roujeau J.C.
      • Benichou J.
      • et al.
      A comparison of two regimens of topical corticosteroids in the treatment of patients with bullous pemphigoid: a multicenter randomized study.
      ). A total of 312 patients with moderate or extensive bullous pemphigoid were randomized to treatment with either high-dose clobetasol (40 g/day) or low-dose clobetasol (10–30 g/day). An important methodological component of any such trial is the a priori estimation of sample size, which calculates the number of subjects needed to detect significant differences in effects between interventions. The 2009 study was designed to have 80% power to detect a 33% difference in event-free survival between the two groups, with a one-sided log-rank test and type I error of 5%. Simply put, the statistical power of the study is the probability of finding a significant difference that does exist between the two groups; increasing the power of the study while holding other parameters equal will increase the number of experimental samples needed to reach the same level of significance. A type I error occurs when a difference between the two groups is claimed, although one does not actually exist. The probability of a type I error is known as α. Decreasing α—and thus reducing the probability of making such an error—while holding other parameters equal will require a larger sample size.
      The bullous pemphigoid study cited above used the log-rank test for analysis of event-free and disease-free survival between patients in the two treatment groups (Figure 1). This test is used to assess differences between populations in the probability of an event over time, such as death or disease recurrence, and is often used for comparisons of survival between experimental groups (
      • Bland J.M.
      • Altman D.G.
      The logrank test.
      ). Such data are routinely plotted in Kaplan–Meier curves, which display time on the x-axis and percentage of surviving or unaffected individuals on the y-axis. Joly and colleagues reported no significant difference in overall event-free survival (patients unaffected by life-threatening adverse events or death) between the two treatment groups (P value = 0.95, Figure 1a). Significantly fewer side effects were seen in the lower-dose group. However, there was a significantly higher rate of disease relapse in subjects given the lower dose of steroids (P value = 0.012, Figure 1b). The authors concluded that the lower-steroid regimen demonstrated comparable clinical effectiveness with fewer side effects and lower overall costs that outweigh the slightly higher rate of relapse, a powerful lesson for both patients and their clinicians going forward. When addressing multiple treatment interventions in CER, selecting the appropriate statistical model and test is critical for a valid interpretation of the results and to limit research wastage caused by flawed designs or methodology (
      • Williams H.C.
      • Delavalle R.P.
      The growth of clinical trials and systematic reviews in informing dermatological patient care.
      ).
      Figure thumbnail gr1
      Figure 1Kaplan–Meier curves. These curves demonstrate event-free and disease-free survival in patients treated with different topical steroid regimens for bullous pemphigoid.
      From
      • Joly P.
      • Roujeau J.C.
      • Benichou J.
      • et al.
      A comparison of two regimens of topical corticosteroids in the treatment of patients with bullous pemphigoid: a multicenter randomized study.
      .

      CER IN DERMATOLOGY: DIAGNOSTIC TOOLS AND HEALTH-CARE TECHNOLOGIES

      As a field in which clinical activities span physical diagnosis, medical and surgical treatments, long-term patient follow-up, and population health, clinical dermatology integrates several technologies into its routine practice. In addition to comparisons of pharmacologic treatments, CER can also be used to assess diagnostic instruments and other health-care technologies used in the practice of dermatology. There is a great need to develop outcomes in dermatology that are valid and reliably measure the diagnosis of a disease (e.g., screening questionnaires used in large epidemiology studies or diagnostic criteria used during a physical exam), disease severity (e.g., the Eczema Area and Severity Index), subphenotypes of disease (e.g., localized alopecia areata versus alopecia areata totalis), and health-related quality-of-life indices. Beyond development of indices and outcomes, CER is needed to better assess the use of measures in various settings (e.g., clinic vs. clinical trial). For example, efforts have begun to systematically review the process by which skin cancer is diagnosed by specialist and nonspecialist clinicians and to better understand the clinical impact of noninvasive technologies such as dermoscopy and photography on skin cancer diagnosis (
      • Parsons S.K.
      • Chan J.A.
      • Yu W.W.
      • et al.
      Noninvasive diagnostic techniques for the detection of skin cancers.
      ).
      Another recent study compared the Skindex-29 with the Skindex-17, two health-related quality-of-life survey instruments that quantify the patient burden of dermatologic disease (
      • Sampogna F.
      • Spagnoli A.
      • Di Pietro C.
      • et al.
      Field performance of the Skindex-17 Quality of Life Questionnaire: a comparison with the Skindex-29 in a large sample of dermatological outpatients.
      ). The Skindex-17 consists of a subset of questions that are derived from those in the longer 29-item questionnaire. Data from 2,487 patients who completed Skindex-29 surveys administered in a single institution were used to compute corresponding Skindex-17 scores. The mean values from these two sets of scores were then compared (Table 1). Sampogna and colleagues used intraclass correlation coefficients (ICCs) to compare the scores on the two questionnaires. The ICC is a statistical test used to calculate the reproducibility of measurements (or scores) of two different instruments measuring the same entity. The correlation coefficient is reported from 0 to 1. A high ICC represents a high degree of agreement between the scoring instruments. The two Skindex scores were found to have an ICC of 0.957 for questions regarding symptoms and an ICC of 0.94 for questions regarding psychosocial impact of disease. The high level of concordance observed between the instruments led the researchers to suggest that the shorter questionnaire may be effectively used to measure health-related quality of life, thereby reducing challenges associated with a longer survey, such as great respondent burden. Such research adds meaningful data by addressing the very metrics by which we assess dermatologic disease and also inspires further investigation of the effectiveness of our research tools and methodologies.

      FUTURE DIRECTIONS FOR CER IN DERMATOLOGY

      The current attention focused on CER by federal and international agencies looking to enhance the quality of medical-care delivery argues strongly for increased CER efforts within dermatology. Identifying meaningful future directions for such research should help translate into better, evidence-driven, more effective patient care. Surveys of clinicians actively engaged in the treatment of patients with psoriasis have identified particular therapeutic interventions that these practitioners would like to see compared in future CER studies (
      • Wan J.
      • Abuabara K.
      • Troxel A.B.
      • et al.
      Dermatologist preferences for treatments to compare in future randomized controlled comparative effectiveness trials for moderate to severe psoriasis.
      ), providing guidance for additional work in the field. Recent studies comparing electronic-health and teledermatology visits for the management of chronic conditions such as acne (
      • Watson A.J.
      • Bergman H.
      • Williams C.M.
      • et al.
      A randomized trial to evaluate the efficacy of online follow-up visits in the management of acne.
      ) and atopic dermatitis (
      • van Os-Medendorp H.
      • Koffijberg H.
      • Eland-de Kok P.C.
      • et al.
      E-health in caring for patients with atopic dermatitis: a randomized controlled cost-effectiveness study of Internet-guided monitoring and online self-management training.
      ) begin to address the value of technology-based care delivery in the practice of clinical dermatology. With proper planning and analysis, CER studies represent a powerful addition to the investigative dermatologist's toolkit for answering an array of complex questions. Current attention to better, more efficient, and lower-cost health-care delivery in the United States may be the burning platform for CER. With increasing demand for reducing variation and clinical process improvement, CER may finally receive the attention required to propel the next group of large studies. The ultimate challenge for the practicing clinician will be to translate these studies into better care for patients with dermatologic disease.
      Figure thumbnail fx3

      Supplementary MATERIAL

      Answers and a PowerPoint slide presentation appropriate for journal club or other teaching exercises are available at http://dx.doi.org/10.1038/jid.2012.497.

      REFERENCES

        • Bland J.M.
        • Altman D.G.
        The logrank test.
        Br Med J. 2004; 328: 1073
        • Chosidow O.
        • Giraudeau B.
        • Cottrell M.S.
        • et al.
        Oral ivermectin versus malathion lotion for difficult-to-treat head lice.
        N Engl J Med. 2010; 362: 896-905
        • Gelfand J.M.
        • Wan J.
        • Callis Duffin K.
        • et al.
        Comparative effectiveness of commonly used systemic treatments or phototherapy for moderate to severe plaque psoriasis in the clinical practice setting.
        Arch Dermatol. 2012; 148: 487-494
        • Joly P.
        • Roujeau J.C.
        • Benichou J.
        • et al.
        A comparison of two regimens of topical corticosteroids in the treatment of patients with bullous pemphigoid: a multicenter randomized study.
        J Invest Dermatol. 2009; 129: 1681-1687
        • Mosterd K.
        • Krekels G.A.
        • Nieman F.H.
        • et al.
        Surgical excision versus Mohs' micrographic surgery for primary and recurrent basal-cell carcinoma of the face: a prospective randomised controlled trial with 5-years' follow-up.
        Lancet Oncol. 2008; 12: 1149-1156
        • Ozolins M.
        • Eady E.A.
        • Avery A.J.
        • et al.
        Comparison of five antimicrobial regimens for treatment of mild to moderate inflammatory facial acne vulgaris in the community: randomised controlled trial.
        Lancet. 2004; 364: 2188-2195
        • Parsons S.K.
        • Chan J.A.
        • Yu W.W.
        • et al.
        Noninvasive diagnostic techniques for the detection of skin cancers.
        In: AHRQ Comparative Effectiveness Reviews. Agency for Healthcare Research and Quality, Rockville, MD2011
        • Sampogna F.
        • Spagnoli A.
        • Di Pietro C.
        • et al.
        Field performance of the Skindex-17 Quality of Life Questionnaire: a comparison with the Skindex-29 in a large sample of dermatological outpatients.
        J Invest Dermatol. 2012; 133: 104-109
        • van Os-Medendorp H.
        • Koffijberg H.
        • Eland-de Kok P.C.
        • et al.
        E-health in caring for patients with atopic dermatitis: a randomized controlled cost-effectiveness study of Internet-guided monitoring and online self-management training.
        Br J Dermatol. 2012; 166: 1060-1068
        • Wan J.
        • Abuabara K.
        • Troxel A.B.
        • et al.
        Dermatologist preferences for treatments to compare in future randomized controlled comparative effectiveness trials for moderate to severe psoriasis.
        Arch Dermatol. 2012; 148: 539-541
        • Watson A.J.
        • Bergman H.
        • Williams C.M.
        • et al.
        A randomized trial to evaluate the efficacy of online follow-up visits in the management of acne.
        Arch Dermatol. 2010; 146: 406-411
        • Williams H.C.
        Evidence-based dermatology: everyone's business.
        Am J Clin Dermatol. 2011; 12: 357-359
        • Williams H.C.
        • Delavalle R.P.
        The growth of clinical trials and systematic reviews in informing dermatological patient care.
        J Invest Dermatol. 2012; 132: 1008-1017