Advertisement

What Is a Pragmatic Clinical Trial?

      INTRODUCTION

      This article summarizes the scientific concepts underlying pragmatic clinical trials as a research technique that is worthy of wider use in dermatology.

      PRAGMATIC TRIALS

      Pragmatic clinical trials seek to determine the effectiveness of an intervention in a real-world setting to inform clinical decision making (
      • Roland M.
      • Torgerson D.J.
      • et al.
      What are pragmatic trials?.
      ). Researchers designing pragmatic trials take particular care to ensure that the study population is as similar as possible to the population on which the intervention is meant to be used (external validity), reflecting the normal range of diversity in disease severity, comorbidities, age, sex, and social and ethnic groups seen in everyday clinical practice. Pragmatic trials also ensure that the sorts of interventions tested can be plausibly rolled out in clinical practice and that the outcomes used to assess effectiveness are valid and easily understood by a range of users, including clinicians, patients, policy makers, and health commissioners. Pragmatic clinical trial patients may also be used to test “strategies” or treatment policies rather than one specific drug at a time. For example, the BLISTER (Bullous Pemphigoid Steroids and Tetracyclines Study) randomized controlled trial tests the policy of starting treatment for bullous pemphigoid patients with either doxycycline or prednisolone (
      • Chalmers J.R.
      • Wojnarowska F.
      • Kirtschig G.
      • et al.
      A randomised controlled trial to compare the safety and effectiveness of doxycycline (200 mg/day) with oral prednisolone (0.5 mg/kg/day) for initial treatment of bullous pemphigoid: a protocol for the Bullous Pemphigoid Steroids and Tetracyclines (BLISTER) Trial.
      ). The policy evaluates the trade-off between the short-term smaller benefit for blister control, as might be anticipated for doxycycline, and the long-term safety concerns that may disadvantage patients randomized to prednisolone. It does not matter whether the dose of prednisolone is altered during the study as would normally occur in clinical practice, nor does it matter if some of the patients initially randomized to the strategy of starting with doxycycline are switched subsequently to prednisolone—what matters is a comparison of the two strategies to which the participants were originally randomized. Cost-effectiveness analysis is usually a key component of pragmatic trials to enable care providers to make informed decisions on value for money (
      • Thomas K.S.
      • Keogh-Brown M.K.
      • Chalmers J.R.
      • et al.
      Effectiveness and cost-effectiveness of salicylic acid and cryotherapy for cutaneous warts. An economic decision model.
      ).
      Figure thumbnail fx1

      EXPLANATORY TRIALS

      Explanatory clinical trials, on the other hand, seek to determine the efficacy of an intervention under ideal conditions. Participants are often a highly selected and homogenous group exhibiting good compliance, and are usually recruited in secondary or tertiary care (
      • Treweek S.
      • Zwarenstein M.
      • et al.
      Making trials matter: pragmatic and explanatory trials and the problem of applicability.
      ). Participants are more likely to remain in the study, typically have only the target condition, and are subject to strict dosing schedules and monitoring. Explanatory trials are deliberately designed to give the maximum chance of showing an effect, if one is present. Outcomes may include cellular markers that explore disease mechanisms (
      • Papp K.A.
      • Reid C.
      • Foley P.
      • et al.
      Anti-IL-17 receptor antibody AMG 827 leads to rapid clinical response in subjects with moderate to severe psoriasis: results from a phase I, randomized, placebo-controlled trial.
      ), as well as composite clinical scales for assessing clinical efficacy. Per protocol analysis tends to be carried out and cost-effectiveness analyses are unusual. Explanatory trials thus typically answer the question “can this treatment work under ideal conditions?” They usually precede pragmatic trials which then ask “we now know it can work, but how well does it work in real world clinical practice?” Table 1 shows the contrasting features between a typical pragmatic and explanatory trial in terms of patients, intervention, comparison, outcome and time of assessment (PICOT framework).
      Table 1PICOT comparison—a comparison of pragmatic versus explanatory trials in dermatology in terms of their five main components: patients, intervention, comparison, outcome, and time of assessment
      Table thumbnail fx3

      THE PRAGMATIC/EXPLANATORY CONTINUUM AND THE PRECIS WHEEL

      Having described the main differences between pragmatic and explanatory trials to aid understanding of their underlying concepts, it is important to recognize the limitations of such a dichotomous approach because most trials contain components of both approaches. This concept of a continuum has led to the development of a useful method to assess the degree of pragmatism when designing a clinical trial, called the Pragmatic–Explanatory Continuum Indicator Summary (PRECIS) tool (
      • Thorpe K.E.
      • Zwarenstein M.
      • Oxman A.D.
      • et al.
      A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers.
      ). Figure 1 shows how investigators of the BLISTER trial (www.blistertrial.co.uk) rated the degree of pragmatism for 10 features on the PRECIS tool (
      • Bratton D.J.
      • Nunn A.J.
      • Wojnarowska F.
      • et al.
      The value of the pragmatic–explanatory continuum indicator summary wheel in an ongoing study: the Bullous Pemphigoid Steroids and Tetracyclines Study.
      ). For example, flexibility of the experimental intervention that permitted investigators to alter the dose of oral corticosteroids for bullous pemphigoid patients in a way that reflected normal clinical practice resulted in a mean score toward the outer pragmatic boundary of the PRECIS wheel, whereas the domain of “eligibility criteria” scored somewhere between the pragmatic outer boundary and the explanatory center of the wheel—probably as a result of exclusion of patients with dementia. The resultant diagrams, which resemble spiderwebs, provide an immediate visual guide as to how pragmatic a study is overall and where the most pragmatic elements reside. Highly explanatory trials show a “web” that is closely tucked into the center, whereas pragmatic trials are dispersed toward the periphery. The PRECIS wheel can be used both at the design stage and when trying to assess the degree of pragmatism in an ongoing or published study (Bratton et al., 2012).
      Figure thumbnail gr1
      Figure 1PRECIS wheel showing the mean of the scores given by BLISTER trial management group members (solid line). Also presented are the most explanatory scores (inner line) and most pragmatic scores (outer line) given in each domain. Scores were plotted on the wheel using a simple pictureediting program (Bratton et al., 2012).

      LIMITATIONS OF THE TECHNIQUE

      There is a perception that pragmatic trials are equivalent to saying that “anything goes.” Although this is true for some aspects, such as allowing flexible dosing for reasons of safety and to mimic how the intervention might be delivered in clinical practice, all critical aspects to reduce bias—e.g., randomization, blinding, treating both groups equally, and analyzing all those originally randomized—must be adhered to (
      • Williams H.C.
      • Gilchrest B.
      • et al.
      Clinical trials submitted to the JID: place your bet and show us your hand.
      ). Sometimes blinding of the intervention (as in surgical procedures) is simply not possible, yet it is almost always possible to ensure a blinded evaluation of outcome. Pragmatic clinical trials are usually more expensive to conduct than explanatory trials because they are typically larger in order to identify minimally clinically important differences in real-life settings. They often require longer follow-up and additional cost-effectiveness analyses, which can add to study complexity and cost. Feasibility work or explanatory trials are often needed before embarking on such studies. Like all study designs, there are good and bad pragmatic studies, and like all clinical trials published in the JID and elsewhere, they must be fully registered and reported fully using the pragmatic-trials extension of the Consolidated Standards of Reporting Trials (CONSORT; http://www.consort-statement.org/extensions?ContentWidgetId=556) so that the reader can judge study quality.

      WHICH IS BEST?

      Despite the widespread publication and acceptance of pragmatic clinical trials as the cornerstone of primary research for health technology assessment in the top general medical journals (
      • Thomas K.S.
      • Crook A.M.
      • Nunn A.J.
      • et al.
      Penicillin to prevent leg cellulitis recurrence.
      ), they are still not widely understood or conducted in dermatology, where explanatory trials still appear to be the norm (
      • Williams H.C.
      • Dellavalle R.P.
      • et al.
      The growth of clinical trials and systematic reviews in informing dermatological patient care.
      ). For example, a PubMed search (11 February 2015) using the terms “dermatology AND randomized AND pragmatic” found only 15 citations, of which only 5 were completed studies, compared with an estimated 5,000 randomized controlled clinical trials in dermatology as a whole. Perhaps the small yield of pragmatic studies in the above search is due to the fact that many such studies fail to mention the term “pragmatic” in the title or study text, a feature that can be righted easily by making the design clear in the study title and description. The choice of an explanatory or pragmatic design clearly depends on the perspective of the question and where in the research cycle the technology being tested is positioned. New drugs or devices typically require explanatory trials when testing whether the intervention demonstrates clinical benefit against vehicle or placebo. Placebo-controlled explanatory trials rarely inform clinical practice because it is unusual to use placebos in clinical practice and because patients are not typically perfectly adherent to the prescribed treatment for various reasons. Those interventions that have shown efficacy against placebo should then be tested for effectiveness against active comparators using pragmatic designs. Pragmatic clinical trials are the cornerstone of the comparative effectiveness agenda (Agency for Healthcare Research and Quality), which has gained significant funds from the 2009 US Government American Recovery and Reinvestment Act (
      • Sox H.C.
      • Greenfield S.
      • et al.
      Comparative effectiveness research: a report from the Institute of Medicine.
      ). Most of the definitive national clinical trials funded by the UK National Institute of Health Research Health Technology Assessment Board are also pragmatic in nature (http://www.nets.nihr.ac.uk/programmes/hta).
      Figure thumbnail fx2

      ACKNOWLEDGMENTS

      The corresponding author wishes to thank Dr. Gudula Kurtschig for suggesting the topic of pragmatic trials.

      CME ACCREDITATION

      This activity has been planned and implemented in accordance with the Essential Areas and Policies of the Accreditation Council for Continuing Medical Education through the joint sponsorship of the Duke University School of Medicine and Society for Investigative Dermatology. The Duke University School of Medicine is accredited by the ACCME to provide continuing medical education for physicians. To participate in the CME activity, follow the link provided. Physicians should only claim credit commensurate with the extent of their participation in the activity.
      To take the online quiz, follow the link below:

      SUPPLEMENTARY MATERIAL

      A PowerPoint slide presentation appropriate for teaching purposes is available at http://dx.doi.org/10.1038/jid.2015.134.

      REFERENCES

        • Bratton D.J.
        • Nunn A.J.
        • Wojnarowska F.
        • et al.
        The value of the pragmatic–explanatory continuum indicator summary wheel in an ongoing study: the Bullous Pemphigoid Steroids and Tetracyclines Study.
        Trials. 2012; 13: 50
        • Chalmers J.R.
        • Wojnarowska F.
        • Kirtschig G.
        • et al.
        A randomised controlled trial to compare the safety and effectiveness of doxycycline (200 mg/day) with oral prednisolone (0.5 mg/kg/day) for initial treatment of bullous pemphigoid: a protocol for the Bullous Pemphigoid Steroids and Tetracyclines (BLISTER) Trial.
        Br J Dermatol. 2015; (e-pub ahead of print 13 February 2015)
        • Papp K.A.
        • Reid C.
        • Foley P.
        • et al.
        Anti-IL-17 receptor antibody AMG 827 leads to rapid clinical response in subjects with moderate to severe psoriasis: results from a phase I, randomized, placebo-controlled trial.
        J Invest Dermatol. 2012; 132: 2466-2469
        • Roland M.
        • Torgerson D.J.
        • et al.
        What are pragmatic trials?.
        BMJ. 1998; 316: 285
        • Sox H.C.
        • Greenfield S.
        • et al.
        Comparative effectiveness research: a report from the Institute of Medicine.
        Ann Intern Med. 2009; 151: 203-205
        • Thomas K.S.
        • Keogh-Brown M.K.
        • Chalmers J.R.
        • et al.
        Effectiveness and cost-effectiveness of salicylic acid and cryotherapy for cutaneous warts. An economic decision model.
        Health Technol Assess. 2006; 10: iii, ix-87
        • Thomas K.S.
        • Crook A.M.
        • Nunn A.J.
        • et al.
        Penicillin to prevent leg cellulitis recurrence.
        N Engl J Med. 2013; 368: 1695-1703
        • Thorpe K.E.
        • Zwarenstein M.
        • Oxman A.D.
        • et al.
        A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers.
        J Clin Epidemiol. 2009; 62: 464-475
        • Treweek S.
        • Zwarenstein M.
        • et al.
        Making trials matter: pragmatic and explanatory trials and the problem of applicability.
        Trials. 2009; 10: 37
        • Williams H.C.
        • Dellavalle R.P.
        • et al.
        The growth of clinical trials and systematic reviews in informing dermatological patient care.
        J Invest Dermatol. 2012; 132: 1008-1017
        • Williams H.C.
        • Gilchrest B.
        • et al.
        Clinical trials submitted to the JID: place your bet and show us your hand.
        J Invest Dermatol. 2015; 135: 325-327