- Open Access
An overview of the impact of rare disease characteristics on research methodology
© The Author(s). 2018
- Received: 13 September 2017
- Accepted: 29 December 2017
- Published: 19 January 2018
About 30 million individuals in the United States are living with a rare disease, which by definition have a prevalence of 200,000 or fewer cases in the United States ([National Organization for Rare Disorders], [About NORD], ). Disease heterogeneity and geographic dispersion add to the difficulty of completing robust studies in small populations. Improving the ability to conduct research on rare diseases would have a significant impact on population health. The purpose of this paper is to raise awareness of methodological approaches that can address the challenges to conducting robust research on rare diseases.
We conducted a landscape review of available methodological and analytic approaches to address the challenges of rare disease research. Our objectives were to: 1. identify algorithms for matching study design to rare disease attributes and the methodological approaches applicable to these algorithms; 2. draw inferences on how research communities and infrastructure can contribute to the efficiency of research on rare diseases; and 3. to describe methodological approaches in the rare disease portfolio of the Patient-Centered Outcomes Research Institute (PCORI), a funder promoting both rare disease research and research infrastructure.
We identified three algorithms for matching study design to rare disease or intervention characteristics (Gagne, et.al, BMJ 349:g6802, 2014); (Gupta, et.al, J Clin Epidemiol 64:1085-1094, 2011); (Cornu, et. al, Orphet J Rare Dis 8:48,2012) and summarized the applicable methodological and analytic approaches. From this literature we were also able to draw inferences on how an effective research infrastructure can set an agenda, prioritize studies, accelerate accrual, catalyze patient engagement and terminate poorly performing studies. Of the 24 rare disease projects in the PCORI portfolio, 11 are randomized controlled trials (RCTs) using standard designs. Thirteen are observational studies using case-control, prospective cohort, or natural history designs. PCORI has supported the development of 9 Patient-Powered Research Networks (PPRNs) focused on rare diseases.
Matching research design to attributes of rare diseases and interventions can facilitate the completion of RCTs that are adequately powered. An effective research infrastructure can improve efficiency and avoid waste in rare disease research. Our review of the PCORI research portfolio demonstrates that it is feasible to conduct RCTs in rare disease. However, most of these studies are using standard RCT designs. This suggests that use of a broader array of methodological approaches to RCTs --such as adaptive trials, cross-over trials, and early escape designs can improve the productivity of robust research in rare diseases.
- Rare disease
- Research design
- Orphan disease
- Experimental designs
Rare diseases are defined by the Rare Disease Act of 2002 as diseases affecting 200,000 individuals or fewer in the United States . Current estimates indicate that there are close to 7000 rare diseases and that about 30 million individuals in the United States are living with a rare disease . While individually, each rare disease impacts a small population, collectively, a large number of individuals are affected by these conditions. Therefore, improving the ability to conduct research on rare diseases would have a broad population impact.
Research on treatments or management strategies for rare diseases can be challenging primarily due to the limited number of individuals who will be eligible to participate in any given study and uncertainty about or heterogeneity in the natural history of the disease. These smaller populations of eligible individuals can vary in their disease presentation, severity, progression, and exposure to prior treatment and can be geographically dispersed. These issues are not unique to rare diseases but are often magnified for these conditions. The size and characteristics of each affected population impact the type and number of studies that can be conducted due to their influence on factors such as study design, sample size, and power.
to identify algorithms for matching study design to rare disease attributes and to summarize the methodological approaches applicable to these algorithms;
to draw inferences on how research communities and infrastructure can contribute to the efficiency of research on the treatment and management of rare diseases; and
to describe the use of the above approaches in the rare disease portfolio of the Patient-Centered Outcomes Research Institute (PCORI), a funder which is promoting both rare disease research and research infrastructure.
The PCORI is a not-for-profit organization that was created through the Patient Protection and Affordable Care Act of 2010 to fund patient-centered comparative effectiveness research (CER) that provides clinicians, patients, and other stakeholders with the information they need to make informed health care decisions . As part of its charge, PCORI is tasked with supporting CER on rare diseases. PCORI funds both research projects and a national infrastructure to support CER - the National Patient-Centered Clinical Research Network (PCORnet) program. PCORnet is comprised of 13 Clinical Data Research Networks and 20 Patient Powered Research Network. Clinical Data Research Networks are networks that originate in healthcare systems and securely collect health information as part of routine clinical care, whereas Patient Powered Research Networks are networks governed by patients, caregivers, clinicians, researchers, and others focused on a sharing and collecting information particular health care conditions .
We conducted a landscape review to address the first two objectives, but did not attempt a formal systematic review for practical reasons. First, the purpose of this project was to provide timely information to the PCORI rare disease advisory panel . Second, the potential scope of the literature is vast and our purpose was to get an overview of methodologic approaches rather than to resolve a specific question . The search strategies used to identify relevant articles to address the first two objectives described above are included in the Appendix.
Objective 1: Algorithms for matching study design to rare disease attributes and main methodological approaches applicable to these algorithms
To identify relevant methodological and analytic approaches to conducting rare disease research, two authors (DW and SP) reviewed the search results and categorized the literature discussing methodological and analytic approaches into three distinct groups: (a) articles that provided a general, high level discussion of different research methods, (b) articles that described the advantages and limits of different methodological and analytic techniques, and (c) articles that discussed the application of research methods in particular clinical settings. The most relevant articles for the purposes of this paper were those within category two that presented algorithms or frameworks designed to help select appropriate study designs for research on rare disease. Other articles were used to generate a list of methodological and analytic approaches used in rare disease research. Where necessary, this literature was supplemented with literature describing the methodological and analytic approaches identified through the literature search.
Objective 2: Contributions of research communities and infrastructure to the efficiency of research on the treatment and management of rare diseases
To draw inferences on how research communities and infrastructure can contribute to efficiency, the authors first generated examples of how research infrastructure could be leveraged to support rare disease research. The main categories included recruitment and retention, agenda setting, executing a research agenda, terminating poorly performing studies, and executing new studies. Using this list, two authors (DW and SP) then reviewed all articles identified through the literature search, coding those that discussed one or more of the uses of research infrastructure and excluding those that simply described the structure of an existing registry or network.
Objective 3: Description of approaches in the PCORI rare disease research and infrastructure portfolio
To describe the types of research designs currently being used to support PCORI-funded research, the authors identified currently funded rare disease research projects and extracted the study design and main analytic technique used. We also identified the PCORnet Patient Powered Research Networks that focus on one or more rare diseases and identified the stated purpose of the network or registry.
Several areas that are important to rare disease research were outside the scope of this project. These include pharmacokinetic and pharmacodynamics study designs, guideline development, and the application of evidence-grading methods . Additionally, it was outside the scope of this project to provide descriptions of the study designs. Study design descriptions are widely available in sources on methods both for researchers and interested stakeholders, such as patients, caregivers, clinicians, policy makers, and health system leaders.
Objective 1a: Algorithms for matching study design to rare disease attributes
Feature of disease, intervention, or outcome measures
Impact on Study Design
Diseases that are life threatening
In placebo controlled RCTs, time on placebo should be minimized
Diseases in which individuals are often diagnosed when they first have the condition
Prospective inception cohort designs may be useful in establishing temporality among study variables
Diseases that have an unpredictable disease course
Several experimental designs cannot be used including crossover designs, latin square designs, n-of-1 trials, and randomized withdrawal designs
Whether the anticipated response to the intervention is non-reversible
Several experimental designs cannot be used including crossover designs, latin square designs, n-of-1 trials, randomized withdrawal designs, early escape, and delayed start designs
Whether the anticipated response to the intervention is delayed rather than immediate
Several experimental designs cannot be used including crossover designs, latin square designs, n-of-1 trials, early escape designs, and designs that involve adaptive randomization
Whether the effects on the outcomes are influenced by the order of interventions received*
Several experimental designs cannot be used including crossover designs, latin square designs, and n-of-1 trials
Outcome and prognostic tool characteristics
Whether meaningful surrogate outcomes or composite measures are available or whether statistical techniques for analyzing repeated outcome measures are applicable
In these situations, it may be possible to reduce the sample size needed to answer the study question
Whether tools are available that can be used to accurately predict prognosis
In these situations, risk-based allocation designs are feasible and it may be possible to reduce the sample size needed if the study focuses on recruiting only patients who are at high risk of progressing. However, enrolling only high risk patients will also reduce the pool of eligible individuals.
Whether existing research infrastructure exists for the condition of interest, such as a patient registry
In situations where there is existing infrastructure, that infrastructure may be leveraged to recruit eligible participants more rapidly and to implement a study more efficiently
Acceptable levels of uncertainty
Whether decision-makers expected to use the study data are willing accept results from a trial with an alpha >0.05
In these situations, it may be possible to reduce the sample size needed to address the study question are the study would not need to be powered at an alpha ≤0.05
Gagne and colleagues aimed “to identify innovative approaches to research that have been, or can be, applied to overcome the methodological challenges inherent in the study of rare diseases” . To reduce the required sample size, outcomes that occur more frequently or that occur sooner can be selected. One way to accomplish this is to use composite outcomes or surrogate endpoints. In some cases, using repeated measures in the same individual or using continuous outcome variables may enhance statistical efficiency, depending on the properties of the outcome measures or statistical techniques used . Use of an adaptive trial that allows for pre-specified changes to the study design based on treatment response can improve the efficiency of testing for efficacy by optimizing dosage and delivery without the need for additional trials. Although α ≤ 0.05 is the generally accepted standard for statistical significance in clinical trials, where treatment options are limited, patients may be willing to accept greater uncertainty with consequent reductions in sample size needed to adequately power the study. While we disagree with the proposition that underpowered studies are acceptable because they may ultimately contribute to a meta-analysis, underpowered studies can contribute to a Bayesian model (described below) to inform a future trial that can provide definitive results. Studies in which all participants eventually receive the intervention can attract eligible individuals, potentially improving recruitment and retention.
Where there is an obstacle to randomization due to strong patient preferences, for example a reluctance a placebo or an invasive intervention, the use of an observational design may make recruitment feasible. Potential designs include self-controlled observational study designs (which are similar to crossover designs but do not involve random assignment), case-control designs, and prospective inception cohorts. Propensity scores have been used in an effort to account for confounding due to measured variables. However, Gagne et al. note the inherent limitations of observational methods and state that “greater attention to innovative methods for using observational data to study rare disease health outcomes is needed” .
In the second article, Gupta and colleagues summarized a variety of study designs that could be used for studies of rare diseases, including the pros and cons of each design, and developed a framework to help investigators determine when different designs are appropriate . The study designs considered include parallel group designs, crossover designs, n-of-1 trials, adaptive designs, and design combinations. Their framework takes investigators through a series of yes and no questions to assess the usefulness of alternative designs in particular situations. The framework suggests, for example, that crossover and n-of-1 trials should only be used in situations where three conditions are met: the intervention has a predictable and short duration of effect, the disease course is stable over at least two intervention periods, and participants can be retained for at least two intervention periods. However, if the investigators believe it will not be possible to adequately power a crossover trial, than an n-of-1 design should be considered. If the above three conditions are not met and the time between enrollment and outcome assessment is relatively long compared to the time needed to accrue all participants, the framework suggests that investigators should consider a conventional parallel group randomized controlled trial (RCT) design. If the above three conditions are not met and the time between enrollment and outcome assessment is relatively short compared to the time needed to accrue all participants, the framework suggests that investigators should consider adaptive trial designs.
Cornu and colleagues propose a framework where outcomes and responses are key decision points. Investigators are asked to consider first whether the outcomes of interest are reversible or irreversible, then how rapidly individuals are expected to respond to the study intervention(s), followed by whether it is possible to minimize time on placebo, and, finally, whether it is possible to treat all participants enrolled in the study . The algorithm includes twelve possible study designs (parallel group RCT, crossover design, latin square design, n-of-1 trials, randomized placebo phase, stepped wedge, randomized withdrawal, early escape designs, delayed start designs, three stage designs, and adaptive randomization).
In situations where the outcomes included in the trial are reversible and the response to the intervention is relatively quick (within a few weeks), all study designs are possible. If the outcomes are reversible and the response to the intervention is slow, crossover designs including latin square and n-of-1 trials are not feasible, nor are early escape, delayed start, three stage designs, or designs involving adaptive randomization. In situations where outcomes are not reversible but the response to the intervention is relatively fast, crossover designs, including latin square and n-of-1 trials are not feasible, nor are randomized withdrawal, early escape, delayed start, or three stage designs.
If the response to the intervention is relatively slow, in addition to the designs listed above, adaptive randomization is also no longer feasible. The remaining decision nodes ask the investigators to consider the feasibility of minimizing time on placebo and, if it is feasible, whether it is also feasible to implement a design that ensures that all participants will receive active treatment by the end of the study. Similar to Gagne and colleagues, these authors note that designs that can minimize time on placebo and that ensure that all participants will receive a treatment are more attractive to eligible individuals. Study designs that minimize participant time on placebo include delayed start, randomized placebo phase, stepped wedge, randomized withdrawal, early escape, three stage, and adaptive randomization designs. Study designs that also ensure that all participants receive active treatment by the end of the study include delayed start, randomized placebo phase trials, and stepped wedge designs .
Objective 1b. Main methodological approaches
Possible Study Designs for Rare Disease Research
• Patients are guaranteed to receive active treatment and spend less time on placebo
• Patients receive two interventions in sequence randomly, with a washout period between interventions
• Each participant serves as his or her own control, thereby reducing sample size requirements
• Latin square allows for multiple interventions in randomized sequence
• Each intervention appears only once in each sequence and intervention period
• Design variations include N-of-1 trials
• This design can increase the proportion of patients assigned to the more favorable treatment, which can contribute to a greater number of willing participant
• Adaptive treatment allocation designs test the null hypothesis in a series of interim analyses; these analyses then influence subsequent randomization in the next phase
• Bayesian analyses (allowing updates of prior probabilities) or frequentist approaches can be used
• Adaptive treatment allocation designs allow the probability of being randomized to an intervention to change during the enrollment period; the probability of being randomized will increasingly favor the arm with the more promising results (play the winner) or increasingly penalize the arm with less promising results (drop the loser
• Adaptive designs can be used to narrow from a selection of doses (ranking and selection designs) rather than rejecting a null hypothesis
• Adaptive designs can be used to select among subpopulations and thereby balance covariates (covariate-adaptive randomization) and help address underlying heterogeneity
• Design variations include sequential RCTs and ranking and selection RCTs
Randomized Placebo Phase
• This design minimizes the length of time that patients are on placebo with all patients receiving treatment in the end. Limited time on placebo is beneficial for conditions with a rapid unfavorable evolution
• Design variations include stepped wedge, randomized withdrawal, and three-stage trials. In stepped wedged designs, randomization occurs at crossover to different treatment
• Low-risk patients are randomized to high-dose and standard treatment, high-risk patients receive high-dosetreatment, thereby addressing concerns about the ethics of withholding treatment from high-risk patients
• A combined analysis allows the prediction of the added benefit of high-dose treatment
• This design offers patients with diseases that have long latency periods the opportunity to participate in research
• Study participants with the disease (cases) are compared to participants without the disease (controls) in an effort to identify factors that may contribute to a particular outcome
• To address concern that cases and controls may differ in characteristics other than the condition studied, cases and controls can be matched on other characteristics (ex. Age, race, sex)
Cohorts with historic controls [Natural History Studies]
• This design provides patients with the opportunity to learn about different treatments
• Comparison of prospectively treated patients with historic controls reduces recruitment burden for the control arm
• Patients receive usual care or standard intervention followed by tested treatment. Patients receive an active treatment throughout the course of the study
• Requires a detailed understanding of the natural history of the disease to avoid issues of regression to the mean
In addition to the study designs described in Table 2, other experimental designs include repeated measurement designs, factorial designs, and “early escape” in RCT. In a repeated measurement study, multiple observations of response variables are taken from each participant, allowing for within-subject comparisons and increasing the number of data points . In what is essentially a four arm RCT, factorial design involves double randomization in which two comparisons are made concurrently as if conducting two simultaneous studies in the same patient population with the assumption that there is no interaction between the two treatments (i.e. the biologic effect of the first intervention is not mediated or modified by the second intervention) . The benefit of a factorial design is that it allows investigators to answer two research questions within the same trial. Applicable to various trial designs, in “early escape” designs patients can withdraw from the trial either by choice or if they meet a priori criteria listed in the protocol, possibly leading to enhanced retention and power . A prospective inception cohort is another relevant study design of interest but is non-experimental. In this design, cohort inception takes place at the time of medical diagnosis or start of treatment, allowing researchers to establish temporality among study variables, such as baseline confounders and exposures, and to capture outcomes that occur shortly after a participant enters the cohort .
The research challenges posed by the characteristics of rare diseases also impact analytic methods. In the literature reviewed on analytic methods in rare disease research, Bayesian analysis was by far the most frequently discussed technique [8, 9, 14–18, 25]. Bayesian analysis provides formal incorporation of prior information or external evidence into the analysis, allowing a greater amount of information to be gained from a smaller number of subjects. A central component of Bayesian analysis is prior probability distribution of the variable of interest from the external data. This distribution is integrated with the distribution of the internal data, yielding the posterior probability distribution . Because of the impact that the prior information has on the analyses, it is important that investigators using Bayesian methods carefully consider the appropriateness of that information before formally incorporating it into the prior distribution or before deciding to use a Bayesian approach . Bayesian methods impact not only the analytic process but also provide a framework that guides the entire research process in both study design and execution.
Objective 2: Contributions of research communities and infrastructure to the efficiency of research on the treatment and management of rare diseases
Research infrastructure such as registries and research networks might promote the efficiency and success of rare disease research. The literature most frequently discussed the benefits of existing infrastructure, such as contact registries, in order to support recruitment and retention efforts. Several authors describe the establishment of contact registries that store information on individuals who have indicated that they are willing to be contacted about clinical research that they may be eligible for [26–28]. Of these articles that describe research and contact registries, several describe the benefits of having an established infrastructure in terms of facilitating the rapid implementation of studies by expediting enrollment [27–31]. One article described how the research team leveraged their registry to support patient and family engagement in designing research studies .
Other potential uses of research infrastructure that were not described in the literature identified through the search but that were identified by the authors of this paper include research prioritization agenda setting, executing a research agenda, and stopping poorly performing studies. A 2010 report by the National Academies of Sciences entitled Rare Diseases and Orphan Products: Accelerating Research and Development described the potential benefits of registries in accelerating research for rare disease. However, the report also notes that currently “no uniform, accepted standards govern the collection, organization, or availability of these data. The result is sometimes wasteful duplication and sometimes underuse of information or samples contributed by patients or research participants” . To remedy this issue, the report suggests that there is a need to move toward common standards and a “freely available platform for creating or restructuring patient registries and biorepositories for rare diseases and for sharing de-identified data” . One such initiative that aims to accomplish these objectives is RD-CONNECT. RD-CONNECT was funded in 2012 by the European Union’s Seventh Framework Program under the International Rare Diseases Research Consortium (IRDiRC). Two of the stated goals of this initiative are to develop “an integrated platform to host and analyze genomic and clinical data from research projects” and “common infrastructures and data elements for rare disease patient registries” .
Objective 3: Description of approaches in the PCORI rare disease research and infrastructure portfolio
PCORI Rare Disease Portfolio: Study Designs and Analytic Techniques
Project Title (Condition in Bold)
Home or Away from Home: Comparing Clinician and Patient/Family-Centered Outcomes Relevant to the Care of Pediatric Acute Myeloid Leukemia during Periods of Neutropenia
Retrospective and prospective observational cohort
(Also has a qualitative focus group component)
Propensity score analysis; regression analysis (linear mixed effect models); thematic qualitative analysis
Intervention and Outcomes in Duarte Galactosemia
Standard Multivariate analysis; generalized estimating equations
Treatment Alternatives in Adult Rare Disease; Assessment of Options in Idiopathic Subglottic Stenosis
Prospective observational cohort
Standard multivariate analysis; survival or Kaplan Meier Analysis
Collaborative Assessment of Pediatric Transverse Myelitis: Understand, Reveal, Educate (CAPTURE) Study
Prospective observational cohort
Propensity scores; sensitivity analysis; mixed effects linear models
The Impact of Self-Management with Probiotics on Urinary Symptoms and the Urine Microbiome in Individuals with Spinal Cord Injury (SCI) and Spina Bifida (SB)
Prospective observational cohort
Linear mixed-effects modeling
Anti-TNF Monotherapy versus Combination Therapy with Low Dose Methotrexate in Pediatric Crohn’s Disease
Randomized, double-blind, multicenter pragmatic clinical trial
Standard multivariate analysis; Survival or Kaplan Meier Analysis
Comparative Effectiveness of CARRA Treatment Strategies for Polyarticular Juvenile Idiopathic Arthritis
prospective observational cohort
Bayesian Analytic Approach, including logistic regression model, sensitivity analysis, linear mixed effects model
Comparative Effectiveness of a Decision Aid for Therapeutic Options in Sickle Cell Disease
RCT; Also descriptive observation study
Specifics not discussed in app. Endpoints for RCT include decisional process and decisional bias
Patient Centered Comprehensive Medication Adherence Management System to Improve
Effectiveness of Disease Modifying Therapy with Hydroxyurea in Patients with Sickle Cell Disease
Linear mixed effects regression models
Comparing patient centered outcomes in the management of pain between emergency departments and dedicated acute care facilities for adults with sickle cell disease
prospective observational cohort
Multilevel regression modelling - Propensity scores
Posterior fossa decompression with or without duraplasty for Chiari type I malformation with syringomyelia.
Plan to use weighted analyses if there is a balance in prognostic factors; Heterogeneity of Treatment Effect (exploratory analysis)
Relapsed childhood neuroblastoma as a model for parental end-of-life decision-making
Prospective longitudinal cohort study (multicenter)/nested case-control study
Standard multivariate analysis; nested case control analysis;
Taking Charge of Systemic Sclerosis: Improving Patient Outcomes Through Self-Management
Standard univariate and multivariate analysis
Comparative Efficacy of Therapies for Eosinophilic Esophagitis
Sensitivity analysis; heterogeneity of treatment effect analyses; standard multivariate techniques
Decision Support for Parents Receiving Genetic Information about Child’s Rare Disease
Prospective observational cohort
Thematic Analysis (of interviews, focus groups, surveys)
Individualized Patient Decision Making for Treatment Choices among Minorities with Lupus
Tools and Information to Guide Choice of Therapies in Older & Medically Infirm Patients with Acute Myeloid Leukemia
Prospective observational cohort
Sensitivity Analysis; Propensity Scores; survival-Kaplan modeling; heterogeneity of Treatment Effect
Comparative effectiveness of therapy in rare diseases: Liver transplantation vs. conservative management of urea cycle disorders.
Prospective observational cohort with retrospective components
Propensity scores, survival or Kaplan Meier Analysis
A randomized controlled trial of anterior versus posterior entry site for cerebrospinal fluid shunt insertion (hydrocephalus)
Survival-Kaplan Model; Regression Analysis
Enhancing Genomic Laboratory Reports to Enhance Communication and Empower Patients (Genetic Diseases)
Randomized, single-blinded pre-post intervention trial with crossover
Comparative effectiveness and safety of inhaled corticosteroids and antimicrobial compounds for non-CF bronchiectasis
Retrospective observational cohort
Propensity scores; survival or Kaplan Meier Analysis
A Comparison: High Intense periodic vs. Every week therapy in children with cerebral palsy
Linear mixed modeling; multivariate linear regression; sensitivity analysis
Developmental Trajectories of Impairments, Health, and Participation of Children with Cerebral Palsy
Prospective longitudinal cohort
Cluster analysis; linear and non-linear mixed effects modelling;
A stakeholder-driven comparative effectiveness study of treatments to prevent coronary artery damage in patients with resistant Kawasaki disease.
Fisher’s exact test; sensitivity analysis; mixed model repeated measures
PCORnet Patient Powered Research Networks focusing on Rare Diseases
Collaborative Patient-Centered Rare Epilepsy Network
Goal of the network is to build patient/caregiver-centered database designed to increase research opportunities for patients and caregivers
Inventory and collect information from existing patient registries, advocacy groups and design common elements; create a social network platform that enables communication between patients and researchers
The Vasculitis Patient Powered Research Network
Directly engages patients in order to explore research questions that matter most to patients – involve patients in study design, increase study eligibility
Phelan-McDermid Syndrome Data Network
Encourage active participation from patient families; develop multiple data feeds to extract and link data from patient cohorts
Empowering Patients and Families for Community-Driven Research: The DuchenneConnect Patient-Report Registry Infrastructure Project
Duchenne and Becker Muscular Dystrophies
Balance robust data collection with reducing burden and increasing benefits for registrants; integrate EHRs; evaluate patient-reported outcome accuracy; improve coding and standardize information exchange
NephCure Kidney Network for Patients with Nephrotic Syndrome
Change nature of NKN from static cross-sectional data to patient-reported outcomes database; establish network governance with active patient participation; collect data that are interoperable across research networks
The Patients, Advocates and Rheumatology Teams Network for Research and Service (PARTNERS) Consortium
Juvenile Rheumatic Disease
Extend current registry; create patient-centered learning health system; patients involved in governance structure
PI Patient Research Connection: PI-CONNECT
Primary Immunodeficiency Diseases
Create venue for researcher and patients to communicate about proposed research; use mobile apps to engage population; integrate existing medical records into the network; identify potential markers for risk stratification
Community Engaged Netowrk for All (CENA)
Alström syndrome; Dyskeratosis congenital; Gaucher disease; Hepatitis; Inflammatory breast cancer; Joubert syndrome; Klinefelter syndrome and associated conditions; Metachromatic leukodystrophy; Pseudoxanthoma elasticum (PXE); Psoriasis
Launch or upgrade online registries; participants determine to whom and for what purpose their information is shared; participant-led governance model
Two of the projects illustrate how infrastructure can support rare disease research. The first study, which compares the outcomes of different entry sites for shunt insertion surgeries to treat hydrocephalus, leveraged the existing Hydrocephalus Clinical Research Network to prioritize the study question, to design the study, and to accelerate enrollment. The second study, which compares a combination therapy to monotherapy anti-TNF in the pediatric Crohn’s Disease, leverages the ImproveCareNow network to recruit participants who are starting anti-TNF treatment.
Table 4 summarizes the focus and goal of each of the 9 rare disease PCORNet patient-powered research networks (PPRNs). The PPRNs currently support 16 studies. The network infrastructure supports both identification of potential participants and patient engagement in the development of research questions and study design. In addition, some PPRNs have undertaken projects to improve methods of data collection. For example, the DuchenneConnect Patient-Report Registry Infrastructure Project has as its goal to reduce the burden of data collection, evaluate the accuracy of patient-reported outcomes, improve coding, and standardize information exchange. The NephCure Kidney Network for Patients with Nephrotic Syndrome aims to improve the interoperability of data collection across networks.
Our landscape review of the literature describes available methodological and analytic approaches that could be used to address the challenges of conducting research in rare diseases, especially the challenge of conducting research with small patient populations. We identified three frameworks that attempted to tie attributes of interventions and rare diseases to specific methodological approaches. As displayed in Table 1, specific features of diseases, interventions, and outcomes have implications for study design choices. In some case, these attributes are limitations and exclude certain designs. In other cases, variations on the standard randomized controlled trial can ameliorate such barriers. For example, as shown in Table 2, both crossover and adaptive RCTs can minimize the time that patients spend on placebo or suboptimal treatments. In any disease, especially those that are serious or life threatening, minimizing exposure to placebo or suboptimal interventions can increase patients’ willingness to be recruited into and be retained in a trial. While the literature we identified focuses predominantly on pharmacologic research, many of the principles that apply to pharmacologic studies also apply to device studies. The exception is that in device studies, blinding of participants and research staff is more difficult and may require, for example, independent outcome assessment.
A central problem of rare disease research is how to avoid conducting underpowered studies. Underpowered studies are “waste in research” [34–36]. While we found limited analysis of the benefits of existing infrastructure in the literature we reviewed, we were able to draw inferences about the benefits of leveraging the capabilities of a research community. A prominent use of networks and registries was to promote contact databases for identifying and recruiting eligible participants. In a well-organized research community, effective research prioritization and agenda setting can reduce waste in research. If a research network or registry engages the specialist community for a particular rare disease, that community can collectively determine which research questions are most critical. For example, the Hydrocephalus Clinical Research Network (hcrn.org) lays out a research agenda on its website and the investigators collaborate to complete research initiatives that fulfill the agenda, thus ensuring that there are not multiple studies competing for the same small patient population. Moreover, defining core outcome sets to be used in registries and studies of rare diseases facilitates aggregation of data over time and comparison across interventions and subpopulations. A corollary role for a research network is to terminate poorly performing studies. If it is clear early on that a study will not be successful, those resources can be redirected to more fruitful avenues. However, while disease registries and the communities that maintain them can be critical to advancing rare disease research, propriety data arrangements can complicate the creation and sustainability of a robust registry.
Our review of the PCORI-funded research portfolio shows 11 RCTs in progress, most using standard RCT approaches, which demonstrates that it is possible to conduct randomized comparative trials in rare diseases. Use of a broader array of methodological approaches could expand the range of diseases feasible to study under PCORI funding. Work underway by the PPRNs shows how a network of engaged patients and researchers might make durable improvements to the research infrastructure, as is demonstrated by the examples we provided of the DuchenneConnect Patient-Report Registry Infrastructure Project and the NephCure Kidney Network for Patients with Nephrotic Syndrome.
Opportunities for methods development that were beyond the scope of this review are designs for studying complex interventions and health system-level interventions in rare diseases. Healthcare delivery and behavioral interventions that seek to improve health care management may improve the quality of care delivered to patients with a variety of rare diseases and may therefore have a cross-cutting impact on patient outcomes. However, these interventions are often complex, involving multiple components, and their success is often dependent on the environment in which they are implemented. Presently, the PCORI Methodology Committee is developing standards on comparative effectiveness research on complex interventions .
Improving the ability to conduct research on rare diseases would have a significant impact on population health. While each rare disease affects a relatively small population, collectively a large number of individuals are affected by these conditions. Disease heterogeneity and geographic dispersion further contribute to the difficulty of completing robust studies in small populations. To raise awareness among key stakeholders of methodological and analytic approaches to these challenges we reviewed algorithms for matching study design to rare disease characteristics and summarized applicable methodological and analytic approaches. Use of these approaches can facilitate the completion of RCTs that are adequately powered. From this literature we were also able to draw inferences on how an effective research infrastructure can set an agenda, prioritize studies, accelerate accrual, catalyze patient engagement, and avoid waste in research. Reviewing the Patient Centered Outcomes Research Institute portfolio of funded studies on rare disease, there were 11 RCTs, most using standard designs. This suggests that use of broader array of methodological approaches to RCTs-- such as adaptive trials, cross-over trials, and early escape designs can improve the productivity of robust research in rare diseases.
The authors would like to acknowledge the following groups and individuals for their review of this manuscript: the PCORI Rare Disease Advisory Panel, the PCORI Methodology Committee, Dr. David Hickam, Dr. Bryan Luce, Dr. Lea Drye, and Gwendolyn Claybourne.
The views, statements, and opinions in this work are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee. While Danielle Whicher is now employed by the National Academy of Medicine, her work on this paper was primarily completed while she was employed at PCORI. The views expressed in this paper do not represent the views of the NAM or the National Academies of Sciences, Engineering, and Medicine.
Availability of data and materials
Data sharing not applicable to this article as no datasets were generated or analysed during the current study.
DW and SP reviewed the literature search results on methodological approaches for rare disease research and the role of infrastructure in supporting rare disease research. DW was a major contributor in writing the manuscript, especially the results section on existing algorithms for selecting appropriate study designs. SP was a major contributor in writing the manuscript, especially the application to the PCORI Rare Disease portfolio. NA provided extensive input on the framing and direction of the manuscript and was a major contributor in writing the manuscript. All authors read and approved the final manuscript.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Rare Disease Act of 2002, Pub. L. No. 107–280, 116 Stat. 1988 (2002).Google Scholar
- U.S. Department of Health & Human Services National Center for Advancing Translational Sciences. Genetic and Rare Diseases Information Center FAQs About Rare Diseases. 2010. Retrieved from: https://rarediseases.info.nih.gov/diseases/pages/31/faqs-about-rare-diseases.
- Selby JV, Beal AC, Frank L. The Patient-Centered Outcomes Research Institute (PCORI) National Priorities for research and initial research agenda. JAMA. 2012;307(15):1583–4.View ArticlePubMedGoogle Scholar
- The National Patient-Centered Clinical Research Network. About PCORNet. 2017. Retrieved from: http://www.pcornet.org/about-pcornet/.Google Scholar
- Patient-Centered Outcomes Research Institute. Advisory panel on rare disease. 2017. Retrieved from: https://www.pcori.org/engagement/engage-us/join-advisory-panel/advisory-panel-rare-disease.Google Scholar
- Lynn, K. Difference between a systematic review and a literature review. (2013). Retrieved from: https://libguides.sjsu.edu/c.php?g=230370&p=1528399Google Scholar
- Academy Health. Evaluating complex health interventions: challenges, goals, and proposals for progress. June 2016. Retrieved from: http://www.academyhealth.org/files/phsr/AH_Report_Evaluating_Complex_HSI_June2016_FINAL.pdfGoogle Scholar
- Gagne JJ, Thompson L, O’Keefe K, Kesselheim AS. Innovative research methods for studying treatment for rare diseases: methodological review. BMJ. 2014;349:g6802.View ArticlePubMedPubMed CentralGoogle Scholar
- Gupta S, Faughnan ME, Tomlinson GA, Bayoumi AMA. Framework for applying unfamiliar trial designs in studies of rare diseases. J Clin Epidemiol. 2011;64(10):1085–94.View ArticlePubMedGoogle Scholar
- Cornu C, Kassai B, Fisch R, et al. Experimental designs for small randomised clinical trials: an algorithm for choice. Orphet JRare Dis. 2012;8:48.View ArticleGoogle Scholar
- Detry MA. Analyzing repeated measurements using mixed models. JAMA. 2016;315(4)Google Scholar
- Bogaerts J, Sydes MR, Keat N, McConnell A, Benson A, Ho A, Seymour M. Clinical trial designs for rare diseases: studies developed and discussed by the international rare cancers initiative. Eur J Cancer. 2015;51(3):271–81.View ArticlePubMedPubMed CentralGoogle Scholar
- Chow SC, Chang M. Adaptive design methods in clinical trials - a review. Orphanet J Rare Dis. 2008;3:11.View ArticlePubMedPubMed CentralGoogle Scholar
- Facey K, Granados A, Guyatt G, Kent A, Shah N, van der Wilt GJ, Wong-Rieger D. Generating health technology assessment evidence for rare diseases. Int J Technol Assess Health Care. 2014;30(4):416–22.View ArticlePubMedGoogle Scholar
- Gerss JW, Kopcke W. Clinical trials and rare diseases. Adv Exp Med Biol. 2010;686:173–90.View ArticlePubMedGoogle Scholar
- Johnson SR, Feldman BM, Pope JE, Tomlinson GA. Shifting our thinking about uncommon disease trials: the case of methotrexate in scleroderma. J Rheumatol. 2009;36(2):323–9.PubMedGoogle Scholar
- Kesselheim AS, Gagne JJ. Strategies for postmarketing surveillance of drugs for rare diseases. Clin Pharmacol Ther. 2014;95(3):265–8.View ArticlePubMedGoogle Scholar
- Prasad V, Oseran A. Do we need randomised trials for rare cancers? Eur J Cancer. 2015;51(11):1355–7.View ArticlePubMedGoogle Scholar
- Tudur Smith C, Williamson PR, Beresford MW. Methodology of clinical trials for rare diseases. Best Pract Res Clin Rheumatol. 2014;28(2):247–62.View ArticlePubMedGoogle Scholar
- van der Lee JH, Wesseling J, Tanck MW, Offringa M. Efficient ways exist to obtain the optimal sample size in clinical trials in rare diseases. J Clin Epidemiol. 2008;61(4):324–30.View ArticlePubMedGoogle Scholar
- Kinder B, McCormack FX. Clinical trials for rare lung diseases: lessons from Lymphangioleiomyomatosis. Lymphat Res Biol. 2010;8(1):71–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Lagakos SW. Clinical trials and rare diseases. N Engl J Med. 2003;348(24):2455–6.View ArticlePubMedGoogle Scholar
- Gerß, J. W. O., & Köpcke, W. (2010). Clinical Trials and Rare Diseases. In M. Posada de la Paz & C. S. Groft (Eds.), Rare Diseases Epidemiology. Dordrecht: Springer Netherlands. 2010. (pp. 173–190).Google Scholar
- Adams, M., Berkman, N., Bishop, E., Lohr, K., Pan, H., Ringer, D., Viswanathan, M., & Whithead, N. Landscape review on rare disease research registries: final report. 2015. Retrieved from: http://www.pcori.org/sites/default/files/PCORI-Report-Landscape-Review-On-Rare-Disease-May-2015.pdfGoogle Scholar
- Abrahamyan L, Diamond IR, Johnson SR, Feldman BM. A new toolkit for conducting clinical trials in rare disorders. J Popul Ther Clin Pharmacol. 2014;21(1):e66–78.PubMedGoogle Scholar
- de Blieck EA, Augustine EF, Marshall FJ, et al. Methodology of clinical research in rare diseases: development of a research program in juvenile neuronal ceroid lipofuscinosis (JNCL) via creation of a patient registry and collaboration with patient advocates. Contemporary Clinical Trials. 2013;35(2):48–54.View ArticlePubMedPubMed CentralGoogle Scholar
- Richesson RL, Cuthbertson HSL, Lloyd J, Young K, Krischer JP. An automated communication system in a contact registry for persons with rare diseases: scalable tools for identifying and recruiting clinical research participants. Contemporary Clinical Trials. 2009;30(1):55–62.View ArticlePubMedGoogle Scholar
- Richesson R, Sutphen R, Shereff D, Kischer J. The rare diseases clinical research network contact registry update: features and functionality. Contemporary Clinical Trials. 2012;33(4):647–56.View ArticlePubMedPubMed CentralGoogle Scholar
- Hilbert JE, Kissel JT, Luebbe EA, et al. If you build a rare disease registry, will they enroll and will they enroll? Methods and data from the National Registry of Myotonic dystrophy (DM) and Facioscapulohumeral muscular dystrophy (FSHD). Contemporary Clinical Trials. 2012;33(2):302–11.View ArticlePubMedGoogle Scholar
- Bellgard MI, Macgregor A, Janon F, et al. A modular approach to disease registry design: successful adoption of an internet-based rare disease registry. Human Mutation, Brief. 2012;33:E2356–66.View ArticleGoogle Scholar
- Krischer JP, Gopal-Srivastava R, Groft SC, et al. The rare diseases clinical research Network’s organization and approach to observational research and health outcomes research. J Gen Intern Med. 2014;29(Suppl 3):S739–44.View ArticlePubMedGoogle Scholar
- Institute of Medicine. Rare diseases and orphan products: accelerating Research and Development. 2010. Washington, DC: the: National Academies Press; 2010.Google Scholar
- Thompson R, Johnston L, Taruscio D, et al. RD-Connect: an integrated platform connecting databases, registries, biobanks and clinical bioinformatics for rare disease research. J Gen Intern Med. 2014;29(Suppl 3):780–7. https://doi.org/10.1007/s11606-014-2908-8.View ArticlePubMed CentralGoogle Scholar
- Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.View ArticlePubMedGoogle Scholar
- Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.View ArticlePubMedGoogle Scholar
- Ioannidis JPA, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75.View ArticlePubMedPubMed CentralGoogle Scholar
- Patient-Centered Outcomes Research Institute. Methodology Committee. 2017. Retrieved from: https://www.pcori.org/about-us/governance/methodology-committee.