Directions for new developments on statistical design and analysis of small population group trials
- Ralf-Dieter Hilgers†1Email authorView ORCID ID profile,
- Kit Roes†2,
- Nigel Stallard†3 and
- for the IDeAl, Asterix and InSPiRe project groups
© The Author(s). 2016
Received: 25 March 2016
Accepted: 3 June 2016
Published: 14 June 2016
Most statistical design and analysis methods for clinical trials have been developed and evaluated where at least several hundreds of patients could be recruited. These methods may not be suitable to evaluate therapies if the sample size is unavoidably small, which is usually termed by small populations. The specific sample size cut off, where the standard methods fail, needs to be investigated. In this paper, the authors present their view on new developments for design and analysis of clinical trials in small population groups, where conventional statistical methods may be inappropriate, e.g., because of lack of power or poor adherence to asymptotic approximations due to sample size restrictions.
Following the EMA/CHMP guideline on clinical trials in small populations, we consider directions for new developments in the area of statistical methodology for design and analysis of small population clinical trials. We relate the findings to the research activities of three projects, Asterix, IDeAl, and InSPiRe, which have received funding since 2013 within the FP7-HEALTH-2013-INNOVATION-1 framework of the EU. As not all aspects of the wide research area of small population clinical trials can be addressed, we focus on areas where we feel advances are needed and feasible.
The general framework of the EMA/CHMP guideline on small population clinical trials stimulates a number of research areas. These serve as the basis for the three projects, Asterix, IDeAl, and InSPiRe, which use various approaches to develop new statistical methodology for design and analysis of small population clinical trials. Small population clinical trials refer to trials with a limited number of patients. Small populations may result form rare diseases or specific subtypes of more common diseases. New statistical methodology needs to be tailored to these specific situations.
The main results from the three projects will constitute a useful toolbox for improved design and analysis of small population clinical trials. They address various challenges presented by the EMA/CHMP guideline as well as recent discussions about extrapolation. There is a need for involvement of the patients’ perspective in the planning and conduct of small population clinical trials for a successful therapy evaluation.
KeywordsEMA/CHMP Guideline on clinical trials in small populations Statistical methods Statistical design Statistical analysis Small population clinical trials Rare disease
Most statistical design and analysis methods for clinical trials have been developed in the setting of confirmatory trials with relatively large sample sizes, perhaps several hundreds or even thousands of patients. These methods may not be suitable to evaluate therapies in small populations, for example when the size of the population is limited because a disease is rare, or a treatment is targeted at a particular genetic subgroup or a small paediatric population. Such cases, where there is a limited number of patients that could potentially be enrolled in the trial, raise a number of specific statistical challenges and can lead to slow approval of orphan drugs for marketing and poorly designed studies .
The Asterix, IDeAl and InSPiRe projects
In the light of these challenges, in 2013 the European Commission set up a unique call for new methodologies for clinical trials for small population groups within the FP7-HEALTH-2013-INNOVATION-1  framework. This builds on other initiatives from around the world aimed at improving research, including methodology, in rare diseases . The objective of the research is to develop new or improved statistical methodologies for clinical trials for the assessment of treatments for small population groups, in particular for rare diseases or personalised (stratified or individualised) medicine. Research is expected to be multidisciplinary and should involve all relevant stakeholders including industry and patient advocacy groups as appropriate.
In the EU, a disease is defined as rare if the prevalence is not more than 5 in 10,000. New drug treatments for these rare diseases may be eligible for orphan drug designation, also depending on other conditions – such as the status of existing treatments. The number of rare diseases is estimated to be around 7000 . Thus although the diseases themselves are rare, the total number affected in the EU is estimated to exceed 30 million  with 6 % of the global population affected by a rare disease at some stage in their life , resulting in considerable total health care cost . More than 1120 orphan drug designations have been granted by the EMA since 2000, and 114 orphan drugs received market authorization by January 2016 . This seem to be small compared to the objective of the international rare disease research consortium to develop 200 new therapies by 2020 . Although many factors complicate the development of new medicines for orphan diseases, the main issue setting it apart from drug development for common diseases is the challenge of generating acceptable evidence from clinical trials in the clinical research phase in which recruitment is necessarily limited.
In the EU clinical trials in drug development for small population groups should take into account the EMA/CHMP guideline  together with a recently published reflection paper on extrapolation of efficacy and safety in medicine development . In the US the FDA drafted guideline on orphan drug approval has been published , whereas in Japan no such specific guidance for clinical trials in small populations exists. To summarize, the EMA/CHMP guideline states, that “No methods exist that are relevant to small studies that are not also applicable to large studies. However, it may be that in conditions with small and very small populations, less conventional and/or less commonly seen methodological approaches may be acceptable if they help to improve the interpretability of the study results” . Further it is recommended to use as much information as possible for designing a clinical trial and extract as much information as possible from a clinical trial to enable valid benefit risk assessment. Additionally the EMA/CHMP guideline  suggests avoiding unnecessary clinical trials e.g., by extrapolation from a larger source population to a smaller target population when this is appropriate.
This paper is structured around the headings in the EMA/CHMP guideline ; “Levels of evidence”, “Pharmacological considerations”, “Choice of endpoints” and “Methodological and statistical considerations”. We add “extrapolation” and “patient involvement and ethical considerations” as additional points, but do not consider specifically “Choice of control groups” except when this arises in association with the other headings. Under each heading we give a brief overview of the key statistical and methodological challenges and explain how the three EU funded projects Asterix, IDeAl, and InSPiRe address these challenges and will stimulate uptake of new methodology in practice. Some further aspects will be considered, where we feel that extensions are helpful, e.g., taking into account health economic aspects.
Biostatistical Research by Asterix, IDeAl and InSPiRe
Levels of evidence
An approach will be developed based on Bayesian decision theory to obtain optimised clinical trial designs that account for prior belief regarding the benefits of treatment and the population size.
Methods to relax the traditionally applied levels of type 1 and type 2 errors based on the anticipated finite (and relatively small) future population to treat will be developed, essentially based on a mix of Bayesian and frequentist methods.
Value of information methods will be applied to assess appropriate sample sizes for clinical trials in small populations.
In small clinical trials properties of randomization procedure relying on approximate arguments fail to suffice. Recommendations for selection of the best practice randomization procedure maintaining the significance level in the presence of selection bias will be derived.
Determination of appropriate levels of evidence for decision-making in small population clinical trials will be investigated.
When evidence from randomised controlled trials is limited or absent, evidence from lower levels in the hierarchy, including observational studies or case-reports, may become of greater importance together with systematic review and synthesis of all available information, whatever its source. The formal synthesis of evidence from sources of differing quality, briefly evidence synthesis, using either Bayesian or frequentist meta analysis methods, can present particular statistical challenges.
Evidence synthesis methods for small populations and rare diseases will be developed to support the planning, analysis and interpretation of a single randomised controlled trial. The feasibility and utility of the newly developed methods will be assessed in small population settings.
Generalized evidence synthesis approaches will be applied to paediatric studies and compounds developed for potentially multiple rare indications.
Evidence synthesis methods across trials (of similar or different design) that take into account the sequential nature of drug development will be developed.
The “pharmacological considerations” section of the EMA/CHMP guideline suggests that the design of a clinical trial could be improved by detailed pharmacological knowledge of the disease and the drug under consideration . As elaborated in the guideline, knowledge about the variability is essential for efficient study design and use of the best available techniques to obtain and analyse information is crucial. It is well known that poor designs may be a unnecessary source of variation.
Many drugs are metabolized in a nonlinear e.g., exponential fashion. Thus the description of pharmacological processes needs the application of complex statistical models, such as non-linear mixed effects models. These models allow the evaluation of drug profiles resulting from repeated drug concentration measurements from a group of patients while considering individual variation. As these models are extensions of the usual linear mixed effects models, they are nonlinear in the parameters to be estimated, which complicates the optimal design problem.
Pharmacometric  approaches will be considered by modelling the disease – therapy relation using non-linear mixed effect models. These theoretical models are used for improved sample size estimation in small clinical trials allowing for uncertainty, defining appropriate outcome measures and investigating analysis methods.
Optimal design techniques, resulting in time points of observation based on minimization of the variance of the effects estimators in nonlinear mixed-effects models will be developed by means of using preliminary estimates from interim looks to improve the final estimator. New numerical techniques in case of small samples where asymptotic assumptions are no longer valid.
Innovative designs for early phase clinical trials taking into account safety, efficacy and pharmacokinetic/pharmacodynamic (PK/PD) measures will be developed to better estimate the dose level to be recommended based on limited sample sizes and subgroups with continuous and binary outcomes.
The performance of the novel methods will be evaluated in terms of information gain, number of subjects, efficiency, and robustness. Designs for within patient data modelling will be developed to allow as much information as possible to be used for benefit risk assessment.
Improved methods for identification of genetic prognostic factors will be derived, leading to efficient clinical trial design and analysis.
Choice of endpoints
A suitable framework for validation of biomarkers as surrogate endpoints in small clinical trials will be developed and optimal designs investigated. Hereby, the most appropriate approach, e.g., the causal inference and meta analytical paradigm will be evaluated within the context of small samples.
The impact of unbalanced small data sets on numerical accuracy for mixed effects models used for surrogate endpoint validation will be investigated.
Recommendations for models to describe the reliability of measurements in longitudinal studies, where the number and measurement times differs between subjects and the number of longitudinal measurements is large compared to the number of subjects of the trial will be established.
The possibility of using an individualized outcome measurement instrument, called Goal Attainment Scale , will be investigated. This may enable generation of evidence in diseases for which patient variability precludes the use of conventional variables to demonstrate efficacy.
Methodological and statistical considerations
Specific designs which are recommended in small population clinical trials will be evaluated concerning, whether it is possible to estimate the treatment effect without bias.
Once the treatment estimate could be estimated without bias, optimal designs minimizing the variation of the estimates, e.g., in non-linear mixed effects models to analyse data in small population groups will be derived.
The impact of selection and time trend bias on the validity of trial results depending on the selected randomisation procedure will be investigated and recommendations derived for different endpoints such as continuous or time to event.
A framework for scientific arguments to select the “best” randomisation procedure reflecting the practical clinical constraints will be derived.
A bias corrected randomisation test will be developed.
The advantage of covariate or stratified randomisation procedures in small population clinical trials will be examined.
The benefits and limitations of response adaptive randomisation in small population clinical trials will be evaluated.
Adaptive designs such as multi-arm multi-stage trials suitable for small populations will be developed.
Modelling and extrapolation tools will be incorporated in adaptive designs for confirmatory studies.
Adaptive designs that take into account multiple endpoints, biomarkers or surrogate endpoints efficiently will be developed.
The use of adaptive designs within a trial will be compared with the use of adaptive strategies across trials in a drug development programme.
The use of adaptive estimation in non-linear mixed effects models to gain in precision and sample size will be explored.
The extent of bias due to interim analysis will be investigated.
Pharmacogenetic information for tailored therapeutics including its application in crossover trials, n-of-1 trials and enrichment trials will be investigated.
Randomisation and bias arguments for the validity of n-of-1 trials will be elaborated and randomisation based inference will be developed.
The use of randomisation-based inference in small clinical trials, where asymptotic behaviour of parametric tests is questionable will be evaluated. In particular, randomisation tests for complex statistical models and response adaptive randomisation procedures will be developed
The gain in efficiency of stratification in small clinical trials will be investigated.
The use of non-linear and linear mixed effects models for longitudinal, hierarchical, and clustered data in small data sets and for unbalanced data will be considered. In particular, non- or semi-parametric methods that do not rely, or rely less, on asymptotic arguments will be developed.
More complex models that include genetic factors that influence the response to therapy in small population groups will be studied.
Longitudinal data analysis methods will be improved with respect to non-parametric methods to overcome potential weaknesses caused by the asymptotic theory.
Frequentist and decision theoretic methods will be developed to predict patients’ responses to targeted treatments based on genetic features or other biomarkers enabling subgroups of patients for which the benefit risk balance of a treatment is positive to be identified and confirmed.
The decision analysis framework will be expanded by explicitly studying the inter-relationship between the decision analyses made by health technology assessors, regulators, patients and trial sponsors.
The way in which decision analysis affects the design of a clinical trial will be evaluated.
The uncertainty in dose–response information when knowledge of disease and drug is used to extrapolate from a large population to a much smaller target population will be quantified.
The use of extrapolation within a Bayesian framework will be explored.
Adaptive design strategies for extrapolation studies will be considered.
Guidance on the data to include in registries to support extrapolation and control group data for trial design will be developed.
Patient involvement and ethical considerations
A “Patient Think Tank” is established within one of the projects to directly involve the patient voice in a project and to work together on models for patient involvement in design stages of a trial. Patient organizations are represented in this Think Tank.
Involvement of patients at the design stage is also aimed to assess impact of design features (such as assessment schedules, informed consent procedures, multiple treatment arms) on willingness and ability to participate.
To what extent does our ethical framework need to be adjusted to accommodate using innovative trial designs in the context of research on orphan diseases?
Could we ethically accept less stringent standards of evidence in the context of rare diseases in order to benefit people with orphan diseases sooner?
Ethical perspectives are reflected in the projects in particular in the extended advisory boards. In answering these questions, the framework of Emmanuel  is used as starting point.
Regulatory impact and dissemination
For regulatory impact it is essential that specific guidance can be given on the use and acceptability of new methods. This is where the current guidance may fall short. As there are several thousands of rare diseases development of disease specific guidelines is impractical. To support regulator impact we propose an intermediate approach based on clustering of diseases sharing similar characteristics that determine similar methodological approaches to their study. While previous approaches have focused on the selection of the best design for a single disease, through algorithms guided on disease and treatment characteristics [31, 32], the aim of our clustering is wider, in the sense of serving as the framework to produce methodological and regulatory recommendations that can be valid not only for a single study or disease, but for the general approach to product development of a group of diseases sharing similar traits.
Awareness of biostatistical methods
A major goal of all three projects is to raise awareness and to disseminate newly developed statistical methods throughout the scientific community involved in clinical research. We will undertake dissemination activities such as teaching relevant groups and informing the scientific community and general public about the project through our websites (www.asterix-fp7.eu, www.ideal.rwth-aachen.de, www.warwick.ac.uk/inspire). We will publish our findings in peer-reviewed journals and stimulate the process of revision or development of regulatory guidelines as necessary. Information about the newly developed results will be targeted towards all relevant groups, e.g., regulators, patient advocacy groups, industry, the scientific community, etc. Further dissemination activities will include giving information about our methods to patient, families, carers, clinicians, social services, regulators, etc., and training of early stage researchers and other interested parties. Results are discussed on a regular basis with the projects’ Advisory Boards that include representatives from EMA, eurordis and patient organisations, amongst others. Although funding is organised around three projects, the scientists involved collaborate closely to ensure that work progresses efficiently and without duplication. Some pairs of projects have members in common, as do project advisory boards, and representatives of the three projects meet regularly at conferences and scientific meetings, where current and planned research directions are presented and discussed. In some cases, this has led to closer links between projects with groups comprising scientists from more than one of the funded projects collaborating on a specific topic.
The three projects address open statistical methodological design and analysis problems for clinical trials in small population groups, encompassing areas like rare diseases in paediatrics, nephrology, etc. and as well as subgroups of populations, and individualised therapy evaluation.
The description above shows the wide spread and interwoven structure of the research, which has to be addressed. Obviously, additional research will be stimulated by the investigations. The main gain will be that new methods are available with proven efficiency for small population clinical trials. The derived methods will serve as a basis for small population clinical trials in the future.
This paper aims to raise awareness of the ongoing research and stimulate other groups to work on statistical methodology on design and analysis of clinical trials in small populations. Having well informed researchers, physicians and biostatisticians will result in the use of more efficient methods to conduct a clinical trial in small population groups and thus bring approved treatments faster to our patients.
ASTERIX, Advances in Small Trials dEsign for Regulatory Innovation and eXcellence; EMA, European Medical Agency; EU, European Union; EURORDIS, European Organisation for Rare Diseases; FDA, U.S. Food and Drug Administration; IDeAl, Integrated Design and Analysi of small population group trials; InSPiRe, innovative methodology for small populations research; PD, Pharmacodynamic; PK, Pharmacokinetic; pocri, Patient-Centered Outcomes Research institute
This work has received funding from the European Union Seventh Framework Programme [FP7 2007–2013] under grant agreement no.: Health-F5-2013-603160 (Asterix project), 602552 (IDEAL project) and 602144 (InSpiRe project).
The Asterix, IDeAl and InSPiRe investigators are Corinne Alberti (InSPiRe, PhD, INSERM, Paris, France), Caroline van Baal (Asterix, PhD, University Medical Center Utrecht, The Netherlands), Norbert Benda (InSPiRe, PhD, BfArM, Berlin, Germany), Egbert Biesheuvel (Asterix, PhD, EFSPI, UK), Carl Fredrik Burmann (IDeAl, PhD, Chalmers University of Technology, Göteborg, Sweden), Malgorzata Bogdan, (IDeAl, PhD, Warschau University, Warschau, Polen), Emmanuelle Comets (InSPiRe, PhD, INSERM, Paris, France), Simon Day (InSPiRe, PhD, CTCT Ltd, Buckingham, UK), Holger Dette (IDeAl, PhD, Ruhr University Bochum, Germany), Alex Dmitrienko (InSPiRe, PhD), Tim Friede (InSPiRe, PhD, University Medical Center, Göttingen), Alexandra Graf (InSPiRe, PhD, Medical University Vienna, Austria), Mats Karlsson (IDeAl, PhD, UPPSALA University, Uppsala, Sweden), Armin Koch (Asterix, PhD, Hannover Medical School, Germany),Franz König (IDeAl, PhD, Medical University Vienna, Austria), Johanna H. van der Lee (Asterix, MD, PhD, Amsterdam Medical Center, The Netherlands),Frederike Lentz (InSPiRe, PhD, BfArM, Berlin, Germany), Jason Madan (InSPiRe, PhD, Warwick Medical School, UK), Christoph Male (IDeAl, PhD, Medical University Vienna, Austria), France Mentré (IDeAl, PhD, INSERM Paris, France), Frank Miller (InSPiRe, PhD, Stockholm University, Sweden), Geert Molenberghs (IDeAl, PhD, I-BioStat, UHasselt, Hasselt &, KU Leuven, Leuven, Belgium), Beat Neuenschwander (InSPiRe, PhD, Novartis, Basel, Switzerland), Martin Posch (Asterix, InSPiRe, PhD, Medical University Vienna, Austria), Cor Oosterwijk (Asterix, PhD, VSOP), Christian Röver (InSPiRe, PhD, University Medical Center, Göttingen), Stephen Senn (IDeAl, PhD, LIH Luxembourg, Luxembourg), Ferran Torres (Asterix, University of Barcelona (UAB), Spain), Sarah Zohar (InSPiRe, PhD, INSERM, Paris, France).
All three authors contributed equally in conceiving the manuscript and read and approved the final manuscript. RDH is principal investigator and responsible for designing the IDeAl research project. KR is principal investigator and responsible for designing the Asterix research project. NS is principal investigator and responsible for designing the InSPiRe research project.
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Joppi R, Bertele V, Garattini S. Orphan drug development is progressing too slowly. Br J Clin Pharmacol. 2006;61(3):355–60. doi:10.1111/j.1365-2125.2006.02579.x.View ArticlePubMedPubMed CentralGoogle Scholar
- EU. FP7-HEALTH-2013-INNOVATION-1. [Online] 2013. [Cited: 23 03 2016.] https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/fp7/calls/fp7-health-2013-innovation-1.html. Accessed 8 June 2016.
- Molster C et al. Key outcomes from stakeholder workshops at a symposium to inform the development of an Australian national plan for rare diseases. Orphanet J Rare Dis. 2012;7:50. doi:10.1186/1750-1172-7-50.View ArticlePubMedPubMed CentralGoogle Scholar
- Orphanet. Orphanet Report Series - List of rare diseases - listed in alphabetical number. [Online] December 2012. [Cited: February 1, 2013.] http://www.orpha.net/orphacom/cahiers/docs/GB/List_of_rare_diseases_in_alphabetical_order.pdf. Accessed 8 June 2016.
- Eurordis. Fact Sheet. [Online] 2007. [Cited: February 1, 2013.] http://www.eurordis.org/sites/default/files/publications/Fact_Sheet_RD.pdf. Accessed 8 June 2016.
- Colledge VL, Solly J. The rare disease challenge and how to promote a productive rare disease community: Case study of Birt-Hogg-Dube Symposia. Orphanet J Rare Dis. 2012;7:63. doi:10.1186/1750-1172-7-63.View ArticlePubMedPubMed CentralGoogle Scholar
- Breithaupt C. Contribution to the treatment of rare diseases: orphan drugs. J Eur Medical Writers Assoc Write Stuff. 2009;18(1):21–3.Google Scholar
- EMA. Orphan Medicines Figures 2000. Orphan Medicines Figures 2000/2014 Update 1/1/2015. [Online] 01 01 2015. [Cited: 26 02 2015.] www.ema.europa.eu/docs/en_GB/document_library/Other/2015/04/WC500185766.pdf. Accessed 8 June 2016.
- irdirc. Current Results of Research. Current Results of Research. [Online] 2016. [Cited: 25 05 2016.] http://www.irdirc.org/rare-diseases-research/current-results-of-research/. Accessed 8 June 2016.
- CHMP. Guideline on clinical trials in small populations. [Online] 2007. [Cited: February 1, 2013.] www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003615.pdf. Accessed 8 June 2016.
- EMA. Concept paper on extrapolation of efficacy and safety in medicine development. [Online] 2012. [Cited: 1 February 2013.] www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129285.pdf. Accessed 8 June 2016.
- FDA. Rare Diseases: Common Issues in Drug Development Guidance for Industry. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM458485.pdf. [Online] 08 2015. [Cited: 28 02 2016.]. Accessed 8 June 2016.
- Karlsson, K E. Benefits of pharmacometric model-based design and dnalysis of clinical trials. Acta Universitatis Upsaliensis. 2010;133. http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A360473&dswid=1152. Accessed 8 June 2016.
- Barrett J. Pharmacometrics: a multidisciplinary field to facilitate critical thinking in drug development and rranslational research settings. J Clin Pharmacol. 2008;48:632.View ArticlePubMedGoogle Scholar
- Burzykowski T, Molenberghs G, Buyse M. The Evaluation of Surrogate Endpoints. New York: Springer; 2005.View ArticleGoogle Scholar
- pcori. Patient-Centered Outcomes Research Institute. Patient-Centered Outcomes Research Institute. [Online] 2016. [Cited: 25 may 2016.] http://www.pcori.org. Accessed 8 June 2016.
- Frank L, Basch E, Selby JV. The PCORI perspective on patient-centered outcomes research. J Am Med Assoc. 2014;312(15):1513–4. doi:10.1001/jama.2014.11100.View ArticleGoogle Scholar
- Gargon E et al. Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One. 2014;9:6. doi:10.1371/journal.pone.0099111.View ArticleGoogle Scholar
- Kiresuk TJ, Sherman RE. Goal attainment scaling: A general method for evaluating comprehensive community mental health programs. Community Ment Health J. 1968;4(6):443–53. doi:10.1007/BF01530764.View ArticlePubMedGoogle Scholar
- Pocock SJ, Geller NL, Tsiatis AA. The analysis of multiple endpoints in clinical trials. Biometrics. 1987;43(3):487–98.View ArticlePubMedGoogle Scholar
- Posch M, Futschik A. A uniform improvement of Bonferroni-type tests by sequential tests. J Am Stat Assoc. 2008;103(481):299–308. doi:10.1198/016214508000000012.View ArticleGoogle Scholar
- Bretz F et al. A graphical approach to sequentially rejective multiple test procedures. Stat Med. 2009;28(4):586–604. doi:10.1002/sim.3495.View ArticlePubMedGoogle Scholar
- eurordis. European Organisation for Rare Diseases.[Online] [Cited: 23 03 206.] http://www.eurordis.org/. Accessed 8 June 2016.
- Kennes LN et al. The impact of selection bias on test decisions in randomized clinical trials. Stat Med. 2011;30(21):2573–81. doi:10.1002/sim.4279.PubMedGoogle Scholar
- Bretz F et al. Adaptive designs for confirmatory clinical trials. Stat Med. 2009;28(8):1181–217. doi:10.1002/sim.3538.View ArticlePubMedGoogle Scholar
- Gerss JW, Köppke W. Clincal trials and rare diseases. Adv Exp Med Biol. 2010;686:173–90. doi:10.1007/978-90-481-9485-8_11.View ArticlePubMedGoogle Scholar
- FDA. Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials. [Online] 2010. http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm071121.pdf. Accessed 8 June 2016.
- Wästfelt M, Fadeel B, Henter JI. A journey of hope: lessons learned from studies on rare diseases and orphan drugs. J Intern Med. 2006;260(1):1–10.View ArticlePubMedGoogle Scholar
- van der Graaf R, Roes KC, van Delden JJ. Adaptive trials in clinical research: scientific and ethical issues to consider. JAMA. 2012;307(22):2379–80. doi:10.1001/jama.2012.6380.PubMedGoogle Scholar
- Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA. 2000;283(20):2701–11.View ArticlePubMedGoogle Scholar
- Gupta S et al. A framework for applying unfamiliar trial designs in studies of rare diseases. J Clin Epidemiol. 2011;64(10):1085–94. doi:10.1016/j.jclinepi.2010.12.019.View ArticlePubMedGoogle Scholar
- Cornu C et al. Experimental designs for small randomized clinical trials: an algorithm for choice. Orphanet J Rare Dis. 2013;8:48. doi:10.1186/1750-1172-8-48.View ArticlePubMedPubMed CentralGoogle Scholar