- Open Access
Patient reported outcome measures in rare diseases: a narrative review
Orphanet Journal of Rare Diseases volume 13, Article number: 61 (2018)
Rare diseases can lead to a significant reduction in quality of life for patients and their families. Ensuring the patients voice is central to clinical decision making is key to delivering, evaluating and understanding the efficacy of therapeutic interventions. Patient reported outcome measures (PROMs) are used to capture the patient’s views about their health status and facilitate our understanding of the impact of these diseases and their treatments on patient’s quality of life and symptoms.
This review explores some of the current issues around the utilisation of PROMs in rare diseases, including small patient populations and dearth of valid PROMs. Difficulties in validating new or current PROMs for use in clinical trials and research are discussed. The review highlights potential solutions for some of the issues outlined in the review and the implementation of PROMs in research and clinical practice are discussed.
Patient input throughout the development of PROMs including qualitative research is essential to ensure that outcomes that matter to people living with rare disease are appropriately captured. Given the large number of rare diseases, small numbers of patients living with each condition and the cost of instrument development, creative and pragmatic solutions to PROM development and use may be necessary. Solutions include qualitative interviews, modern psychometrics and resources such as item banking and computer adaptive testing. Use of PROMs in rare disease research and clinical practice offers the potential to improve patient care and clinical outcomes.
There are approximately 7000 identified distinct rare diseases affecting around 350 million people worldwide . In the European Union, a disease is deemed to be rare when it affects less than 1 per 2000 persons . In the United States, a disease is defined as rare when it affects fewer than 200,000 people at any given time. An estimated 80% of rare diseases have a genetic origin and the remaining 20% are as a result of infections (bacterial or viral), environmental factors, allergies, proliferative and degenerative causes . Approximately 75% of rare diseases affect children, 30% of whom die before their 5th birthday . Despite this, approximately 95% of rare diseases have no Food and Drug Administration (FDA) approved treatment. While only 140 orphan medicines have been authorised for use in the European Union since 2001, and 60% of these were designated for use in paediatric conditions [1, 4].
The challenges faced by patients with rare disease
The impact of rare diseases can be diverse and include loss of physical function, cognitive and communication impairments, as well as emotional problems and social isolation. This may present challenges for patients in their social interactions, work, and education and can also have an adverse impact on the quality of life of other family members .
Many patients with a rare disease find themselves with unmet clinical needs due to a number of issues associated with access to clinical care and information [6, 7]. Patients with the same diagnosis may present differently because of genetic variations and timing of presentations. This can result in a delay to diagnosis, or misdiagnosis, because they cannot access specialists with knowledge of their disease [7, 8]. Even after receiving a diagnosis, accessing therapeutic interventions and appropriate drugs may be difficult as treatment options maybe non existent or restricted and often costly [8, 9]. Specialist clinics are often held infrequently, are limited in number and are located in regional centres, therefore patients have to travel considerable distances to access expert care [7, 9]. This can result in significant financial burdens associated with travel costs and the need to take time off work to undergo therapeutic procedures or clinic visits and may prevent some patients from accessing the expert care they need [8, 9]. Accessing quality information can also be difficult for both patients and their families because little may be known or published about their rare disease, therefore limiting their access to healthcare options or informed care [7, 10].
The impact of rare diseases on a patient’s daily activities means individuals are dependent on families or carers to manage their everyday needs . This loss of autonomy can have a negative impact on their quality of life (QoL) and has implications for their family and carers, given the burden of care and the young age of many of those diagnosed . As a result, families have reported psychosocial concerns and feelings of isolation and depression . The diverse presentations and progressive nature of many rare diseases make it important to chronicle an individual’s natural history and the extent to which their health and social care needs are changing .
What are patient reported outcome measures?
One way of monitoring changes in the natural history and disease progression of a rare disease is through the use of patient reported outcome measures (PROMs) [12, 13]. PROMs are reports directly obtained from patients about their health status/condition or treatment without interpretation by a clinician [14, 15]. PROMs are used in research and clinical settings to provide the patient’s views about a range of issues including symptom severity, function, psychological problems, treatment satisfaction and health-related quality of life (HRQoL) .
PROMs are generally characterized as generic or disease-specific measures. Generic measures are designed to assess any disease condition or general population sample. These measures can be used to compare the health status across and between different disease groups [12, 17]. However, generic measures may exclude important aspects that are relevant to specific groups . Condition-specific measures are designed to assess the severity of symptoms, or functional limitations specific to a particular health condition or diagnostic group. Condition-specific PROMs may be more sensitive when monitoring symptoms and detecting changes in health within a specific homogeneous group .
It is also recognised that for a condition specific PROM to have content validity it should have been developed with input from service users who have the lived experience of the condition [19, 20]. The European Medical Agency (EMA)  and the Food and Drugs Agency (FDA) have recognised the importance of including the patient’s perspective in development of PROMS and the FDA guidelines state that patient input is a requirement for applications using PROMs to support medical labelling claims [15, 22]. It has also been demonstrated that PROMs developed using the patient’s perspective are more robust and provide more sensitive and specific measurements .
PROMs available for use in patients with rare diseases
PROMs data, if collected, analysed and reported appropriately, can inform shared-decision making, pharmaceutical labelling claims, clinical guideline development and health policy . PROMs also have an important role as primary or secondary endpoints in clinical trials and other epidemiological studies, developing and evaluating new therapies . However, there are a number of issues relating to the use of PROMs in rare diseases. These include: a dearth of PROMs validated for the target population, especially for children and adolescents; sampling issues, data collection and statistical analysis issues secondary to small sample sizes and heterogenous study populations; and diverse contexts for recruitment such as multi-centre international studies.
PROMs in rare disease research
Despite these issues, however, PROMs are being used to assess the impact of rare diseases on quality of life. A systematic review on quality of life in rare genetic conditions concluded that the Short Form-36 item health survey (SF-36), Cystic fibrosis questionnaire and Dermatologic-specific QoL scales were the most frequently used measures in primary studies . Another review focussing on children and adolescents with rare diseases, identified that the Child Health and Illness Profile (CHIP), pediatric quality of life inventory (PedsQL)  and KIDSCREEN  questionnaires were the measures of choice for measuring HRQoL .
As well as these generic HRQoL tools, there are a number of PROMs that have been developed specifically for rare diseases, these include: the phenylketonuria-specific Quality-of-Life questionnaire (PKU-QoL) [28, 29]; Birdshot chorioretinopathy (BCR) Birdshot Disease & Medication Symptoms Questionnaire (BD & MSQ), quality of life impact of BCR (QoL-BCR) and the QoL impact of BCR medication (Qol-Meds) ; the Kids’ ITP tool used for children with immune thrombocytopenic purpura (ITP) ; and the Fabry-specific Paediatric Health and Pain Questionnaire (FPHPQ) .
Despite these examples, most rare diseases lack disease-specific PROMs that can be used to gain a better understanding of the issues that patients experience . A systematic review focussing on the impact of enzyme replacement therapy on HRQoL in Fabry disease (FD) found HRQoL was lower in the FD population and efficacy of current therapies was inconclusive . They found limitations in some of the generic HRQoL PROMs used in the study, and suggested that a FD specific PROM would be more accurate when monitoring patients.
Why PROMs are important in clinical practice
Routine use of PROMs in clinical practice has been shown to improve patient satisfaction with their care, symptom management, quality of life and survival rates [35,36,37]. It may also benefit clincians by improving clinician satisfaction, helping reduce burnout and increasing workflow efficiency, as routine screening questions may be asked in the waiting room and clinicians can use results from PROMs to inform the consultation . This can help promote patient-centred care by improving communication between clinicians and patients about their disease progression and the impact of prescribed treatments . PROMS can also support clinical decision-making when prescribing therapeutic interventions and managing side effects. In some specialities (e.g. cancer, mental health), PROMs are used to assess disease severity, monitor symptoms and responses to therapeutic interventions . PROMs can also identify adverse events and screen patients for physical and emotional problems requiring additional clinical intervention [40, 41]. While these are considerable advantages, PROMs data is still not routinely collected in all areas of clinical practice especially in the case of rare diseases .
Understanding what matters to patients living with rare diseases
In order for PROMs to be effective in clinical practice they have to capture the disease characteristics that matter to the patient. Selection of suitable PROMs should reflect these properties and domains as well as reflecting the natural disease history and prognosis. This can be challenging because of the heterogeneity of patients experiences, and differences in presentations caused by cultural influences, genetic and phenotype variations . Therefore its important to understand these variances when establishing key characteristics and selecting suitable PROMs. A recent taskforce report suggested one way of identifying patient preferences in rare diseases was through the use of qualitative interviews with patients, carers and their families . These could be used to elicite common core domains and symptoms. They also suggested key areas for research focus should include:
Outcomes that matter to patients (including PROMs)
The views of patient and clinicians regarding the use of PROMs in research and routine clinical practice.
Addressing these key areas would ensure patients voices are central to research and clinical practice. This would enhance our understanding of the experiences of people living with a rare disease, and help ensure that outcomes that matter are encapsulated in research and clinical practice. Patient Centred Outcome Measures (PCOMs) as proposed by Morel and Cano  can also provide additional benefits to health care providers by addressing the important issues that matter to patients, whilst providing a measure of patient benefit.
In addition to qualitative interviews other evidence to inform PROM development may include: literature reviews; media access, genotype-phenotype databases; observational studies; case histories; rare disease registries; and observer information e.g. parents and carers . Access to patient forums and blogs may also help researchers to gain insights into the issues that matter to patients . Patient and public involvement (PPI), and patient advocacy groups also present additional opportunities for accessing patients’ viewpoints [45, 46]. Encapsulating patients viewpoints into PROMs and collaborations between industry, academics and patient partners can improve the quality of data collected [47,48,49]. As demonstrated by Mesa et al.  who identified which symptoms were important to patients with myelofibrosis (a rare myeloproliferative neoplasm) through qualitative interviews. They used this information to create and test an appropriate PROM for use in a clinical trial of ruxolitinib . Using a PROM that was meaningful to the patients resulted in more than 95% compliance in the trial and the results demonstrated that the experimental group had a 50% or greater improvement in comparison to the control group . This resulted in approval of ruxolitinib by the FDA and EMA .
Using PROMs in patient registries
Patient registries are also a useful tool for the development of rare disease research and can help improve patient care and services, as well as development of new treatments [2, 51]. Rare disease registries have often grown organically and are usually initiated by organisations including patients, family, carers and advocacy groups, as well as industry partners, health services and clinicians . Registries are important resources for information on the natural history of diseases, supporting enrollment of patients to research and to assess the impact of new therapies in real-world settings . PROMs are increasingly being used in patient registries to understand the patient perspective and disease management strategies .Incorporation of PROMs in registries can improve our understanding of the impact of symptoms and QoL over the course of the disease and treatment . There are a large number of registries for rare diseases and and a number of them are collecting PROM data . Examples of these registries can be found online, including the UK Cystic Fibrosis registry  and the Cystic Fibrosis Foundation United States (US) . The lysosomal storage disorder registry is also an example of how conditions with a common link can be accomodated together e.g. Fabry disease,  Pompe  and Gaucher . This approach allows common themes to be identified and pools resources for families, clinicians and researchers.
Proms for assessing service delivery
As well as being an important tool in clinical care PROMs can be used to evaluate service delivery as demonstrated by the use of the EQ-5D for audit/benchmarking purposes in the UK National Health Service (NHS) since 2009. The EQ-5D in conjunction with disease specific measures are used to measure HRQoL change pre and post-surgery in Hip and knee replacements, hernia repairs and varicose vein surgery [60, 61]. PROM data collected at a national level in this way can inform patients’ decisions regarding the selection of their preferred health care provider. However such an approach is not without challenges, for example reducing levels of missing data and providing information on PROM results to patients in an easily accessible way can be difficult to achieve [62, 63]. Using PROMs as part of routine care would give patients an important say in their treatment and could be used as a performance metric to compare the quality and effectiveness of care providers. This is especially important given the complexities associated with their care pathways and dearth of specialist services .
The diversity of PROMs currently used in clinical trials and practice makes it difficult to compare results when different outcomes have been utilised. Consequently, there have been moves to develop recommended core outcome measures for different conditions for use in clinical trials and service evaluation. This approach has been developed by the International Consortium for Health Outcomes Measurement (ICHOM), their remit is to incorporate patient and clinician view points and develop standardised sets of outcome measures for specific medical conditions based on concerns that matter to patients, these can then be used to evaluate and compare services . Other approaches to developing core outcome sets include those developed by Core Outcome Measures in Effectiveness Trials (COMET) which brings together experts and researchers in specific conditions to develop core outcome sets recommended for use in trials and clinical practice. Another group taking a similar route includes the Outcome Measures in Rheumatology (OMERACT) initiative which also aims to develop a consensus on which outcome measures should be used in autoimmune and musculoskeletal conditions .
Selecting reliable and valid PROMS
Having identified which issues matter most to patients it then becomes important to ensure that selection of potential PROMs reflects those issues. Additional considerations include the psychometric validity of the PROM and the context in which PROM data are being collected. Ensuring the reliability, validity and acceptability of the measure is a crucial component of selecting PROMs for use in rare diseases. Evaluating the measurement properties of PROMs in rare disease is challenging because of the small numbers of patients and heterogenous populations. Therefore, some of the usual options for choosing, adapting or developing PROMs are not always appropriate or available for this group of patients and may require adaptation . Key to this process is understanding the construct of interest (COI) and a clearly defined rationale for using PROMs within a research study in order to reach a meaningful conclusion about the outcome/endpoints that will be measured . When deciding on the PROM to use, the COI should meet the aim of the research study e.g. symptom burden, or HRQoL . Additionally, the measure should be selected according to the population of interest e.g. disease characteristics. Information gleaned from qualitative interviews on which symptoms and outcomes are important to the patients can inform this process. However, given the challenges of recruitment in rare diseases it may be more pragmatic to use existing measures with additional items to capture specific disease issues or domains .
A systematic review may be required to identify the most suitable PROMs . Guidelines developed by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative aims to help improve the selection of outcome measurement for research and clinical practice . This has included the development of a critical appraisal tool for evaluating the methodological quality of studies evaluating health measures such as PROMs. Systematic reviews can be used as tools to identify gaps in knowledge on the quality/measurement properties of PROMs or potential PROMs for use in rare diseases . An example of how COSMIN guidelines can be used in practice is demonstrated in a systematic review of PROMs used in Primary Sclerosing Cholangitis (PSC), conducted by the authors. The review found that a wide diversity of PROMs were used to measure HRQoL and symptom burden in patients with PSC. However, the quality of the evidence to support reliability and validity in the most commonly used measures was generally poor and only one had been validated within this population following FDA guidelines [70, 71].
Additional sources of information include databases such as ePROVIDE™ which has a range of information on PROMs and includes some critical review on the measurement properties for each PROM . Additional databases include the Patient-Reported Outcomes Measurement Information System (PROMIS) which is a co-operative group program of research which aims to develop, validate and standardise item banks to capture PROM data across a wide range of conditions and domains for use in a range of chronic disorders and conditions .
PROMIS uses item response theory (IRT) and computer adaptive testing (CAT) technology to generate and validate item banks for specific domains . Access to electronic forms of data collection using different psychometric models offer some potential solutions to the issues that arise from using traditional methods such as classical test theory (CTT). These models include IRT, and Rasch measurement theory (RMT). Increasingly these different methods are being used to modify or evaluate PROMs for use in rare diseases [43, 75].
While it is important to collect a comprehensive range of information using PROMs often data includes a wide range of domains. Collecting information on every PROM for every domain can be burdensome and result in PROM fatigue and missing data. CAT technology is effective in reducing this burden, by only presenting a small but specific number of questions, sequentially selected based on responses to previous questions . This approach also means that condition specific items as highlighted by the patients living with the rare disease can be incorporated with readily available tools . This allows tailoring of the PROM to a patient’s specific symptoms or needs without the need to develop a completely new instrument. One of the drawbacks of this method are the large sample sizes required to carry out psychometric analysis for validation of the PROMs . Using RMT is a potential solution to this issue. Rare group populations can be combined with populations with similar disease presentations [78, 79]. Examination of differential item functioning (DIF) analysis by diagnostic group could identify if responses from these groups are equivocal, if they are then both samples can be used to validate PROMs . Items where DIF is an issue can be split and used with the appropriate population in an item bank.
Mattera et al.  used concept elicitation and cognitive interviews to refine potential items from the PROMIS physical function (PF) item bank for use with patients with Fibrodysplasia Ossificans Progressiva (FOP) (an ultra-rare and progressive disease). Concept elicitation interviews were used to identify the impact of FOP on daily life and the concepts that mattered to the participants living with FOP. Themes and concepts from the interviews were identified and items that reflected these were established from the selected PROMIS PF items. Cognitive interviews were used to ensure that selected items were relevant and understood by the participants. The findings of the study demonstrated that concept elicitation and cognitive interviews could support development or modification of instruments for use with rare diseases .
Cultural adaptation of PROMS
While pooling data maybe one option, participants recruited for clinical trials might not only be recruited from multiple-centres but also different countries . Therefore PROMs will be required in different languages in order to accommodate this type of rescruitiment. This raises a number of issues. Firstly, PROMs should have been translated using industry standard methods, as both language and cultural adaptations are required . Cultural adaptation ensures that the context and content is appropriate for the population. This aspect of translation is often overlooked when translating PROMs and this can have a negative impact on attrition rates and levels of missing data. There are a number of recommendations for adapting PROMs to ensure cross cultural validity and equivalence [81,82,83]. Pooling data from international studies maybe the only way of recruiting sufficient sample sizes to test the psychometric properties of a PROM or to establish suitable therapeutic interventions. However, because of the influences of cultural adaptations it then becomes important to ensure the cross-cultural validity and equivalence of PROMs before pooling data. One approach to doing this could be through the use of RMT. By using RMT it is possible to establish whether DIF by country or language is an issue . If DIF by country is not identified then cross-cultural equivalence can be assumed and pooled data results can be utilised in clinical trials . Regnault and Herdman (2015) also suggest using factor analysis and the Universalist model to establish whether the theoretical model fits the data. Confirmatory factor analysis and qualitative interviews are used to evaluate conceptual and cross cultural equivalence of PROMs . Utilising psychometric methods alongside qualitative methods can facilitate identification of the most appropriate PROMs for use in rare diseases and conditions of interest .
Technological advances and prom data collection
The rapid progression of alternative technologies, increased connectivity and ease of administration has seen the increasing use of electronic PROMs (ePROMs), allowing data to be collected internationally, locally and in real time. It also gives patients the freedom to complete PROMs using an electronic device of their choosing e.g. tablets, smartphones etc. and at a time that is convenient for them . This is particularly important for patients with unstable conditions or high symptom burden who are likely to deteriorate rapidly . Patients can also opt to complete PROMs prior to clinical visits . Technological advances also offer alternative and adaptive methods for data collection such as audio options for people with low vision and touch screens that eliminate the need for computer skills . Wearable technology and Telehealth are used to capture clinical data, allowing patients remote access to expert clinical care without the need for expensive travel. Incorporating PROM data capture in addition to clinical data would allow clinicians to evaluate the health status of patients between clinic appointments and monitor the impact or side effects of drug from therapeutic interventions [87, 88].
PROMs can be integrated into electronic healthcare records and the use of predetermined threshold absolute and change scores can generate an automatic notification that can be provided both to the patient and the clinical team where follow-up is required . Appointments can be changed if patients appear to be deterioriating or postponed if patients appear stable. Thereby improving access to timely health care interventions while reducing patients’ health and financial burdens, reducing service costs by utilising health service resources appropriately . The Patient-Centred Outcomes Research Institute (PCORI) have developed a comprehensive guide on how to integrate ePROMS into electronic health records .
This paper has highlighted some of the issues that have been identified in using PROMs in rare diseases, however, they are not insurmountable and we have offered a number of potential solutions. Identifying the issues for people living with rare diseases and ensuring that their views are central to health care and policy is paramount. It is also important to remember that having an appropriate PROM in place does not necessarily mean that data can be collected from all patients. Physical limitations, cognitive impairments and age may prevent some patients from completing the PROM. In these instances, alternate methods such as interviews or observer collected data maybe required to enable completion. The International Society for Pharmoeconomics and Outcomes Research (ISPOR) has recently published comprehensive guidance on how this can be achieved . There is also an increasing emphasis on integrating quality PROM data into clinical care enabling dialogue between the clinician and patients and facilitating joint decision making around care options. In some respects this is more important for patients with rare diseases, than other long-term conditions as often their voices are in the minority and their clinical options limited. The use of PROMs potentially provides a more complete picture of their experiences and enables better communication with the clinical team.
Rare diseases can be chronically debilitating or life-threatening conditions, leading to a significant reduction in QoL for both the patient and their families. Small patient populations and a relative scarcity of medical knowledge and specialists in rare diseases make it challenging to understand the burden of these diseases in patients. The importance of ensuring that the patients voice is central to clinical decision making is key to delivering, evaluating and understanding therapeutic interventions. However, this requires ways of evaluating outcomes that reflect patients needs and concerns as well as clinically driven measures and outcomes. The dearth of valid PROMs for use in many rare conditions and difficulties in validating currently available PROMs makes evaluating effective treatments problematic. Patient input throughout the development of PROMs including qualitative research is essential to ensure that outcomes that matter to people living with rare disease are appropriately captured. Given the large number of rare diseases, small numbers of patients living with each condition and the cost of instrument development, creative and pragmatic solutions to PROM development and use may be necessary, such as the development of item banks and use of CAT. Use of PROMs in rare disease research and clinical practice offers the potential to improve patient care and outcomes.
Birdshot disease & medication symptoms questionnaire
Computer adaptive testing
Child health and illness profile
Concept of interest
Core outcome measures in effectiveness trials
Consensus-based standards for the selection of health measurement instruments
Classical test theory
Differential item functioning
European medicines agency
Food and drug administration
Fibrodysplasia ossificans progressiva
Fabry-specific paediatric health and pain questionnaire
Health-related quality of life
International consortium for health outcomes measurement
Item response theory
International society of quality of life research
International society for pharmoeconomics and outcomes research
Immune thrombocytopenic purpura
Lysosomal storage diseases
National health service
National institutes of health
Patient centred outcome measures
patient-centred outcomes research institute
Paediatric quality of life inventory
Phenylketonuria - quality-of-life questionnaire
Patient and public involvement
Patient-reported outcomes measurement information system
Patient reported outcome measures
Primary sclerosing cholangitis
Quality of life
Quality of life impact of BCR
The QoL impact of BCR medication
Rasch measurement theory
36-item short form health survey
Global Genes. Rare diseases: facts and statistics. https://globalgenes.org/rare-diseases-facts-statistics/. Accessed 2 Nov 2017.
EURORDIS. The voice of rare disease patients in Europe. https://www.eurordis.org/about-eurordis. Accessed 15 Sept 2017.
Limb L, Nutt S, Sen A. Experiences of rare diseases: an insight from patients and families: UK RD; 2010. https://www.raredisease.org.uk/media/1594/rduk-family-report.pdf. Accessed 9 Sept 2017.
EMA. Orphan medicines in the EU-leaving no patient behind. 2017. Accessed 8 Jan 2018.
Johansen H, Dammann B, Andresen I-L, Fagerland MW. Health-related quality of life for children with rare diagnoses, their parents’ satisfaction with life and the association between the two. Health Qual Life Outcomes. 2013;11:152.
Schalet BD, Pilkonis PA, Yu L, Dodds N, Johnston KL, Yount S, Riley W, Cella D. Clinical validity of PROMIS depression, anxiety, and anger across diverse clinical samples. J Clin Epidemiol. 2016;73:119–27.
Spillmann RC, McConkie-Rosell A, Pena L, Jiang Y-H, Schoch K, Walley N, Sanders C, Sullivan J, Hooper SR, Shashi V. A window into living with an undiagnosed disease: illness narratives from the undiagnosed diseases network. Orphanet J Rare Dis. 2017;12:71.
EURORDIS. The Voice of 12,000 Patients: Experiences and Expectations of Rare Disease Patients on Diagnosis and Care in Europe: EURORDIS Rare Diseases Europe; 2009. https://www.eurordis.org/IMG/pdf/voice_12000_patients/EURORDISCARE_FULLBOOKr.pdf. Accessed 9 Sept 2017
Angelis A, Tordrup D, Kanavos P. Socio-economic burden of rare diseases: a systematic review of cost of illness evidence. Health Policy. 2015;119:964–79.
Silibello G, Vizziello P, Gallucci M, Selicorni A, Milani D, Ajmone PF, Rigamonti C, De Stefano S, Bedeschi MF, Lalatta F. Daily life changes and adaptations investigated in 154 families with a child suffering from a rare disability at a public Centre for rare diseases in northern Italy. Ital J Pediatr. 2016;42:76.
Dellve L, Samuelsson L, Tallborn A, Fasth A, Hallberg LR. Stress and well-being among parents of children with rare diseases: a prospective intervention study. J Adv Nurs. 2006;53:392–402.
Benjamin K, Vernon MK, Patrick DL, Perfetto E, Nestler-Parr S, Burke L. Patient-Reported Outcome and Observer-reported outcome assessment in rare disease clinical trials - emerging good practices: an ISPOR COA emerging good practices report. Value Health. 2017;7:838–55.
Sabino G, Mills A, Jonker A, Lau LPL, Aartsma-Rus A, Arora J, Calvert M, Cano SJ, Denegri S, Hass S et al. Patient-centered outcome measures in the field of rare Diseases 2016. http://www.irdirc.org/wp-content/uploads/2017/12/PCOM_Post-Workshop_Report_Final.pdf. Accessed 2 Sept 2017.
Rotenstein LS, Huckman RS, Wagle NW. Making patients and doctors happier — the potential of patient-reported outcomes. N Engl J Med. 2017;377:1309–12.
FDA. Guidance for industry. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. US Department of Health and Human Services. 2009. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf. Accessed 7 Nov 2017.
Howell D, Molloy S, Wilkinson K, Green E, Orchard K, Wang K, Liberty J. Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol. 2015;26:1846–58.
Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use, 4th edn. Oxford: Oxford University Press; 2008.
Hendriksz CJ, Lavery C, Coker M, Ucar SK, Jain M, Bell L, Lampe C. Burden of disease in patients with Morquio a syndrome: results from an international patient-reported outcomes survey. Orphanet J Rare Dis. 2014;9:32.
Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha S, O'Neill R, Kennedy DL. Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health. 2007;10:S125–S37.
Basch E, Geoghegan C, Coons S, et al. Patient-reported outcomes in cancer drug development and US regulatory review: perspectives from industry, the food and drug administration, and the patient. JAMA Oncol. 2015;1:375–9.
EMA. Appendix 2 to the guideline on the evaluation of anticancer medicinal products in man. The use of patient-reported outcome (PRO) measures in oncology studies: EMA; 2016. http://www.ema.europa.eu/docs/en_GB/document_library/Other/2016/04/WC500205159.pdf. Accessed 29 Jan 2018
EMA. Reflection Paper on the use of patient reported outcome (PRO) measures in oncology studies: European Medicines Agency CfMPfHUC; 2014. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2014/06/WC500168852.pdf. Accessed 8 Jan 2018
Calvert M, Kyte D, Duffy H, Gheorghe A, Mercieca-Bebber R, Ives J, Draper H, Brundage M, Blazeby J, King M. Patient-reported outcome (PRO) assessment in clinical trials: a systematic review of guidance for trial protocol writers. PLoS One. 2014;9:e110216.
Deshpande PR, Rajan S, Sudeepthi BL, Abdul Nazir CP. Patient-reported outcomes: a new era in clinical research. Perspect Clin Res. 2011;2:137–44.
Cohen JS, Biesecker BB. Quality of life in rare genetic conditions: a systematic review of the literature. Am J Med Genet A. 2010;152A:1136–56.
Varni JW, Seid M, Rode CA. The PedsQL: measurement model for the pediatric quality of life inventory. Med Care. 1999;37:126–39.
Ravens-Sieberer U, Herdman M, Devine J, Otto C, Bullinger M, Rose M, Klasen F. The European KIDSCREEN approach to measure quality of life and well-being in children: development, current application, and future advances. Qual Life Res. 2014;23:791–803.
Bosch AM, Burlina A, Cunningham A, Bettiol E, Moreau-Stucker F, Koledova E, Benmedjahed K, Regnault A. Assessment of the impact of phenylketonuria and its treatment on quality of life of patients and parents from seven European countries. Orphanet J Rare Dis. 2015;10:80.
Regnault A, Burlina A, Cunningham A, Bettiol E, Moreau-Stucker F, Benmedjahed K, Bosch AM. Development and psychometric validation of measures to assess the impact of phenylketonuria and its dietary treatment on patients’ and parents’ quality of life: the phenylketonuria – quality of life (PKU-QOL) questionnaires. Orphanet J Rare Dis. 2015;10:59.
Barry JA, Folkard A, Denniston AK, Moran E, Ayliffe W. Development and validation of quality-of-life questionnaires for birdshot Chorioretinopathy. Ophthalmology. 2014;121:1488–9. e2
Zhang H, Wang L, Quan M, Huang J, Wu P, Lu Q, Fang Y. Health-related quality of life in children with chronic immune thrombocytopenia in China. Health Qual Life Outcomes. 2016;14:45.
Ramaswami U, Stull DE, Parini R, Pintos-Morell G, Whybra C, Kalkum G, Rohrbach M, Raluy-Callado M, Beck M, Chen W-H, et al. Measuring patient experiences in Fabry disease: validation of the Fabry-specific pediatric health and pain questionnaire (FPHPQ). Health Qual Life Outcomes. 2012;10:116.
International Rare Diseases Consortium (IRDiRC). Patient-centred outcome measures initiatives in the field of rare diseases 2016. http://www.irdirc.org/wp-content/uploads/2016/03/PCOM_Post-Workshop_Report_Final.pdf. Accessed 7 Nov 2017.
Arends M, Hollak CEM, Biegstraaten M. Quality of life in patients with Fabry disease: a systematic review of the literature. Orphanet J Rare Dis. 2015;10:77.
Hjollund INH, Larsen PL, Biering K, Johnsen PS, Riiskjær E, Schougaard ML. Use of patient-reported outcome (PRO) measures at group and patient levels: experiences from the generic integrated PRO system, WestChronic. Interact J Med Res. 2014;3:e5.
Jensen RE, Rothrock NE, DeWitt EM, Spiegel B, Tucker CA, Crane HM, Forrest CB, Patrick DL, Fredericksen R, Shulman LM, et al. The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Med Care. 2015;53:153–9.
Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, Rogak L, Bennett AV, Dueck AC, Atkinson TM, et al. Symptom monitoring with patient-reported outcomes during routine Cancer treatment: a randomized controlled trial. J Clin Oncol. 2016;34:557–65.
Snyder CF, Aaronson NK. Use of patient-reported outcomes in clinical practice. Lancet. 2009;374:369–70.
Luckett T, Butow PN, King MT. Improving patient outcomes through the routine use of patient-reported data in cancer clinics: future directions. Psychooncology. 2009;18:1129–38.
Basch E, Jia X, Heller G, Barz A, Sit L, Fruscione M, Appawu M, Iasonos A, Atkinson T, Goldfarb S, et al. Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. J Natl Cancer Inst. 2009;101:1624–32.
Trotti A, Colevas AD, Setser A, Basch E. Patient-reported outcomes and the evolution of adverse event reporting in oncology. J Clin Oncol. 2007;25:5121–7.
Trujillano D, Oprea GE, Schmitz Y, Bertoli-Avella AM, Abou Jamra R, Rolfs A. A comprehensive global genotype–phenotype database for rare diseases. Mol Genet Genomic Med. 2017;5:66–75.
Morel T, Cano SJ. Measuring what matters to rare disease patients – reflections on the work by the IRDiRC taskforce on patient-centered outcome measures. Orphanet J Rare Dis. 2017;12:171.
patientslikeme. The value of Openess. http://blog.patientslikeme.com/2007/12/13/the-value-of-openness/. Accessed 13 Nov 2017.
Newman SC, Bland RC, Orn H. A comparison of methods of scoring the general health questionnaire. Compr Psychiatry. 1988;29:402–8.
NIHR. INVOLVE. http://www.invo.org.uk/. Accessed 5 Sept 2017.
Basch E, Bennett AV. Patient-reported outcomes in clinical trials of rare diseases. J Gen Intern Med. 2014;29:801–3.
Mesa RA, Schwager S, Radia D, Cheville A, Hussein K, Niblack J, Pardanani AD, Steensma DP, Litzow MR, Rivera CE, et al. The myelofibrosis symptom assessment form (MFSAF): an evidence-based brief inventory to measure quality of life and symptomatic response to treatment in myelofibrosis. Leuk Res. 2009;33:1199–203.
Verstovsek S, Mesa RA, Gotlib J, Levy RS, Gupta V, DiPersio JF, Catalano JV, Deininger M, Miller C, Silver RT, et al. A double-blind placebo-controlled trial of Ruxolitinib for myelofibrosis. N Engl J Med. 2012;366:799–807.
Mesa RA, Kantarjian H, Tefferi A, Dueck A, Levy R, Vaddi K, Erickson-Viitanen S, Thomas DA, Cortes J, Borthakur G, et al. Evaluating the serial use of the myelofibrosis symptom assessment form for measuring symptomatic improvement. Cancer. 2011;117:4869–77.
Rodwell C, Ayme S. 2014 Report on the state of the art of rare disease activities in Europe. Scientific Secretariat of the European Union Committee of Experts on Rare Diseases (EUCERD) 2014. http://www.eucerd.eu/upload/file/Reports/2014ReportStateofArtRDActivities.pdf. Accessed 13 Nov 2017.
Gliklich RE, Dreyer NA, Leavy MB. Registries for evaluating patient outcomes: a user’s guide https://www.ncbi.nlm.nih.gov/books/NBK208609/. Accessed 14 Sept 2017.
Renal Association. How to routinely collect data on patient reported outcomes and experience measures in renal registries in Europe: an expert consensus meeting. https://www.renalreg.org/documents/how-to-routinely-collect-data-on-patient-reported-outcome-and-experience-measures-in-renal-registries-in-europe-an-expert-consensus-meeting/. Accessed 29 Sept 2017.
Orphanet. Rare Disease Registries in Europe. Research IFNIfHaM. 2017. http://www.orpha.net/orphacom/cahiers/docs/GB/Registries.pdf. Accessed 11 Nov 2017.
Cystic Fibrosis Trust. UK CF Registry. https://www.cysticfibrosis.org.uk/the-work-we-do/uk-cf-registry. Accessed 15 Sept 2017.
Cystic Fibrosis Foundation. Cystic Fibrosis Foundation Registry. https://www.cff.org/Research/Researcher-Resources/Patient-Registry/. Accessed 15 Nov 2016.
Fabry Registry. Fabry Registry: Creating knowledge on Fabry disease. https://www.fabrazyme.com/healthcare/Resources/Fabry-Registry.aspx. Accessed 11 Sept 2017.
Pompe Community. Pompe Registry. https://www.pompe-disease.eu/en. Accessed 19 Sept 2017.
Gaucher Care. The Gaucher Registry. https://www.gaucher-disease.eu/en/hcp/resources/gaucher-registry. Accessed 11 Sept 2017.
Devlin NJ, Parkin D, Browne J. Patient-reported outcome measures in the NHS: new methods for analysing and reporting EQ-5D data. Health Econ. 2010;19:886–905.
Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.
Calvert M, Thwaites R, Kyte D, Devlin N. Putting patient-reported outcomes on the ‘Big data road Map’. J R Soc Med. 2015;108:299–303.
Kyte D, Cockwell P, Lencioni M, Skrybant M, von Hildebrand M, Price G, Squire K, Webb S, Brookes O, Fanning H, et al. Reflections on the national patient-reported outcome measures (PROMs) programme: where do we go from here? J R Soc Med. 2016;109(12):441–5.
Frosch DL. Patient-reported outcomes as a measure of healthcare quality. J Gen Intern Med. 2015;30:1383–4.
ICHOM. http://www.ichom.org/. Accessed.
OMERACT. OMERACT resources. https://omeract.org/resources. Accessed Dec 2017.
Prinsen CA, Vohra S, Rose MR, Boers M, Tugwell P, Clarke M, Williamson PR, Terwee CB. How to select outcome measurement instruments for outcomes included in a “Core outcome set” - a practical guideline. Trials. 2016;17:449.
Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM, de Vet HCW. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21:651–7.
Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, Bouter LM, De Vet HCW. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010;10:22.
Isa F, Turner G, Kyte D, Slade A, Pankhurst T, Kerecuk L, Keeley T, Ferguson J, Calvert M. Patient reported outcome measures used in patients with primary sclerosing cholangistis: systematic review. Copenhagen: ISOQOL; 2016.
Younossi ZM, Afendy A, Stepanova M, Racila A, Nader F, Gomel R, Safer R, Lenderking WR, Skalicky A, Kleinman L, Myers RP, Subramanian GM, McHutchison JG, Levy C, Bowlus CL, Kowdley K, Muir AJ. Development and Validation of a Primary Sclerosing Cholangitis-Specific Patient-Reported Outcomes Instrument: The PSC PRO. Hepatology. 2018. Accepted Author Manuscript. https://doi.org/10.1002/hep.29664.
Mapi Research Trust. ePROVIDE Online Support in Clinical Outcome Assessments. https://eprovide.mapi-trust.org/search?form[searchText]=&form[ezxform_token]=vm5mY6mwXBfdtdI8lIZbV5vT3P8YndbDBnIiXAa3gLI. Accessed 15 Sept 2017.
Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries JF, Bruce B, Rose M. The patient-reported outcomes measurement information system (PROMIS): progress of an NIH roadmap cooperative group during its first two years. Med Care. 2007;45:S3–S11.
Bevans M, Ross A, Cella D. Patient-reported outcomes measurement information system (PROMIS(®)): efficient, standardized tools to measure self-reported health and quality of life. Nurs Outlook. 2014;62:339–45.
van der Beek NAME, Hagemans MLC, van der Ploeg AT, van Doorn PA, Merkies ISJ. The Rasch-built Pompe-specific activity (R-PAct) scale. Neuromuscul Disord. 2013;23:256–64.
Mattera MS, Kaplan FS, Pignolo RJ, Grogan D, Revicki DA. Patient-reported physical function outcome measure for adults with Fibrodysplasia Ossificans Progressiva: intelligent test design based on Promis item banks. Value Health. 2015;18:A165.
Hermans MCE, Hoeijmakers JGJ, Faber CG, Merkies ISJ. Reconstructing the Rasch-built myotonic dystrophy type 1 activity and participation scale. PLoS One. 2015;10:e0139944.
Wolfe E. Equating and item banking with the Rasch model. J Appl Meas. 2000;1:409.
Pallant JF, Tennant A. An introduction to the Rasch measurement model: an example using the hospital anxiety and depression scale (HADS). Br J Clin Psychol. 2007;46:1–18.
Du Y, Yates F. When to adjust for differential item functioning. In: Rasch Measurement Transactions vol 9; 1995. p. 414.
Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, Erikson P. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005;8:94–104.
Wild D, Eremenco S, Mear I, Martin M, Houchin C, Gawlicki M, Hareendran A, Wiklund I, Chong LY, von Maltzahn R, et al. Multinational trials—recommendations on the translations required, approaches to using the same language in different countries, and the approaches to support pooling the data: the ISPOR patient-reported outcomes translation and linguistic validation good research practices task force report. Value Health. 2009;12:430–40.
Regnault A, Herdman M. Using quantitative methods within the universalist model framework to explore the cross-cultural equivalence of patient-reported outcome instruments. Qual Life Res. 2015;24:115–24.
Tennant A, Penta M, Tesio L, Grimby G, Thonnard JL, Slade A, Lawton G, Simone A, Carter J, Lundgren-Nilsson A, et al. Assessing and adjusting for cross-cultural validity of impairment and activity limitation scales through differential item functioning within the framework of the Rasch model - The PRO-ESOR project. Med Care. 2004;42:37–48.
Cano SJ, Mayhew A, Glanzman AM, Krosschell KJ, Swoboda KJ, Main M, Steffensen BF, BÉRard C, Girardot F, Payan CA, et al. Rasch analysis of clinical outcome measures in spinal muscular atrophy. Muscle Nerve. 2014;49:422–30.
Aiyegbusi OL, Kyte D, Cockwell P, Marshall T, Dutton M, Slade A, Marklew N, Price G, Verdi R, Waters J, et al. Using patient-reported outcome measures (PROMs) to promote quality of care and safety in the management of patients with advanced chronic kidney disease (PRO-trACK project): a mixed-methods project protocol. BMJ Open. 2017;7:e016687.
Zhu X, Cahan A. Wearable technologies and telehealth in Care Management for Chronic Illness. In: Weaver CA, Ball MJ, Kim GR, Kiel JM, editors. Healthcare Information management systems: cases, strategies, and solutions. Cham: springer international publishing; 2016. p. 375–98.
PCORI. Can Telehealth improve care? https://www.pcori.org/research-in-action/can-telehealth-improve-care. Accessed 8 Jan 2018.
PCORI. Patient-Centred Outcomes Research Institute. https://www.pcori.org/document/users-guide-integrating-patient-reported-outcomes-electronic-health-records. Accessed 9 Jan 2018.
We received funding for the study from the Metchley Park Medical Society.
Availability of data and materials
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
Ethics approval and consent to participate
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.