Skip to main content
Download PDF

PROs for RARE: protocol for development of a core patient reported outcome set for individuals with genetic intellectual disability

Orphanet Journal of Rare Diseases volume 19, Article number: 354 (2024) Cite this article

Abstract

Introduction

Rare genetic neurodevelopmental disorders and intellectual disability (ID), collectively called genetic ID (GID), can profoundly impact daily functioning and overall well-being of affected individuals. To improve our understanding of the impact of GID and advancing both care and research, measuring relevant patient reported outcomes (PROs) is crucial. Currently, various PROs are measured for GID. Given the shared comorbidities across disorders, we aim to develop a generic core PRO set for children and adults with GID.

Methods and results

Developing the generic core PRO set entails the following steps: 1) providing an overview of potentially relevant PROs by scoping reviews and qualitative research; 2) integrating and conceptualizing these PROs (i.e., describing the content of the PROs in detail) into a pilot generic core PRO set; and 3) prioritizing relevant PROs by a European Delphi survey and consensus meetings.

Conclusions

This protocol presents the steps for developing a generic core PRO set for children and adults with GID. The next step involves selecting suitable patient reported outcome measures (PROMs) to adequately measure these PROs: the generic core PROM set. This generic core PROM set needs validation in the GID population, and eventually implementation in care and research, facilitating the aggregation and analysis of PRO data and guaranteeing continuous integration of the patient perspective in both care and research.

Background

Rare genetic neurodevelopmental disorders occur in about 1% of the general population [1, 2]. By European definition, disorders are classified as ‘rare’ when affecting less than one in 2000 individuals [3, 4]. Rare genetic neurodevelopmental disorders can be associated with intellectual disability (ID) [5,6,7], which is defined as an IQ < 70 and affects up to 2.5% of the population [8]. With current techniques, a genetic or other biological etiology can now be identified in about half of the individuals with ID. Genetic etiology varies from point mutations in one of the more than 1500 ID-related genes identified [9,10,11], to chromosomal copy number variants encompassing multiple genes, short tandem repeat expansion, or other structural variation to epigenetic anomalies. Rare genetic neurodevelopmental disorders and ID, henceforth called together genetic ID (GID), often have a negative impact on daily functioning and overall well-being of affected individuals due to the wide spectrum of physical and neuropsychiatric manifestations [12, 13]. These manifestations may include pain, epilepsy, psychiatric disorders such as autism spectrum disorder, and complex behaviors such as self-injurious behavior and aggression [14,15,16,17]. Due to these complex manifestations, individuals often require lifelong support on all life domains, including intensive medical and psychological care [18].

Understanding how the manifestations of GID affect daily functioning is an important part of disease management and crucial for delivering the best possible care for individuals with GID. To gain more insight into the impact of GID on daily functioning, measuring relevant patient reported outcomes (PROs) is essential. PROs refer to aspects of a patient’s health status (e.g., pain, anxiety, or fatigue) that are directly reported by the patient themselves or a proxy (e.g,. caregiver), without interference of a clinician or someone else [19]. PROs can be multidimensional, encompassing multiple PRO constructs (e.g., perceived health), or unidimensional, including only one specific PRO construct (e.g., pain or anxiety). PROs can be measured with patient reported outcome measures (PROMs) which are standardized questionnaires completed by the patient or a proxy (e.g., caregiver) [20]. PROMs can be generic (i.e., measuring health concepts that are relevant to a broad range of conditions or the general population, such as the Pediatric Quality of Life Inventory [21]), condition-specific (i.e., measuring health concepts relevant to a specific condition, such as the tuberous sclerosis complex (TSC) PROM [22]), or individualized (i.e., measuring health concepts relevant to a specific individual, such as the Patient-Specific Complaint Questionnaire [23]) [24, 25] (see Additional file 1).

PROMs can be used in both care and research and may serve as a valuable tool to guide health policy (Fig. 1). In care, PROMs can be used for the individual with GID to monitor overall functioning, screen for problems, and establish personalized treatment. The use of PROMs in care has already shown benefits within a wide range of fields like oncology, primary care, and psychiatry [26, 27], but is still not routinely used in care for GID [28]. In research involving individuals with GID, PROMs can be used to assess the efficacy and safety of treatments, as well as to gain more insight into the natural course of the disorder. This also helps in identifying new research areas, all from the perspective of the affected individuals or their caregivers [29, 30]. Moreover, PROMs are gaining escalating significance in patient registries for rare diseases, including GID, due to the growing recognition of the necessity to incorporate patients' perspectives to fully capture the burden of a disease and its treatment [31]. Lastly, PROMs can contribute to a more patient-centered health policy, ensuring that health policies are in harmony with the needs of patients. PROMs could, for instance, serve as an indicator to evaluate quality of the often time- and labor-intensive care delivered for individuals with GID [32]. Additionally, PRO data gathered with PROMs could inform the development of clinical guidelines and influence decisions regarding funding healthcare [24, 33].

Fig. 1
figure 1

Potential benefits of utilizing PROMs in care, research, and health policy

Full size image

Currently, various PROs are measured for GID, especially in research. A recent study identified more than 200 different PROMs, which were being utilized in 312 clinical trials involving individuals with GID [34]. Although this overload of PROs measured is not surprising due to the wide spectrum of manifestations of GID, it does hamper the possibility to combine and compare PRO data across different genetic or other subgroups [30]. Additionally, analyzing and interpreting PRO data can be complex due to the multidimensionality of PROs (e.g., adaptive functioning), differences in terminology used for similar PROs (e.g., participation, engagement in meaningful activity, community involvement), and unclear PROs due to lack of definition (e.g., syndrome-specific symptoms). Lastly, it is unknown whether the PROs measured are relevant to individuals with GID.

To overcome these challenges, we aim to develop a comprehensive, universally applicable core PRO set for both children and adults with GID to be used in care and research. The frequently shared comorbidities in GID can result in similar patient reported problems across disorders, enabling the development of a generic core PRO set. This corresponds to the trend seen toward standardized measurement of generic PROs across conditions in the general population, where PROs such as anxiety, depression, pain, and fatigue are often shared [35]. In this protocol, the steps for developing the generic core PRO set are described, following the Core Outcome Set-Standardized Protocol Items (COS-STAP) Statement [36] in order to align with the official steps for developing a Core Outcome Set (COS).

Methods and results

Stakeholders

The project team consists of a Study Management Group and a Study Advisory Board. The Study Management Group consists of researchers; one researcher (NvS) who will coordinate the day-to-day management of the project, two PROM experts and medical psychologists (MvM, LH), and two GID experts including an ID physician and a genetic metabolic pediatrician (AvE, CvK). The Study Advisory Board consists of GID experts. The Study Management Group will also collaborate with patient representatives and experts of the European Reference Network on Intellectual Disability, TeleHealth and Congenital Anomalies (ERN-ITHACA), which is a patient-centered network that is established with the goal of improving collaboration on rare neurodevelopmental disorders, both with and without biological diagnosis. Throughout this study, individuals with GID and their caregivers will be actively engaged and included where feasible and relevant. Individuals with ID without known genetic etiology will also be included within this study, since they exhibit overlapping comorbidities with GID. They will be subsumed under the term GID.

Conceptual framework

PROs will be classified within a conceptual framework (Table 1), based on the model of Valderas and Alonso [25] (combination of the classification model by Wilson and Cleary [37] and the International Classification of Functioning, Disability and Health (ICF) [38]), and the Patient Reported Outcomes Measurement Information System (PROMIS) [39]. This model is further refined during expert meetings regarding PRO(M)s in the Netherlands [35]. A PRO domain represents an overarching PRO (e.g., physical functioning), and a PRO subdomain represents a PRO that falls under a PRO domain (e.g., mobility). Each symptom (e.g., pain) represents its own a PRO domain. The PRO conceptualization represents the description of the content of a PRO (e.g., ability to perform daily activities). Describing PROs in as much detail as possible before selecting PROMs is essential [40]. This conceptual framework should enable the classification of all PROs for GID.

Table 1 Conceptual framework
Full size table

Study design

Developing the generic core PRO set will consist of three distinct steps as described in Fig. 2.

Fig. 2
figure 2

Steps for developing the generic core PRO set. PROs, patient reported outcomes; PROM(s), patient reported outcome measure(s); COS(s), core outcome set(s). *Target number of participants

Full size image

Step 1: Providing an overview of potentially relevant PROs

Step 1.1: Scoping review; PROs included in clinical trials

PROs used in clinical trials with GID will be identified. Data will be extracted from a previous published scoping review on outcomes and outcome measurement instruments used in clinical trials with individuals with genetic neurodevelopmental disorders and ID. The method and search strategy can be found in the original publication [34].

In our data extraction, only studies focusing on PROs and/or PROMs will be included. Studies including individuals with inborn errors of metabolism will be excluded, as these individuals lie outside the intended scope of the generic core PRO set. The subsequent data will be extracted: PRO(s), dimensionality PRO(s), PROM(s), type of PROM report (self-report or proxy-report), and type of PROM (generic, condition-specific, or individualized). PROs will be classified within the conceptual framework by the Study Management Group.

Step 1.2: Scoping review; PROs included in COSs

In the scoping review, PROs included in COSs for specific rare genetic neurodevelopmental disorders and/or ID will be identified. This review will be performed according to methodology published in advance in the Open Science Framework (OSF) [41].

MEDLINE (Ovid), PsycINFO, Embase, and the COMET database will be searched. A list of GID will be composed using the human phenotype ontology (HPO) database on https://hpo.jax.org/app/. All terms describing a genetic disorder assigned to the subontology ID will be included. Furthermore, a search strategy for ID of unknown cause will be used in combination with terms for a COS.

Any type of COS development studies for individuals with a rare genetic neurodevelopmental disorder or ID will be included. COS studies for application in research studies, for the use in care, and for the use of quality assessment will be included. COS studies covering all types of interventions will be included.

All titles and abstracts will be screened. Of the relevant articles, full texts will be reviewed and the same data will be extracted as in step 1.1. PROs will be classified within the conceptual framework by the Study Management Group. The psychometric properties of the PROMs will be checked for reliability and validity among individuals with GID.

Step 1.3: Qualitative study; Dutch and European focus groups and interviews

Focus groups and interviews will be conducted with affected individuals, their caregivers, healthcare professionals (e.g., medical doctors, psychologists, and nurses), and European patient representatives in order to qualitatively identify relevant PROs for individuals with GID. Affected individuals and caregivers will be recruited via their healthcare professional at the involved healthcare organizations (Amsterdam UMC and ‘s Heeren Loo). Healthcare professionals will be recruited through a call to participate in a focus group or interview at the Amsterdam UMC and ‘s Heeren Loo, and will be asked to disseminate the call to colleagues. European patient representatives will be recruited via ERN-ITHACA. Participants will be assessed for socio-demographic characteristics to ensure a representative sample.

Participants with GID will be selected using purposive sampling, aiming for diversity in age, genetic diagnoses, and level of ID [42]. To be eligible for inclusion in either a focus group or interview, individuals must be affected by GID, possess effective communication skills to express their thoughts and feelings to other participants or the interviewer, be over 12 years of age, and be proficient in Dutch (speaking, reading, and writing). Caregivers who provide care and support to an individual with a GID, as well as healthcare professionals with a minimum of five years’ experience working with individuals with a GID, will also be invited to participate. European patient representatives should have at least five years’ experience working for a patient organization dedicated to GID. The final number of participants recruited for the focus groups and interviews will be around 40, but the final number will depend on adequately sampling of the group until data sufficiency is reached.

Dutch focus groups and interviews will be performed at the Amsterdam UMC in the Netherlands via live and/or online meetings. European focus groups will be performed online. The duration of focus groups will be about 1.5 hours and interviews 30 minutes. The focus groups and interviews will consist of two parts:

  • Part 1 – By using the ‘Complain and Cheer wall’ technique [43] (i.e., writing down negative aspects on the ‘Complain wall’ and positive aspects on the ‘Cheer wall’), the impact of a GID on daily life will be discussed.

  • Part 2 – By using the ‘Brainstorming’ technique [43] (i.e., generating and discussing new ideas together), relevant topics regarding physical, mental and social functioning to discuss with the healthcare professional during a consultation will be discussed.

The PROs emerging from the focus groups and interviews will be classified within the conceptual framework by the Study Management Group.

Step 2: Integrating and conceptualizing PROs

Step 2.1 Expert group meeting

The PROs established and classified in step 1.1 to step 1.3 will be integrated into a pilot generic core PRO set. PROs will be eliminated where appropriate (e.g., duplicate or no PRO, such as behavior) and multidimensional PROs will be separated into unidimensional PROs whenever possible. For example, the multidimensional PRO ‘adaptive functioning’ will be separated into unidimensional PROs: ‘communication’, ‘daily living skills’, and ‘social functioning’. The PROs will be conceptualized to ensure clear PRO definition. Shared experiences identified by the focus groups and interviews, as well as those found in the literature and guidelines, will be used to conceptualize the PROs. This approach aims to ensure the applicability of the PRO conceptualizations to all GID groups. As experiences are meant to be generic, we expect them to remain manageable and describable within the conceptualization of a generic PRO. Eventually, a reduced and clearly defined list of PROs will remain, classified into the conceptual framework: the pilot generic core PRO set. Step 2.1 will be performed by the Study Management Group and two PROM/COS experts.

Step 3: Prioritizing the relevant PROs

Step 3.1: European Delphi survey

A European Delphi survey will be performed to reach consensus on which PROs should be included in the final generic core PRO set. A Delphi survey is a technique used to gain consensus among relevant experts or stakeholders. The Delphi method typically involves multiple rounds in which participants remain anonymous in the surveys [44].

Participant groups will align with those in the focus groups and interviews. Healthcare professionals and European patient representatives will be merged into one stakeholder group: experts. Participants of the Dutch and European focus groups and interviews will be contacted again if they have given their consent. Additional individuals with GID, caregivers, and experts will be contacted through Dutch patient organizations dedicated to a specific GID and ERN-ITHACA. To ensure a representative sample, the socio-demographic characteristics of the participants will be checked.

There is currently no established guideline for determining the optimal sample size for a Delphi survey [45, 46]. The Study Management Group will aim at including 60 participants in the Delphi survey in order to represent the GID population as best as possible, ensuring the representation of 20 individuals with GID, 20 caregivers, and 20 experts. The pilot generic core PRO set will be presented in the Delphi survey. The survey will be kept as concise as possible. Additionally, participants, especially affected individuals, will have the option to complete the survey at multiple times to prevent overburdening. The Delphi survey will first be pilot-tested with one member of each participant group to ensure that the survey is clearly understandable.

Round one

In the first survey round, PROs will be prioritized. The pilot generic core PRO set will be presented to participants, including a list of excluded PROs or other outcomes along with the reason for their exclusion. Participants will be asked to determine why a PRO is important enough to be included in the generic core PRO set, or why a PRO should be excluded: response options will be ‘Yes’, ‘Unsure/I do not know’, and ‘No’. They may also provide a rationale for their decision, respond to the proposed conceptualization of the PROs to ensure that no shared experiences are overlooked, and will be asked for additional PROs.

The ‘70/15%’ consensus definition (i.e., including an outcome in the COS when at least 70% of participants score an outcome as ‘very important’ and less than 15% as ‘not important’), which is described by Williamson et al. (2012) [47], has previously been used by other COS developers [48,49,50]. In addition, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative usually use a consensus threshold of 67% in Delphi studies [51, 52]. However, given the considerable diversity within and between GID subgroups, it is reasonable to anticipate a wide range of relevant PROs. For this reason, the consensus threshold will not be excessively high: the consensus threshold will be set at 60% or more of all stakeholder groups responding ‘Yes’ for a PRO to include the PRO in the final generic core PRO set. If 60% or more of all stakeholder groups respond ‘No’ for a PRO, it will not be included in the final generic core PRO set. The responses of round one will be analyzed for each participant group. The PROs that have reached consensus for inclusion or exclusion from the final generic core PRO set will be shown again in the second round to allow participants to reassess them and ensure their inclusion or exclusion. If at least two participants suggest the same modification to the conceptualization of a PRO or suggest the same additional PRO in the open-ended question, it will be included in the second round.

Round two

In the second survey round, PROs will be reprioritized with the aim to reach consensus on which PROs should be included in the final generic core PRO set. The pilot generic core PRO set, along with feedback from the first round, will be presented to participants. The results for each PRO will be aggregated for each participant group, and summary statistics and reason for inclusion or exclusion will be presented. Response options will be the same as in the first round. The responses of round two will be analyzed again for each participant group. Responses will be carefully examined to ensure no strong objections are raised against the prevailing group response.

Step 3.2: Two consensus meetings

Two separate consensus meetings will be organized to reach consensus on the undecided PROs: one online meeting with three Dutch individuals with GID and three caregivers, and one online meeting with six experts. At the end of Delphi round two, participants will be invited to join a consensus meeting. Those who wish to participate will be checked on socio-demographic characteristics to ensure diverse representation in the consensus meetings. PROs that have not reached consensus during the Delphi will be subjected to voting in these two meetings. A PRO receiving at least 60% positive votes during both meetings will be included in the final generic core PRO set.

Discussion

To our knowledge, the current study is the first to develop a generic core PRO set for children and adults with GID. An innovative and thorough methodology is used, involving the identification, classification, and conceptualization of PROs within a comprehensive conceptual framework. This methodology may also be used for the development of generic core PRO sets for other (rare) health conditions. With this study, we aim to reach an important milestone in addressing the needs of individuals with GID. The most relevant PROs for the whole GID population will be identified. Subsequently, a generic core PRO set will be developed that should be consistently measured in both care and research, all from the perspective of affected individuals or their caregivers.

For the development of the generic core PRO set, an all-encompassing conceptual framework will be used to classify the PROs for GID. An essential and novel step is integrating and conceptualizing PROs into this conceptual framework in step 2.1 (the pilot generic core PRO set). PROs often lack precise definition or exhibit multidimensionality, a common occurrence when they are presented in Delphi surveys. This makes it challenging for Delphi participants to grasp the exact meaning of the PROs. By integrating and conceptualizing PROs, we aim to include a reduced, well-defined, but still all-encompassing list of unidimensional PROs (whenever possible) in the Delphi survey. This approach not only ensures that participants understand and recognize the meaning of the PROs, enabling them to provide thoughtful responses for each PRO, it also facilitates the process of eventually selecting suitable PROMs, as it becomes very clear ‘what’ needs to be measured. While this is a crucial step, we are aware it will be challenging to incorporate and conceptualize all potentially relevant PROs for the whole GID population. Nevertheless, diligently performing the steps outlined in this protocol should result in a classified and well-defined generic core PRO set relevant to the whole GID population. Moreover, it will also become clear which outcomes do not qualify as a PRO, and should therefore be measured with alternative instruments. The next step involves the selection of suitable PROMs with optimal psychometric properties which sufficiently measure the generic core PRO set: the generic core PROM set. As GID is a heterogeneous population, condition-specific PROs will be necessary for specific GID subgroups. Additionally, there may also be a need for specific PROs for other clinically significant subgroups (e.g., age, ID level) that are not sufficiently covered by the generic core PRO set. Therefore, it is crucial to explore these subgroups to identify potential condition-specific or other subgroup-specific PROs. If necessary, specific questions or PROMs for specific genetic conditions or other subgroups will be added. Subsequently, the generic core PROM set will be validated within the GID population (e.g., by measuring reliability, internal consistency, and responsiveness of the PROMs), and implemented in care and research. The validation and implementation of the core PROM set will be done in collaboration with experienced methodologists and implementation scientists [53,54,55]. Consideration will be given to known PROM implementation barriers [55,56,57].

Necessity of a core PRO(M) set

The development of the generic core PRO(M) set forms the first step in standardizing personalized measurement in care and research for children and adults with GID [58]. This novel and state-of-the-art generic core PRO(M) set will enable consistent and standardized measurement of relevant PROs across different GID groups in both care and research. This reduces the great diversity of PROs measured, simplifying the aggregation and analysis of PRO data. Moreover, this approach ensures an ongoing integration of patient perspective in care and research, leading to a better understanding of the overall impact and the variations in impact on daily functioning among different GIDs, or other subgroups of interest such as those affected by shared comorbidity such as autism spectrum disorders. The necessity to integrate PRO data, especially in research areas such as drug development and disease-modifying therapies [59], resounds not only from patients themselves, but also from healthcare professionals and policymakers including the European Medicines Agency (EMA) and the Food and Drug Administration (FDA) [60,61,62,63]. Furthermore, utilizing the generic core PRO(M) set will also tackle the ongoing challenge in finding suitable PROMs for individuals with GID. PROMs can be cognitively demanding, as many affected individuals encounter difficulties understanding and responding to questions [64], and PROMs frequently consist of lengthy and difficult questions. In addition, PROMs are often not tested and validated within the GID population or only in a specific population group [30]. This raises the questions about whether the current PROMs used for GID accurately measure real treatment effects and progress of affected individuals [65]. At last, utilizing the generic core PRO(M) in care and research settings may offer many benefits for the individual with GID and the GID population as a whole (Fig. 1).

Conclusions

This protocol outlines the steps for developing a generic core PRO set for children and adults with GID. The subsequent step focuses on the careful selection of PROMs to effectively measure these PROs, forming the generic core PROM set. This generic core PROM set needs to be validated in the GID population. Future utilization of such a generic core PROM set in both care and research, will facilitate aggregation and comparison of PRO data and guarantee the integration of the patient perspective.

Availability of data and materials

Not applicable.

References

  1. Castrén E, Elgersma Y, Maffei L, Hagerman R. Treatment of neurodevelopmental disorders in adulthood. J Neurosci. 2012;32(41):14074–9.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Levy G, Barak B. Postnatal therapeutic approaches in genetic neurodevelopmental disorders. Neural Regen Res. 2021;16(3):414.

    Article  CAS  PubMed  Google Scholar 

  3. EURORDIS, 2013. The voice of rare disease patients in Europe. https://download2.eurordis.org/the-voice-of-rare-disease-patients.pdf.

  4. Genes G. Rare diseases: facts and statistics. Statistics and Figures on Prevalence of Genetic and Rare Diseases Available at: https://globalgenes.org/rare-diseases-facts-statistics/ Accessed. 2017;5.

  5. Swillen A, McDonald‐McGinn D. Developmental trajectories in 22q11. 2 deletion syndrome. American Journal of Medical Genetics Part C: Seminars in Medical Genetics. 2015;169(2):172–81.

  6. Juriaans AF, Kerkhof GF, Hokken-Koelega AC. The spectrum of the Prader–Willi-like pheno-and genotype: a review of the literature. Endocrine Rev. 2022;43(1):1–18.

    Article  Google Scholar 

  7. Salcedo-Arellano MJ, Hagerman RJ, Martínez-Cerdeño V. Fragile X syndrome: clinical presentation, pathology and treatment. Gac Med Mex. 2020;156(1):60–6.

    PubMed  Google Scholar 

  8. Patel DR, Merrick J. Neurodevelopmental disabilities: Introduction and epidemiology. Neurodevelopmental disabilities: clinical care for children and young adults. Dordrecht: Springer; 2011:1–13.

  9. Wang J, Wang Y, Wang L, Chen WY, Sheng M. The diagnostic yield of intellectual disability: combined whole genome low-coverage sequencing and medical exome sequencing. BMC Med Genomics. 2020;13:1–15.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Maia N, Nabais Sá MJ, Melo-Pires M, de Brouwer AP, Jorge P. Intellectual disability genomics: current state, pitfalls and future challenges. BMC Genomics. 2021;22(1):1–17.

    Article  Google Scholar 

  11. Pekeles H, Accogli A, Boudrahem-Addour N, Russell L, Parente F, Srour M. Diagnostic yield of intellectual disability gene panels. Pediatric Neurol. 2019;92:32–6.

    Article  Google Scholar 

  12. Butcher NJ, Chow EW, Costain G, Karas D, Ho A, Bassett AS. Functional outcomes of adults with 22q11.2 deletion syndrome. Genet Med. 2012;14(10):836–43.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Williams K, Jacoby P, Whitehouse A, Kim R, Epstein A, Murphy N, et al. Functioning, participation, and quality of life in children with intellectual disability: an observational study. Dev Med Child Neurol. 2021;63(1):89–96.

    Article  PubMed  Google Scholar 

  14. McGuire BE, Defrin R. Pain perception in people with Down syndrome: a synthesis of clinical and experimental research. Front Behav Neurosci. 2015;9:194.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Robertson J, Hatton C, Emerson E, Baines S. Prevalence of epilepsy among people with intellectual disabilities: a systematic review. Seizure. 2015;29:46–62.

    Article  PubMed  Google Scholar 

  16. Lubbers K, Stijl EM, Dierckx B, Hagenaar DA, Ten Hoopen LW, Legerstee JS, et al. Autism symptoms in children and young adults with fragile x syndrome, angelman syndrome, tuberous sclerosis complex, and neurofibromatosis type 1: a cross-syndrome comparison. Front Psychiatry. 2022;13:852208.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Arron K, Oliver C, Moss J, Berg K, Burbidge C. The prevalence and phenomenology of self-injurious and aggressive behaviour in genetic syndromes. J Intellect Disabil Res. 2011;55(2):109–20.

    Article  CAS  PubMed  Google Scholar 

  18. Cooper S-A, McLean G, Guthrie B, McConnachie A, Mercer S, Sullivan F, Morrison J. Multiple physical and mental health comorbidity in adults with intellectual disabilities: population-based cross-sectional analysis. BMC Fam Pract. 2015;16:1–11.

    Article  Google Scholar 

  19. Health USDo, Human Services FDACfDE, Research, Health USDo, Human Services FDACfBE, Research, et al. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes. 2006;4(1):79.

    Article  Google Scholar 

  20. McKenna SP. Measuring patient-reported outcomes: moving beyond misplaced common sense to hard science. BMC Med. 2011;9(1):1–12.

    Article  Google Scholar 

  21. Varni JW, Burwinkle TM, Seid M, Skarr D. The PedsQL 4.0 as a pediatric population health measure: feasibility, reliability, and validity. Ambul Pediatr. 2003;3(6):329–41.

    Article  PubMed  Google Scholar 

  22. Müller AR, Luijten MA, Haverman L, de Ranitz-Greven WL, Janssens P, Rietman AB, et al. Understanding the impact of tuberous sclerosis complex: development and validation of the TSC-PROM. BMC Med. 2023;21(1):1–15.

    Google Scholar 

  23. Beurskens AJ, de Vet HC, Kökeb AJ, Lindeman E, van der Heijden GJ, Regtope W, Knipschild PG. A patient-specific approach for measuring functional status in low back pain. J Manipulative Physiol Ther. 1999;22(3):144–8.

    Article  CAS  PubMed  Google Scholar 

  24. Churruca K, Pomare C, Ellis LA, Long JC, Henderson SB, Murphy LE, et al. Patient-reported outcome measures (PROMs): A review of generic and condition-specific measures and a discussion of trends and issues. Health Expect. 2021;24(4):1015–24.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Valderas JM, Alonso J. Patient reported outcome measures: a model-based classification system for research and clinical practice. QualLife Res. 2008;17(9):1125–35.

    Google Scholar 

  26. Gibbons C, Porter I, Goncalves-Bradley DC, Stoilov S, Ricci-Cabello I, Tsangaris E, et al. Routine provision of feedback from patient-reported outcome measurements to healthcare providers and patients in clinical practice. Cochrane Database Syst Rev. 2021;10:CD011589.

    PubMed  Google Scholar 

  27. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol. 2016;34(6):557.

    Article  CAS  PubMed  Google Scholar 

  28. Greenberg KK, Schwartz AE, Kramer JM. Adoption of patient‐reported outcome measures with youth with intellectual/developmental disabilities: Contextual influences and practice patterns. Child Care Health Dev. 2021;47(4):501–8.

    Article  PubMed  Google Scholar 

  29. Benjamin K, Vernon MK, Patrick DL, Perfetto E, Nestler-Parr S, Burke L. Patient-reported outcome and observer-reported outcome assessment in rare disease clinical trials: an ISPOR COA emerging good practices task force report. Value Health. 2017;20(7):838–55.

    Article  PubMed  Google Scholar 

  30. Slade A, Isa F, Kyte D, Pankhurst T, Kerecuk L, Ferguson J, et al. Patient reported outcome measures in rare diseases: a narrative review. Orphanet J Rare Dis. 2018;13:1–9.

    Article  Google Scholar 

  31. Kölker S, Gleich F, Mütze U, Opladen T. Rare disease registries are key to evidence-based personalized medicine: highlighting the European experience. Front Endocrinol. 2022;13:832063.

    Article  Google Scholar 

  32. Basch E, Torda P, Adams K. Standards for patient-reported outcome–based performance measures. Jama. 2013;310(2):139–40.

    Article  CAS  PubMed  Google Scholar 

  33. Rivera SC, Kyte DG, Aiyegbusi OL, Slade AL, McMullan C, Calvert MJ. The impact of patient-reported outcome (PRO) data from clinical trials: a systematic review and critical analysis. Health Qual Life Outcomes. 2019;17:1–19.

    Article  Google Scholar 

  34. Müller AR, van Silfhout NY, den Hollander B, Kampman DH, Bakkum L, Brands MM, et al. Navigating the outcome maze: a scoping review of outcomes and instruments in clinical trials in genetic neurodevelopmental disorders and intellectual disability. Ther Adv Rare Dis. 2024;5:26330040241245720.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Oude Voshaar M, Terwee CB, Haverman L, van der Kolk B, Harkes M, van Woerden CS, et al. Development of a standard set of PROs and generic PROMs for Dutch medical specialist care: Recommendations from the Outcome-Based Healthcare Program Working Group Generic PROMs. Qual Life Res. 2023;32(6):1595–605.

  36. Kirkham JJ, Gorst S, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core outcome Set-STAndardised protocol items: the COS-STAP statement. Trials. 2019;20:1–7.

    Article  Google Scholar 

  37. Wilson IB, Cleary PD. Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA. 1995;273(1):59–65.

    Article  CAS  PubMed  Google Scholar 

  38. Ustun TB, Chatterji S, Bickenbach J, Kostanjsek N, Schneider M. The international classification of functioning, disability and health: A new tool for understanding disability and health. Disabil Rehabil. 2003;25(11–12):565–71.

    Article  CAS  PubMed  Google Scholar 

  39. Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45(5 Suppl 1):S3-s11.

    Article  PubMed  PubMed Central  Google Scholar 

  40. de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in Medicine: A Practical Guide. Cambridge: Cambridge University Press; 2011.

  41. Foster ED, Deardorff A. Open science framework (OSF). J Med Libr Assoc. 2017;105(2):203.

    Article  PubMed Central  Google Scholar 

  42. American Psychiatric Association D, Association AP. Diagnostic and statistical manual of mental disorders: DSM-5. Washington, DC: American psychiatric association; 2013.

    Book  Google Scholar 

  43. Temme BBA. Leidraad groepsbijeenkomsten. Den Haag: B&A Groep; 1999.

  44. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311(7001):376.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Williams PL, Webb C. The Delphi technique: a methodological discussion. J Adv Nurs. 1994;19(1):180–6.

    Article  CAS  PubMed  Google Scholar 

  46. Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol. 2005;5(1):1–12.

    Article  Google Scholar 

  47. Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, Tugwell P. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13(1):1–8.

    Article  Google Scholar 

  48. Potter S, Holcombe C, Ward J, Blazeby J, Brookes S, Cawthorn S, et al. Development of a core outcome set for research and audit studies in reconstructive breast surgery. J Brit Surg. 2015;102(11):1360–71.

    Article  CAS  Google Scholar 

  49. Blazeby J, Macefield R, Blencowe N, Jacobs M, McNair A, Sprangers M, et al. Core information set for oesophageal cancer surgery. J Brit Surg. 2015;102(8):936–43.

    Article  CAS  Google Scholar 

  50. Harman NL, Bruce IA, Kirkham JJ, Tierney S, Callery P, O’Brien K, et al. The importance of integration of stakeholder views in core outcome set development: otitis media with effusion in children with cleft palate. PloS one. 2015;10(6):e0129514.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, Terwee CB. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1147–57.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Mokkink LB, Boers M, Van Der Vleuten C, Bouter LM, Alonso J, Patrick DL, et al. COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study. BMC Med Res Methodol. 2020;20:1–13.

    Article  Google Scholar 

  53. Klaufus LH, Luijten MAJ, Verlinden E, van der Wal MF, Haverman L, Cuijpers P, et al. Psychometric properties of the Dutch-Flemish PROMIS(®) pediatric item banks Anxiety and Depressive Symptoms in a general population. Qual Life Res. 2021;30(9):2683–95.

    Article  CAS  PubMed  Google Scholar 

  54. Luijten MA, van Litsenburg RR, Terwee CB, Grootenhuis MA, Haverman L. Psychometric properties of the Patient-Reported Outcomes Measurement Information System (PROMIS®) pediatric item bank peer relationships in the Dutch general population. Qual Life Res. 2021;30:2061–70.

  55. van Muilekom MM, Teela L, van Oers HA, van Goudoever JB, Grootenhuis MA, Haverman L. Patients’ and parents’ perspective on the implementation of Patient Reported Outcome Measures in pediatric clinical practice using the KLIK PROM portal. Qual Life Res. 2022;31(1):241–54.

  56. Teela L, van Muilekom MM, Kooij LH, Gathier AW, van Goudoever JB, Grootenhuis MA, et al. Clinicians’ perspective on the implemented KLIK PROM portal in clinical practice. Qual Life Res. 2021;30(11):3267–77.

    Article  PubMed  Google Scholar 

  57. van Muilekom MM, Teela L, van Oers HA, van Goudoever JB, Grootenhuis MA, Haverman L. Patients’ and parents’ perspective on the implementation of Patient Reported Outcome Measures in pediatric clinical practice using the KLIK PROM portal. Quality of Life Research. 2021;31:241-254.

  58. van Eeghen AM, Bruining H, Wolf NI, Bergen AA, Houtkooper RH, van Haelst MM, van Karnebeek CD. Personalized medicine for rare neurogenetic disorders: can we make it happen? Mole Case Stud. 2022;8(2):a006200.

    Google Scholar 

  59. Hoytema van Konijnenburg EM, Wortmann SB, Koelewijn MJ, Tseng LA, Houben R, Stöckler-Ipsiroglu S, Treatable inherited metabolic disorders causing intellectual disability, et al. Treatable inherited metabolic disorders causing intellectual disability: 2021 review and digital app. Orphanet J Rare Dis. 2021;2021(16):1–35.

    Google Scholar 

  60. Administration USFaD. FDA Patient-Focused Drug Development Guidance Series for Enhancing the Incorporation of the Patient’s Voice in Medical Product Development and Regulatory Decision Making. https://www.fda.gov/drugs/development-approval-process-drugs/fda-patient-focused-drug-development-guidance-series-enhancing-incorporation-patients-voice-medical2023.

  61. Agency EM. Patient experience data in medicines development and regulatory decision-making. https://www.ema.europa.eu/en/events/multi-stakeholder-workshop-patient-experience-data-medicines-development-regulatory-decision-making#documents-section2022.

  62. Compendium P. Guide To Patient Involvement In Rare Disease Therapy Development. https://everylifefoundation.org/wp-content/uploads/2022/01/Guide-to-Patient-Involvement-FINAL-COMPLETE-GUIDE-Rev.pdf2022.

  63. Administration USFaD. Rare diseases: common issues in drug development: guidance for industry (draft guidance). Washington, D.C.: US FDA2015.

  64. Nicolaidis C, Raymaker DM, McDonald KE, Lund EM, Leotti S, Kapp SK, et al. Creating accessible survey instruments for use with autistic adults and people with intellectual disability: Lessons learned and recommendations. Autism Adulthood. 2020;2(1):61–76.

    Article  PubMed  Google Scholar 

  65. Mokkink L, Terwee C, de Vet H. Key concepts in clinical epidemiology: Responsiveness, the longitudinal aspect of validity. J Clin Epidemiol. 2021;140:159–62.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank all members of ERN-ITHACA, in particular all members of the Patient Council and Anne Hugon, for their cooperation within this project. AvE is a member of ERN-ITHACA. CvK is a member of Metab-ERN.

Funding

This research project was funded by the ForWis(h)dom foundation and ‘s Heeren Loo. They have no role in the conceptualization, design, decision to publish, or preparation of the manuscript.

Author information

Author notes

  1. Lotte Haverman and Agnies M. van Eeghen contributed equally to this manuscript.

Authors and Affiliations

  1. Department of Child and Adolescent Psychiatry & Psychosocial Care, Emma Children’s Hospital, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands

    Nadia Y. van Silfhout, Maud M. van Muilekom & Lotte Haverman

  2. Department of Pediatrics, Amsterdam Gastroenterology Endocrinology Metabolism, Emma Children’s Hospital, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands

    Nadia Y. van Silfhout, Clara D. van Karnebeek & Agnies M. van Eeghen

  3. Emma Center for Personalized Medicine, Amsterdam UMC, Amsterdam, The Netherlands

    Nadia Y. van Silfhout, Clara D. van Karnebeek & Agnies M. van Eeghen

  4. Amsterdam Public Health Research Institute, Mental health and Personalized Medicine, Amsterdam, The Netherlands

    Nadia Y. van Silfhout & Maud M. van Muilekom

  5. Amsterdam Reproduction & Development Research Institute, Child Development, Amsterdam, The Netherlands

    Nadia Y. van Silfhout, Clara D. van Karnebeek & Agnies M. van Eeghen

  6. Amsterdam Public Health Research Institute, Mental health and Digital Health, Amsterdam, The Netherlands

    Lotte Haverman

  7. Amsterdam Public Health Research Institute, Aging & Later life and Personalized Medicine, Amsterdam, The Netherlands

    Agnies M. van Eeghen

  8. Advisium’s Heeren Loo, Amersfoort, The Netherlands

    Agnies M. van Eeghen

Authors
  1. Nadia Y. van Silfhout
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maud M. van Muilekom
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Clara D. van Karnebeek
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lotte Haverman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Agnies M. van Eeghen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

NvS, MvM, LH, and AvE conceived and designed the study. NvS wrote the first version of the manuscript. MvM, LH, AvE, and CvK provided important revision on the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Agnies M. van Eeghen.

Ethics declarations

Ethics approval and consent to participate

A formal review and waiver was provided by the Medical Ethics Committee of the Amsterdam UMC (W22_345 # 22.413). Informed consent will be obtained from all participants included in the focus groups and interviews, the European Delphi survey, and the consensus meetings.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

van Silfhout, N.Y., van Muilekom, M.M., van Karnebeek, C.D. et al. PROs for RARE: protocol for development of a core patient reported outcome set for individuals with genetic intellectual disability. Orphanet J Rare Dis 19, 354 (2024). https://doi.org/10.1186/s13023-024-03264-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13023-024-03264-0

Keywords

Orphanet Journal of Rare Diseases

ISSN: 1750-1172