Our pilot study provides evidence for the feasibility and construct validity of using PROMIS instruments to record important components of the disease experience among individuals with OI. The quantitative results from both PROMIS and PODCI can provide clinicians and researchers with a yardstick for assessing overall need for treatment and determining the success (or failure) of an intervention. In addition, our results confirm that the scores from most domains differ significantly from the general US population: individuals with OI have higher disease burden and worse functioning.
There were two exceptions of “no differences” from the general population: the parent-proxy assessments of depression and fatigue. At least two reasons could explain this finding. One is that the reference population for the PROMIS Pediatric and the PROMIS Proxy measures is a mix of both representatives of the US general population and representatives of children with other clinical conditions or disorders. Thus, it may be that the children with OI are similar to this mixed clinical/general population on these two PROs. The second reason may be that parents have more difficulty in evaluating their child’s depression and fatigue levels since these are unobservable symptoms (as opposed to more observable outcomes such as mobility).
We found few ceiling effects. Five instruments demonstrated floor effects between 11 and 15% which is in acceptable range given that floor effects for symptom measures reflect the patient is not experiencing the symptom, such as pain.
For children and adolescents, all six parent proxy PROMIS instruments showed some correlation with at least one PODCI element, providing evidence for convergent validity. The Parent Proxy – Mobility PROMIS instrument correlated strongly with the PODCI which has been used in the LCRC, the BBDC, as well as this project. The results of the correlation analysis between the PODCI and six PROMIS® instruments speak to the potential value of expanding the use of PROMIS instruments for children and adolescents in the BBDC. Although the PODCI and PROMIS instruments are not measuring precisely the same aspects of HRQOL, and although the number of relevant respondents in this pilot study was relatively small, we judge that these results support the validity of PROMIS CAT instruments in measuring disease severity in children and adolescents for OI and that use of PROMIS CAT may possibly lead to higher completion rates of on-line questionnaires in future studies because they ask fewer questions of participants than fixed-length questionnaires like PODCI.
PROMIS CATs were not able to differentiate individuals by OI severity. Patients with OI and clinicians often disagree on the level of disease burden experienced. Anecdotal evidence suggests that patients with milder types of OI may have different expectations for quality of life than patients with more affected or severe types. For example, the young man with mild OI may feel severely burdened because he cannot play football with his peers and/or the individual with severe III OI may feel extremely mobile and unrestricted because they have acquired a top-of-the-line wheelchair which allows them to participate in school, family, job, and social activities easily. There is an extensive literature that illustrates that, across numerous chronic diseases, people adapt to their situation or condition, and then, within that context have expectations for quality of life that might seem surprising to people not suffering from that condition [41, 42]. It is also possible that since OI type was self-reported in this study, incorrect identification of OI type is contributing to these findings.
In addition, our pilot study demonstrates that the BBDC Contact Registry for People with OI can be used to recruit participants for online surveys regarding health status. This capability will allow researchers to better understand the OI community’s perspective on health status and quality of life; it will also permit users to capture quantitative PROs using tools such as PROMIS® or PODCI to explore domains that patients identify as important. However, it also underscores some of the important challenges to using registries that must be addressed. Researchers cannot simply seek to expand our understanding of the health issues faced by individuals with OI, but must also develop methodologies to communicate these findings back to the OI community, especially those who do not have access to major OI clinical centers.
Self-knowledge of OI type
A significant proportion of respondents did not know their OI type. This was particularly prevalent in older individuals, a troubling problem as we seek to expand our knowledge of the natural history of OI in adulthood. We believe this result reflects confusion regarding the current and historical classification of OI. Prior to the publication of the 1979 Sillence classification of OI, patients were typically described as having “congenita” or “tarda” (fractures and bowing occurring after birth) disease. This has led many older patients to describe themselves as having type 1 or 2 disease. Although 18 OI types have been described in the research literature, Sillence and others recently pleaded to simplify the classification of OI to mild, moderate, and severe. The rationale is that OI type is artificial and sometimes not helpful. They contend that this simplified categorization, focusing on broad clinical findings, particularly clinical and historical data, fracture frequency, bone densitometry, level of mobility, and patient report tools [19, 43,44,45,46,47], can be used to enhance communication between patients and professionals.
We believe that a critical next step for the BBDC research team is leveraging the clinical data of the BBDC Natural History Study to validate a PRO question set that establishes whether patients have mild, moderate, or severe disease. Patient reports can be validated against known clinical data as well as their clinicians’ perception of disease severity. Broad dissemination of this question set and inclusion of its contents at the beginning of all surveys and studies will standardize data collection, and, most important, help members of the OI community have a better understanding of which study results are relevant to them.
Moving forward, PRO results will be particularly helpful if they demonstrate an association between patients’ perspectives of their own disease experience and objective clinical data stratified by OI type. To date, only a limited number of PROMIS instruments address physiologic symptoms, such as “breathlessness”. However, new instruments are becoming available. Where there are no PROMIS measures, there must be a decision either to develop a new measure (time consuming and costly) or use another PRO measure available in the field that yields good psychometric properties.
Recruitment
The Contact Registry succeeded in attracting a meaningful number of patients with OI who were willing to complete an extensive set of questionnaires. However, while the demographic data on those enrolled in the registry closely approximated participation in this study, neither the Registry nor the study cohorts mirror what is known about the epidemiology of the disorder in the community. Participants in this pilot study were far less diverse in terms of sex and race/ethnicity than the known OI community. Expanded recruitment efforts will clearly be needed for future studies.
Similarly, the international prevalence of persons with OI is estimated to be between 1 and 2 per 20,000 individuals. For the United States, this translates to more than 32,000 possible respondents, yet, at the time we solicited participants in this pilot study, we had only attracted approximately 1000 US subjects to join the BBDC Contact Registry [48]. The OIF has already taken note of this finding and is redoubling its efforts to increase enrollment. Of note, 23% of enrollees in this pilot study were from outside the United States, which speaks to the need to ensure that research findings can be communicated back to the broad OI community.
The original call to participate in this pilot study recruited 100 individuals in a day, but another 2 months were needed to fulfil the recruitment goal of 300. This suggests that although a small cohort of individuals is eager to participate in research, this enthusiasm may not extend to the entire OI community. Moreover, participants were limited to English speakers and those who had access to computers. Some members of the community may have been prevented from participating because of disease limitations or personal time constraints.
In the same vein, we had planned to link the data gathered by this pilot study to existing clinical data collected through the BBDC, so that we could associate the range of clinical findings found on examination with the range of T-scores noted in PROMIS-assessed PROs. However, only 21 of the respondents are known to have participated in the BBDC and had data available for linkage and analysis. Although the registry is a component of the BBDC, we did not specifically recruit individuals who are participating in the BBDC OI Longitudinal Study for this study. One plausible explanation for this shortfall is that those already involved in the Longitudinal Study may believe that they are sufficiently involved and making an adequate contribution to the knowledge base about OI.
The match (or mismatch) between self- and proxy- responses for children
Parent-proxy reported measures of PRO results may well differ from child self-reported results, especially for adolescents and possibly for older children [49]. For this study, we did not ask children to complete self-reported instruments; we relied on parent-proxy instruments. This prevented analysis of congruency between the self-perceived disease experiences of children and the estimations of caretakers. Future studies will need to explore the similarities and differences found in parent-proxy and patient PRO responses, at least for the pediatric age groups for which reliable and valid self-report measures exist. Including tools that can be answered by older children and adolescents is essential.
Respondent burden
CATs instruments were developed to reduce respondent burden by limiting the number of questions that participants need to answer, and our results underscore the importance of that philosophy [36]. The PROMIS CATs for this study usually required between about four and six answers for any given domain. Indeed, in examining the 15 PROMIS instruments alone, the average number of questions answered per individual, per instrument was generally fewer than six questions. Across all the self-report (or proxy-report) domains in PROMIS, adults probably had to respond to 40–50 items. Understandably, completion rates for PODCI were lower. It required response to up to 88 questions (covering 6 domains). Adult participants were not required to complete any instrument as burdensome as the PODCI.
Data regarding the average time needed to administer our study might have enhanced our understanding of study completion rates. Unfortunately, our platform was not able to time participants as they completed the survey instruments. Thus, we cannot know how much faster, on average, participants might be able to complete a PROMIS® instrument than the PODCI.
Recognizing the voice of the patient
Finally, but perhaps most important, the issues of concern to patients with OI frequently differ from the typical clinician focus. Indeed, the PROMIS instruments chosen for the study are not typically topics covered in a routine clinic visit. The wide range of existing PROMIS instruments offers an opportunity to explore a variety of health concerns for which there may not be time to discuss during a standard clinic visit.