Skip to main content

Validating online approaches for rare disease research using latent class mixture modeling

Abstract

Background

Rare disease patients are geographically dispersed, posing challenges to research. Some researchers have partnered with patient organizations and used web-based approaches to overcome geographic recruitment barriers. Critics of such methods claim that samples are homogenous and do not represent the broader patient population—as patients recruited from patient organizations are thought to have high levels of needs. We applied latent class mixture modeling (LCMM) to define patient clusters based on underlying characteristics. We used previously collected data from a cohort of patients with congenital hypogonadotropic hypogonadism who were recruited online in collaboration with a patient organization. Patient demographics, clinical information, Revised Illness Perception Questionnaire (IPQ-R) scores and Zung self-rating depression Scale (SDS) were used as variables for LCMM analysis. Specifically, we aimed to test the classic critique that patients recruited online in collaboration with a patient organization are a homogenous group with high needs. We hypothesized that distinct classes (clinical profiles) of patients could be identified—thereby demonstrating the validity of online recruitment and supporting transferability of findings.

Results

In total, 154 patients with CHH were included. The LCMM analysis identified three distinct subgroups (Class I: n = 84 [54.5%], Class II: n = 41 [26.6%], Class III: n = 29 [18.8%]) that differed significantly in terms of age, education, disease consequences, emotional consequences, illness coherence and depression symptoms (all p < 0.001) as well as age at diagnosis (p = 0.045). Classes depict a continuum of psychosocial impact ranging from severe to relatively modest. Additional analyses revealed later diagnosis (Class I: 19.2 ± 6.7 years [95% CI 17.8–20.7]) is significantly associated with worse psychological adaptation and coping as assessed by disease consequences, emotional responses, making sense of one’s illness and SDS depressive symptoms (all p < 0.001).

Conclusions

We identify three distinct classes of patients who were recruited online in collaboration with a patient organization. Findings refute prior critiques of patient partnership and web-based recruitment for rare disease research. This is the first empirical data suggesting negative psychosocial sequelae of later diagnosis (“diagnostic odyssey”) often observed in CHH.

Background

Patients with rare diseases are dispersed geographically posing significant challenges to research in rare diseases [1, 2]. As such, many rare disease publications report relatively small sample sizes and/or cohorts amassed at individual centers. While geographic distance hampers prospective studies [3], it also contributes to the sense of isolation and marginalization experienced by rare disease patients [4]. The internet has been a powerful tool for rare disease patients to find information, obtain peer-to-peer support and locate online patient organizations [5]. Researchers also have leveraged the internet to enhance prospective recruitment of rare disease patients [6,7,8]. Additionally, researchers have collaborated with patient advocacy groups (i.e. support organizations) to enhance recruitment [8,9,10]. Further, some investigators have shifted from traditional transactional research paradigms to one that accepts patients as partners and key stakeholders (i.e. community-based participatory methods) [11]. One example of using community engagement and online methods comes from research on congenital hypogonadotropic hypogonadism (CHH).

Congenital hypogonadotropic hypogonadism (CHH, ORPHA174590) is a rare, genetic endocrine disorder characterized by absent/incomplete puberty and infertility resulting from deficient secretion (or action) of gonadotropin releasing hormone (GnRH). A range of non-reproductive phenotypes are associated with CHH (i.e. midline defects, skeletal/dental anomalies, unilateral renal agenesis, synkinesia/mirror movement), and approximately half of patients exhibit diminished/absent sense of smell (anosmia)—termed Kallmann syndrome (KS, ORPHA478) [12]. Incidence of CHH is estimated to be 1:48,000 [13] with a striking sexual discordance (4 males:1 female) [14]. Unlike many rare diseases, effective treatments are available. Hormonal therapies (i.e. sex steroids) induce secondary sexual characteristics and gonadotropin therapy or pulsatile GnRH can induce fertility in approximately 75–80% of cases [12]. While CHH/KS does not shorten life expectancy, there is evidence of life altering effects and significant impact on wellbeing and health-related quality of life [15, 16]. To reach geographically dispersed patients with CHH/KS, we have previously partnered with a patient organization and used online data collection to conduct patient needs assessments [17]. After identifying unmet needs, we collaborated with patients to co-create education materials responding to unmet patient informational needs, then evaluated the materials online [18, 19]. A common criticism of such participatory projects is that recruiting patients online in collaboration with a patient advocacy group creates a biased sample that is not representative of the broader patient community [20, 21]. Specifically, critics posit that identifying patients through a patient organization skews the sample—as these individuals are thought to be a homogenous group with disproportionately higher levels of need [22].

In this study we apply a novel statistical approach (latent class mixture modeling, LCMM) [23] to analyze an existing rare disease (CHH/KS) data set previously obtained by partnering with a patient organization using online data collection. We aimed to determine if distinct subgroups (classes) of patients could be identified based on demographic, clinical and patient-reported outcome data. Identifying multiple classes would refute the critique that patients recruited over the web via patient organizations are homogenous (a single class) and not representative of the larger patient population. Notably, demonstrating multiple subgroups in the CHH/KS cohort would support the validity of such recruitment approaches and bolster evidence of transferability of findings. Moreover, such evidence could strengthen methodologic rigor for internet recruitment conducted in collaboration with patient organizations, and therefore has implications for the broader rare disease research community.

Methods

The study is a secondary analysis of de-identified quantitative data previously collected as part of a cross-sectional, multiple methods (quantitative and qualitative) needs assessment of patients with CHH/KS. The original study received ethics approval and all participants provided opt-in electronic informed consent prior to competing an online survey. Findings of the needs assessments have been previously reported [14, 17, 24, 25].

Participants

The original study utilized a community based participatory research framework [26, 27]. Briefly, we partnered with CHH/KS patient community leaders to develop survey content, beta test the online survey and aid in recruitment (for details see [17]). Participants were recruited for the quantitative online survey via social media and patient-oriented sites (i.e. Facebook, Rareconnect.org), an online patient-led forum (CHH/KS chat room) as well as postings on www.clinicaltrials.gov and www.gnrhdeficiency.eu (COST Action BM1105). Men and women with CHH/KS [28] (18–70 years old) were included in the study. Diagnosis was confirmed in a random sampling (40% of subjects) to ensure accuracy of self-reported diagnosis.

Instruments

The online survey collected demographic information (e.g. age, education), clinical information (i.e. age at diagnosis, seen at specialized academic medical center) and participants completed several validated instruments. The Illness Perception Questionnaire-Revised (IPQ-R) includes 38 items (scored on 5-point a Likert scale) assessing emotional and cognitive representations of illness [29]. For the present study we utilized composite scores on three dimensions: consequences (i.e. negative consequences of the disease, 6 items, range 6–30, higher scores indicate greater negative disease consequences), emotional representations (i.e. emotional responses generated by the illness, 6 items, range 6–30, higher scores indicate greater emotional impact of the disease), and illness coherence (i.e. personal understanding and making sense of the disease, 5 items, range 5–25, higher scores indicate greater understanding of the disease) [29]. The Zung Self-Rating Depression Scale (SDS) is a validated, 20-item instrument that is used widely to quantify the severity of affective, somatic, psychomotor, and psychological depressive symptoms [30, 31]. Scores range from 20 to 80 with higher scores indicating more severe depressive symptoms. A score of < 50 is considered normal, 51–60 mild clinically meaningful depressive symptoms, 61–70 moderate to major depressive symptoms (e.g. dysthymia) and > 70 akin to severe major depressive symptoms [30, 31]. For the present study we used the composite SDS score.

Statistical analyses

We employed latent class mixture modeling (LCMM) to test our hypothesis that multiple distinct subgroups (classes) of patients could be identified within a rare disease cohort (Fig. 1). Briefly, LCMM is a versatile analytic strategy used to identify previously-unobserved subgroups (i.e. classes) in cross-sectional data [23]. LCMM utilizes multiple measured variables (continuous or binary/categorical) to identify subgroups of latent (unmeasured) constructs. Mixture refers to the presence of multiple subgroups (classes) with unique characteristics within a sample. LCMM produces a novel categorical variable, the hitherto unobserved classes, wherein subjects are assigned to a specific class based on the statistically greatest likelihood of belonging to the particular subgroup. Findings can be used to identify membership in a respective class as well as identify what variable(s) predict class membership. Given the sample size (n = 154), we identified eight variable for LCMM analysis (i.e. n = 20 subjects per variable). We followed step-by-step procedures for LCMM to identify subgroups (classes) as described by Ram and Grimm [32]. Mplus software [33] was employed for LCMM analyses and SPSS Version 25 (IBM) was used for other statistical analyses (i.e. χ2, ANOVA, Sheffe post hoc tests, multiple linear regression). As an exploratory step, we used multivariate linear regression, with age at diagnosis as the dependent variable, to assess the relationship between age at diagnosis and patient-reported outcome measures. Results are reported as mean ± standard deviation and p values < 0.05 were considered statistically significant.

Fig. 1
figure1

Schematic of latent class mixture modeling for the CHH/KS cohort (n = 154). The latent categorical variable (i.e. distinct class) is measured by eight (y1–6, u1–2). Continuous variables are depicted by “y”, binary/categorical variables “u” and “ε” indicates error. The categorical variable “C” indicates the most likely class for each case based on conditional probabilities. Class membership can be modeled as a function of multiple characteristics (X). Class membership can be used to predict continuous and categorical (Y/U) variables

Results

In total, 154 participants (99 males, 55 females) were included in the analysis. Characteristics of the participants as shown in Table 1. To evaluate the null hypothesis (i.e. patients are a single, homogenous, monolithic group) we tested an initial model with two unobserved subgroups compared to a single class model followed by sequential increases in classes (i.e. 3 then four) to identify the best fit model. Based on the Ram criteria in general (i.e. Vuong–Lo–Mendell–Rubin likelihood ratio test, Lo–Mendell–Rubin adjusted likelihood ratio test), Bayesian information criteria (BIC = 6656.46) and parametric bootstrapped likelihood ratio test (p = 0.03) in particular, the three class solution provided more information compared with two and four class solutions. Increasing the number of classes to four failed to reach statistical significance and was abandoned in favor of the model with three subgroups.

Table 1 Participant characteristics (n = 154)

Accordingly, latent class analysis revealed the model with three subgroups demonstrated the best fit. The 154 subjects were classified as being a member of class I (n = 84 [54.5%]), class II (n = 41 [26.6%]) or Class III (n = 29 [18.8%]). We used maximum likelihood estimation with robust standard errors in an iterative process to determine parameters within the three classes and to generate probabilities of each participant belonging to each class. The classification probabilities for the most likely latent class membership (i.e. posterior probabilities) were acceptable (class I = 0.836, II = 0.906, III = 0.937, entropy = 0.80). Radar graphs of the three distinct profiles are shown in Fig. 2. Mean values with 95% confidence intervals for each continuous variable are shown in Table 2. In terms of the categorical variable education, high school education had a weak, negatively association with Class I membership (χ2 = − 0.575, p = 0.024) while having post-graduate education was strongly associated (χ2 = 4.392, p < 0.001). Having a college/university education was positively associated with membership in Class III (χ2 = 1.869, p = 0.028). Having been seen at a specialty/academic medical center was not significantly associated with membership in any of the classes.

Fig. 2
figure2

Three latent classes of patients with CHH/KS (n = 154). The LCMM analysis identified distinct subgroups based on demographic, clinical and patient-reported outcome data. a Class I (n = 84) was diagnosed significantly later (p = 0.045) and exhibits high SDS, disease consequences and emotional impact scores and low illness coherence (making sense of one’s disease). b Class II (n = 41) exhibited less severe psychosocial outcomes and greater illness coherence (all p < 0.001 vs. Class I). c Class III (n = 29) was diagnosed the earliest and exhibited relatively modest psychosocial impact. Dx diagnosis, SDS self-rating depression scale, IPQR Illness Perception Questionnaire-Revised

Table 2 Mean values for continuous variables by class

Compared to other classes, class I was diagnosed significantly later (Sheffe post hoc p < 0.05) and had significantly more IPQ-R consequences, greater IPQ-R emotional impact, and lower IPQ-R illness coherence (i.e. how one makes sense of their disease) (all p < 0.001) (Table 2). Class I also exhibited significantly higher Zung SDS scores (measuring depressive symptoms) than either of the other subgroups (Sheffe post hoc p < 0.001). Class II and III exhibit SDS scores in the normal reference range (i.e. 20–39) yet class I SDS scores (95% CI 47.6–51.7) fell squarely in the rage of moderate depressive symptoms (SDS range 48–55) akin to dysthymia or depression typically seen in the ambulatory setting [31]. These empirical data point to psychosocial sequelae associated with later diagnosis. As an exploratory step, we performed linear regression to identify predictors of older age at diagnosis among the patient reported outcome (i.e. IPQR consequences, IPQR emotional representations, IPQR illness coherence, Zung SDS). With the stepwise model selection procedures, only illness coherence was retained. Thus, the multivariate linear regression model is equivalent to a Pearson’s correlation. Illness coherence was negatively correlated with age at diagnosis (r = − 0.192, p = 0.009), consistent with a small-to-medium effect size (i.e. 0.1–0.3). Thus, older age at diagnosis is associated with making less sense of the illness (CHH/KS).

Discussion

Herein we present findings of LCMM on a previously recruited cohort (n = 154) of patients with CHH/KS. The iterative Bayesian analytic approach identified three distinct subgroups (classes) of patients who were recruited online in collaboration with a patient support organization. These findings refute prior critiques of online community-based participatory methods for recruiting rare disease patients. Specifically, we identified three subgroups spanning a range of ages and psychosocial adaption (i.e. illness perceptions and depression symptoms). The present findings suggest this research methodology does not recruit a biased, homogenous sample with disproportionately higher needs than the general patient population. Prior work demonstrates that rare disease patients are internet “power-users” who frequently go online to seek information about their condition and find peer-to-peer support [5]. Indeed, a number of studies point to the important role the internet and social media has for patients and families living with rare diseases [17, 34,35,36,37]. Given the avid use of the internet by rare disease patients, researchers have utilized this avenue expand recruitment [6,7,8]. Moreover, the European Reference Network on Rare Endocrine Conditions (ENDO-ERN) highlights effective partnerships with patient organizations for conducting needs assessments [14, 17, 24, 25] and co-creating patient-facing materials [18, 19] as a model for facilitating clinic trials and improving clinical care for rare diseases [38].

As initially depicted in the 2011 landmark EURORDIS report [4], the so-called “diagnostic odyssey” is a common experience across rare diseases. Importantly, recent reports demonstrate the problem of delayed diagnosis has persisted in the field of rare diseases [39, 40]. Published literature on CHH/KS has suggested that later diagnosis is associated with poorer psychosocial outcomes (i.e. wellbeing and health-related quality of life) [12]. The cohort presented here is the largest group of prospectively recruited CHH/KS patients with measures relating to psychosocial outcomes (i.e. IPQ-R, SDS). The LCMM analysis of the CHH/KS cohort (n = 154) identified a subgroup (class I) was diagnosed significantly later (19.2 ± 6.7 years, 95% CI 17.8–20.7, p = 0.045) and had significantly worse patient-reported outcomes relating to psychosocial function (all p < 0.001). These data provide the first empirical evidence of the negative psychosocial sequelae related to later diagnosis. These data provide further impetus for increased attention to timely diagnosis [12, 15, 41]. Notably, the present findings provide new insights into coping and adaptation that were recently highlighted in a publication co-authored with a patient support group leader outlining a roadmap for supporting psychological adaptation related to CHH/KS [15].

Limitations of the present study include the limited sample size (n = 154). However, given the rarity of CHH/KS (i.e. 1:48,000 [13]), this cohort represents the largest prospectively recruited cohort in the literature. The LCMM findings demonstrate the utility of this analytic approach for identifying subgroups within cohorts despite limited sample size. Future directions may include broader application of LCMM to identify patient subgroups to inform tailored approached to treatment of rare diseases. Similarly, latent growth mixture modeling could be employed as a data-centered analytic strategy to identify distinct trajectories (i.e. natural history, treatment response) in rare disease populations [42].

Conclusions

Rare disease research has traditionally been hampered by geographically dispersed patient populations resulting in studies with limited sample size and power. Partnerships with patient organizations combined with online data collection have emerged as approaches to overcome geographic roadblocks in rare disease research. Using LCMM, we counter critiques of such approaches. Rare disease patients recruited online in partnership with patient organizations are not a monolithic group. We identify three distinct latent subgroups (classes) spanning a spectrum of age, clinical experiences (age at diagnosis) and measures of coping (illness perceptions and depressive symptoms). These data support the validity of using community based participatory methods and online data collection for rare disease research. Moreover, we show the first empirical evidence that later age of CHH/KS diagnosis is associated with worse psychosocial outcomes. These findings underscore the importance of timely identification and initiation of treatment for improving health-related quality of life for patients with CHH/KS.

Availability of data and material

De-identified data will be made readily available upon request for research purposes to qualified individuals within the scientific community.

Abbreviations

AMC:

Academic medical center

CHH:

Congenital hypogonadotropic hypogonadism

COST:

European Cooperation in Science and Technology

ENDO-ERN:

European Reference Network on Rare Endocrine Conditions

EUCERD:

European Union Committee of Experts on Rare Diseases

EURORDIS:

European Organization for Rare Diseases

GnRH:

Gonadotropin releasing hormone

KS:

Kallmann syndrome

LCMM:

Latent class mixture modeling

SD:

Standard deviation

References

  1. 1.

    Kempf L, Goldsmith JC, Temple R. Challenges of developing and conducting clinical trials in rare disorders. Am J Med Genet A. 2018;176:773–83.

    PubMed  Article  Google Scholar 

  2. 2.

    Day S, Jonker AH, Lau LPL, Hilgers RD, Irony I, Larsson K, Roes KC, Stallard N. Recommendations for the design of small population clinical trials. Orphanet J Rare Dis. 2018;13:195.

    PubMed  PubMed Central  Article  Google Scholar 

  3. 3.

    Rees CA, Pica N, Monuteaux MC, Bourgeois FT. Noncompletion and nonpublication of trials studying rare diseases: a cross-sectional analysis. PLoS Med. 2019;16:e1002966.

    PubMed  PubMed Central  Article  Google Scholar 

  4. 4.

    EURORDIS: The voice of 12,000 patients: Experiences and expectations of rare disease patients on diagnosis and care in Europe. In Book The voice of 12,000 patients: Experiences and expectations of rare disease patients on diagnosis and care in Europe (Editor ed.^eds.). City; 2009.

  5. 5.

    Fox S. Peer-to-Peer Healthcare. Washington: Pew Internet; 2011.

    Google Scholar 

  6. 6.

    Johnson KJ, Mueller NL, Williams K, Gutmann DH. Evaluation of participant recruitment methods to a rare disease online registry. Am J Med Genet A. 2014;164A:1686–94.

    PubMed  Article  Google Scholar 

  7. 7.

    Krischer J, Cronholm PF, Burroughs C, McAlear CA, Borchin R, Easley E, Davis T, Kullman J, Carette S, Khalidi N, et al. Experience with direct-to-patient recruitment for enrollment into a clinical trial in a rare disease: a web-based study. J Med Internet Res. 2017;19:e50.

    PubMed  PubMed Central  Article  Google Scholar 

  8. 8.

    Applequist J, Burroughs C, Ramirez A Jr, Merkel PA, Rothenberg ME, Trapnell B, Desnick RJ, Sahin M, Krischer JP. A novel approach to conducting clinical trials in the community setting: utilizing patient-driven platforms and social media to drive web-based patient recruitment. BMC Med Res Methodol. 2020;20:58.

    PubMed  PubMed Central  Article  Google Scholar 

  9. 9.

    Forsythe LP, Szydlowski V, Murad MH, Ip S, Wang Z, Elraiyah TA, Fleurence R, Hickam DH. A systematic review of approaches for engaging patients for research on rare diseases. J Gen Intern Med. 2014;29(Suppl 3):S788-800.

    PubMed  Article  Google Scholar 

  10. 10.

    Merkel PA, Manion M, Gopal-Srivastava R, Groft S, Jinnah HA, Robertson D, Krischer JP. Rare Diseases Clinical Research N: the partnership of patient advocacy groups and clinical investigators in the rare diseases clinical research network. Orphanet J Rare Dis. 2016;11:66.

    PubMed  PubMed Central  Article  Google Scholar 

  11. 11.

    Hoekstra F, Mrklas KJ, Khan M, McKay RC, Vis-Dunbar M, Sibley KM, Nguyen T, Graham ID, Panel SCIGPC, Gainforth HL. A review of reviews on principles, strategies, outcomes and impacts of research partnerships approaches: a first step in synthesising the research partnership literature. Health Res Policy Syst. 2020;18:51.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  12. 12.

    Boehm U, Bouloux PM, Dattani MT, de Roux N, Dode C, Dunkel L, Dwyer AA, Giacobini P, Hardelin JP, Juul A, et al. Expert consensus document: European Consensus Statement on congenital hypogonadotropic hypogonadism–pathogenesis, diagnosis and treatment. Nat Rev Endocrinol. 2015;11:547–64.

    PubMed  Article  Google Scholar 

  13. 13.

    Laitinen EM, Vaaralahti K, Tommiska J, Eklund E, Tervaniemi M, Valanne L, Raivio T. Incidence, phenotypic features and molecular genetics of Kallmann syndrome in Finland. Orphanet J Rare Dis. 2011;6:41.

    PubMed  PubMed Central  Article  Google Scholar 

  14. 14.

    Dzemaili S, Tiemensma J, Quinton R, Pitteloud N, Morin D, Dwyer AA. Beyond hormone replacement: quality of life in women with congenital hypogonadotropic hypogonadism. Endocr Connect. 2017;6:404–12.

    PubMed  PubMed Central  Article  Google Scholar 

  15. 15.

    Dwyer AA, Smith N, Quinton R. Psychological aspects of congenital hypogonadotropic hypogonadism. Front Endocrinol. 2019;10:353.

    Article  Google Scholar 

  16. 16.

    Dwyer AA. Psychosexual effects resulting from delayed, incomplete, or absent puberty. Curr Opin Endocr Metab Res. 2020;14:15–21.

    PubMed  Article  Google Scholar 

  17. 17.

    Dwyer AA, Quinton R, Morin D, Pitteloud N. Identifying the unmet health needs of patients with congenital hypogonadotropic hypogonadism using a web-based needs assessment: implications for online interventions and peer-to-peer support. Orphanet J Rare Dis. 2014;9:83.

    PubMed  PubMed Central  Article  Google Scholar 

  18. 18.

    Badiu C, Bonomi M, Borshchevsky I, Cools M, Craen M, Ghervan C, Hauschild M, Hershkovitz E, Hrabovszky E, et al. Developing and evaluating rare disease educational materials co-created by expert clinicians and patients: the paradigm of congenital hypogonadotropic hypogonadism. Orphanet J Rare Dis. 2017;12:57.

    PubMed  PubMed Central  Article  Google Scholar 

  19. 19.

    Dwyer AA, Au MG, Smith N, Plummer L, Lippincott MF, Balasubramanian R, Seminara SB. Evaluating co-created patient-facing materials to increase understanding of genetic test results. J Genet Couns. 2020;30(2):598–605.

    PubMed  Article  Google Scholar 

  20. 20.

    Bombak AE, Hanson HM. A critical discussion of patient engagement in research. J Patient Cent Res Rev. 2017;4:39–41.

    PubMed  PubMed Central  Article  Google Scholar 

  21. 21.

    Whitaker C, Stevelink S, Fear N. The use of facebook in recruiting participants for health research purposes: a systematic review. J Med Internet Res. 2017;19:e290.

    PubMed  PubMed Central  Article  Google Scholar 

  22. 22.

    Benedict C, Hahn AL, Diefenbach MA, Ford JS. Recruitment via social media: advantages and potential biases. Digit Health. 2019;5:2055207619867223.

    PubMed  PubMed Central  Google Scholar 

  23. 23.

    Lee CS, Faulkner KM, Thompson JH. Identifying subgroups: part 1: patterns among cross-sectional data. Eur J Cardiovasc Nurs. 2020;19:359–65.

    PubMed  Article  Google Scholar 

  24. 24.

    Dwyer AA, Quinton R, Pitteloud N, Morin D. Psychosexual development in men with congenital hypogonadotropic hypogonadism on long-term treatment: a mixed methods study. Sex Med. 2015;3:32–41.

    PubMed  Article  Google Scholar 

  25. 25.

    Dwyer AA, Tiemensma J, Quinton R, Pitteloud N, Morin D. Adherence to treatment in men with hypogonadotrophic hypogonadism. Clin Endocrinol (Oxf). 2017;86:377–83.

    CAS  Article  Google Scholar 

  26. 26.

    Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. Am J Public Health. 2010;100(Suppl 1):S40-46.

    PubMed  PubMed Central  Article  Google Scholar 

  27. 27.

    Kwon SC, Tandon SD, Islam N, Riley L, Trinh-Shevrin C. Applying a community-based participatory research framework to patient and family engagement in the development of patient-centered outcomes research and practice. Transl Behav Med. 2018;8:683–91.

    PubMed  Article  Google Scholar 

  28. 28.

    Young J. Approach to the male patient with congenital hypogonadotropic hypogonadism. J Clin Endocrinol Metab. 2012;97:707–18.

    CAS  PubMed  Article  Google Scholar 

  29. 29.

    Moss-Morris R, Weinman J, Petrie K, Horne R, Cameron L, Buick D. The revised illness perception quesionnaire (IPQ-R). Psychol Health. 2002;17:1–16.

    Article  Google Scholar 

  30. 30.

    Zung WW. A Self-Rating Depression Scale. Arch Gen Psychiatry. 1965;12:63–70.

    CAS  PubMed  Article  Google Scholar 

  31. 31.

    Zung WW. The role of rating scales in the identification and management of the depressed patient in the primary care setting. J Clin Psychiatry. 1990;51(Suppl):72–6.

    PubMed  Google Scholar 

  32. 32.

    Ram N, Grimm KJ. Methods and measures: growth mixture modeling: a method for identifying differences in longitudinal change among unobserved groups. Int J Behav Dev. 2009;33:565–76.

    PubMed  PubMed Central  Article  Google Scholar 

  33. 33.

    Muthén LK, Muthén BO: Mplus User’s Guide. 8th edn. Los Angeles: Muthén & Muthén; 1998–2017.

  34. 34.

    Tozzi AE, Mingarelli R, Agricola E, Gonfiantini M, Pandolfi E, Carloni E, Gesualdo F, Dallapiccola B. The internet user profile of Italian families of patients with rare diseases: a web survey. Orphanet J Rare Dis. 2013;8:76.

    PubMed  PubMed Central  Article  Google Scholar 

  35. 35.

    Nicholl H, Tracey C, Begley T, King C, Lynch AM. Internet use by parents of children with rare conditions: findings from a study on parents’ web information needs. J Med Internet Res. 2017;19:e51.

    PubMed  PubMed Central  Article  Google Scholar 

  36. 36.

    Titgemeyer SC, Schaaf CP. Facebook support groups for rare pediatric diseases: quantitative analysis. JMIR Pediatr Parent. 2020;3:e21694.

    PubMed  PubMed Central  Article  Google Scholar 

  37. 37.

    Morgan T, Schmidt J, Haakonsen C, Lewis J, Della Rocca M, Morrison S, Biesecker B, Kaphingst KA. Using the internet to seek information about genetic and rare diseases: a case study comparing data from 2006 and 2011. JMIR Res Protoc. 2014;3:e10.

    PubMed  PubMed Central  Article  Google Scholar 

  38. 38.

    Hiort O, Cools M, Springer A, McElreavey K, Greenfield A, Wudy SA, Kulle A, Ahmed SF, Dessens A, Balsamo A, et al. Addressing gaps in care of people with conditions affecting sex development and maturation. Nat Rev Endocrinol. 2019;15:615–22.

    PubMed  Article  Google Scholar 

  39. 39.

    Boycott KM, Rath A, Chong JX, Hartley T, Alkuraya FS, Baynam G, Brookes AJ, Brudno M, Carracedo A, den Dunnen JT, et al. International cooperation to enable the diagnosis of all rare genetic diseases. Am J Hum Genet. 2017;100:695–705.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  40. 40.

    Gainotti S, Mascalzoni D, Bros-Facer V, Petrini C, Floridia G, Roos M, Salvatore M, Taruscio D. Meeting patients’ right to the correct diagnosis: ongoing international initiatives on undiagnosed rare diseases and ethical and social issues. Int J Environ Res Public Health. 2018;15(10):2072.

    PubMed Central  Article  Google Scholar 

  41. 41.

    Dwyer AA, Jayasena CN, Quinton R. Congenital hypogonadotropic hypogonadism: implications of absent mini-puberty. Minerva Endocrinol. 2016;41:188–95.

    PubMed  Google Scholar 

  42. 42.

    Lee CS, Faulkner KM, Thompson JH. Identifying subgroups: part 2: trajectories of change over time. Eur J Cardiovasc Nurs. 2020;19:444–50.

    PubMed  Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank the patients who generously gave their time in participating in the original needs assessment (online survey). Special thanks to patient community leader Mr. Neil Smith.

Funding

This work was supported by the National Institutes of Health National Center for Advancing Translational Sciences (1R03TR003533-01). Dr. Dwyer also receives support from the National Institutes of Health Eunice Kennedy Shriver National Institute of Child Health and Human Development (1P50HD104224-01).

Author information

Affiliations

Authors

Contributions

Andrew A. Dwyer conceived and designed the work, acquired and curated the data, oversaw data analysis and drafted the manuscript. Ziwei Zeng conducted the data analysis, contributed to interpreting the data and provided critical feedback on the manuscript. Christopher S. Lee made substantial contributions to the design of the work, contributed to interpreting the data and provided critical feedback on the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Andrew A. Dwyer.

Ethics declarations

Ethics approval and consent to participate

The initial study (protocol #233/13) was reviewed and approved by the Commission Cantonale d’ethique de la recherche sur l’être humain which is the institutional review board (Ethics Committee) associated with the University of Lausanne. All survey participants provided opt-in electronic consent prior to completing the online evaluation.

Consent for publication

Not applicable.

Competing interests

The authors have no financial or non-financial competing interests to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dwyer, A.A., Zeng, Z. & Lee, C.S. Validating online approaches for rare disease research using latent class mixture modeling. Orphanet J Rare Dis 16, 209 (2021). https://doi.org/10.1186/s13023-021-01827-z

Download citation

Keywords

  • Community based participatory research
  • Diagnostic odyssey
  • Hypogonadotropic hypogonadism
  • Kallmann syndrome
  • Patient organization
  • Rare disease