Open Access

Developing and evaluating rare disease educational materials co-created by expert clinicians and patients: the paradigm of congenital hypogonadotropic hypogonadism

  • COST Action BM1105,
  • Corin Badiu1,
  • Marco Bonomi2, 3,
  • Ivan Borshchevsky4,
  • Martine Cools5,
  • Margarita Craen5,
  • Cristina Ghervan6,
  • Michael Hauschild7,
  • Eli Hershkovitz8,
  • Erik Hrabovszky9,
  • Anders Juul10,
  • Soo-Hyun Kim11,
  • Phillip Kumanov12,
  • Beatriz Lecumberri13,
  • Manuel C. Lemos14,
  • Vassos Neocleous15,
  • Marek Niedziela16,
  • Sandra Pekic Djurdjevic17,
  • Luca Persani3, 18, 19,
  • Franziska Phan-Hug7,
  • Duarte Pignatelli20,
  • Nelly Pitteloud21,
  • Vera Popovic22,
  • Richard Quinton23,
  • Nicos Skordis24, 25,
  • Neil Smith26,
  • Magdalena Avbelj Stefanija27,
  • Cheng Xu21,
  • Jacques Young28 and
  • Andrew A. Dwyer29Email author
Orphanet Journal of Rare Diseases201712:57

https://doi.org/10.1186/s13023-017-0608-2

Received: 14 December 2016

Accepted: 6 March 2017

Published: 20 March 2017

Abstract

Background

Patients with rare diseases face health disparities and are often challenged to find accurate information about their condition. We aimed to use the best available evidence and community partnerships to produce patient education materials for congenital hypogonadotropic hypogonadism (CHH) and the olfacto-genital (Kallmann) syndrome (i.e., CHH and defective sense of smell), and to evaluate end-user acceptability. Expert clinicians, researchers and patients co-created the materials in a multi-step process. Six validated algorithms were used to assess reading level of the final product. Comprehensibility and actionability were measured using the Patient Education Materials Assessment Tool via web-based data collection. Descriptive statistics were employed to summarize data and thematic analysis for analyzing open-ended responses. Subsequently, translation and cultural adaption were conducted by clinicians and patients who are native speakers.

Results

Co-created patient education materials reached the target 6th grade reading level according to 2/6 (33%) algorithms (range: grade 5.9–9.7). The online survey received 164 hits in 2 months and 63/159 (40%) of eligible patients completed the evaluation. Patients ranged in age from 18 to 66 years (median 36, mean 39 ± 11) and 52/63 (83%), had adequate health literacy. Patients scored understandability at 94.2% and actionability at 90.5%. The patient education materials were culturally adapted and translated into 20 languages (available in Additional file 1).

Conclusions

Partnering with patients enabled us to create patient education materials that met patient- identified needs as evidenced by high end-user acceptability, understandability and actionability. The web-based evaluation was effective for reaching dispersed rare disease patients. Combining dissemination via traditional healthcare professional platforms as well as patient-centric sites can facilitate broad uptake of culturally adapted translations. This process may serve as a roadmap for creating patient education materials for other rare diseases.

Keywords

Congenital hypogonadotropic hypogonadism Kallmann syndrome Rare diseases E-health Community based participatory research Patient education Patient participation Patient-centered care Nursing

Background

The landmark 2009 report from the European Organization for Rare Diseases (EURORDIS) brought to light the many challenges faced by patients with rare diseases [1]. Delays in diagnosis, difficulty finding information about their condition and inadequate access to expert care are frequent patient experiences. Indeed, some have posited that living with a rare disease places one in the realm of health disparities [2]. Physical and psychological morbidity can be significant and feelings of isolation and powerlessness can further undermine quality of life [3]. Importantly, potential means to overcome these challenges include using the internet to connect dispersed patients with expert care and community engagement to help empower patients who feel marginalized by the healthcare system [46].

One such rare disorder is congenital hypogonadotropic hypogonadism (CHH, ORPHA174590). Based on a study of French conscripts, CHH occurs in approximately one in 4,000-10,000 [7]. It is clinically characterized by incomplete (or absent) puberty and infertility resulting from insufficient secretion or action of gonadotropin releasing hormone (GnRH) - the master hormone of the reproductive axis [8]. Genetic defects that affect olfactory axon and olfactory bulb development, thus leading to absent or defective sense of smell (anosmia or hyposmia), usually also affect the migration of neuroendocrine GnRH cells from the nose to the brain during embryonic life, thus leading to CHH. This condition is known as the olfacto-genital syndrome or Kallmann syndrome (ORPHA478). Some genetic and phenotypic overlap may exist between isolated CHH and Kallmann syndrome, yet patients with this syndrome are much more likely to exhibit additional non-reproductive phenotypes (i.e., skeletal defects, renal agenesis, cleft lip/palate, deafness, etc) than normosmic CHH patients. In the context of an international network of leading clinicians/geneticists/researchers focused on CHH [9], we have previously developed patient partnerships and conducted a needs assessment that leveraged engagement with patient support groups, social media and online data collection [10]. In parallel, we developed a web-based platform [9] with resources for patients to find expert clinicians and peer-to-peer support. Additionally, consensus guidelines for the diagnosis and treatment of CHH were created using an evidence-based approach [8].

Engagement and co-creation have been effectively used in diverse fields including business, design and computer science (i.e. user-centered design) as a means to spur innovation, adoption and foster sustainability [11, 12]. Therefore, the aim of the present study was to partner clinical experts and patients to co-create high-quality patient education materials (PEM) that respond to the issues and questions most important and relevant to patients. Secondary aims included evaluating the readability of the PEM and end-user acceptability (i.e. understandability and actionability) as well as to disseminate these materials widely across different countries and cultures.

Methods

PEM development

A community based participatory research framework was selected to guide the development of the patient education material (PEM) for its relevance to patient empowerment and health disparities [13]. The Patient advocacy Working group of the European network focused on CHH (COST Action BM1105, “GnRH Deficiency: Elucidation of the neuroendocrine control of human reproduction”) [9] worked closely with online patient community leaders (i.e. moderators of online patient support sites) to identify key PEM content areas and topics based on the most frequently asked questions on social media sites (Additional file 1) as well as from a previously conducted patient needs assessment [10]. Clinical information was drawn from the evidence-based consensus statement on the approach to diagnosis and treatment of CHH [8]. The PEM development was an iterative process (Fig. 1) involving multiple stakeholders including patients, patient support groups, clinicians and researchers spanning the fields of endocrinology, andrology, nursing and genetics. At each step, input and feedback were used to refine and modify the PEM.
Fig. 1

Study Schema. PEM were co-created in a multi-step process. (a) Three main sources were used for PEM development. (b) Members of the Patient Advocacy Working Group and patient collaborators identified topics for the PEM in an iterative process. (c) The initial draft was created and revised based on patient input. (d) PEM (V2.0) was circulated to the Clinical Working Group and Genetics Working Group members for comment and revised accordingly with patient validation in two rounds. (e) PEM (V4.0) were evaluated by patients recruited via social media (private/closed Facebook groups), patient support meetings and via RareConnect [12]. (f) Following evaluation materials were culturally adapted and translated to 20 languages and distributed in avenues targeting healthcare professionals and patients. PEM: patient education materials, V: version

Readability assessment

To assess reading level of the produced PEM, we subjected the final version to several validated measures evaluating readability: Flesch Reading Ease Formula (evaluates sentence length and number of syllables per word), Flesch Kincaid Grade Level (converts the Flesch reading ease formula to a grade level), Gunning Fox Index (calculates a weighted average of the number of words per sentence and long words to determine grade level), Coleman Liau Index (uses number of characters rather than syllables to determine grade level), Simple Measure of Gobbledygook (SMOG, a modification of the Gunning-Fog Index it calculates grade level based on the number of words with 3 or more syllables) and the Automated Readability Index (ratio of difficult words and sentences to provide an estimated age range and grade level) [14].

End-user acceptability

To evaluate end-user perspectives of adults with CHH (18 years and older), we used an online data collection (SurveyGizmo™) and recruited a convenience sample of patients via postings on closed/private CHH social media group (Facebook™), as well as notifications in patient support group meetings and RareConnect [15]. This social media approach has been previously shown to be an effective means of recruitment for this rare disease patient population [10]. The survey included questions on patient demographics, past healthcare interactions and a brief assessment of healthcare literacy that has been validated against longer gold-standards metrics [16, 17].

After reviewing a pdf of the PEM, participants were asked to complete the Patient Education Materials Assessment Tool (PEMAT). This instrument was developed and validated by the U.S. Department of Health & Human Services Agency for Health Research & Quality to evaluate print and audiovisual educational materials [18]. The unique aspect of the PEMAT is that it incorporates other additional elements that are not assessed in traditional readability formulas. Patients select agree, disagree or not-applicable for 17 items relating to understandability (the ability to process key messages) and 7 items on actionabilty (the ability to identify what one can do to manage their condition). Items rated as agree are given a score of 1, disagree 0 and cumulative scores are expressed as a percentage (total score/possible total X 100). Initial psychometric evaluation of the PEMAT has demonstrated strong internal consistency, good reliability, and initial evidence of construct validity [19]. Survey respondents were also given an opportunity to provide free text comments (i.e. critiques and suggestions) after completing the PEMAT questions.

Statistical analyses

The survey was alpha tested by patients in two rounds to identify and correct any bugs prior to online launch. Descriptive statistics were used to report summary findings. To assess for potential response bias, Student’s t test and Chi square test were used to compare demographic characteristics of patients who completed the evaluation with those who did not (partial completion). Thematic analysis [20] was employed to codify and analyze open-text responses NVivo11 (QSR International PSY Ltd., Melbourne Australia). The study was reviewed and approved by the ethics committee of the University of Lausanne and participants provided opt-in online consent.

Dissemination

The final step of this process was to disseminate the PEM to reach the broadest possible audience. This included using native speakers (i.e. expert clinicians, medical translators) from across the European network to provide versions in multiple languages. Particular attention was given to finding appropriate terms and examples for the translated PEM to make them culturally sensitive and not simply verbatim translations. The final materials will be distributed via traditional means to reach healthcare professionals (i.e. peer-review publication, professional meetings, individual providers’ websites, and via the COST Action website [9]). In parallel PEMs will be distributed via patient support groups including online social media (Facebook™, Twitter™, patient blogs) and publicized on internet platforms targeting the rare disease community including the EURORDIS initiative RareConnect [15].

Results

PEM development

Patient partnerships were used to identify key topics and to target issues most important to patients as well as to contribute content. A working group of the network (Patient Advocacy Working Group) created a topic list based on the consensus statement guidelines [8]. Additional items were drawn from focus group discussions with patients held in the context of patient support meetings (organized with patient leaders) as part of the prior needs assessment [10] (Fig. 1A). Patient collaborators also contributed lists of “frequently asked questions” as well as topics that were recurrent in social media threads and chat room discussions. Common questions include: what causes CHH? Why didn’t I go through puberty? Why can’t I smell? Is it curable? Can I have children? Will my children have CHH? (see Additional file 1). The drafted material version 1.0 (V1.0) went through two subsequent revisions to refine language, wording and selection of images via email and the PEM was finalized in a face-to-face meeting prior to vetting with the broader network (Fig. 1D). During this development process it was sometimes challenging to balance the input and feedback from clinicians and patients to find the right balance and depth of information provided. Indeed, the face-to-face meeting was valuable for arriving at consensus as opinions were conflicting at times during the process and this was not easy to reconcile via email.

Readability levels

Readability was evaluated using 6 different validated algorithms that are widely used to assess reading level (Table 1). These employ different formulas that use word length and complexity (i.e. the number of characters or syllables in words, sentence length) to calculate an average grade level needed to understand the material. Most patients read at an 8-9th grade reading level [21]. However, expert recommendation has identified the target reading level at 6th grade (i.e. an 11 year-old child) [22]. Two of the six algorithms scored the PEM at the target grade level (Flesch Kincaid Grade Level: 5.9, Automated Readability Index: 6.1). The mean grade level across instruments was 8.0 (Table 1), indicating that more work could be done to enhance readability. However, one challenge in doing this is the number of complex words (i.e. hypogonadism, cryptorchidism, infertility, etc) that were deemed important by clinicians and patients alike to include and define in lay terms.
Table 1

Summary of readability of co-created patient education material

Summary statistics (5 pages, 203 sentences + images)

 Word count = 1,654

 Character count = 8,251

 Complex words (≥3 syllables) = 268

 Average characters per word = 5

 Average words per sentence = 8.1

Instrument

Score

Grade level

 Flesch Reading Ease

67.6

8-9th

 Flesch-Kincaid Reading Level

5.9

6th

 Gunning Fog Index

9.7

9-10th

 Coleman Liau Index

8.8

8-9th

 SMOG

9.7

9th

 Automated Readability Index

6.1

6th

 Mean

8.0

8th grade

Participants

Following patient alpha testing to identify and correct bugs in the online evaluation, the survey was launched and remained open for 8-weeks. During this period, 164 hits were registered. In total, 38 (23%) were “one-click” entries who passed the opt-in consent but did not enter demographic information. Responses of five participants were excluded (age <18 years). More than a third of respondents (58/164, 35%) partially completed the evaluation (i.e. demographics up to viewing the PEM) and 63 (38%) completed the entire PEM evaluation (Fig. 1E). Characteristics of survey respondents are depicted in Table 2. Notably, the predominance of male responders (2:1) is keeping with the striking sexual discordance in CHH [8]. Overall, patients were well-educated (46/63, 73% achieving university or higher) and by-and-large exhibited adequate health literacy (52/63, 82%). Notably, the mean age of diagnosis was 20.9 ± 6.4 years (range: 10–40, median 19) suggesting that many patients are diagnosed quite late. In terms of prior healthcare interactions, more than half (39/63, 62%) had either a consultation or had received care at a specialized academic center. In total, 36/63 (56%) had undergone genetic testing yet only 12/63 (19%) reported having had genetic counseling. We found no significant differences between those who completed the evaluation and the partial completers in terms of age (p = 0.30), sex (p = 0.37), education (p = 0.94), health literacy (p = 0.15), or being seen at an academic center (p = 0.09).
Table 2

Patient characteristics (n = 63)

Sex

n (%)

 Male

42 (67%)

 Female

21 (33%)

Age

 

 18–29

13 (21%)

 30–39

24 (38%)

 40–49

17 (27%)

 50–59

5 (8%)

 60+

4 (6%)

Children

 

 None

42 (66%)

 Biologic children

14 (22%)

 Adopted children

7 (11%)

Education level

 

 High school/vocational

17 (27%)

 University

25 (40%)

 Post-Graduate

21 (33%)

Health literacya

 

 Adequate

52 (83%)

 Inadequate

11 (17%)

 Health experiences

 

 Seen at academic center

39 (62%)

 Had genetic testing

36 (57%)

 Received genetic counseling

12 (19%)

ahealth literacy as assessed by [16, 17]

End-user acceptability

Patients gave the co-created PEM high scores on understandability (range: 88.9–97.5%, total mean: 94.2%) which includes content, word choice/style, use of numbers, organization, layout/design and visual aids (Table 3). The lowest rating (88.9%) was linked with being uncluttered which was commented on in the free text field by three patients (i.e. having more white space). Similarly, patients gave high scores on actionability (overall mean: 90.5%). The lowest score was assigned to explaining how to use charts, graphs, or diagrams to take action and manage the condition. Together the high scores on both understandability and actionability indicate high end-user acceptability.
Table 3

PEMAT Understandability and actionability of co-created materials (n = 63)

PEMAT topic/category

% agree

Content

92.1%

 The material makes its purpose completely evident.

 The material does not include information or content that distracts from its purpose.

90.5%

Word choice & style

93.7%

 The material uses common, everyday language.

 Medical terms are used only to familiarize the audience with the terms.

 When used, medical terms are defined.

93.7%

 The material uses the active voice (e.g. action verbs).

95.2%

Use of numbers

96.8%

 Numbers appearing in the material are clear and easy to understand.

 The material does not expect the user to perform calculations.

96.8%

Organization

93.7%

 The material breaks or “chunks” information into short sections.

 The material’s sections have informative headers.

95.2%

 The material presents information in a logical sequence.

93.7%

 The material provides a summary.

96.8%

Layout & design

95.2%

 The material uses visual cues to draw attention to key points.

Visual aids

95.2%

 The material uses visual aids whenever they could make content more easily understood.

 The material’s visual aids reinforce rather than distract from the content.

92.1%

 The material’s visual aids have clear titles or captions.

95.2%

 The material uses illustrations and photographs that are clear and uncluttered.

88.9%

 The material uses simple tables with short and clear row and column headings.

97.5%

Total understandability

94.2%

Actionability

95.2%

 The material clearly identifies at least one action the user can take.

 The material addresses the user directly when describing actions.

92.1%

 The material breaks down any action into manageable, explicit steps.

92.5%

 The material provides a tangible tool (e.g. checklists) whenever it could help the user take action.

94.5%

 The material explains how to use the charts, graphs, tables, or diagrams to take actions

70.7%

 The material uses visual aids whenever they could make it easier to act on the instructions.

92.0%

Total actionability

90.5%

Overall we received comments from 45/63 (71.4%) patients. Comments were coded according to themes and sorted into categories. In total, 52 concepts were identified from the 45 comments clustering into five categories (Table 4). The most frequent sentiments were expressions of thanks/approval (n = 19, 37%) followed by content (i.e. treatment, infertility, and psychological aspects) n = 11 (21%), format (i.e. use of simple language, spacing) n = 10 (19%), personal concerns (n = 9, 17%) and three comments underscored the importance of translating the PEM to make it available to more patients.
Table 4

Patient comments (n = 52) regarding the co-created materials

Category

Representative quote(s)

Thanks/approval

n = 19

• “I was very impressed and I think my friends and family will find it easy to understand”

• “I am glad that there is a clear male/female explanation. Often materials I find focus predominantly on the males”

Content

n = 11

• “Elaborate on infertility and treatment”

• “I think you may be under discussing the life-long emotional and psychological impact”

Formatting

n = 10

• “It felt like the pages were a bit full but I can appreciate it must have been hard to provide all the information necessary in only 5 pages”

• “I think if there was a way to click on each section for more detailed info that might help”

Personal concerns

n = 9

• “Other rare conditions that can also be evident in patients with KS/CHH… …Explain that patients can have additional illnesses besides KS/CHH”

• “It might be helpful to state that anosmia is permanent. Of course it is most important to focus on the hormonal component, but there are definitely considerations in dealing with anosmia as well”

Dissemination

n = 3

• “It would be great if you could translate into several languages to reach more people”

Broad dissemination

Native speakers from across the European network made culturally adapted translations. In some instances local patients contributed to this translation and adaption process. The translated PEMs required cultural adaptation in some instances to help make them more relevant for the target audience. For instance, a small cherry was used to describe the size of the pituitary gland in the Hungarian version, the Chinese version was altered as “what you should know” was not culturally appropriate, and terms describing depression were adapted in the Polish version to enhance comprehension by the lay public. Every effort was made to keep the entire content of the PEM in the translated versions. When text length expanded the images were adjusted accordingly to maintain a 5-page document. PEM are now available in 20 languages: English, Bulgarian, Chinese, Danish, Dutch, French, German, Greek, Hebrew, Hungarian, Italian, Korean, Polish, Portuguese, Romanian, Russian, Serbian, Slovenian, Spanish, and Turkish (Additional file 2). Dissemination plans will target healthcare professionals and patient-centric avenues such as social media and patient support sites.

Discussion

The aim of this study was to engage patients and co-create PEM that respond to what matters most to patients. Subsequently, we evaluated the readability and end-user acceptability of the PEM and sought to widely disseminate the translated PEM across different countries and cultures. Patients living with a rare disease face health disparities [2] and patient engagement has been identified as potential means to empower this patient population [46]. Interestingly, patient engagement has recently been gaining attention in the context of orphan drug development [23]. However, the extent of patient engagement varies widely. A 2014 systematic review of patient engagement for research on rare diseases found engagement is typically unidirectional - involving patients in consultative roles and rarely in creative aspects or in terms of dissemination [24]. The present study is unique in that we used a participatory process to co-create PEM with patients; we then evaluated the PEM produced by this collaboration, and worked with patient groups to facilitate dissemination to the largest possible audience.

We previously partnered with online patient community leaders to identify the unmet health and informational needs of patients with congenital hypogonadotropic hypogonadism (CHH) and Kallmann syndrome [10]. In the present study, the partnership was more clearly bi-directional as patients were not simply providers of opinions; rather they contributed directly in co-creating the PEM in an iterative process. Notably, patient knowledge and expertise emerges from the day-to-day experiences of living and coping with a rare condition and therefore is inherently different from the expertise of healthcare professionals [25]. Recently, a study examining online exchanges among patients with rare adrenal disorders found that information and support were central elements in peer-to-peer exchanges [26]. Moreover, the authors noted that patient-centered care could be enhanced by better integrating patient knowledge with the care provided by professionals. In the present study, developing the PEM was a true partnership that recognized patient expertise as unique and complementary to expert clinician knowledge. We believe that this co-creation contributed to the high acceptability ratings by patients.

This evaluation process of the co-created PEM has limitations. The evaluation was only conducted on the English version. As such, the findings are not completely transferable to the other translated versions despite the inclusion of patients in developing some of the translations. Moreover, the additional validation step of back translating the other versions was not conducted and this could be viewed as a limitation. We only assessed readability once the materials had been finalized, not during the development process. In future studies, this testing could be incorporated earlier in the development process to improve the reading level of developed PEM. While the evaluation was overwhelmingly positive and a fairly sizeable sample was reached (for a rare disease population), the patients completing the evaluation were quite well-educated and exhibited high levels of health literacy. Accordingly, our ability to draw inferences to a broader population of lower literacy patients is limited. This may reflect a bias of using a web-based survey - as perhaps those using the web may have higher literacy levels. However, recruiting sufficient numbers of patients for rare disease studies has been a long-standing challenge [27, 28]. Therefore, we used a web-based approach to overcome this barrier but note that such an approach entails a potential risk of bias.

The Pew Foundation’s published report on health and the internet indicates that patients living with a rare disease are internet power users who are most likely to seek information about their condition online and find support from other patients using social media [29]. Based on our previous success combining patient partnerships and social media for the online needs assessment [10], we employed a similar approach in the present study to reach a relatively large sample (n = 63) over 8-weeks. These experiences suggest that web-based platforms are an effective means to reach and connect rare disease patients. Thus, the opportunities afforded by the internet and social media may provide novel avenues for crowdsourcing solutions as well as offering a shared venue for either clinician- or patient-led collaborations to improve quality and add value to the healthcare system [5, 30]. The European Union Committee of Experts on Rare Diseases (EUCERD) recommendations for Centers of Expertise underscore the importance of collaboration with patient organizations to provide information that is at once accessible and adapted to patient needs [31]. For many rare diseases, such as CHH/Kallmann syndrome, formal organized patient support organizations do not exist. As such, web-based approaches and social media provide a critical means to broadly reach patients, identify priorities and incorporate their perspectives and knowledge into care. This may be particularly advantageous in light of the movement to form European Reference Networks for rare diseases [32, 33].

The final step in this co-creation process was to engage in bi-directional dissemination. This has been identified as a shortcoming in much of the patient engagement research conducted in the context of rare diseases [24]. Through the work of members of the Network and patients alike, materials were adapted and translated into 20 languages by native speakers. This collaborative process is essential for ensuring that information provided to patients is culturally adapted and sensitive – a key element for Centers of Expertise [31]. In parallel to traditional healthcare professional outlets (e.g. scientific meetings, peer-review publication) patient participants are distributing materials directly to other patients via social media and postings on centralized patients sites [15]. The co-created PEM (in multiple languages) is a critical component of the list of patient resources available on the website of the European network comprising a virtual empowerment toolkit for patients and families [9]. Available information includes listings of international specialized referral centers, genetic testing labs, clinical trials, and peer-to-peer support as well as a portal for a patient registry. We are utilizing both professional-oriented avenues and more patient-oriented social media outlets to hopefully reach unprecedented numbers of patients and clinicians and overcome traditional roadblocks of implementation into practice [3436].

Conclusions

Partnering with patients enabled co-creation of high-quality PEM while social media and web-based data collection facilitated timely evaluation by a dispersed patient population. We believe that partnering with expert patients was an empowering experience and provides valuable contributions for developing patient-centered approaches to care. We envision this work will serve as a roadmap for those wishing to engage in a co-creation process and will help inform projects aimed at improving care for patients living with a rare disease.

Abbreviations

CHH: 

Congenital hypogonadotropic hypogonadism

EUCERD: 

European Union Committee of Experts on Rare Diseases

EURORDIS: 

European Organization for Rare Diseases

COST: 

European Cooperation in Science and Technology

PEM: 

Patient education materials

PEMAT: 

Patient education materials assessment tool

SMOG: 

Simple measure of Gobbledygook

Declarations

Acknowledgements

The authors wish to thank the patients who generously gave their time in helping to develop the patient education materials and we acknowledge the Working Group members of COST Action BM1105 for their contributions. Special thanks to Ms. Virginia Hughes for her critical input during the development process, Prof. Linan Cheng of the Shanghai Institute of Planned Parenthood Research for her validation of the Chinese translations and Ms. Özlem Firat for her help in coordinating the translated materials. This work was supported by the Rare Disease Foundation and the BC Children’s Hospital Foundation (# 17-26) and COST Action BM1105.

Funding

This work was supported by the COST Action BM1105 and received funding from the Rare Disease Foundation and the BC Children’s Hospital Foundation (# 17-26).

Availability of data and materials

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Authors’ contributions

AD conceived the study and participated in study design, conduct, analyses and drafted the manuscript. RQ, NP and NS helped draft the educational materials with patients. All authors helped revise the materials and contributed to developing culturally relevant translations. All authors read and approved the final manuscript.

Competing interests

The authors have no financial or non-financial competing interests to declare.

Consent for publication

Not applicable.

Ethics approval and consent to participate

This study (protocol #233/13) was reviewed and approved by the Commission Cantonale d’ethique de la recherche sur l’être humain which is the institutional review board (Ethics Committee) associated with the University of Lausanne. All survey participants provided opt-in electronic consent prior to completing the online evaluation.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Endocrinology, National Institute of Endocrinology, C. Davila University of Medicine and Pharmacy
(2)
Deptartment of Clinical Sciences & Community Health and the Division of Endocrine and Metabolic Diseases & Laboratory of Endocrine and Metabolic Research, University of Milan
(3)
Ospedale San Luca, IRCCS Istituto Auxologico Italiano
(4)
Patient Advocacy Working Group and the International Medical Interpreters Association
(5)
University Hospital Ghent Department of Pediatric Endocrinology, Ghent University
(6)
University of Medicine and Pharmacy “Iuliu Hatieganu”
(7)
Endocrinology, Diabetes & Obesity Service of the Department of Pediatric Medicine and Surgery, Children’s Hospital of Lausanne
(8)
Pediatric Endocrinology and Metabolism Unit, Faculty of Health Sciences, Soroka Medical Center, Ben-Gurion University of the Negev
(9)
Laboratory of Endocrine Neurobiology of the Institute of Experimental Medicine of the Hungarian Academy of Sciences
(10)
Department of Growth and Reproduction GR, Rigshospitalet section 5064
(11)
St. George’s Medical School of the University of London, Molecular & Clinical Sciences Research Institute, Cell Biology & Genetics Research Centre
(12)
Clinical Center of Endocrinology, Medical University
(13)
Autónoma University of Madrid, Hospital La Paz Institute of Health Research (IdiPAZ), Endocrinology and Nutrition Service of La Paz University Hospital
(14)
CICS-UBI, Health Sciences Research Centre, University of Beira Interior
(15)
Department of Molecular Genetics of the Cyprus Institute of Neurology and Genetics
(16)
Department of Pediatric Endocrinology and Rheumatology, Poznan University of Medical Sciences
(17)
School of Medicine, University of Belgrade & Clinic of Endocrinology, Diabetes and Metabolic Diseases, University Clinical Center Belgrade
(18)
Department of Clinical Sciences & Community Health, University of Milan
(19)
Division of Endocrine and Metabolic Diseases & Laboratory of Endocrine and Metabolic Research
(20)
Department of Endocrinology, Hospital S João
(21)
University of Lausanne and the Endocrinology, Diabetes & Metabolism Service, Lausanne University Hospital
(22)
School of Medicine, University of Belgrade
(23)
University of Newcastle-upon-Tyne, Institute of Genetic Medicine and the Royal Victoria Infirmary
(24)
Division of Pediatric Endocrinology, Paedi Center for specialized Pediatrics
(25)
St George’s University of London Medical School at the University of Nicosia
(26)
Patient Advocacy Working Group
(27)
University Medical Centre Ljubljana, University Children’s Hospital, Dept. of pediatric endocrinology, diabetes and metabolism
(28)
Service d’Endocrinologie et des Maladies de la Reproduction, Hôpital Bicêtre
(29)
University of Lausanne, Institute of Higher Education & Research in Healthcare and the Endocrinology, Diabetes & Metabolism Service of the Lausanne University Hospital

References

  1. EURORDIS. The voice of 12,000 patients: Experiences and expectations of rare disease patients on diagnosis and care in Europe. France: Boulogne-Billancourt; 2009.Google Scholar
  2. Holtzclaw Williams P. Policy framework for rare disease health disparities. Policy Polit Nurs Pract. 2011;12:114–8.View ArticlePubMedGoogle Scholar
  3. Cohen JS, Biesecker BB. Quality of life in rare genetic conditions: a systematic review of the literature. Am J Med Genet A. 2010;152A:1136–56.View ArticlePubMedPubMed CentralGoogle Scholar
  4. Ayme S, Kole A, Groft S. Empowerment of patients: lessons from the rare diseases community. Lancet. 2008;371:2048–51.View ArticlePubMedGoogle Scholar
  5. Vicari S, Cappai F. Health activism and the logic of connective action. A case study of rare disease patient organisations. Inf Commun Soc. 2016;19:1653–71.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Sofolahan-Oladeinde Y, Mullins CD, Baquet CR. Using community-based participatory research in patient-centered outcomes research to address health disparities in under-represented communities. J Comp Effectiveness Res. 2015;4:515–23.View ArticleGoogle Scholar
  7. Fromantin M, Gineste J, Didier A, Rouvier J. Impuberism and hypogonadism at induction into military service. Statistical study. Problemes Actuels d’Endocrinologie et de Nutr. 1973;16:179–99.Google Scholar
  8. Boehm U, Bouloux PM, Dattani MT, de Roux N, Dode C, Dunkel L, Dwyer AA, Giacobini P, Hardelin JP, Juul A, et al. Expert consensus document: European Consensus Statement on congenital hypogonadotropic hypogonadism--pathogenesis, diagnosis and treatment. Nat Rev Endocrinol. 2015;11:547–64.PubMedGoogle Scholar
  9. COST Action BM1105 [http://www.gnrhnetwork.eu/]. Accessed 14 Mar 2017.
  10. Dwyer AA, Quinton R, Morin D, Pitteloud N. Identifying the unmet health needs of patients with congenital hypogonadotropic hypogonadism using a web-based needs assessment: implications for online interventions and peer-to-peer support. Orphanet J Rare Dis. 2014;9:83.View ArticlePubMedPubMed CentralGoogle Scholar
  11. Crawford MJ, Rutter D, Manley C, Weaver T, Bhui K, Fulop N, Tyrer P. Systematic review of involving patients in the planning and development of health care. BMJ. 2002;325:1263.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94:392–429.View ArticlePubMedPubMed CentralGoogle Scholar
  13. Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. Am J Public Health. 2010;100 Suppl 1:S40–46.View ArticlePubMedPubMed CentralGoogle Scholar
  14. CheckText [http://www.readabilityformulas.com]. Accessed 14 Mar 2017.
  15. RareConnect [https://www.rareconnect.org]. Accessed 14 Mar 2017.
  16. Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med. 2004;36:588–94.PubMedGoogle Scholar
  17. Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, Bradley KA, Nugent SM, Baines AD, Vanryn M. Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med. 2008;23:561–6.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Shoemaker SJ, Wolf MS, Brach C, (Prepared by Abt Associates IuCNHI: The Patient Education Materials Assessment Tool (PEMAT) and User’s Guide. In Book The Patient Education Materials Assessment Tool (PEMAT) and User’s Guide (Editor ed.^eds.). Maryland: Agency for Healthcare Research and Quality; 2013.Google Scholar
  19. Shoemaker SJ, Wolf MS, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns. 2014;96:395–403.View ArticlePubMedPubMed CentralGoogle Scholar
  20. Saldana J. Coding manual for qualitative researchers. Thousand Oaks, CA: Sage; 2009.Google Scholar
  21. Safeer RS, Keenan J. Health literacy: the gap between physicians and patients. Am Fam Physician. 2005;72:463–8.PubMedGoogle Scholar
  22. Doak CC, Doak LG, Root JH. Teaching Patients with Low Literacy Skills. 2nd ed. Philadelphia, PA: J.B. Lippincott Company; 1996.Google Scholar
  23. Bronstein MG, Kakkis ED. Patients as key partners in rare disease drug development. Nat Rev Drug Discov. 2016;15:731–2.View ArticlePubMedGoogle Scholar
  24. Forsythe LP, Szydlowski V, Murad MH, Ip S, Wang Z, Elraiyah TA, Fleurence R, Hickam DH. A systematic review of approaches for engaging patients for research on rare diseases. J Gen Intern Med. 2014;29 Suppl 3:S788–800.View ArticlePubMedGoogle Scholar
  25. Hartzler A, Pratt W. Managing the personal side of health: how patient expertise differs from the expertise of clinicians. J Med Internet Res. 2011;13:e62.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Kauw D, Repping-Wuts H, Noordzij A, Stikkelbroeck N, Hermus A, Faber M. The contribution of online peer-to-peer communication among patients with adrenal disease to patient-centered care. J Med Internet Res. 2015;17:e54.View ArticlePubMedPubMed CentralGoogle Scholar
  27. Kruer MC, Steiner RD. The role of evidence-based medicine and clinical trials in rare genetic disorders. Clin Genet. 2008;74:197–207.View ArticlePubMedGoogle Scholar
  28. Watson MS, Epstein C, Howell RR, Jones MC, Korf BR, McCabe ER, Simpson JL. Developing a national collaborative study system for rare genetic diseases. Genet Med. 2008;10:325–9.View ArticlePubMedGoogle Scholar
  29. Fox S. Peer-to-peer healthcare: many people - especially those living with chronic or rare diseases - use online connections to supplement professional medical advice. Washington, D.C.: Pew Internet; 2011.Google Scholar
  30. Nambisan P, Nambisan S. Models of consumer value cocreation in health care. Health Care Manag Rev. 2009;34:344–54.View ArticleGoogle Scholar
  31. (EUCERD) EUCoEoRD: Recommendations on quality criteria for centres of expertise for rare diseases in member states. In: Book Recommendations on Quality Criteria for Centres of Expertise for Rare Diseases in Member States (Editor ed.^eds.). City; 2011.Google Scholar
  32. Diseases EUCoEoR. Recommendations to the European Commission and the Member States on European Reference Networks for Rare Diseases. 2013.Google Scholar
  33. European Reference Network [http://ec.europa.eu/health/rare_diseases/european_reference_networks/erf/index_en.htm]. Accessed 14 Mar 2017.
  34. Sung NS, Crowley Jr WF, Genel M, Salber P, Sandy L, Sherwood LM, Johnson SB, Catanese V, Tilson H, Getz K, et al. Central challenges facing the national clinical research enterprise. JAMA. 2003;289:1278–87.View ArticlePubMedGoogle Scholar
  35. Thornicroft G, Lempp H, Tansella M. The place of implementation science in the translational medicine continuum. Psychol Med. 2011;41:2015–21.View ArticlePubMedGoogle Scholar
  36. Bellgarda MI, Sleemanc MW, Guerrerod FD, Fletcher S, Baynam G, Goldblatt J, Rubinstein Y, Bell C, Groft S, Barreroa R, et al. Rare disease research roadmap: navigating the bioinformatics and translational challenges for improved patient health outcomes. Health Policy Technol. 2014;3:325–35.View ArticleGoogle Scholar

Copyright

© The Author(s). 2017

Advertisement