- Open Access
Evaluation of DSD training schools organized by cost action BM1303 “DSDnet”
Orphanet Journal of Rare Diseasesvolume 13, Article number: 227 (2018)
The Differences of Sex Development network (DSDnet) aims to establish interactive relationships between clinicians, scientists, support groups and people with a difference of sex development (DSD) to improve the overall care for people affected by such condition. DSDnet has hosted three Training Schools (TSs) in Ghent, Bologna and Budapest between 2015 and 2017 with the primary purpose of providing multidisciplinary training to young professionals and encouraging ongoing activity in the field of DSD. The aim of our study was to evaluate the success and long-term effect effectiveness of these three TSs.
Methods and results
Eighty-seven trainees (70 women, 17 men) attended one of three TSs. The distribution of trainees according to their professional field was: 47 (54.0%) from Pediatrics/Endocrinology, 13 (14.9%) from Biology/Genetics, 12 (13.8%) from Psychology/Psychiatry and 15 (17.2%) from Surgical Professions. All trainees were asked to complete an evaluation form on the last day of the TS to gain feedback on how to improve the next one. A further survey was sent at the end of 2017 to provide information about the overall long-term impact of the TSs. Seventy-eight (89.7%) trainees completed evaluation forms at the end of the respective TSs. Replies to the subsequent survey were received from 76 (87.4%) of trainees. A total of 72/76 (94.7%) responders reported that they continue to be active in the field of DSD. The vast majority (64/68, 94.1%) reported that the TSs had enlarged their professional networks. Among the 76 respondent trainees, 11.8% (n = 9) had applied for a research grant and 10.5% (n = 8) had received a fellowship related to DSD since their TS attendance.
According to our results, the majority of TS participants continue to be active in the field of DSD and have enlarged their professional networks following participation at the TS. These findings indicate the need of this type of educational program and justify ongoing efforts to provide postgraduate multidisciplinary training in rare diseases such as DSD.
Difference (Disorders) of Sex Development (DSD) are rare conditions in which there has been atypical development of the external and/or internal genitalia, the reproductive system and eventually other organs. DSD phenotypes are highly variable and are often associated with lifelong medial problems.
Due to the relatively low frequency of DSDs, there remains limited data regarding optimal management and outcomes for affected individuals and until the formation of international multidisciplinary working groups in the field of DSD around 2000, management has been variable across Europe. Therefore, adequate training of young professionals is needed both for standardized management as well as spreading of expertise. Although many professional organizations such as the European Society of Endocrinology and the European Society of Paediatric Endocrinology organize different types of trainings in the framework of their structural educational program, the DSDnet training school exclusively concentrated on the topic of DSD.
The DSDnet  in frame of the Action of COST (European Cooperation in Science and Technology)  is an EU-funded program that was created to enable researchers to establish interdisciplinary research networks in Europe and beyond. Funds from COST are available to organize conferences, meetings, training schools, short scientific exchanges or other networking activities in a wide range of scientific topics. The COST Action DSDnet: “A systematic elucidation of differences of sex development (DSD)” was started in 2014 by professionals in the field of DSD, together with representatives from patient organizations. The aims of DSDnet were to obtain new knowledge on the biological pathways of sex development in humans, to standardize management and inform physicians and psychologists caring for people who have a DSD, and to actively collaborate with support groups with the aim of improving patient-centered multidisciplinary care.
To achieve these goals, DSDnet created five working groups, each with different objectives. The working group “Dissemination and Capacity Building” used part of its funds to provide training in DSD for eligible young scientists and therefore organized training schools (TSs) for multidisciplinary healthcare professionals in the field of DSD. Each TS faculty was comprised of approximately 30 students and 10 teachers and the schools took place over five half days (2.5 full days in total). The format of each TS included plenary lectures from the teachers, selected case reports or research presentations by the students delivered in plenary style and interactive training sessions in small groups leaving ample time for discussions.
The Program Organizing Committee included a Scientific Committee, the local organizers and, from the second TS onwards, representatives of the previous TS. The draft of each program was presented, discussed and approved during a preparatory meeting of the Actions Management Committee. The three TSs took place in Ghent, Belgium (2015), Bologna, Italy (2016), and Budapest, Hungary (2017).
The title, topics and trainee target groups for each TS varied as specified below:
“Holistic care and research in DSD”: The aim of this first TS was to provide training in various steps of management of DSDs, considering a holistic view on this complex entity. Trainers and trainees were selected from different specialties and professions, ranging from basic and clinical science to psychology and social science. Among the trainers were also a parent and a patient. This meeting provided a range of educational sessions with presentation topics ranging from genetics and development to gender aspects. In addition, there were formal opportunities for trainees to improve their communication skills . The 5th International Symposium on DSD organized by the International Disorders of Sex Development (I-DSD) Registry , followed this TS, enabling the extension of the faculty to include trainers from outside the European Union.
“People with DSD: holistic care and research through the lifespan”: The aim of this TS was to connect young professionals and experts from multidisciplinary backgrounds with each other in an interactive training environment . Attached to this TS, a dedicated workshop was organized for patients and professionals with the aim of jointly developing basic principles of multidisciplinary care .
“Promoting Research in DSD”: The aim of this TS was to strengthen research collaborations between basic scientists, biologists, geneticists and clinical endocrinologists in the field of DSD . Lecturers gave cutting edge presentations ranging from gonadal dysgenesis and new pathways of steroidogenesis to the research potential of ongoing clinical studies.
To attend the TSs, potential attendees had to complete an application form, which was scored using the following standardized criteria: if they were an early stage researcher (< 8 years post PhD); if they were from an Eastern European country; the quality and the presentation of the application; the overall scientific or clinical merits of the application; the benefit for the Researcher/DSD specific merits and the benefit for the Home Institution. Before the TSs, successful applicants were asked to complete preparatory work including e-learning modules, preliminary reading, a case report and/or a presentation of their own research. The e-learning modules used within the first TS and analysis of their development have been described in two recent papers by Kranenburg et al. [8, 9].
The budgets of the three TSs were 36.970 euros, 23.670 euros and 29.020 euros. Travel grants for trainees were between 150 and 780 euros, depending on the local costs for accommodation and travel distances from the home countries of the trainees to the location of the respective TS. The speakers received reimbursement of their travel expenses according to COST rules for reimbursement . Trainees arriving from countries of the European Union, International Partner Country (IPC), Near Neighboring Country (NNC) and Inclusiveness Target Countries (ITC) were supported according to the COST rules .
The primary long-term aim of each TS was to encourage participants to continue and improve involvement in DSD patient care. The second purpose was to promote engagement in the national and international DSD network, for example through participation in the I-DSD registry , through grant and fellowship applications in the field of DSD and by becoming active members of the developing European Reference Network for Rare Endocrine Conditions .
The aim of this study was to confirm the effectiveness of the TS model and to determine whether the TSs achieved their long term aims.
All participants were asked to fill in an anonymous evaluation form about how they had experienced the DSDnet TS on the last day of each meeting. The form was conceived by the Program Organizing Committee of Ghent and included three open questions and 12 responses scored from strongly agree (5) to strongly disagree (1) or from excellent (5) to very poor (1) (Additional file 1: Table S1). Questions were focused particularly on whether the TS met with the expectations of trainees, how they could apply the knowledge gained from the TS in their daily practice, whether the faculty was sufficiently knowledgeable and their overall rating of the TS social program. Following each TS, all the presentations of trainers and case reports of trainees were available on the DSDnet website, with possibilities for further discussion.
A further anonymous survey was sent to all TS trainees at the end of 2017 using the online system Survey Monkey (Table 1). The questionnaire was composed by the authors of the present paper based on Donaldson’s report about the outcome of the European Society for Paediatric Endocrinology (ESPE) Winter Schools . In this survey the participants were first asked about their specialty, job title and training status both at the time of the TS and currently. Participants were also asked about whether they were still active in the field of DSD; whether their professional network was enlarged due to the TS and whether they had applied for or had received any kind of fellowship or grant related to the DSD field since the TS.
The results of these questionnaires were analyzed anonymously by the authors. For the purposes of this study, the term Early Career Investigators refers to researchers within 8 years from the date he/she obtained a PhD/doctorate.
Since the evaluation form and survey were anonymous and involved only healthcare professionals with no clinically sensitive questions, ethical approval was not required.
In 2015, 2016 and 2017 the number of trainees attending the DSDnet training school was 29 (applied: 64), 29 (applied: 48), 29 (applied: 31), with a total of 87 trainees (one attending twice). Median (range) [interquartile range] age was 33 (24–51) , with 15 (17.2%) participants aged 20–30 and 14 (16.1%) aged > 40 years. The gender distribution of the 87 trainees favored women with 70 (80.4%) women and 17 (19.5%) men.
Twenty-five European countries, one International Partner Country (Indonesia) and one Near Neighboring Country (Egypt) were represented (Fig. 1.). There were 30 (34.4%) trainees from Inclusiveness Target Countries and 66 (75.8%) Early Career Investigators present. To minimize the chances for conflict of interest, scholarship awardees were selected by the respective organizing committees according to pre-identified criteria. Applicants and trainees were specialists in the fields of Pediatrics/Endocrinology, Surgery/Urology/Gynecology, Biology/Genetics, Psychology/Psychiatry as shown in Fig. 2.
Results of the evaluation form
A total of Seventy-eight of 87 (89.7%) trainees completed the evaluation form at the end of each TS of whom 47.4% (37/78) reported that they strongly agreed that the TS had met their expectations; 48.7 (38/78) reported that the TS had met their expectations and 3.8% (3/78) felt neutral about this item. A total of 29/78 (37.1%) strongly agreed that they could use the acquired knowledge from the TS in their daily practice; 43/78 (55.1%) agreed and 6/78 (7.7%) were neutral about this. The faculty were felt to be sufficiently knowledgeable with 72% (56/78) strongly agreeing about this and 25.6% (20/78) agreeing. Overall, the social programs and the TSs as a whole received excellent average ratings. There were no questions scored below 3.
Results of the survey
The follow-up survey was completed by 76 of the 87 trainees (87.4%), with participants of the three TSs equally represented. There was a 100% response rate to all questions except for one regarding whether the trainees felt their professional network had enlarged after the TS, which was completed by 68 (89.5%) respondents. As the data collection was anonymous we do not have exact information about the nine non-responders.
A total of 94.7% (n = 72/76) respondents reported that they continue to be active in the field of DSD. Of the 4 participants who were no longer active in this field, the reason reported for discontinuation of DSD activity was incompatibility with their current job role, which had changed since the TS.
The question about job status was completed by 76 (87.4%) trainees including one inconclusive response, therefore 75 responses were analyzed. The declared job status had changed after the third TS in 19/75 trainees (25.3%) (Fig. 3a). 18/19 of these trainees had advanced in their career according to their own specialties and the system of their own countries. All of them stayed or moved into the direction of DSD except one junior researcher, who ended an academic career and became self-employed. The distribution of trainees that changed job status after the third TS is shown in Fig. 3b.
A total of 68/76 (89.5%) trainees responded to being asked about whether their professional network had been enlarged thanks to the TS. Of these, participants were more likely to stay in contact with trainees alone compared to trainers alone (28 vs 8, p = 0.0002) or with both trainees and trainers rather than trainers alone (26 vs 8, p = 0.0006) (Fig. 4.). A total of 5 (7.3%) (3 from the Ghent TS in 2015 and 2 from the Budapest TS in 2017) reported that they remained in contact with other members of the DSDnet COST action, all of whom also maintained contact with patient associations.
Nine (11.8%) of the TS attendants applied for a DSD grant after attendance of a TS. Three of these grants were related to congenital adrenal hyperplasia (CAH), one was claimed within the COST project, one was for DSD related genetics, one was provided by the European Society of Paediatric Endocrinology and no details were supplied regarding the other three. Eight additional trainees (10.5%) reported that they had received a fellowship related to DSD: five in the field of pediatric endocrinology, one related to the COST Action, one in the field of pediatric urology and with no further information submitted regarding one. Of the nine applications for grants, six were from students from the 2015 TS, one from the 2016 TS and two from the 2017 TS. The majority of scholarships were therefore awarded to four students who had attended the first TS, with a further two from the 2016 TS and two from the 2017 TS.
Since the beginning of the COST Action “DSDnet” in 2014, three TSs have been organized providing two and a half days of intensive teaching and including free travel and accommodation to 87 trainees from European countries (85), Indonesia (1, IPC), and Egypt (1, NNC). In order to assess the success, usefulness and long-term outcomes of the meetings, we first used an evaluation form, which was completed on the last day of the TS, followed by a survey, which was completed several months after the third TS. It is possible that the initial evaluation form may have been influenced by immediate feelings of gratitude for participation in the TS. This evaluation form was primarily used to gain feedback regarding how to improve the next TS and we received some constructive advice and opinions from the trainees, which were then used for the subsequent TSs. The ratings of the usefulness, the success, the meeting with expectations, the social program and the TS overall were high. No negative responses were received from any trainee. Free suggestions for improvement included: “Lectures could be shorter with more interaction between trainers/trainees”, “Genetics could be simpler”, “More cases with discussion by multidisciplinary experts would be good” and “More interaction with patients would be good”. From the first TS, it became clear that it is difficult to reconcile basic knowledge and expectations from disciplines as different as basis science and psychology. Therefore, the second and third TSs focused on providing holistic care and understanding the biology of DSD respectively. From the second TS onwards, trainees of previous TSs participated in the program organizing committee to ensure a trainee centered focus. Due to the different scopes of the TSs, the professional occupation of trainees varied per training school. For example, only one psychologist applied to participate in the third TS, due to its dedicated topic “Promoting research in DSD”. In the end, this person did not attend as she did not meet the requirements. For this TS, there was also a remarkable decrease in applications. Possibly the proposed team “Promoting research in DSD” has attracted less professionals working in the field of DSD. Alternatively, it is possible that a large proportion of early-career health care workers with an interest in DSD already had the opportunity to participate in one of the previous TSs. Future possibilities for enhanced interaction with patients were explored in a dedicated workshop that took place attached to the 2nd (Bologna) TS .
In the second part of our investigation we used a survey, which was sent after the third TS. We hope that this survey therefore gives a more objective evaluation and provides information about the long-term outcome of trainees after having attended one of the TSs. Our survey showed that about a quarter (25.3%) of trainees reported that their job status had changed by the time of the survey. Apart from one trainee who declared termination of an academic career, the others reported career advancements both in clinical and research fields. It seems that the timing of TS attendance might be crucial, as the majority of those with different job status (73.7%) attended the first TS that was held in Ghent in 2015 (Fig. 3a and b).
Given that 94.7% of people enrolled in the survey claim to be still active in the DSD field at the time of the survey, we can assume that trainees who were selected to attend the TSs had a strong motivation to operate in the field and had already made a choice on their career direction prior to TS attendance. Indeed, already working in and being motivated to stay in the DSD field was one of the applied selection criteria for trainees, and our results reflect adequate recruitment of trainees with regard to this criterion. We are convinced that this stringent selection procedure also contributed to the overall success of the TSs and the positive ratings that were received.
In our survey, 89.5% of responders reported that they felt their professional network had enlarged thanks to the TS regardless of which TS was attended. In an increasingly digital age, it is easier than before to maintain contact with colleagues from around the globe and it is hoped that this ongoing contact between TS trainees and trainers will result in productive collaborations, leading to the advancement of care of DSD patients over time. Of note, TS participants were more likely to stay in contact with other trainees rather than with trainers (Fig. 4.). Studies suggest that friendships within organizations such as medical schools or training schools are relatively easily formed due to an increased likelihood of shared common interests . There are several benefits reported in friendships developing between peers working together, including the ability to share knowledge and information  and enhanced academic engagement . The high rate of ongoing contact between trainees should therefore be seen as a success of the TS program. This is particularly important for rare disease networks, as peers and partners may not be readily available at their own institution and multicenter collaboration is crucial. In addition, careers involved in rare diseases tend to be hampered by less economic success and also loneliness regarding the field of interest. Therefore, a TS can be highly motivating, and may facilitate successful grant applications and building up one’s personal network.
The numbers of DSD related grants and scholarships obtained after the TSs is limited (11.8 and 9.2%) but the time between the TSs and the survey is short while application times for scholarships and grants are typically much longer. Indeed, as we can see from the results, the majority of the applications for grants (66.7%) and fellowships (50%) were done by students of the first TS in 2015 and it is therefore expected that a number of applications are still to be done.
General recommendations for the organisation of TSs in the field of rare diseases
In-depth and interactive discussion of case reports is highly appreciated by participants and a good way to transfer knowledge
Special attention must be given to make very specialised knowledge accessible for all disciplines involved
It is important to ensure that the size of groups representing different disciplines is balanced to ensure fruitfull discussions as well as to be able to work in subgroups corresponding to multidisciplinary teams
More specific topics may reduce the number of applicants
A social program is an important part of the TS and has to be prepared well also by the organizer
Evaluation forms provide very useful feedback that can be incorporated in the organisation of subsequent TS on the same topic
Based on our experience, relevant indicators of long-term success are
Changes in clinical practice demonstrate the successful transfer of the knowledge during the TS
Asking about any applications for grants or fellowships in the field of the TS demonstrates the ongoing committment and interest in the fied of the candidates
Asking about the change of the job status in the direction of the topic of the TS can also present its positive influence.
Retrospectively, we consider the lack of a question assessing the capacity of the TSs to effectively change clinical practice a missed opportunity. For the evaluation of future similar initiatives, we recommend to incorporate at least one question with this purpose, e.g. Did your participation in the TS lead to a change in your clinical practice? If yes, how was this achieved?
In conclusion, this study highlights the success of the COST Action BM1303“DSDnet” TS model through demonstration of high levels of satisfaction with participation in the TS, high rates of ongoing activity in the field of DSD and a promising DSD related grant and fellowship application rate shortly after termination of the last TS.
A Systematic Elucidation on Differences of Sex Development BMBS COST Action BM1303. http://www.dsdnet.eu/. Accessed 11 Dec 2018.
COST European Cooperation in Science and Technology: http://www.cost.eu/. Accessed 11 Dec 2018.
1st DSDnet Training School 2015. http://www.dsdnet.eu/educationtraining/training-school/training-school-2015.html. Accessed 11 Dec 2018.
I-DSD Registry. www.i-dsd.org. Accessed 11 Dec 2018.
2nd DSDnet Training School 2016. http://www.dsdnet.eu/educationtraining/training-school/training-school-2016.html. Accessed 11 Dec 2018.
Sanders C, Hall J, Sanders C, Dessens A, Bryce J, Callens N, Cools M, Kourime M, Kyriakou A, Springer A, Audi L, Balsamo A, Iotova V, Mladenof V, Krawczynski M, Nordeskjold A, Rozas M, Chaasen-van der Grinten H, Hiort O, Riedl S, Ahmed SF. Involving individuals with disorders of sex development and their parents in exploring new models of shared learning: proceedings from a DSDnet COST action workshop. Sex Dev. 2018; in press.
3rd DSDnet Training School 2017. http://www.dsdnet.eu/educationtraining/training-school/training-school-2017.html. Accessed 11 Dec 2018.
Kranenburg LJC, Reerds STH, Cools M, Alderson J, Muscarella M, Magrite E, Kuiper M, Abdelgaffar S, Balsamo A, Brauner R, Chanoine JP, Deeb A, Fechner P, German A, Holterhus PM, Juul A, Mendonca BB, Neville K, Nordenstrom A, Oostdijk W, Rey RA, Rutter MM, Shah N, Luo X, Grijpink K, Drop SLS. Global application of the assessment of communication skills of Paediatric endocrinology fellows in the Management of Differences in sex development using the ESPE E-Learning.org Portal. Horm Res Paediatr. 2017;88:127–39 Epub 2017 Jul 7.
Kranenburg L, Reerds STH, Cools M, Alderson J, Muscarella M, Grijpink K, Quigley CA, Drop SLS. Global application of assessment of competencies of Paediatric endocrinology fellows in the Management of Differences of sex development (DSD) using the ESPE e-learning.org portal. Med Sci Educ. 2016. https://doi.org/10.1007/s40670-016-0333-9.
European Reference Network on Rare Endocrine Conditions. https://endo-ern.eu. Accessed 11 Dec 2018.
Donaldson MDC. Outcome of doctors attending the European Society for Paediatric Endocrinology Winter Schools 2008-2014. Horm Res Paediatr. 2017;87:396–9 Epub 2017 May 9.
Pogrebin LC. Among friends: who we like, why we like them, and what we do with them. NewYork: McGraw-Hill; 1987.
Morrison EW. Longitudinal study of the effects of information seeking on newcomer socialization. J Appl Psychol. 1993a;78:173–83.
Ethington CA. Influences of the normative environment of peer groups on community college students’ perceptions of growth and development. Res High Educ. 2000;41:703–22.
The author thanks all of the TS trainees who responded to the survey, all of the trainers who gave their much appreciated contribution to the Schools, and Renate Wagner for her help in statistics of TSs, as well as her enormous efforts for the administrative work for the TSs. We also thank for their commitment in the preparation of the three events, the members of the organizing committees:-Ghent: Ellie Magritte (UK), Franco D’Alberton (IT), Alexander Springer (AU), Anna Lauber-Biason (CH), Andreas Kyriakou (GR), Stenvert Drop (NL);-Bologna: Marta Berra (IT), Silvano Bertelloni (IT), Franco D’Alberton (IT), Stenvert Drop (NL), Paolo Ghirri (IT), Zofia Kolesinska (PL), Angela Lucas-Herald (UK), Laetitia Martinerie (FR); Maria Cristina, Meriggiola (IT); -Budapest: Olaf Hiort (DE), Anna Lauber Biason (CH), Zofia Kolesinska (PL), Mojca Zenjav Tansek (SL).
This article is based upon work from COST Action BM1303 DSDnet, supported by COST (European Cooperation in Science and Technology).
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Table S1. Evaluation form. (PDF 151 kb)