Skip to main content
Download PDF

Guidance to develop a multidisciplinary, international, pediatric registry: a systematic review

Orphanet Journal of Rare Diseases volume 18, Article number: 296 (2023) Cite this article

Abstract

Aim

The European Reference Network for craniofacial anomalies and ear, nose and throat disorders (ERN-CRANIO) aims to improve craniofacial care on a European scale. Within ERN-CRANIO, the cleft lip and palate (CL/P) work stream seeks to ameliorate health outcomes for patients with CL/P. This work stream acknowledged the need for a European wide registry for comparable outcome measures and therapy endpoints to achieve this goal. This review aimed to provide a scientific basis for the conceptualization of this registry by studying previous registry initiatives.

Methods

This review performed thematic analysis on twenty-four articles through narrative synthesis. An iterative process was used to identify key-themes required for prolonged registry success.

Results

Analysis of the literature resulted in twenty-one distinct headings including quantitative and qualitative data. Quantitative data including registry characteristics were visualized in a table. The analysis of qualitative data resulted in the identification of fourteen key-themes, which have been summarized and visualized in a guidance.

Conclusion

This review has successfully identified key-themes required for the development of an international, multidisciplinary, pediatric registry for pan-European cleft care. The guidance provided by this review applies to the goals of ERN-CRANIO, but can be used by any initiative developing a registry.

Key points

  • Twenty-four previous multidisciplinary, pediatric registry initiatives were analyzed using thematic analysis in an iterative process to identify key-themes required for registry success.

  • Fourteen key-themes for the successful development of an international, multidisciplinary, pediatric registry were identified.

  • This study provides the scientific basis for the development of the ERN-CRANIO CL/P registry, however this study is of use to any initiative seeking to develop a registry.

Introduction

The European Reference Network for Craniofacial anomalies and ear, nose, and throat disorders (ERN CRANIO) is a Europe-wide initiative to improve craniofacial care [1]. ERN CRANIO seeks to achieve health goals unattainable by a single country, by combining the disease-specific expertise, knowledge, and resources of the ERN CRANIO member healthcare centers. Within ERN CRANIO, workstreams for distinctive craniofacial diagnosis were created. One of these workstreams is the cleft lip and palate and orodental anomalies workstream (CL/P workstream). This workstream is comprised of specialists working at the ERN CRANIO member healthcare centers, and includes experts of all specialisms required in cleft care.

The multidisciplinary treatment of patients with CL/P is challenging, as it starts almost immediately after birth and lasts up to adulthood. Patients with CL/ might face numerous surgical procedures; speech therapy; hearing problems; orthodontic and dentition problems; psychological support and more during childhood and adolescence [2,3,4,5,6].

One of the main issues with cleft care is the lack of scientific evidence for optimal treatment protocols. Consequently, many different treatment protocols have been implemented within European healthcare centers over the last decades. Therefore, the comparison of treatment outcomes between centers remains impeded. Accurate data collection is also hampered by the variation of outcome measures used in studies, which complicates the comparison of treatment outcomes between different protocols [7,8,9,10]. For the same reason, benchmarking between cleft centers is impeded. All these factors slow down innovation and progress in this field.

The lack of sound scientific evidence about optimal treatment of CL/P favors undesirable practice variation even in well-resourced countries [11]. To facilitate good comparative studies, the definition of uniform outcome measures and treatment endpoints is vital. The need to systematically register outcome measures on a European level is widely recognized within the CL/P work stream. For this purpose, ERN CRANIO will take the initiative to develop a common dataset and a European cleft registry.

To support the conceptualization of the registry, a systematic review of the literature was conducted to report on previous experiences in this field. The aim of this review is to identify the pitfalls in the development and implementation as well as factors influencing long-term success of a multidisciplinary, international registry for cleft care on a global scale. However, the findings and guidance reported in this review are meaningful for all (international) multidisciplinary collaborations seeking to register data of a pediatric patient population.

Methods

Search strategy and selection criteria

A systematic review was performed according to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) [12]. Members of the research team determined the search terms with a medical librarian, who developed a comprehensive search strategy. Embase, Web of Science, Medline, and ABI/INFORM Collection – ProQuest were consulted from inception up to June 2021. Online Appendix 1 of Additional file 1 contains the search strategies for each database.

Studies were included if their primary or secondary aim was to describe the design and/or the methods to develop or maintain a multidisciplinary registry that involves pediatric patients or are specifically designed for pediatric patients. As our purpose was to identify all information relevant to the development of an international database for pediatric patients, studies based on poor methodological quality or publications from before 2000 were not excluded, since these could still provide useful information on high quality registry development and design. Therefore, no method for quality assessment apart from the aforementioned exclusion criteria was used. Studies were excluded if they primarily analyzed the content of a registry (i.e., completeness or comparisons in outcomes) without assessing the design, development or maintenance of the registry itself. Studies written in any other language than English were excluded as well. In case the described registries contained information about both pediatric patients and adults, the study had to include a specific description for the pediatric population within the registry, or the differences between the registration of pediatric and adult patients had to be addressed.

Search results from all databases were merged in Endnote. Two authors independently screened for titles and abstracts according to a standardized, blinded process [13]. Full texts were reviewed independently by two authors as well. In case of disagreement, a third author was consulted. Authors were contacted by e-mail in case of any uncertainty about suitability, or in case of missing information. To find any other relevant articles, reference lists were scanned from all articles that were included after full text review.

Data extraction and synthesis

The guidance of Popay et al. was used to establish a narrative synthesis [14]. Key themes for data extraction were first identified through thematic analysis. Two reviewers analyzed the included study, and a data extraction table was made in excel containing twenty-one headings (Table 1). The process of thematic analysis was done in an inductive and iterative manner, and several meetings were held during the process. Further, relevant information was extracted from each study by one reviewer, whereas a second reviewer cross-checked results. Data was assigned to the appropriate key theme. According to the guidance, the qualitative data collected within each key theme was then summarized. Structured narratives of each key theme are included in the results. For the extracted data that were unsuitable to be summarized into a narrative, like quantitative data or registry characteristics, tabulation was used to present a clear overview (Table 2).

Table 1 Collected data headings with description
Full size table
Table 2 Quantitative data and characteristics of reviewed registries
Full size table

Results

In total 1804 references were identified via Embase, Web of Science, Medline, and ABI/INFORM Collection – ProQuest up until August 2022. In total 761 duplicates were removed, and 142 publications were screened. 118 publications were excluded. Figure 1 shows the Preferred Reporting Items for Systematic Review and Meta-Analysis, PRISMA flow-chart. The complete data extraction table is available as a supplementary Excel file (Additional file 2). The headings used in the Excel file match the headings used in the article.

Fig. 1
figure 1

Prisma diagram

Full size image

Key themes

Purpose

The most common purpose of pediatric registries was to monitor the incidence of congenital malformations in newborns (n = 7) [15,16,17,18,19,20,21]. Other purposes were data-collection for performance reporting (n = 3) [17, 22, 23], research (n = 4) [17, 24,25,26], health monitoring (n = 2) [17, 27], and complications monitoring (n = 1) [24]. Two registries described the main purpose of creating an inter-operable multicenter framework for data collection [28, 29]. Prince et al. reported the change of a paper-based registry into electronic-based registry as the main goal of their study [30]. Other motivations for developing pediatric registries were improving international cooperation between specialized centers [28] assisting parent or caretakers of pediatric patients to access resources for support [16] Gauvrit et al. defined the purpose of the registry to report general indicator of cochlear implants in pediatric and adult patients [31].

Funding

Funding was essential for the development and durability of a registry [23]. Continues funding was necessary to train, hire and retain registry team members [17]. From the included studies, the majority received funding deriving from a local or national government (N = 6) [17, 20, 22, 25, 32, 33]. Other sources were nonprofit organizations [34], or via a combination of multiple sources such as government run grants, and private (industry) trusts [21, 26,27,28, 30,31,32, 35]. However, a lack of funding, or biased funding could both be detrimental to the functioning of a registry [23]. Nembhard et al. described that the change from a temporary research grant to long term government funding greatly benefitted the overall success of the registry [17].

Governance

In total, nine studies reported on the governance of a registry [17, 25, 28,29,30,31,32,33, 35]. Two studies noted a registry committee consisting of both patients and experts to ensure governance aligned with the purpose of the registry [28, 29]. Nembhard et al. noted that clinicians entering data felt more comfortable entering data once ownership of the registry was relinquished to government [17]. One study mentioned joint ownership of the data between two or more centers [23]. Four studies mentioned that data was owned and stored at one medical center [28, 30, 33, 35]. One study mentioned that the entire dataset was owned by the principal researcher [32], whilst another noted that data was owned by a private company funded by a patient foundation [29].

Legal

Seven studies noted the importance of a solid legal basis for the collection, storage, and use of data [17, 28,29,30, 32, 35, 36]. Three studies mentioned the registry gaining approval for implementation after consulting national authorities pertaining to medical legislation [17, 26, 28, 30, 31]. Furthermore, four studies noted that consent forms were required prior to inclusion of patients into the registry [28,29,30, 35]. The study by Hassan et al. mentions that implementation of the registry was performed in accordance with local jurisdiction, whilst not attaining to the medical and health research act of 2008 [32]. Lazem et al. ensured all participating centers signed data sharing and cooperation agreements as well as the national pediatric nephrology society [25].

Security

When storing patient data on an electronic database, multilevel security was advised when identifiable patient data were included [37]. These security measures consisted of passwords (n = 2) [17, 35], limiting access to the database to members of the registry team and only allowing de-identified patient data to be used by researchers (n = 7) [17, 22, 28, 30, 32, 34, 35]. Furthermore, in one instance data logs were used to monitor the access and behavior of users [30]. In case of a paper-based framework, secure storage, limited access, and legal identification prior to data access were most frequently used (n = 3) [21, 32, 38]. Finally, one study advised contracts regarding privacy legislation to be signed by affiliated centers prior to participation into the registry [30]. Four studies mentioned the use of servers for data storage [22, 28,29,30]. Most of these studies used MySql for as the management system for the database [22, 29, 30]. Hassan et al. describes the use of the DHIS2 software package [32].

Registry team

Labor needed for the successful running of a registry range between 0.2 full time equivalents (FTE) to 5.0 FTE [17, 22, 32, 34]. Members of registry teams reported are medical experts; principal investigators and co-researchers; and patient representatives [22, 28,29,30]. Responsibilities of a registry team entailed mainly data quality controls [17, 32, 34] Other responsibilities were: to improve the overall quality by updating patient case information on a regular basis, and to ensure continued completeness of data [17]. Auditing of researchers prior to relinquishing data was mentioned to be valuable to ensure data was used ethically and securely [17, 28, 33]. Furthermore, centralized registry teams produced periodic benchmark reports and structured feedback to and from participating centers. [22, 23, 30, 33]. Finally, documenting procedures and policies by the current registry team helps potential successors to work effectively after retirement of previous registry team members [17].

Design

Most of the studies described registries that were based on an electronic database (n = 12) [22, 23, 25,26,27, 29,30,31, 33,34,35,36] or a combination of paper-based entries and direct entries into an electronic database (n = 8) [17, 19, 20, 24, 28, 32, 38, 39]. However, two studies report the use of exclusively paper-based data entries. When classifying countries using the World Bank classification [40]. After 2005,

electronic registries used to be more frequent in high-income countries than in middle income countries and low-income countries. The popularity of electronic databases in low-income countries has increased in the past decade. However, not without issues, as Mallon et al. describe frequent power outages and poor Wi-Fi connections even after distributing laptops to participating centers [26]. Furthermore, Hassan et al. stated the implementation of an electronic database was difficult in LMIC due to the lack of existing electronic frameworks [32]. Having the option of both paper-based and electronic-based entries helped include more centers in participating with the collection of registry data [15,16,17, 19, 20, 24, 28, 32, 33, 38, 39]. Electronic databases were therefore an appropriate choice in countries with a solid electronic healthcare framework and had many benefits, such as direct data entries and high data quality [18, 22, 23, 27, 29, 30, 33,34,35,36]. Paper-based data entries hindered participation and had lower efficiency compared to electronic-based data entries [17]. Furthermore, initial design and conceptualization of the registry needed to be profound since later changes to the system could be simple in concept but proved difficult in execution [39]. However, changes in outcome measures after implementation of the registry should be expected in pediatric registries [33]. Shahian et al. even described regular updates of common data elements to evaluate current variables and adding new variables [33]. Finally, three studies stated that registry design flaws caused innate bias and incomplete definition of complex malformations by using low-quality data sources [16, 18, 19].

Quality checks

Eighteen registries discussed the implementation of data quality checks. Our thematic analysis identified four categories: manual data checks prior to the data inclusion; periodic manual data reviews; periodic auditing visitations; automatic data quality checks. Manual data checks performed [16,17,18, 20,21,22,23, 25,26,27,28,29,30,31,32,33,34,35]by the registry staff upon data-entry into the registry were the most common (n = 10) [16, 20,21,22, 25, 28, 31,32,33,34]. Periodic auditing visitations were described in three studies [21, 28, 32]. Alternatives mentioned were automated data quality checks, consisting of plausible ranges, mandatory answer options and duplicate removal (n = 4) [22, 28, 33, 35]. Periodic manual data reviews were also used in two examples [23, 26, 32]. Furthermore, in one registry, after initial data entry, a summary was presented to the participant to confirm the correctness of the data [30]. Regarding internal validity of data in the registry, erroneous or inaccurate data entries should be anticipated and prevented [15,16,17,18, 21, 34]. On top of that, several studies advised training of the medical staff at the affiliated centers [21, 26, 28, 32].

Answer options

Concerning answer options, multiple designs were suggested. The use of some form of standardized answer options (SAO) was mentioned in eighteen studies[15,16,17,18,19, 21,22,23, 25, 26, 28,29,30, 33,34,35, 38, 39]. Several variations were described, of which tick-box answers, and international classification of Disease (ICD) codes [41, 42] were commonly used [16, 18, 21, 32, 38].

Three studies reported the use of SAO exclusively, while eight studies implemented both questions with free text answer options as well as SAO. Facilitating both options allowed for more profound and descriptive information to be captured [15, 17,18,19, 22, 25, 26, 28,29,30, 33, 34, 39]. In registries concerning more complex multi-malformations or diseases with phenotypical variety, this strategy was reported to be especially useful [35]. As a supplement to written descriptions, the inclusion of drawings, photographs or x-photographs were of added value [18]. Changing the framing of questions substantially changed the outcome of the questions and should thus be done cautiously according to Minassian et al. [27].

Data-entries

Various data-entry options were discussed in the included studies. Most registries (n = 9 reported that data was entered exclusively by clinicians or medical professionals [18, 20, 21, 23, 25, 26, 28, 30,31,32, 34, 39]. Two studies reported the data entries were performed by a dedicated registry team [17, 28]. Seidlin et al. reported that data was entered solely by patients [29]. In the latter case, continues monitoring of data entries was advised, since disease specific surveys can contain questions prone to misunderstanding. To increase correct data entries, patients were provided with a glossary and photographs of the most major manifestations of the disease [29]. Moreover, Hassan et al. concluded that the amount of data that should be entered has to be considered as well: extensive lists of variables that were not part of routine documentation prior to the registry were viewed as tedious by medical professionals. This caused missing data entries and even low-quality data entries in the registry [32].

Linkage

In total, eleven studies mentioned some form of linkage. Most of these registries allowed for linkage to more than one database or population-based registry, to increase data completeness (N = 8) [17, 19, 20, 24, 27, 33, 36, 38]. Designing the registry with personal identifiers was advised in three studies [19, 20, 34], since this allows for linkage to other registries or population-based databases. Linkage of databases was described as a powerful tool to improve data completeness [15, 17, 19, 20, 24, 27, 33, 36, 38].

Completeness

Completeness of data was limited by missing data; incomplete data; failure to complete data after linkage or failure to follow-up during data quality checks (n = 6) [16, 21, 22, 24, 27, 28]. Six registries contacted and implored participating centers to correct dubious data entries and false entries, or complete missing entries [20, 22, 28, 32, 33, 35]. Furthermore, to promote participating centers to enter high quality and complete data, two studies used immediate feedback forms after data entries to allow for instantaneous correction by the participants [30, 33]. Completion was improved via routine linkage to other databases or registries in three studies, [17, 18, 27] or via active data ascertainment by a dedicated research team [17]. In case of repeated incomplete or dubious data entries, one registry used a regional coordinator to provide support to the participating center [35]. In case of patient entered data, frequent reminders to complete or update personal data increases data entries were sent to patients [29]. Finally, Mallon et al. report that regular site visits to the participating centers helped increase data completeness [26].

Participation

Findings regarding participation were noted in sixteen studies. The thematic analysis showed that participation was influenced by different factors. First, the affiliated centers must feel that the active participation positively affects health outcomes, without causing a disproportionate increase in workload [32]. To ensure involvement of affiliated centers, two studies organized periodic meetings to inform on the functioning and future perspectives of the registry at the center’s site [20, 39] Nembhard et al. suggested active feedback ascertainment [17]. This appeared to improve participation of medical professionals and patients, as well as data quality. Financial reimbursements for completing data entries were mentioned to improve overall motivation of clinicians to participate [34]. Mallon et al. reported financially reimbursing participating centers after more than fifty new case entries to compensate for time [26]. Lastly, the study by Seidlin found a correlation between patient entered data entries and a yearly reminder e-mail to complete data entries [29].

Benchmarking

The use of periodic benchmarking reports was mentioned in five studies [22, 23, 30, 32, 33]. The advantages reported were the allowance for continued comparisons between participating centers, for caveats in performance to be addressed and to improve resource utilization [22, 23, 30, 33, 39]. Furthermore, benchmarking or performance reports allowed participating centers to routinely review data in their respective database [22, 25]. In one study, participating centers could voluntarily choose to share personal benchmarking reports with patients or other participating centers. This transparency allowed patients to make better decisions regarding healthcare providers [33].

Discussion

In this systematic review, through narrative analysis, the key themes for developing a successful multidisciplinary registry for pediatric patients have been identified. While the goal of our research was to provide information for the design of the ERN-CRANIO initiative of a new European registry for patients with a cleft. However, the information that is provided in this review can be applied to a wider pediatric patient group and provides support in the development of other registries.

Summary of key themes

This systematic review successfully identified the key themes necessary for the long-term success of an international, multidisciplinary registry for pediatric patients. All key themes are important challenges on their own. In order to provide a hands-on, stepwise approach for the development of a registry, the key themes have been subdivided in three categories: “fundamentals”, “considerations”, and “amplifiers”. This is visualized in Fig. 2. The key themes included in the “fundamentals” category needs to be fully developed before the conceptualization faze of a registry. If one or more of these themes is not completely formulated, registry success will be unlikely. The themes categorized in the “considerations” subgroup are concepts that may differ depending on the purpose of the registry. Therefore, these themes require careful consideration prior to implementation and during the running of the registry. Finally, the themes discussed in “amplifiers” are important to increase the quality of the registry. The better the key themes categorized in the “amplifiers” subgroup are implemented, the higher the data quality and completeness will be. The themes per category will be discussed in more detail in the following paragraph in the form of a written guidance, a visual guidance useful during conceptualization of an international pediatric registry is available in the online appendix 2 of Additional file 1.

Fig. 2
figure 2

Key-themes required for a successful international multidisciplinary registry for pediatric patients

Full size image

Guidance of key themes

Fundamentals

The purpose of a registry needs be clear for all participating parties and predetermined prior to the implementation of a registry [43]. This purpose can be diverse, such as health monitoring, quality control or scientific research. Currently, registries are getting used more in research for cost-effective data collection. Furthermore, registries are becoming more popular when flexible research design is needed over randomized controlled trials [43,44,45,46,47]. To warrant durability and functionality of a registry, unbiased funding is required [43]. Funding can be procured via private-sector funding, charities, and non-profit organizations such as the World health organization (WHO) as well as via the government [47]. However, a combination of different sources is not uncommon and expected to become more frequent in the future [43]. Long term funding is especially important for the long-term success of a registry. The design, size and purpose of the registry all influence the amount of funding needed to succeed [43]. Therefore, it is essential to include funding in the entire process of creation and implementation of a registry. To ensure the success of a registry governance is needed. Governance can provide guidance in the form of committees consisting of medical specialists, researchers, and patient representatives. These committees can provide high-level decision making, relating to purpose, funding, and data relinquishing [43]. Furthermore, the WHO suggests publicly stating ownership and governance structure with regards to a registry. (Source) When a registry is operating on an international scale, local data ownership is preferred, due to the legal implications associated with patient data. Privacy laws are strict and can be nation or center specific, thus international ownership of data can prove very difficult practice [48]. Data custody without data ownership can therefore be an appropriate solution to this problem [49]. Furthermore, it is important to consider which data are to be entered in the registry. Variables exceeding standard practice, at least in Europe are subject to more extensive and strict requirements [50]. To guarantee security of patient data, data access should be limited via multilevel security. Limiting full data access to registry to a data committee ensures patient privacy is preserved. Passwords and user logs provide additional protection. Further data protection can be realized by creating data logs and user behavior screening. Numerous free to use software for database provide such services [51,52,53]. Furthermore, for local storage, secure servers hosted by the medical centers themselves are recommended. However, in the future, this could change via the General Data Protection Regulation of the European Union. The GDPR ensures that data protection throughout Europe is compliant to the highest standards [54]. Possibly, this could facilitate centralized data ownership in Europe in the future. Another key factor for long-term success of a registry is a registry team. Such teams can be used for manual quality control of entered data. A registry team can create periodic benchmarking reports and consistently perform visitations to participating centers and audit researchers and perform active data ascertainment. Like other registry committees, registry teams can consist of different members, such as medical experts, researchers, and patient representatives. The required full-time equivalents (FTE) required by the registry team to run a successful registry is highly depended on the purpose and size of the registry [43].

Considerations

The design of a registry is dependent on the purpose for which the registry is created. When designing a registry, it is important to consider what operating framework is best suited for the purpose, the patient population, and the area in which the registry operates This systematic review shows that an electronic based registry is preferred over a paper-based registry provided local ICT facilities allow to do so. An electronic registry allows for higher data quality, higher participation, and immediate data entries. Furthermore, in an international setting, electronic databases allow for instantaneous global functionality. However, it is important to realize that within Europe, internet access and digital skills are not equally distributed throughout the continent [55]. Additionally, even in countries with an electronic infrastructure, health centers can still rely on paper-based records [56]. When multidisciplinary outcome measures are to be captured, the framework of a registry should be accommodating to change, since different specialties might change or expand on the outcome measures. This especially holds true for pediatric registries, since it requires long term follow-up of patients. For instance, tools used to assess patients to change or improve over the years, thereby changing the collected outcome measures over time. However, changing outcome measures should ideally be kept to a minimum since this can be easy in concept but prove difficult in execution. For instance, new data additions might require changes in the framework, training personnel on sight, adjustments to registry protocols and amendments at the medical ethics board. This also results in more financial recourses being used [43]. To ensure the data included in the registry are of high quality and suited for the predetermined purpose of the registry, quality checks need to be implemented. Automated and manual data checks prior to inclusion of data into the registry, periodic manual data checks, periodic auditing visitations and automated data checks are all advised to guarantee continued high data quality. The training of staff at the participating centers is also a viable way to ensure data quality is high. Finally, it is important to consider what answer options are best suited for the registry. By using free-text answer options, SAO, and additional data entries such as photographs, a more profound description of malformations can be achieved. To ensure participants remain motivated to put in the effort of completing data entries over longer time periods, the effort to complete data entries should be reasonable and in line with de purpose of the registry. Therefore, it is important to distinguish between a core data set, ideally consisting of the minimum requirements for achieving the purpose, and additional more extensive data entries useful for further research initiatives. Thus, the purpose of the registry equally dictates the number of variables included in the registry. The WHO suggests appointing a “responsible registrant” to perform and check data-entries to ensure high-quality data is continuously uploaded in the registry [47]. Furthermore, the complexity of clinical outcome measures being captured influences the quality checks required to examine the data entries [57].

Amplifiers

The use of personal identifiers is recommended within a local server system to ensure the possibility of linkage to other local or national databases and registries. This can help increase the overall completeness of the collected data in the registry. High completeness of data is mentioned as an indicator of registry success [58,59,60]. Using entries by both clinicians and patients improves the overall understanding of malformations. However, in case of patient entered data, careful monitoring is necessary to prevent erroneous data to be entered into the registry. This requires a dedicated team [43]. Furthermore, extensive lists of variables are viewed as tedious and are at risk of not being entered correctly. This not only influences data quality since this negatively influences participation of medical centers. However, by organizing periodic meetings to inform on projects and research efforts arising from the data entered in the registry, participation can be increased. Furthermore, participation was reported to increase when active feedback ascertainment was organized by the registry team. Participation can also be increases by allowing participating centers to co-author research initiatives [60, 61]. Lastly, compensating participating centers after completing a pre-determined amount of complete data entries can improve participation [26]. Another frequently noted way of improving completeness and participation was via benchmarking reports. Benchmarking reports are a useful tool to allow for inter-center comparison in performance and help improve participation and health outcomes for patients. Lastly, if possible, financial reimbursements for participating centers after complete and correct data entries are proven to be effective in increasing participation.

Relevance to ERN CRANIO and other research

The ERN CRANIO initiative has recognized the need for a successful, long-term registry to improve cleft care on an international level. A successful registry can facilitate international comparison between centers, improve international research initiatives, provide data for international benchmarking and the improvement of overall healthcare quality [58,59,60]. Additionally, registries are noted to be very suitable for surveillance of rare diseases and for performing studies relating to conditions with complex treatment patterns [57]. An example of such an epidemiological registry in Europe is EUROCAT [62]. This initiative surveys, collects and analyses data on congenital anomalies across populations in Europe since 1979, proving the need and use for such European-wide registries. The ERN CRANIO CL/P registry will also collect epidemiological data; however, it will mainly be focused on collecting comparable clinical and patient reported outcome measures to assess treatment outcomes between centers. Lastly, prospective registries have been noted as cost-effective alternatives for clinical trials [60, 61]. This fact has also been noted by the World Health Organization (WHO), which is currently supporting and launching seventeen distinct trial registries [47]. This systematic review identified the key themes that influence the overall success and longevity of an international multidisciplinary registry for pediatric patients. This information is not only useful in relation to the aims of the ERN CRANIO initiative but can be used by anyone seeking guidance in the development of a registry.

Strengths and limitations

Due to the nature of narrative analysis, the types of included studies varied. The types of publications included: registry reviews, descriptive articles, and data analysis reports. Additionally, publications spanned over forty years. Furthermore, the development of the thematic data extraction set is possibly affected by researcher bias. However, by including a second researcher for the cross-referencing of the publications and the thematic data extraction set it was attempted to mitigate this bias. By including non-international registries into the systematic review, the specificity of the experience of developing such registries may have been diluted. However, by including non-international registries, more overall experience including strengths and pitfalls for successful registries could be analyzed. The inclusion criterion of registries needing to be developed for pediatric patients was not something that was mentioned in the reviewed literature as being distinctly different from developing any other registries. However, we postulate that pediatric registries tend have more assessments over a longer period of time that in adult registries and could therefore be, generally, more complex than adult registries. A strength of this review is that by systematic narrative analysis a scientifically sound basis for the development of a pediatric multidisciplinary registry is provided.

Conclusion

This systematic review provides a scientific basis that aids the ERN initiative in creating a European-wide registry for collecting outcome measures relating to the multidisciplinary treatment of patients with cleft palate. Main pitfalls negating and key themes relating to long-term registry success have been identified via narrative analysis. This review can be useful to any initiative seeking to develop a registry. Furthermore, this review provides methodological tools that can provide help in the development of a registry framework on a wider international and pediatric basis. A future descriptive study reporting on the practical experiences during the development and implementation of the registry could complement this review and provide profound insights for other registry initiatives.

Availability of data and materials

All data generated or analyzed during this study are included in this article and its additional information files.

Abbreviations

ERN-CRANIO:

European Reference Network for craniofacial anomalies and ear, nose and throat disorders

CL/P:

Cleft lip and palate

References

  1. ERN CRANIO [Internet]. [cited 2022 Oct 14]. Available from: https://ern-cranio.eu/.

  2. Imbery TE, Sobin LB, Commesso E, Koester L, Tatum SA, Huang D, et al. Long-term otologic and audiometric outcomes in patients with cleft palate. Otolaryngol Head Neck Surg. 2017;157(4):676–82.

    PubMed  Google Scholar 

  3. Bickham RS, Ranganathan K, Wombacher NR, Shapiro DN, Carlozzi NE, Baker MK, et al. Speech perceptions and health-related quality of life among children with cleft lip and palate. J Craniofac Surg. 2017;28(5):1264–8.

    PubMed  PubMed Central  Google Scholar 

  4. Antoun JS, Fowler PV, Jack HC, Farella M. Oral health–related quality of life changes in standard, cleft, and surgery patients after orthodontic treatment. Am J Orthod Dentofacial Orthop. 2015;148(4):568–75.

    PubMed  Google Scholar 

  5. Feragen KB, Semb G, Heliövaara A, Lohmander A, Johannessen EC, Boysen BM, Havstam C, Lundeborg I, Nyberg J, Pedersen NH, Bogh-Nielsen J. Scandcleft randomised trials of primary surgery for unilateral cleft lip and palate: 10. Parental perceptions of appearance and treatment outcomes in their 5-year-old child. J Plast Surg Hand Surg. 2017;51(1):81–7.

    PubMed  Google Scholar 

  6. Hunt O, Burden D, Hepper P, Stevenson M, Johnston C. Self-reports of psychosocial functioning among children and young adults with cleft lip and palate. Cleft Palate Craniofac J. 2006;43(5):598–605.

    PubMed  Google Scholar 

  7. Shaw WC, Semb G, Nelson P, Brattström V, Mølsted K, Prahl-Andersen B, et al. The eurocleft project 1996–2000: overview. J Craniomaxillofac Surg. 2001;29(3):131–40.

    CAS  PubMed  Google Scholar 

  8. Long RE Jr, Hathaway R, Daskalogiannakis J, Mercado A, Russell K, Cohen M, et al. The Americleft study: an inter-center study of treatment outcomes for patients with unilateral cleft lip and palate part 1. Principles and study design. Cleft Palate Craniofac J. 2011;48(3):239–43.

    PubMed  Google Scholar 

  9. Allori AC, Kelley T, Meara JG, Albert A, Bonanthaya K, Chapman K, et al. A standard set of outcome measures for the comprehensive appraisal of cleft care. Cleft Palate Craniofac J. 2017;54(5):540–54.

    PubMed  Google Scholar 

  10. Semb G, Enemark H, Friede H, Paulin G, Lilja J, Rautio J, et al. A Scandcleft randomised trials of primary surgery for unilateral cleft lip and palate: 1. Planning and management. J Plast Surg Hand Surg. 2017;51(1):2–13.

    PubMed  Google Scholar 

  11. Nederlandse vereniging voor plastische chirurgie. Schisis Richtlijn - Richtlijnendatabase [Internet]. [cited 2022 Mar 16]. Available from: https://richtlijnendatabase.nl/richtlijn/behandeling_van_patienten_met_een_schisis/startpagina_schisis.html.

  12. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700.

    PubMed  PubMed Central  Google Scholar 

  13. Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using endnote. J Med Libr Assoc. 2017;105(1):84–7.

    PubMed  PubMed Central  Google Scholar 

  14. Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, Britten N, Roen K, Duffy S. Guidance on the conduct of narrative synthesis in systematic reviews. Product ESRC Methods Program Ver. 2006;1(1):b92.

    Google Scholar 

  15. Knox EG, Armstrong EH, Lancashire R. The quality of notification of congenital malformations. J Epidemiol Commun Health. 1984;38(4):296–305.

    CAS  Google Scholar 

  16. Marazita ML, Bodurtha JN, Corey L, Rogers A, Barbosa CE, Funkhouser L, et al. Development of the virginia congenital anomalies reporting and education system (VaCARES): two pilot projects. South Med J. 1992;85(6):608–15.

    CAS  PubMed  Google Scholar 

  17. Nembhard WN, Bower C. Evaluation of the Western Australian register of developmental anomalies: thirty-five years of surveillance. Birth Defects Res Part A Clin Mol Teratol. 2016;106(11):894–904.

    CAS  Google Scholar 

  18. Ericson A, Kallen B, Winberg J. Surveillance of malformations at birth: a comparison of two record systems run in parallel. Int J Epidemiol. 1977;6(1):35–41.

    CAS  PubMed  Google Scholar 

  19. Gissler M, Järvelin MR, Hemminki E. Comparison between research data and routinely collected register data for studying childhood health. Eur J Epidemiol. 2000;16(1):59–66.

    CAS  PubMed  Google Scholar 

  20. Gissler M, Teperi J, Hemminki E, Merilainen J. Data quality after restructuring a national medical registry. SCAND J Soc Med. 1995;23(1):75–80.

    CAS  PubMed  Google Scholar 

  21. Anda EE, Nieber E, Voitov AV, Kovalenko AA, Lapina YM, Voitova EA, et al. Implementation, quality control and selected pregnancy outcomes of the Murmansk county birth registry in Russia. Int J Circumpolar Health. 2008;67(4):318–34.

    PubMed  Google Scholar 

  22. Deakyne Davies SJ, Grundmeier RW, Campos DA, Hayes KL, Bell J, Alessandrini EA, et al. The pediatric emergency care applied research network registry: a multicenter electronic health record registry of pediatric emergency care. Appl Clin Inform. 2018;9(2):366–76.

    PubMed  PubMed Central  Google Scholar 

  23. Hammill SC, Kremers MS, Stevenson LW, Kadish AH, Heidenreich PA, Lindsay BD, et al. Review of the registry’s second year, data collected, and plans to add lead and pediatric ICD procedures. Heart Rhythm. 2008;5(9):1359–63.

    PubMed  Google Scholar 

  24. Blenstrup LT, Knudsen LB. Danish registers on aspects of reproduction. Scand J Public Health. 2011;39(7 Suppl):79–82.

    PubMed  Google Scholar 

  25. Lazem M, Sheikhtaheri A, Hooman N. Lessons learned from hemolytic uremic syndrome registries: recommendations for implementation. Orphanet J Rare Dis. 2021;16(1):1–3.

    Google Scholar 

  26. Mallon B, Kaboré R, Couitchere L, Akonde FB, Lukamba R, Dackono TA, Narison ML, Khattab M, Pondy A. The development of an international childhood cancer hospital register database in 13 African countries a project of the French African pediatric oncology group (GFAOP). Pediatric Blood Cancer. 2022;69(3):e29464.

    PubMed  Google Scholar 

  27. Minassian C, Williams R, Meeraus WH, Smeeth L, Campbell OMR, Thomas SL. Methods to generate and validate a pregnancy register in the UK clinical practice research Datalink primary care database. Pharmacoepidemiol Drug Saf. 2019;28(7):923–33.

    PubMed  PubMed Central  Google Scholar 

  28. de Antonio M, Dogan C, Daidj F, Eymard B, Puymirat J, Mathieu J, et al. The DM-scope registry: a rare disease innovative framework bridging the gap between research and medical care. Orphanet J Rare Dis. 2019;14(1):1–4.

    Google Scholar 

  29. Seidlin M, Holzman R, Knight P, Korf B, Rangel Miller V, Viskochil D, et al. Characterization and utilization of an international neurofibromatosis web-based, patient-entered registry: an observational study. PLoS One [Internet]. 2017;12(6):e0178639.

    PubMed  Google Scholar 

  30. Prince FHM, Ferket IS, Kamphuis S, Armbrust W, tenCate R, Hoppenreijs EPAH, et al. Development of a web-based register for the Dutch national study on biologicals in JIA. Rheumatology. 2008;47(9):1413–6.

    CAS  PubMed  Google Scholar 

  31. Gauvrit F, Risoud M, Aubry K, Bordure P, Bozorg-Grayeli A, Deguine O, et al. The French cochlear implant registry (EPIIC): general indicators. Eur Ann Otorhinolaryngol Head Neck Dis. 2020;1(137):S5-9.

    Google Scholar 

  32. Hassan S, Vikanes A, Laine K, Zimmo K, Zimmo M, Bjertness E, et al. Building a research registry for studying birth complications and outcomes in six Palestinian governmental hospitals. BMC Pregnancy Childbirth. 2017;17(1):1.

    Google Scholar 

  33. Shahian DM, Jacobs JP, Edwards FH, Brennan JM, Dokholyan RS, Prager RL, et al. The society of thoracic surgeons national database. Heart. 2013;99(20):1494–501.

    PubMed  Google Scholar 

  34. Anda EE, Nedberg IH, Rylander C, Gamkrelidze A, Turdziladze A, Skjeldestad FE, et al. Implementing a birth registry in a developing country - experiences from Georgia. 2017; Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=medp&AN=29357658.

  35. Ebner K, Feldkoetter M, Ariceta G, Bergmann C, Buettner R, Doyon A, et al. Rationale, design and objectives of ARegPKD, a European ARPKD registry study. BMC Nephrol. 2015;16:22.

    PubMed  PubMed Central  Google Scholar 

  36. Kamper-Jørgensen M, Wohlfahrt J, Simonsen J, Benn CS. The childcare database: a valuable register linkage. Scand J Public Health. 2007;35(3):323–9.

    PubMed  Google Scholar 

  37. Dwyer PA, Jelatis GD, Thuraisingham BM. Multi-level security in database management systems. Comput Secur. 1987;6(3):245–51.

    Google Scholar 

  38. Stiller CA, Allen MB, Eatock EM. Childhood cancer in Britain: the national registry of childhood tumours and incidence rates 1978–1987. Eur J Cancer Part A: General Topics. 1995;31(12):2028–34.

    Google Scholar 

  39. Druschel C, Sharpe-Stimac M, Cross P. Process of and problems in changing a birth defects registry reporting system. Teratology. 2001;64(S1):S30–6. https://doi.org/10.1002/tera.1082.

    Article  PubMed  Google Scholar 

  40. World Bank Country and lending groups – World Bank data help desk [Internet]. [cited 2022 Apr 7]. Available from: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups.

  41. Drösler SE, Weber S, Chute CG. ICD-11 extension codes support detailed clinical abstraction and comprehensive classification. BMC Med Inform Decis Mak. 2021;1:21.

    Google Scholar 

  42. Harrison JE, Weber S, Jakob R, Chute CG. ICD-11: an international classification of diseases for the twenty-first century. BMC Med Inform Decis Mak. 2021;1:21.

    Google Scholar 

  43. Gliklich RE DNLM. Rockville (MD): Agency for healthcare research and quality (US). 2014 [cited 2022 Jun 10]. Planning a registry - registries for evaluating patient outcomes - NCBI bookshelf. Available from: https://www.ncbi.nlm.nih.gov/books/NBK208631/.

  44. Lash TL, Fox MP, Fink AK. Applying quantitative bias analysis to epidemiologic. Am J Epidemiol. 2009;170(10):1316–7.

    Google Scholar 

  45. Hernán MA, Hernández-Diaz S, Werler MM, Mitchell AA. Causal knowledge as a prerequisite for confounding evaluation: an application to birth defects epidemiology. Am J Epidemiol. 2002;155(2):176–84.

    PubMed  Google Scholar 

  46. Alter DA, Venkatesh V, Chong A. Evaluating the performance of the global registry of acute coronary events risk-adjustment index across socioeconomic strata among patients discharged from the hospital after acute myocardial infarction. Am Heart J. 2006;151(2):323–31.

    PubMed  Google Scholar 

  47. World Health Organization. Version 2.1. 2009 [cited 2023 Jan 16]. WHO registry criteria. Available from: https://www.who.int/clinical-trials-registry-platform/network/registry-criteria.

  48. European data protection supervisor. A preliminary opinion on data protection and scientific research. 2020 Jan 6 [cited 2022 Sep 29]; Available from: https://edps.europa.eu/sites/edp/files/publication/20-01-06_opinion_research_en.pdf.

  49. Principles of registry ethics, data ownership, and privacy - registries for evaluating patient outcomes - NCBI Bookshelf [Internet]. [cited 2022 Jun 17]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK208620/.

  50. Mildebrath H. BRIEFING EU legislation in progress proposal for a regulation of the European Parliament and of the council on European data governance (data governance act). [cited 2022 Sep 29]; Available from: https://www.europarl.europa.eu/RegData/etudes/BRIE/2021/690674/EPRS_BRI(2021)690674_EN.pdf.

  51. Molgenis [Internet]. [cited 2022 Jun 17]. Available from: https://www.molgenis.org/.

  52. DHIS2. Home - DHIS2 [Internet]. [cited 2022 Jun 17]. Available from: https://dhis2.org/.

  53. MySQL. MySQL [Internet]. [cited 2022 Jun 17]. Available from: https://www.mysql.com/.

  54. Health | European data protection supervisor [Internet]. [cited 2023 Jul 7]. Available from: https://edps.europa.eu/data-protection/our-work/subjects/health_en.

  55. van Kessel R, Wong BL, Rubinić I, O’Nuallain E, Czabanowska K. Is Europe prepared to go digital? Making the case for developing digital capacity: an exploratory analysis of Eurostat survey data. PLOS Dig Health. 2022;1(2):e0000013. https://doi.org/10.1371/journal.pdig.0000013.

    Article  Google Scholar 

  56. Ćwiklicki M, Schiavone F, Klich J, Pilch K. Antecedents of use of e-health services in Central Eastern Europe: a qualitative comparative analysis. BMC Health Serv Res. 2020;20(1):1–15.

    Google Scholar 

  57. Gliklich RE DNLM. 3rd edition. Rockville (MD): Agency for healthcare research and quality (US). 2014 [cited 2022 Jun 16]. Registry design - registries for evaluating patient outcomes - NCBI Bookshelf. Available from: https://www.ncbi.nlm.nih.gov/books/NBK208632/#ch3.s2.

  58. Breakwell LM, Cole AA, Birch N, Heywood C. Should we all go to the PROM? The first two years of the British spine registry. Bone Joint J. 2015;97-B(7):871–4.

    CAS  PubMed  Google Scholar 

  59. Hickey GL, Grant SW, Cosgriff R, Dimarakis I, Pagano D, Kappetein AP, et al. Clinical registries: governance, management, analysis and applications. Eur J Cardiothorac Surg. 2013;44(4):605–14.

    PubMed  Google Scholar 

  60. Biancari F, Ruggieri VG, Perrotti A, Svenarud P, Dalén M, Onorati F, et al. European multicenter study on coronary artery bypass grafting (E-CABG registry): study protocol for a prospective clinical registry and proposal of classification of postoperative complications. J Cardiothorac Surg. 2015;10(1):1–12.

    Google Scholar 

  61. Elson DW, Dawson M, Wilson C, Risebury M, Wilson A. The UK knee osteotomy registry (UKKOR). Knee. 2015;22(1):1–3.

    PubMed  Google Scholar 

  62. Research | EU RD Platform [Internet]. [cited 2023 Aug 9]. Available from: https://eu-rd-platform.jrc.ec.europa.eu/eurocat/research_en.

Download references

Acknowledgements

The authors would like to thank dr. M.F.M. Engel and dr. W.M. Bramer from the Erasmus MC Medical Library for developing and updating the search strategies. This research is generated within the European Reference Network for rare and/or complex craniofacial anomalies and ear, nose and throat (ENT) disorders (ERN CRANIO). ERN CRANIO is funded by the European Union

Funding

Both authors SO and PG have received funding from the European Reference Network for the current study.

Author information

Authors and Affiliations

  1. Department of Plastic, Reconstructive and Hand Surgery, Erasmus Medical Center, Rotterdam, The Netherlands

    S. Ombashi, P. A. J. van der Goes & S. L. Versnel

  2. Service de Chirurgie Maxillofaciale et Chirurgie Plastique, Hôpital Necker - Enfants Malades, Assistance Publique - Hôpitaux de Paris; Faculté de Médecine, Université Paris Cité, Paris, France

    R. H. Khonsari

  3. Department of Plastic, Reconstructive and Hand Surgery, University Medical Center, Utrecht, The Netherlands

    A. E. Mink van der Molen

  4. Scientific Committee, ERN CRANIO, Rotterdam, The Netherlands

    R. H. Khonsari & A. E. Mink van der Molen

Authors
  1. S. Ombashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. A. J. van der Goes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. L. Versnel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. H. Khonsari
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. A. E. Mink van der Molen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SO, SV and AM conceived the presented idea. SO, and PG performed the data analysis and PG created the visual guidance. SV, RK and AM provided guidance in the design and supervised the findings of this review. All authors discussed the results and contributed to the drafting and revisions of the manuscript. All authors approved the submitted version of the manuscript.

Corresponding author

Correspondence to P. A. J. van der Goes.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

All authors are affiliated with the ERN CRANIO project. RK and AM are both board members of the scientific committee of ERN CRANIO.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Appendix.

Additional file 2.

Standardized basic data infromation supplemented with disease specific data and bio material.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ombashi, S., van der Goes, P.A.J., Versnel, S.L. et al. Guidance to develop a multidisciplinary, international, pediatric registry: a systematic review. Orphanet J Rare Dis 18, 296 (2023). https://doi.org/10.1186/s13023-023-02901-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13023-023-02901-4

Orphanet Journal of Rare Diseases

ISSN: 1750-1172