Quality of care in cystic fibrosis: assessment protocol of the French QIP PHARE-M

Abstract

Background

The PHARE-M care quality improvement program, modeled on the US Cystic Fibrosis Quality Improvement Program, was introduced at 14 cystic fibrosis centers (CFCs) in the French Cystic Fibrosis Network between 2011 and 2013. The pilot phase assessments attested the progressive adherence of the teams and improvements in care management. The PHARE-M Performance research project aims at assessing in 2015 the impact of the PHARE-M program on patient health indicators at trained versus untrained centers. It also sought to identify contextual factors that could account for variability in the performance of the PHARE-M among the trained centers.

Methods

A mixed methodology combining:

• a quantitative experimental study: a comparison, using a mixed model for repeated data (from 2011 to 2015), of the average changes over time in forced expiratory volume in 1 s (FEV1) and body mass index (BMI) between two groups of patients included in a closed cohort (non-transplant patients, continuous follow-up at one participating CFC, and a CF-causing mutation), one having benefitted from the PHARE-M program and the other not having done so, and

• a realistic study: a characterization of the impact on care management and an identification of mechanisms through which the PHARE-M intervention improved the team’s effectiveness in different CFC contexts; this required modeling the intervention, context, and impact on care management with respect to the criteria of the chronic care model (CCM); this was done using a self-administered questionnaire given to professionals and patients/parents supplemented with focus groups.

Conclusion

Although the study population was controlled, it may be difficult to establish a causal relationship between the differences in the changes over time in patient health indicators in the two groups of patients and the PHARE-M intervention as it is often the case in complex interventions rolled out in adaptive environments. The analysis of factors associated with variations in the impact of the PHARE-M at the different trained CFCs required the adoption of instruments validated in other contexts; these could be useful for assessing the performance of other interventions in healthcare practices at CFCs in France.

Background

Cystic fibrosis is the most common rare disease affecting the Caucasian population; it afflicts around 6500 individuals in France, 29,000 in the United States, and 11,000 in the United Kingdom. It is an autosomal recessive genetic disease caused by mutations in the cystic fibrosis transmembrane conductance regulator (CFTR) gene. Among all identified CFTR gene mutations, a list of mutations responsible for cystic fibrosis symptoms has been established and is regularly reviewed by the CFTR2 expert group [1]. Cystic fibrosis mainly affects the respiratory and digestive systems. The thick mucus in the bronchi brings about chronic inflammation and repeated infections, leading to chronic respiratory failure, the major cause of death. The majority of patients have pancreatic insufficiency and show poor nutrient absorption, resulting in an at-risk nutritional status associated with a poorer respiratory state [2]. Since the 1960s, the US Cystic Fibrosis Foundation (CFF) has identified multidisciplinary patient management at specialized centers as an essential factor in care improvement; this has led it to establish criteria for the accreditation of cystic fibrosis centers [3]. In the late 1990s, an increase in the number of adults suffering from cystic fibrosis led the CFF to clarify certain criteria for adult centers by stipulating care management by specialized physicians and a specialized team and a formalized process of transition from a pediatric center to an adult program. The accreditation process not only validates centers but also "fosters continuous improvement efforts within care centers," as "the expectation that each care center have a QI program in place was added to the accreditation and oversight process in 2004." In the 2000s, following the publication by the US Institute of Medicine, of the report on the Quality Chasm [4], the CFF launched a benchmarking study across the US CFCs, which showed a difference of several years in the median survival age between the ten centers having the best patient outcomes and the other centers (unpublished study). This led the CFF to develop and implement a Quality Improvement Program (QIP) in the form of Learning and Leadership collaboratives [5,6,7] with the academic support of The Dartmouth Institute Microsystem Academy (TDIMA). A supplement in BMJ Quality and Safety has been published in May 2014 to present the success of this QI initiative [8].

In 2002, following the generalization of newborn screening in France, the French Ministry of Health designated 49 cystic fibrosis centers (CFCs) [9] and in 2006, the French National Authority for Health (HAS) published the National Diagnosis and Treatment Protocol (PNDS) in Cystic Fibrosis to establish a framework for multidisciplinary care at CFCs. The French public health insurance guarantees that every CF patient is reimbursed 100% for care and authorized drugs related to cystic fibrosis. In 2006, within the framework of the 1st National Plan for Rare Diseases, two centers of expertise for cystic fibrosis were labelled (CF-CERDs), in order to implement six priorities across the CF Network: care expertise, information systems and epidemiology, quality of care, clinical research, network organization and coordination. The Nantes/Roscoff CF-CERD, consisting of the CFCs at the two hospitals in Nantes and Roscoff as well as the transplant center in Nantes and the rehabilitation center in Roscoff, developed its action plan contributing to 5 out of the 6 priorities, covering themes such as therapeutic patient education (care expertise), quality improvement in care processes, information and communication systems, and clinical research on transplantation and in human and social science. The agreement signed by the heads of all CFCs in 2007 included a commitment to “participate in a quality assessment and improvement program to be offered by the CF-CERDs in collaboration with the French Cystic Fibrosis Society (SFM) and the patient organizations in the next five years”.

In 2011, the French national team at the Nantes/Roscoff CF-CERD transposed the PHARE-M quality improvement program from the US CFF QIP model (PHARE-M: Programme Hospitalier d’Amélioration des Résultats et de l’Expertise en Mucoviscidose - Hospital Program to Improve Outcomes and Expertise in Cystic Fibrosis). It was launched in September 2011 with a pilot phase (2011–2012) involving seven volunteer CFCs, which underwent two external assessments, leading to certain adjustments to the initial program. This adjusted version was deployed during a regional expansion phase (2012–2013), including seven more CFCs before its national deployment [10]. The main adjustments consisted in more practical exercises during face-to-face meetings (less theoretical presentations), more on-site coaching to help the quality teams analyze their processes of care, and the designation of a PHARE-M referent in each local team to keep focused on the QI work. These 2 years are called the “experimental phase”, which involved 14 CFCs.

The two evaluations at the end of the one-year pilot phase showed the progressive adherence of the teams and improvements in care management, but a limited impact on patient health outcomes. They also highlighted that the adherence to the program mainly depended on the motivation of the multidisciplinary team (MDT), especially its lead physician. The lack of resources at some CFCs was raised to account for variations in the teams’ engagement as the level of available staff seemed to influence the extent to which the team was effectively enlisted. The participation of a patient or parent in each local quality team varied depending on the cultural context of the centers, some being used to share information with patients/parents, having a patient group in the CF center for years, others being involved in patient therapeutic education while others were acting in a more partenalistic model of care. The support received from the hospital quality department in two hospitals was emphasized as a factor that facilitated the adoption of quality tools by the teams. The recommendation of the assessor was to evaluate the impact of the program on patient outcomes by 2015.

Given the innovative nature of the QIP PHARE-M in France, the cultural differences and various organizational contexts at the CFCs, an assessment of the impact of PHARE-M at the CFCs engaged in the experimental phase was expected after 3 years to continue the enrollment in the program. Will it show favorable changes in the patient outcomes in the group of CFCs engaged in the PHARE-M compared to the other CFCs? What impact on care management can be observed in 2015? Was the period sufficient to show improvements in the two areas? In which contexts is the impact of PHARE-M observed to be the strongest? The PHARE-M Performance research project, submitted at a call for projects of the French Ministry of Health and selected for funding in December 2012, aims at providing answers to these questions.

Methods

A mixed methodology

The rationale of the PHARE-M Performance project is to show evidence of the performance of the PHARE-M program on patient outcomes and care management.

The study is based on a mixed methodology inspired on the one hand by epidemiology, using data from the French Cystic Fibrosis Registry, and on the other hand by the British guidelines on “Process evaluation of complex interventions” [11]:

1) a quantitative study to compare the changes over the 4 years in the patient health indicators of a closed cohort, using data from the French Cystic Fibrosis Patient Registry, between CFCs having benefitted from the intervention during the experimental phase and CFCs not having benefitted from the intervention up to 2015; and.

2) a qualitative study to analyze the contextual elements and mechanisms brought into play by the PHARE-M intervention that could account for a difference in impact among trained CFCs either on patient health indicators or on care management assessed according to the criteria of the chronic care model [12].

Quantitative study

Design

- observational,

- national and multi-center, and

- before/after and here/elsewhere: a comparison of patient health indicators before and after the “PHARE-M training” program at “PHARE-M Group” centers versus “Control Group” centers.

Primary and secondary endpoints

- FEV1%.

- BMI as an absolute value and as a Z-score (standardized normal distribution of the BMI for children under 2 years of age).

For this research in particular, the value selected for these indicators is the only value appearing in the French CF Registry for a given patient and a given year. It will be analyzed by category of patients defined by age, sex, age at diagnosis, and possibly severity of disease expression, treatment, and certain social characteristics (data appearing in the Registry).

Study population

A closed cohort was formed to identify the study population including the patients followed up at CFCs who met the following inclusion criteria according to the 2012 Registry data:

• patients seen at a CFC in 2012

• patients having two of the CF-causing mutations of the CFTR2 list published on Feb 2012

• patients not having received a transplant in 2012

A patient left the cohort if he or she no longer met the inclusion criteria after the annual data were updated in the Registry (2013, 2014, and 2015), i.e.: if he or she was a carrier of a mutation excluded from the CFTR2 list updated on 13/08/2015 [1]; if he or she was followed up at a CFC engaged in the PHARE-M in 2014 or 2015; if he or she changed CFC in the course of the study and in doing so, changed CFC group; if he or she received a transplant between 2013 and 2015 (data up to the transplantation were taken into account), or if the patient died between 2013 and 2015 (data up to the death were taken into account).

The cohort was divided into two groups: the “PHARE-M Group” and the “Control Group”:

• The “PHARE-M Group” consisted of the patients followed up at one of the 14 CFCs trained in the PHARE-M in the experimental phase (1309 patients).

• The “Control Group” consisted of the patients followed up at the CFCs not having benefitted from the intervention in the same period of time (2490 patients).

Pairing of the two “PHARE-M” and “control” groups

A preliminary analysis of the cohort formed from the 2012 Registry data showed significant differences between the two groups of patients, before the PHARE-M intervention, in terms of: 1) distribution by age, 2) distribution by age at diagnosis, and 3) distribution by FEV1% value (see Table 1).

Table 1 Distribution by age, age at diagnosis and FEV1% of the 2012 study population between the two groups of the study cohort before pairing

Consequently, a 1:1 pairing of the patients from the Control Group was decided in an attempt to eliminate certain confounding factors that could be attributed to the type and size of the CFC to which the patient was assigned: each “PHARE-M patient” was associated with a “control patient” followed up at a center of the same type (pediatric, adult, or mixed) caring for a total number of patients belonging to the same interval ([1;50], [51;100], [101;150], [151;200], or [> = 200]). Reunion island CFCs were excluded from the Control Group to reduce heterogeneity in CF care. All “eligible” control patients for each patient in the PHARE-M Group were selected, and one control patient was randomly drawn from that group of eligible control patients (without replacement). The patients in the PHARE-M Group were paired in a random order.

At the end of the process, 1104 patients remained in each of the two paired groups. The Control Group included 20 CFCs. No paired control patients were found for 205 “PHARE-M patients”. As data are collected in the French Cystic Fibrosis Registry for all patients, exposure variables are identical in both groups. Completeness is similar: for FEV1, 20.2% and 24.5% of missing data corresponding to the children below 6 y.o., for whom this measure is not taken, and 0.6% and 3.5% for ZBMI, in the PHARE-M group and the Control group respectively. The two groups had a similar distribution by age (see Fig. 1). However, there remained a significant difference in average age at diagnosis (PHARE-M paired group: 1.9 years; control paired group: 2.5 years; p value: 0.0123); this could be due to the fact that newborn screening was implemented in the 1990s in Brittany, and that seven (out of the 14) CFCs in the PHARE-M Group are located in this region. Furthermore, a significant difference in FEV1% of +3.89% (p value = 0.0012) remained in favor of the PHARE-M patient group before the intervention (see Tables 2 and 3).

Fig. 1

Distribution by population age between the two groups (PHARE-M and control), paired in 2012 data

Table 2 Comparison between the PHARE-M Group and the paired Control Group

Table 3 Comparison of Age at diagnosis between PHARE-M and Control

Analysis of the primary endpoint between the two groups

Changes over 5 years in patient health indicators are measured for 2011 (baseline), 2012, 2013, 2014, and 2015; each patient served as his or her own control. A difference in the rate of decline is expected between the two population groups, PHARE-M and control (see Fig. 2). Changes over time in FEV1% will be modeled and compared in the two groups using a mixed model for repeated data with adjustments for potential confounding variables. Measurements for a subject i at time j is given by the following model, where ε _ij are the normally distributed residual components with mean zero and covariance structure Σ:

Fig. 2

Representations of the analysis of the primary endpoint

${Y}_{ij}={\beta}_0^P+{\beta}_1^P{t}_{ij}+{\varepsilon}_{ij}$ for the PHARE-M group.

${Y}_{ij}={\beta}_0^C+{\beta}_1^C{t}_{ij}+{\varepsilon}_{ij}$ for the CONTROL group

$$ \mathit{\operatorname{cov}}\left({\varepsilon}_{ij},{\varepsilon}_{ik}\right)={\sigma}_{jk} $$

The covariance structure Σ is given by the σ _jk . It allows taking into account correlation between measurements on a same subject. Correlation is assumed to be null between subjects. The choice of a covariance structure will be data driven, but we can expect that the correlation between two measurements will only depend on the time lag between them. The most realistic covariance structure should be the so-called Toeplitz covariance matrix. A special case of the Toeplitz model is the first-order autoregressive model.

The question here is to investigate whether the two slopes are parallel or not, that is to test whether ${\beta}_1^P$ = ${\beta}_1^C$ (H₀) versus ${\beta}_1^P$ ≠ ${\beta}_1^C$ (H₁).

Using this model, the slopes (i.e. decline in FEV1) in the two groups will be calculated and compared. Changes over time in BMI will likewise be analyzed by comparing the changes in the two groups from 2011 to 2015, taking into account the Z-score for children under 2 years of age. The average trends will be calculated and analyzed for different patient categories (such as age, sex, age at diagnosis, severity of disease expression, treatment, and certain social characteristics in the Registry). The changes over time in indicators will be presented for the “PHARE-M Group” population by CFC for crossing with the results of the qualitative study.

Audit of the quality of the data included in the primary endpoints’ calculation

The patient data measured by the CFCs (height, weight, and FEV1 [per L]) for 2012 and 2013 underwent an on-site quality audit at the 14 CFCs in the PHARE-M Group. It was the first on-site audit ever performed to establish the quality of these indicators. The objective was not to comprehensively audit all data for the patients included in the study. Rather, the objective was to comprehensively identify the different causes of error due to failures in the processes of measuring and/or selecting the values transmitted to the Registry in order to identify avenues for improvement of the quality of the data in the Registry. The sample of patients whose data were audited thus had to reflect the distribution by age range of the patients at each CFC (20 records/CFC) in order to cover the different measurement procedures defined by international benchmarks [13,14,15] and the data selection rules defined by the French Patient Registry Steering Committee, and to offer every opportunity to reach saturation of the various causes of error [16]. They will be taken into account in the interpretation of the results of the quantitative study.

Qualitative study

Design

The design refers to the modeling of the intervention [11] including the contextual elements and the mechanisms shown in Fig. 3.

Fig. 3

Modeling of the intervention, context, and mechanisms

The PHARE-M intervention consisted of establishing, training and coaching a quality team (QT) at each CFC comprising a number of professionals from the multidisciplinary CF team and 1 parent or patient from the CFC’s caseload. The members of the QT have been trained in quality methods and tools and coached in changing care processes. The PHARE-M intervention should have directly impacted the ability of the local QT to master QI methods and tools, lead changes in the care processes, and should have generated good appreciation of the utility of the QT efforts. This direct impact of PHARE-M is identified under the heading “QT effectiveness”. QT effectiveness may not only be the result of the PHARE-M intervention but may have been modulated by internal mechanisms, such as the composition of the QT (number of members and disciplines enlisted), its functioning (rigor in the QI work, decision-making, clarity of the roles…) and the parent or patient engagement. Those mechanisms are represented as impacting QT effectiveness (Table 3). Beyond the ability to master the QI methods and tools, the PHARE-M intervention was expected to have an impact on the quality of CF care delivered at the CFC. The Chronic Care Model [12] was deemed appropriate to account for quality of CF care across the 6 dimensions: existing improvement goals, multidisciplinary care, self-management support, decision support (use of evidence-based guidelines), use of information system and electronic patient record, and organization of resources in the patient’s community of life. Finally, an indirect impact of the PHARE-M intervention is expected on the trend in patient outcomes’ evolution as measured in the quantitative part of this study. Moreover, some elements in the CFC contexts, which are external to the PHARE-M intervention and preexisted to its introduction, may have had a major impact both on the adherence of the team to the QI work and on its outputs. The contextual elements that have been brought in this study include the composition of the MDT, the leadership, the patient-centeredness of care, the innovative culture of the team, and the support from the hospital quality department.

The qualitative study will test these hypotheses using a questionnaire to be self-administered, in 2015, to all members of the MDT at the 14 CFCs and to the patients/parents participating in the quality teams.

Quality of care has been defined according to the criteria of the Chronic Care Model [12]; as this model has not been popularized in France nor in cystic fibrosis, we adapted it with 47 items aimed at characterizing CF care. Table 4 presents a list of these items.

Table 4 Criteria for quality of CF care derived from the chronic care model

QT effectiveness has been described in the studies by Lemieux-Charles [17] and Shortell [18]: it is characterized according to 27 items (see Table 5).

Table 5 Effectiveness of a quality team (QT)

QT Internal factors that may have modulated the QT effectiveness: QT functioning [17] is characterized by 22 items classified in 4 categories 1) the organization at work, 2) the decision-making process, 3) the shared improvement goals, and 4) the ability to communicate and get external support. Studies by L. Lemieux-Charles defined these items to analyze the impact of adopting quality improvement practices on the internal functioning of a team. We use the same items to analyze if the team’s functioning could modulate its effectiveness (see Table 6).

Table 6 Internal functioning of the quality team (QT)

The engagement of the patient/parent as characterized in Carman’s framework [19] is assessed by a list of 31 items, prepared as part of this research (see Table 7).

Table 7 Engagement of the patients/parents on the quality team (QT)

The context elements include: the composition of the multidisciplinary team at the beginning of the PHARE-M intervention (2011) because it might have been a limiting factor in assigning staff to the QT; the culture of the microsystem to which the QT belongs [18] i.e. the organizational culture (see Table 8) and patient centeredness and leadership style (see Table 9); the alignment of the PHARE-M QIP with the hospital quality policy as described within the framework of the European QUASER study [20] using eight open questions in an interview with a head of the hospital quality department (see Table 10).

Table 8 Organizational culture

Table 9 Patient-oriented culture and leadership

Table 10 Open-ended questions to the hospital’s quality department

Focus groups with the members of each QT were conducted by the Clinical Research Assistant, designed around four open-ended questions: 1) What changes in the organization of the CFC can be attributed to the PHARE-M? 2) What difficulties were faced at the CFC? 3) What successes were achieved? and 4) What lessons from this experience after 3 to 4 years? The results of these focus groups involving the 14 CFCs will be put in perspective with the results of the survey conducted by one assessor of the pilot phase who interviewed the 7 first CFCs on the following themes: 1) PHARE-M applicability, 2) participation of patients and parents, 3) functioning and coordination, 4) perceived benefits and costs, 5) effect on the team, 6) effect on care management, and 7) recommendations for PHARE-M national deployment.

Development of the instruments of the realistic study

The self-administered questionnaire was developed from the instruments (cited above) translated into French, and new items prepared as part of this research to characterize quality of CF care and the degree of engagement of the patients or parents. The whole questionnaire is proposed to the members of the quality teams. A limited part of the questionnaire is proposed to the members of the MDT not on the quality team. The questionnaire has been prepared from January to June 2014 with clinicians from the Nantes/Roscoff CF-CERD and experts from the Health Education and Practice Laboratory (LEPS) at the Sorbonne Paris Cité University - Paris 13 Bobigny. It has then been tested between July and September 2014 in three teams from the Nantes/Roscoff CF-CERD (pediatric, adult, and mixed) with 29 respondents from all disciplines and the patients/parents participating in the QT. As a result of these tests, the questionnaire has been slightly adapted, essentially by rewording parts of the French translation and adding free text fields (Questionnaire available upon request to the corresponding author).

On-site investigations

The investigations conducted by the clinical research assistant at the 14 PHARE-M centers take place over the course of 2.5 consecutive days per CFC. The questionnaire is self-administered successively under the supervision of the clinical research associate according to a schedule established with the team at the site, with no possibility of communication or consultation among respondents. The questionnaires and responses are managed in SurveyMonkey Software and subsequently exploited using SAS and Excel Software. The focus group is conducted at the end of the visit. Each focus group is recorded using audio equipment and transcribed in writing.

Analyses of responses and validation of the questionnaire

Responses to the items of the questionnaire are processed anonymously. Each item receives a score on a Likert scale from one to four based on the degree to which the respondent agrees or disagrees with the proposition: “Completely disagree; Disagree; Agree; Completely agree”. “No” and “Unknown” responses are assigned a score of 0. The score is reset to 100 points and can thus be totaled by theme of the questionnaire and category of respondents. An initial descriptive analysis of the responses by CFC is returned to each quality team in the month following the on-site investigation, via a web conference, in order to validate the interpretation of the scores for the different themes and identify avenues for or obstacles to continuous care quality improvement at the CFC.

A Cronbach’s alpha test will be performed on all responses collected at the centers. Since the anticipated number of respondents is around 130 people in total for the 14 teams, this test will not allow the questionnaire to be modified for use in a larger population of respondents. It mainly aims to validate the French translations of the parts of the questionnaire coming from previous studies in English and discuss the use of the parts created within this research study.

A second level of descriptive analysis will be performed by aggregating the responses (all CFCs, by professional discipline, for resource patients/parents, and for professionals) to search for potential associations between quality of care at the CFC 3 years after the PHARE-M intervention and the effectiveness of the QT and/or the engagement of parents/patients and/or contextual elements.

After the publication of the Registry report presenting the 2015 data, changes in indicators from 2011 to 2015 will be crossed with the results of the realistic part of the study, in an attempt to identify any association in relation with more favorable changes over time in patient outcomes. A “signature” set of factors associated with a maximum/minimum impact of the PHARE-M will be sought.

Analyses of the content of the focus groups

The content of the focus groups will be exploited (coding, categorization), processed (analysis, validity), and interpreted according to the standard thematic content analysis protocol [21]. This will be done by grouping and counting within the framework developed during the pilot phase assessment.

Discussion and conclusion

Scope of the study and generalization

The research program aims at identifying the impact of the PHARE-M quality improvement program 3 years after the intervention at the 14 trained CFCs, situated in different organizational and cultural contexts. It uses a mixed methodology crossing the results of a quantitative analysis based on registry data and the results of a qualitative study designed in accordance with the recommendations for research on complex interventions.

The scope of the PHARE-M intervention and thus of the research concerns the management of a singular disease in a care network organized since 2002, which represents a relatively controlled scope. Therefore, the influence of contextual elements on the PHARE-M program’s impact can be analyzed independently from other confounding factors associated with different organizations for the management of various diseases or different hospital departments running diverse specialties.

Fourteen centers volunteered to engage and test the PHARE-M program; they were not randomized. Moreover, initial assessment highlighted that team motivation is a determinant of the speed of adherence to the program. This pattern of our research, focusing on an experimental phase having enlisted volunteer centers, is to be considered in interpreting the results and developing recommendations for a successful roll- out of the PHARE-M program in the national network.

Finally, the research study on the PHARE-M intervention has a study design that could be applied in the assessment of other complex interventions at healthcare settings. Hence, this research study could inform the assessment of interventions concerning the care of rare and/or chronic diseases and the instruments needed for such assessment.

Limitations identified and initial lessons

As a result of the experimental study based on Registry data, a study population paired between two groups (intervention and control) was defined to eliminate certain confounding factors, especially factors linked to patient age distribution. Despite this pairing, significant differences remained in terms of patient age at diagnosis and primary endpoint (FEV1%) between the two groups before the intervention, in favor of the intervention group. These initial differences could have a favorable effect for the rate of decline in FEV1% in 4 years in the intervention group [22, 23]. The question is to investigate whether the slopes are parallel or not. The difference in FEV1% will be taken into account using two different intercepts in the model, one for the intervention group and one for the control group. The patients belonging to either the “PHARE-M” group or the “Control” group will be identified in the Patient Registry with respect to their group for further analysis of their health outcomes.

Moreover, on-site quality audits of the Registry data included in the calculation of the primary endpoints showed discrepancies, mainly due to the CFCs’ interpretation of the rule for selecting the values to transmit to the Registry [16]. The volume of the discrepancies identified in the data audited could be attributed to the change of the rule applied from the 2011 registry survey. This audit points out the need for a certification process to enable a larger use of this database in epidemiologic studies or for public health or pharmacovigilance purposes.

The survey conducted for the qualitative study of the multidisciplinary teams at the 14 centers should include around 130 respondents, including at most 14 patients/parents. This number of respondents might seem low for having enough statistical power in the statistical validation of the survey instruments, especially for the parts of the questionnaire developed within this research. The survey instruments could be improved within the framework of subsequent research studies aiming, for example, at comparing quality of care between centers trained in the PHARE-M and centers untrained in the program, or at making an assessment of the quality of care before/after another intervention. Therefore, this questionnaire represents an instrument that could have further uses in the network.

Expected results in terms of quality improvement of care

If the research study enables to identify factors promoting the adoption of the PHARE-M QIP and the maximization of its impact at CFCs, attention must be paid to the contextual elements to be worked on before or in parallel with the introduction of this program at the remaining CFCs. In the United States, the CFF has conducted “Leadership Collaborative” programs to develop leadership on multidisciplinary teams. The availability of the MDTs staff at the European standards for the number of patients followed could also represent a pre-requisite for their participation in the PHARE-M. The quality of care assessed after 3 years within the CFCs trained to PHARE-M might also enable to identify new avenues for improvement, including some beyond the scope of the clinical microsystem such as the Information System or the generalization of Guidelines.