Skip to main content

A multi-stakeholder multicriteria decision analysis for the reimbursement of orphan drugs (FinMHU-MCDA study)



Patient access to orphan medicinal products (OMPs) is limited and varies between countries, reimbursement decisions on OMPs are complex, and there is a need for more transparent processes to know which criteria should be considered to inform these decisions. This study aimed to determine the most relevant criteria for the reimbursement of OMPs in Spain, from a multi-stakeholder perspective, and using multicriteria decision analysis (MCDA).


An MCDA was developed in 3 phases and included 28 stakeholders closely related to the field of rare diseases (6 physicians, 5 hospital pharmacists, 7 health economists, 4 patient representatives and 6 members from national and regional health authorities). Initially [phase A], a bibliographic review was conducted to identify the potential reimbursement criteria. Then, a reduced advisory board (8 members) proposed, selected, and defined the final list of criteria that could be relevant for reimbursement. A discrete choice experiment (DCE) [phase B] was developed to determine the relevance and relative importance weight of such criteria according to the stakeholders’ preferences by choosing between pairs of hypothetical financing scenarios. A multinomial logit model was fitted to analyze the DCE responses. Finally [phase C], the advisory board review the results using a deliberative process.


Thirteen criteria were selected, related to 4 dimensions: patient population, disease, treatment, and economic evaluation. Nine criteria were deemed relevant for decision-making and associated with a higher relative importance: Health-related quality of life (HRQL) (23.53%), treatment efficacy (14.64%), availability of treatment alternatives (13.51%), disease severity (12.62%), avoided costs (11.21%), age of target population (7.75%), safety (seriousness of adverse events) (4.72%), quality of evidence (3.82%) and size of target population (3.12%). The remaining criteria had a < 3% relative importance: economic burden of disease (2.50%), cost of treatment (1.73%), cost-effectiveness (0.83%) and safety (frequency of adverse events) (0.03%).


The reimbursement of OMPs in Spain should be determined by its effect on patient’s HRQL, the extent of its therapeutic benefit from efficacy and the availability of other therapeutic options. Furthermore, the severity of the rare disease should also influence the decision along with the potential of the treatment to avoid associated costs.


Rare diseases pose a threat to the health of individuals. They are diseases of low prevalence and high complexity that can lead to death or chronic disability [1] and for which there frequently are no therapeutic options [2]. These generally serious, chronic, and progressive diseases have a high impact on patients, their families, and even society, and are characterized by pain, disability, significant organ damage, and high mortality rates [3].

In Europe, rare diseases are defined as those pathologies that affect less than 5 people per 10,000 inhabitants [4], most of them being extremely rare or ultrarare, affecting less than 1 person per 50,000 inhabitants [2]. Despite their low frequency, since there are many different rare diseases, they affect millions of people. It is estimated that there are more than 6,000 rare diseases [5], affecting between 6 and 8% of the population at some point in their lives [1]. Because of their low prevalence, their specificity, and the high number of people they affect altogether, these pathologies require a comprehensive approach and priority of action to prevent significant morbidity or premature mortality and to improve the quality of life and socioeconomic potential of the people [1].

Orphan medicinal products (OMPs), which are intended to diagnose, prevent, or treat rare diseases, have a shared community procedure for being designated as such in the European Union, and this community approach provides opportunities for research, development, and marketing [4, 6]. Despite regulations at the European level for centralized approval, patient access to medicines depends on the pricing and reimbursement policies established individually in each member state; and these policies vary widely, given the difficulties of evaluating the available evidence during their pricing and reimbursement process [7, 8]. Financing decisions for OMPs are complex and are conditioned by several conflicting factors, including the promotion of equitable and timely access for patients, cost containment strategies to maintain public services, and the reward for innovation [9, 10].

In Spain, once marketing is authorized by the European Commission and after obtaining a national code, the maximum industrial price and the funding conditions of the medicine are established [11]. Only about half of the OMPs authorized by the European Medicines Agency (EMA) [8, 12] are marketed and reimbursed by the Spanish national health care system, fewer than in other European countries such as Germany, France, and Italy, although similar funding decisions were taken in Spain and Italy compared to between other countries [13]. In Spain, as in other European countries, a conditional approval of the drug by the EMA delays and reduces the likelihood of financing by 80%, which is relevant given that a quarter of the cancer OMPs are conditionally approved and, a third of the OMPs for metabolic diseases are approved under exceptional circumstances [13]. The average time between the European Commission’s marketing authorisation decision and the establishment of the price and reimbursement condition of an OMP in Spain is estimated at 23 months [10, 12], and the period between the authorization at the national level by the Spanish Medicines Agency (AEMPS) (national code assignment) and financing is 13.7 months [10]. After the national pricing and reimbursement process, several autonomous agencies and regional and hospital committees re-evaluate the drug and its clinical conditions of use, which results in the establishment of different access criteria. These differences in the availability of medications have caused many people affected by rare diseases to experience serious difficulties in accessing the treatment they need [3].

In most developed countries, including Spain, OMPs are financed under the same criteria as other medicines [14,15,16]. Health-technology assessment processes, in addition to evaluating a drug’s efficacy, safety, and quality, usually consider cost-effectiveness in decisions about pricing and reimbursement, but when evaluating treatments for rare diseases, limitations arising from high prices and high uncertainty regarding efficacy make it difficult to reach the established cost-effectiveness thresholds [7, 17]. Therefore, many consider cost-effectiveness analysis insufficient as a basis for these decisions [18]. However, many of the OMPs, despite not being cost-effective, end up being funded, which implies that in practice, criteria besides efficiency are taken into account [16, 18]. Although the criteria that are used in the decision-making process for the reimbursement of drugs in Spain are included in the current legislation [19], their practical implementation is unknown, and the pricing and reimbursement system needs to be more transparent, in line with the recently published recommendations [20].

To establish the key factors in the reimbursement process, it is necessary to develop specific tools that reflect the priorities and perspectives of all the agents involved in decision-making regarding access to, and financing of medicinal products, and especially OMPs. Multicriteria decision analysis (MCDA) is a set of methods and tools that support complex decision-making, overcoming the limitations inherent to treatments with OMPs and improving the technical quality, ethics, and transparency of decisions on prioritization, inclusion in the therapeutic arsenal, and financing of OMPs. These methods provide the analytical capacity and methodological infrastructure necessary to explicitly add all the elements, interests, criteria, values, and concerns on which decision-making is based. In addition, in an MCDA study, the inclusion of participatory processes is critical for incorporating social value into the different dimensions to be considered and for building decision rules [17], as it is important to involve different stakeholders in the process. Beyond establishing whether policymakers consider a particular criterion relevant when making decisions, MCDA studies can determine how much weight or importance they give to one criterion over another [18].

It is difficult to establish a single overall proposal for the implementation of MCDA since each country or region has different values, capacities, resources, and constraints [18]. Therefore, it is necessary to develop specific tools that reflect the priorities and perspectives of the stakeholders involved when considering the factors relevant to OMP access and financing at the national level.

The objective of this study was twofold: first, to review, discuss, and reach a consensus on the most relevant criteria for decision-making about pricing and financing OMPs in Spain; and second, to prioritize them according to their relative importance based on the preferences stated by different stakeholders, following the MCDA methodology.


The FinMHU-MCDA study consisted of an MCDA based on the international recommendations of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), which establishes the appropriate steps and techniques for such analyses [21, 22].

The work of this study was structured into three different phases. In the initial phase (phase A), we chose and defined the criteria to be considered in reimbursement decisions for OMPs. In phase B, the relevant criteria for financing were established based on their relevance for decision-making and then prioritized by weighting them according to their relative importance. Finally, in phase C, the results were discussed through a deliberative process (Fig. 1).

Fig. 1
figure 1

Results of the literature search for the identification of reimbursement criteria

In this project, there were a total of 28 participants included out of 89 contacts who were invited to participate, belonging to sectors with decision-making capacity and holding opinion-leading positions on orphan drugs and rare diseases, including clinical management, health management, and health technology assessment. These participants were classified into five groups according to the type of stakeholder: Health Authorities (six), with national or regional responsibilities and experience; Hospital Pharmacy (five), with experience in the field of OMPs; Health Economics, including university academics and professionals in the sector (seven); Clinicians, including physicians with experience and knowledge in the management of patients with rare diseases (six); and representatives of Patient Associations (four). Prior the selection of the rest of the participants, a panel of eight experts was chosen to participate in the face-to-face phases (A and C) of the study. This panel included three hospital pharmacists, two health economists, a physician specializing in rare diseases, a representative of health authorities, and a representative of the Spanish Alliance on Rare Diseases (Federación Española de Enfermedades Raras, FEDER).

Phase A: selection and definition of criteria

The objective of this phase was to identify, select, and define the criteria relevant to the financing of OMPs that would serve as the basis for the following phases of the study. This phase included an online questionnaire and a face-to-face meeting of the panel of experts.

To identify possible reimbursement criteria for OMP, a literature review was conducted (July 2019) selecting different nationally and internationally published MCDAs. A search was done in the MEDLINE database through the PubMed search engine using specific terms of interest combined by Boolean operators (AND, OR), with no publication date limit (Fig. 2). We sought studies aimed at decision-making about orphan drugs and rare diseases that were conducted using MCDA methods and describing criteria for drug financing. The search identified a total of 80 publications. Of the articles identified, 22 were eliminated as duplicates or for being the same study conducted in different countries but with the same criteria. The other 58 publications were selected for full-text reading, after which we excluded 39 that did not fit the selection criteria, mainly due to not using MCDA methods or, if they did, for not being related to financing criteria. Finally, 19 references were included for the analysis and extraction of criteria (Fig. 1).

Fig. 2
figure 2

FinMHU-MCDA study: phases of the study

In the studies reviewed, the criteria corresponded to different factors related to orphan drugs, their indicated pathologies, and the patient population for which they are prescribed, including clinical and nonclinical variables. These criteria were characterized according to levels or categories. Each level defined or explained one of the characteristics or values that a criterion could present regarding an orphan drug, a rare disease, or a certain population. These levels could be quantitative or qualitative, they had to be exclusive and differentiated within that criterion in relation to the scope of the study; that is, the difference between whether a criterion was defined at one level or another could reflect a change in decision-making.

Based on the criteria identified in the literature, an online questionnaire was designed. It had an initial proposal of criteria and their levels, and was shared with the panel of experts for validation, with the option of modifying the criteria and suggesting additional ones, if considered. The answers to the questionnaires were shared during a face-to-face meeting of the panel of experts, where a consensus was reached regarding the list of criteria, including their designations, definitions, and levels (Table 1). This list of definitive criteria served as the basis for designing the next phase of the study.

Table 1 Reimbursement criteria for orphan drugs

Phase B: weighting and prioritization of the criteria

Once the potentially relevant criteria for decision-making about the financing of OMPs were agreed upon, an analysis was done using a preference elicitation technique. This aimed to identify which criteria were relevant for decision-making and to prioritize and weight them according to their relative importance.

The selected technique was a discrete choice experiment (DCE), which was developed based on the international recommendations of ISPOR [23, 24]. This method was performed using an online questionnaire addressed to the different stakeholders participating in the study, including the panel of experts from the first phase.

The questionnaire to compile the stated preferences of the participants was designed by combining criteria of different levels that were selected in phase A of the MCDA, by which we generated hypothetical financing situations or scenarios. Each scenario consisted of a unique combination of criteria and levels that described the characteristics of an orphan drug, a rare disease, and a hypothetical target population. Each question of the DCE offered two hypothetical financing scenarios, and the participants had to select the most favorable scenario for OMP reimbursement from the two hypothetical scenarios. To establish the statistical significance of criteria relevant to those decisions, the minimum number of pairs of scenarios that needed to be included in the questionnaire was calculated through an orthogonal design using the “Support.Ces” package.

Two multinomial logit models (general and reduced) were adjusted to analyze questionnaire responses and determine the relevance of the criteria and their levels according to the preferences of the participants. With the general model, we determined the strengths of preferences over the criteria from the coefficients obtained for each criterion. The reduced model determined which criteria were relevant to the selection of one financing scenario over another during the DCE. This reduction of criteria was performed using the Bayesian Akaike information criterion (AIC), eliminating the less informative criteria that were not decisive (p-values > 0.05). Considering n evaluated criteria, the relative importance (WD) was calculated using the following formula given the regression coefficients of each criterion:

$$V_{D} = \frac{{\left| {Coef_{D} } \right|}}{{SE_{D} }}\quad W_{D} = \frac{{V_{D} }}{{\mathop \sum \nolimits_{i = 1}^{n} V_{Di} }} \times 100$$

Coef: coefficient; SE: standard error.

All statistical analyses were performed using R software (version 3.2.3, R Foundation for Statistical Computing, Vienna, Austria) [25].

Phase C: deliberative process

In this last phase of the MCDA, the results of the DCE on the prioritization and weighting of the criteria according to their relative importance were presented in the face-to-face meeting with the panel of experts, with the objective of discussing and interpreting the results through a deliberative process. The conclusions resulting from this process are reflected in the Discussion of this article.


Selection and definition of levels

From the 19 publications selected in the literature review, 225 reimbursement criteria related to the treatment, the disease, and the affected population were identified. Of these, 125 were repeated exactly in more than one publication, resulting in a total of 100 different criteria. This list was simplified to 77 due to similarities between them. Finally, these criteria were grouped into 17 major concepts (Fig. 2). Based on these concepts, an initial proposal of 14 OMP reimbursement criteria for this study was designed, including its categorization on different levels, for review by the panel of experts. After completing the questionnaire, the initial meeting, and several specific reviews with the panel of experts, the 12 final reimbursement criteria of OMPs were established (Table 1). The criteria “Authorized indications” and “Budgetary impact” were removed from the final list, despite being criteria that frequently appear in this type of analysis. The panel of experts considered that the number of indications would not be a relevant criterion for decision-making since reimbursement is granted at indication level. Regarding the budgetary impact, the possibility of double counting was brought up, since this criterion includes both the population of patients to be treated and the cost of treatment, criteria already included individually in the analysis.

Screening, prioritization, and weighting of criteria

Based on the 12 definitive reimbursement criteria, the DCE questionnaire was designed. Due to methodological limitations and to avoid an excessive number of questions that would hinder correct completion, the “Safety” criterion was divided in two: one considering the seriousness (serious/nonserious) and one considering the frequency of adverse events (frequent/infrequent). From the combination of the criteria and their levels, 36 was the minimum number of pairs of hypothetical scenarios necessary to establish the questions of the online questionnaire. This questionnaire was sent to a total of 89 relevant people in the field of rare diseases, of whom 29 (32.6%) completed the questionnaire. One questionnaire had to be excluded from the analysis because it was not complete.

The analysis of the DCE responses showed that 9 of the 13 criteria (counting the double safety criterion) were relevant to decision-making (Table 2). From the whole cohort of participants, economic burden of the disease, frequency of adverse events (safety), cost of treatment, and cost-effectiveness were the criteria that were the least informative at the time of selecting one financing scenario over another; that is, other criteria were more relevant in the choice.

Table 2 Results of the discrete choice experiment

According to the results obtained, preference was given to a scenario that financed OMPs for diseases with a target population of lower prevalence, of nonpediatric age, and prioritizing the most severe pathologies. It was important that the orphan drug presented nonserious versus serious adverse events and that no other treatment options were available. In terms of the efficacy criteria, treatments that were curative or that significantly increased survival were preferred over other benefits, as were treatments that had high-quality evidence from controlled clinical trials with comparators. In addition, treatments that improved the health-related quality of life of patients and that avoided higher disease costs were prioritized, including both direct medical and nonmedical costs and indirect costs associated with loss of productivity.

Considering the reimbursement criteria all together, the relative importance of each criterion with respect to the rest was determined. The impact of treatment on health-related quality of life was the criterion with the greatest weight in decision-making, with 23.53% relative importance, followed by the benefit obtained with efficacy (14.64%), availability of treatment alternatives (13.51%), disease severity (12.62%), and avoided costs (11.21%). The rest of the criteria had a relative importance of less than 10%. For each group of stakeholders, the relative importance of each of the evaluated criteria was also obtained (Table 2). For each group, the criteria of health-related quality of life, efficacy, availability of treatment alternatives, and avoided costs were relevant. In the Health Authorities and Health Economics groups, the three criteria related to economic evaluation (cost of treatment, avoided costs, and cost-effectiveness) were relevant to decision-making, making up more than 25% of the decision in the group of Health Economists. The impact of treatment to health-related quality of life was over 20% in all groups, except for the Patient Associations group, for whom the five most important criteria presented a more uniform distribution.


The FinMHU-MCDA study has established, based on a multi-stakeholder and multi-disciplinary panel of experts in the field, the relative importance of different criteria for the reimbursement process for OMPs in Spain. These criteria and their relative importance weights can serve as a starting point to guide and inspire health authorities and the rest of decision makers involved to promote the development of a specific framework of multicriteria evaluation and move forward to a structured plan for the evaluation of OMPs. All of this with the aim of providing greater clarity and transparency in the access and reimbursement of OMPs in Spain. These criteria and relative importance weights could be further updated to reflect the current situation and the preference of the stakeholders involved in the process. Additionally, this initiative could be considered as a national reference and also contribute for a more uniform OMPs assessment at regional/local levels. In addition, this work establishes a reference framework for the development of future MCDA with direct application on OMPs in specific rare diseases.

The evaluation, selection, and financing of OMPs should be considered in a differentiated way that takes into account the particularities inherent to these treatments with respect to medicines with indications other than rare diseases. For the most part, evidence supporting the efficacy of these treatments is more difficult to gather at the time of marketing authorization than for other drugs because of the conditional and/or accelerated approval granted to benefit the patients receiving these treatments and the limited number of patients due to the low prevalence of these pathologies. Additionally, rare diseases can be serious pathologies that are life-threatening or chronically debilitating and have few or no treatment options. Many of the treatments cannot cure the disease but do improve the general condition of the patients in aspects that can translate into a better quality of life. The limited number of patients to whom they are directed would downplay the cost of treatment compared to other factors. The importance of the cost-effectiveness ratio of these treatments could also be disregarded, since many of the drugs financed are not usually cost-effective [7, 17, 18], so other treatment considerations determine the amount of public financing and their inclusion/exclusion in the therapeutic arsenal. These particularities, which are evident to a greater or lesser extent during the evaluation of the financing of drugs for rare diseases, were reflected in the participants’ preferences regarding the reimbursement criteria. Their responses reflect a high importance of how the treatment improves the quality of life of the patient, without neglecting the efficacy or disease severity or whether there are treatment options, while dismissing the economic criteria, whose relative importance could be greater when evaluating other non-orphan, more traditional, treatments. However, the costs avoided by the treatment would be more relevant because most rare diseases are associated with a significant economic burden, in both direct and indirect costs, the latter comprising a considerable proportion of the total costs [26].

One striking result of this study was the panel’s tendency to prefer a financing scenario for a nonpediatric population. One of the possible interpretations proposed by the expert panel was due to the naming of the criterion levels (pediatric/nonpediatric), which could be difficult to differentiate from a decision context with more criteria involved. Another could be the fact that among clinical decision-makers, there was only one pediatrician, and many rare disease specialists are internists who treat the adult population. In this sense, during the deliberative process, the modification of the levels of the criteria was suggested to distinguish them using different terms such as adults and children or adults and pediatrics (children, adolescents).

One of the strengths of the study is the multi-stakeholder perspective. Although health authorities make the decisions about reimbursement and access to the medications in the National Health System, it is of great importance to facilitate the participation of all sectors with decision-making capacity and opinion-leading positions on rare diseases so as to represent the different existing perspectives, including those of the management/payers, health professionals, and, especially in rare diseases, patients, who bring a unique perspective and are the best experts in their pathologies. Therefore, for the decision to be legitimate, a multi-stakeholder perspective must be incorporated in the development of tools that facilitate decision-making, with MCDA being one of the best approaches to facilitate this participation [27]. Another important element of this study, that strengthens these MCDA studies is the use of decompositional methods for eliciting preferences, such as the DCE, where the criteria are evaluated together [22]. Preferences over criteria are extracted indirectly through the choice between two hypothetical alternatives that embody a set of criteria, as would be done in reality. This allows us to evaluate how the preference for a criterion behaves in the presence of one or another criterion, showing its relative importance. At the same time, one of the possible limitations of this method arises when the number of criteria to consider increases beyond a certain point; too many criteria will increase the number and complexity of the DCE questions [22]. Another limitation of MCDA is that the results will always depend on the number of participants and the proportion of participants belonging to each group of stakeholders. In this sense, we aimed to reach as many participants, and with the most balanced distribution between groups, as we could, but this proportion was finally determined by the acceptance of the invited experts. Ethical criteria related to equity, fairness and justice, population specific interests or priorities, vulnerable populations appear in MCDA frameworks. However, these aspects are often used as contextual criteria, not being included in the core framework where quantitative weights are assessed. In this sense, a potential limitation of our study in the selection and definition of the MCDA criteria could be the non-inclusion of these ethical criteria explicitly, an aspect of great importance in the evaluation of rare diseases.

In the last decade, several analyses have been published evaluating and assessing the criteria that could be considered in the field of OMPs. A recent scoping review [17] compiled the most relevant experiences in recent years regarding the evaluation of orphan drugs in terms of coverage and drug reimbursement in several countries using MCDA, among other approaches [28,29,30,31,32,33,34,35,36,37,38,39,40]. In these studies, a group of 10 criteria was identified that best reflected the particularities of rare diseases as well as the preferences of the different stakeholders regarding the reimbursement decisions for OMPs. The participants in these projects were decision-makers, clinicians, and patient representatives. The criteria of disease severity, available treatment options, and comparative efficacy and safety were the criteria that appeared most frequently, maintaining a high relative importance in all these studies. In contrast, criteria such as innovation, a single indication for the drug, manufacturing complexity, and criteria related to health economics (cost-effectiveness, budgetary impact) appeared less frequently and were considered secondary criteria. The rarity of the disease was a relatively frequent criterion but of low relevance in all included studies. In the studies analyzed, aspects related to the efficacy and therapeutic benefit of the treatment, and the disease severity, were usually considered a priority over less important aspects related to the manufacturing process, technological innovation, or economics, aligning with what is reflected in the FinMHU-MCDA study. Unmet needs were also identified as an important criterion given the lack of knowledge about the natural history of these diseases, their diagnosis, and therefore the absence or scarcity of treatment options [17].

More recently, an MCDA has been performed at the international level that, like the present study, evaluated the relative preferences for the criteria to be included in a decision framework for OMPs, considering a multi-stakeholder perspective via a sample of 120 experts and an expert focus group [41]. That study included stakeholders from different fields, such as the pharmaceutical industry and academia, these two being the majority. It followed a direct rating method based on a total distribution of 100% relative importance among 13 criteria. In both groups of experts, the efficacy and safety of treatment and the severity and unmet needs of the disease were considered the most important criteria, in line with the FinMHU-MCDA study, especially when considering the results of the larger sample of stakeholders. One aspect to highlight in that study was that drug price was not included, because it was considered as a criterion with great influence when evaluating the preferences for the other criteria. However, this criterion was included in the FinMHU-MCDA as it was considered to be important in the decision. Also highlighted in that study was the usefulness of having a discussion group, as the deliberative process generated additional insights regarding the importance of criteria [41]. Regarding the methods used, some studies have used the DCE as a method of eliciting social preferences regarding the reimbursement of orphan drugs [42,43,44,45]. One of them, conducted in several European countries, including Spain, determined that the most preferred criteria were the cost of treatment, improvement in health, value for money, and availability of treatment alternatives [44].

In Spain, there have been some initiatives to evaluate drugs through MCDA methodology [46,47,48] an MCDA has recently been published to encourage the evaluation, positioning, and decision-making regarding OMPs at the national level [46]. In it, 15 criteria were selected for decision-making about OMPs based on the EVIDEM framework and methodology [47]. In this case, the weighting of the criteria was obtained from a previous, nonspecific study of OMPs conducted with 98 health professionals working in drug evaluation committees. The most important criteria were disease severity, comparative efficacy, quality of evidence, and comparative safety/tolerability. Other Spanish studies have used the MCDA method to validate and implement an OMP evaluation framework at the regional level, with evaluators and decision-makers from the Catalan Health Service (CatSalut) [36, 48]. In these studies, 10 quantitative and four contextual criteria were evaluated using the EVIDEM framework, the most important being disease severity, unmet needs, comparative effectiveness, and comparative safety/tolerability. The importance of the present study was that it was performed in all of its phases with a great variety of type of stakeholders related to OMPs bringing a comprehensive approach and conducting a discrete choice experiment, which improves methodological rigor in these studies.

Despite the different methods and the geographical scope, there are similarities between all the aforementioned studies and the FinMHU-MCDA study with respect to the selection of criteria, many of them appearing frequently. Regarding their relative importance, and although there is greater variability between studies, many of the most important criteria in the FinMHU-MCDA study also appear among the most relevant of the studies.


To ensure adequate OMP access and reimbursement, it is necessary that decisions be arrived at through a process in which the preferences over the financing criteria are transparent and explicit, in which all types of agents involved in the field of rare diseases are incorporated, and in which practical tools that favor this process, such as MCDA, are applied.

From a multi-stakeholder perspective, the financing of an orphan drug will be conditioned by its effect on the health-related quality of life, the degree of its therapeutic benefit, and the availability of other treatment options. The severity of the rare disease for which the OMP is indicated is also relevant, as is the extent to which the treatment can avoid the costs associated with this pathology.

Availability of data and materials

The datasets generated or analysed during this study are available on request from Pharmacoeconomics & Outcomes Research Iberia (PORIB), under authorization of the data owner. The data are not publicly available due to restrictions related to confidentiality.



Spanish Medicines Agency (Agencia Española de Medicamentos y Productos Sanitarios)


Akaike Information Criterion


Discrete Choice Experiment


European Medicines Agency


Spanish Alliance on Rare Diseases (Federación Española de Enfermedades Raras), ISPOR: International Society for Pharmacoeconomics and Outcomes Research


Orphan Medicinal Products


  1. Council Recommendation of 8 June 2009 on an action in the field of rare diseases. Official Journal of the European Union (2009/C 151/02). OJ C 151, 3 July 2009, p. 7–10. Document 32009H.

  2. Regulation (EU) No 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/EC. OJ L 158, 27 May 2014, p. 1–76.

  3. Recomendaciones para la búsqueda de soluciones en el ámbito de las enfermedades raras. In: Asociación Española de Laboratorios de Medicamentos Huérfanos y Ultrahuérfanos. 2017. Accessed 11 June 2020.

  4. Regulation (EC) No 141/2000 of the European Parliament and of the Council of 16 December 1999 on orphan medicinal products. OJ L 18, 22 Jan 2000, p. 1–5.

  5. Orphanet. 2020. Accessed 30 July 2020.

  6. Regulation (EC) No 726/2004 of the European Parliament and of the Council of 31 March 2004 laying down Community procedures for the authorisation and supervision of medicinal products for human and veterinary use and establishing a European Medicines Agency. Official Journal of the European Union. OJ L 136, 30 April 2004, p. 1–33.

  7. Gammie T, Lu CY, Babar ZU. Access to orphan drugs: a comprehensive review of legislations, regulations and policies in 35 countries. PLoS ONE. 2015;10(10):e0140002.

    Article  Google Scholar 

  8. Mestre-Ferrándiz J, Iniesta M, Trapero-Bertran M, Espín J, Brosa M. Análisis de la evolución en el acceso a los medicamentos huérfanos en España [Analysis of the evolution in the access to orphan medicines in Spain]. Gac Sanit. 2020;34(2):141–9.

    Article  Google Scholar 

  9. Mincarone P, Leo CG, Sabina S, Sarriá-Santamera A, Taruscio D, Serrano-Aguilar PG, Kanavos P. Reimbursed price of orphan drugs: current strategies and potential improvements. Public Health Genomics. 2017;20(1):1–8.

    Article  Google Scholar 

  10. Badia X, Gil A, Poveda-Andrés JL, Shepherd J, Tort M. Analysing criteria for price and reimbursement of orphan drugs in Spain. Farm Hosp. 2019;43(4):121–7.

    PubMed  Google Scholar 

  11. Rare Diseases Strategy of the Spanish National Health System. Ministry of Health and Social Policy. 2009. Accessed 12 June 2020.

  12. Zamora B, Maignen F, O’Neill P, Mestre-Ferrandiz J, Garau M. Comparing access to orphan medicinal products in Europe. Orphanet J Rare Dis. 2019;14(1):95.

    Article  Google Scholar 

  13. Malinowski KP, Kawalec P, Trabka W, Sowada C, Pilc A. Reimbursement of orphan drugs in Europe in relation to the type of authorization by the European Medicines Agency and the decision making based on health technology assessment. Front Pharmacol. 2018;9:1263.

    Article  CAS  Google Scholar 

  14. Garau M, Mestre-Ferrandiz J. Access mechanisms for orphan drugs: a comparative study of selected European countries. OHE Brief. 2009;52:1–24.

    Google Scholar 

  15. Williams P, Mauskopf J, Lebiecki J, Kilburg A. Using multicriteria decision analysis during drug development to predict reimbursement decisions. J Mark Access Health Policy. 2014;2(1):25270.

    Article  Google Scholar 

  16. Zozaya N, Villoro R, Hidalgo A, Sarria A. Criterios de financiación y reembolso de los medicamentos huérfanos. Instituto de Salud Carlos III—Ministerio de Economía y Competitividad. Agencia de Evaluación de Tecnologías Sanitarias. 2016. Accessed 12 June 2020.

  17. Lasalvia P, Prieto-Pinto L, Moreno M, Castrillón J, Romano G, Garzón-Orjuela N, Rosselli D. International experiences in multicriteria decision analysis (MCDA) for evaluating orphan drugs: a scoping review. Expert Rev Pharmacoecon Outcomes Res. 2019;19(4):409–20.

    Article  CAS  Google Scholar 

  18. Paolucci F, Redekop K, Fouda A, Fiorentini G. Decision Making and priority setting: the evolving path towards universal health coverage. Appl Health Econ Health Policy. 2017;15(6):697–706.

    Article  Google Scholar 

  19. Real Decreto Legislativo 1/2015, de 24 de julio, por el que se aprueba el texto refundido de la Ley de garantías y uso racional de los medicamentos y productos sanitarios. Boletín Oficial del Estado núm. 177 de 25 de julio de 2015. Accessed 12 June 2020.

  20. Advisory Committee for the Reimbursement of the Pharmaceutical Provision or the National Health System (CAPF). Proposal for Strategic Lines of Work of the CAPF for 2020–2021. A Consensus Document. Accessed 13 July 2020.

  21. Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health. 2016;19:1–13.

    Article  Google Scholar 

  22. Marsh K, Ijzerman M, Thokala P, Baltussen R, Boysen M, Kaló Z, et al. ISPOR Task Force. Multiple criteria decision analysis for health care decision making-emerging good practices: report 2 of the ISPOR MCDA emerging good practices task force. Value Health. 2016;19:125–37.

    Article  Google Scholar 

  23. Hauber AB, González JM, Groothuis-Oudshoorn CG, Prior T, Marshall DA, Cunningham C, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health. 2016;19:300–15.

    Article  Google Scholar 

  24. Reed Johnson F, Lancsar E, Marshall D, Kilambi V, Mühlbacher A, Regier DA, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR conjoint analysis experimental design good research practices task force. Value Health. 2013;16:3–13.

    Article  CAS  Google Scholar 

  25. R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2014. Accessed 10 April 2019.

  26. Angelis A, Tordrup D, Kanavos P. Socio-economic burden of rare diseases: a systematic review of cost of illness evidence. Health Policy. 2015;119(7):964–79.

    Article  Google Scholar 

  27. Wagner M, Samaha D, Casciano R, Brougham M, Abrishami P, Petrie C, et al. Moving towards accountability for reasonableness—a systematic exploration of the features of legitimate healthcare coverage decision-making processes using rare diseases and regenerative therapies as a case study. Int J Health Policy Manag. 2019;8(7):424–43.

    Article  Google Scholar 

  28. Hughes-Wilson W, Palma A, Schuurman A, Simoens S. Paying for the Orphan Drug System: break or bend? Is it time for a new evaluation system for payers in Europe to take account of new rare disease treatments? Orphanet J Rare Dis. 2012;7:74.

    Article  Google Scholar 

  29. Naili A. A multi criteria approach for the assessment of drugs for rare diseases. [Internet]. Canada: University of Ottawa; 2016. Accessed 18 June 2020.

  30. Schey C, Krabbe PF, Postma MJ, Connolly MP. Multi-criteria decision analysis (MCDA): testing a proposed MCDA framework for orphan drugs. Orphanet J Rare Dis. 2017;12(1):10.

    Article  CAS  Google Scholar 

  31. Kolasa K, Zwolinski KM, Kalo Z, Hermanowski T. Potential impact of the implementation of multiple-criteria decision analysis (MCDA) on the Polish pricing and reimbursement process of orphan drugs. Orphanet J Rare Dis. 2016;11:23.

    Article  Google Scholar 

  32. Kolasa K, Zwolinski KM, Zah V, Kaló Z, Lewandowski T. Revealed preferences towards the appraisal of orphan drugs in Poland—multi criteria decision analysis. Orphanet J Rare Dis. 2018;13(1):67.

    Article  Google Scholar 

  33. Sussex J, Rollet P, Garau M, Schmitt C, Kent A, Hutchings A. A pilot study of multicriteria decision analysis for valuing orphan medicines. Value Health. 2013;16(8):1163–9.

    Article  Google Scholar 

  34. Iskrov G, Miteva-Katrandzhieva T, Stefanov R. Multi-criteria decision analysis for assessment and appraisal of orphan drugs. Front Public Health. 2016;4:214.

    Article  Google Scholar 

  35. Wagner M, Khoury H, Willet J, Rindress D, Goetghebeur M. Can the EVIDEM framework tackle issues raised by evaluating treatments for rare diseases: analysis of issues and policies, and context-specific adaptation. Pharmacoeconomics. 2016;34(3):285–301.

    Article  Google Scholar 

  36. Gilabert-Perramon A, Torrent-Farnell J, Catalan A, Prat A, Fontanet M, Puig-Peiró R, et al. Drug evaluation and decision making in catalonia: development and validation of a methodological framework based on multi-criteria decision analysis (MCDA) for orphan drugs. Int J Technol Assess Health Care. 2017;33(1):111–20.

    Article  Google Scholar 

  37. Lockhart CM, Hansen RN. Value assessment of orphan drugs for treatment of rare diseases: a systematic review. Value Health. 2016;19(3):A79.

    Article  Google Scholar 

  38. Paulden M, Stafinski T, Menon D, McCabe C. Value-based reimbursement decisions for orphan drugs: a scoping review and decision framework. Pharmacoeconomics. 2015;33(3):255–69.

    Article  Google Scholar 

  39. Short H, Stafinski T, Menon D. A national approach to reimbursement decision-making on drugs for rare diseases in Canada? Insights from across the ponds. Healthc Policy. 2015;10(4):24–46.

    PubMed  PubMed Central  Google Scholar 

  40. Zelei T, Molnár MJ, Szegedi M, Kaló Z. Systematic review on the evaluation criteria of orphan medicines in Central and Eastern European countries. Orphanet J Rare Dis. 2016;11(1):72.

    Article  Google Scholar 

  41. Schey C, Postma MJ, Krabbe PFM, Topachevskyi O, Volovyk A, Connolly M. Assessing the preferences for criteria in multi-criteria decision analysis in treatments for rare diseases. Front Public Health. 2020;8:162.

    Article  Google Scholar 

  42. Mentzakis E, Stefanowska P, Hurley J. A discrete choice experiment investigating preferences for funding drugs used to treat orphan diseases: an exploratory study. Health Econ Policy Law. 2011;6(3):405–33.

    Article  Google Scholar 

  43. Bourke SM, Plumpton CO, Hughes DA. Societal preferences for funding orphan drugs in the United Kingdom: an application of person trade-off and discrete choice experiment methods. Value Health. 2018;21(5):538–46.

    Article  Google Scholar 

  44. López-Bastida J, Ramos-Goñi JM, Aranda-Reneo I, Trapero-Bertran M, Kanavos P, Rodriguez Martin B. Using a stated preference discrete choice experiment to assess societal value from the perspective of decision-makers in Europe. Does it work for rare diseases? Health Policy. 2019;123(2):152–8.

    Article  Google Scholar 

  45. López-Bastida J, Ramos-Goñi JM, Aranda-Reneo I, Taruscio D, Magrelli A, Kanavos P. Using a stated preference discrete choice experiment to assess societal value from the perspective of patients with rare diseases in Italy. Orphanet J Rare Dis. 2019;14(1):154.

    Article  Google Scholar 

  46. Badia X, Chugani D, Abad MR, Arias P, Guillén-Navarro E, Jarque I, et al. Development and validation of an MCDA framework for evaluation and decision-making of orphan drugs in Spain. Expert Opin Orphan Drugs. 2019;7(7–8):363–72.

    Article  Google Scholar 

  47. EVIDEM. A reflective framework to advance health. 2017. Accessed 18 June 2020.

  48. Guarga L, Badia X, Obach M, Fontanet M, Prat A, Vallano A, et al. Implementing reflective multicriteria decision analysis (MCDA) to assess orphan drugs value in the Catalan Health Service (CatSalut). Orphanet J Rare Dis. 2019;14(1):157.

    Article  Google Scholar 

Download references


FinMHU-MCDA Group: Alba Ancochea (Federación Española de Enfermedades Raras, Madrid, Spain); Fernando Antoñanzas (Universidad de La Rioja, Logroño, Spain); Santiago Bonanad (Hospital Universitari i Politècnic La Fe, Valencia, Spain); Encarnación Cruz (Asociación Española de Medicamentos Biosimilares, Madrid, Spain); Teresa Caballero (Hospital Universitario La Paz, IdiPaz, CIBERER U754, Madrid, Spain); Juan Manuel Cabasés (Universidad Pública de Navarra, Pamplona, Spain); Miguel Ángel Calleja (Hospital Universitario Virgen Macarena, Sevilla, Spain); Jordi Cruz (Asociación de Mucopolisacaridosis España (MPS España), Barcelona, Spain); Olga Delgado (Hospital Universitario Son Espases, Sociedad Española de Farmacia Hospitalaria; Palma de Mallorca, Spain); Jaime Espín (Escuela Andaluza de Salud Pública, Centro de Investigación Biosanitaria (IBS) y CIBERESP, Granada, Spain); Manuel García-Goñi (Universidad Complutense de Madrid, Madrid, Spain); Ricardo Gil (Hospital Universitari i Politècnic La Fe (Medicina Interna), Valencia, Spain); Pedro Gómez Pajuelo (Economista de la Salud, Madrid, Spain); Maria Queralt Gorgas Torner (Hospital Universitari Vall d’Hebron, Barcelona, Spain); Antonio López Andrés (Subdirección de Farmacia, Servicio Navarro de Salud-Osasunbidea, Pamplona, Spain); Mónica López Rodríguez (Hospital Universitario Ramón y Cajal, Universidad de Alcalá, IRYCIS, Madrid, Spain); Adela Marín Ballvé (Hospital Clínico Universitario Lozano Blesa, Zaragoza, Spain); María Isabel Martín Herranz (Instituto de Investigación Biomédica de A Coruña (INIBIC), Complexo Hospitalario Universitario de A Coruña (CHUAC), Universidade da Coruña (UDC), A Coruña, Spain); Jorge Mestre-Ferrándiz (Consultor independiente; Universidad Carlos III de Madrid, Madrid, Spain); Alberto Morell Baladrón (Hospital Universitario de La Princesa; Madrid, Spain); Carlos Mur (Consejería de Sanidad, Madrid, Spain); Francesc Palau (Hospital Universitari Sant Joan de Déu y CIBERER, Barcelona, Spain); Matilde P. Machado (Universidad Carlos III de Madrid, Madrid, Spain); Fernando Ignacio Sánchez Martínez (Universidad de Murcia, Murcia, Spain); Alba R. Santos (Asociación NUPA, Madrid, Spain); Mónica Suárez (Federación Española de Enfermedades Neuromusculares, Federación ASEM, Barcelona, Spain); José Luis Trillo (Departamento de Salud Clínico Malvarrosa, Valencia, Spain).


This project was carried out with an unrestricted grant from AELMHU (Asociación Española de Laboratorios de Medicamentos Huérfanos y Ultrahuérfanos). AELMHU was not involved in the design of the study and collection, analysis, and interpretation of data and writing the manuscript.

Author information

Authors and Affiliations




FdAN, RDH and MACas made substantial contributions to the conception and design of the work, generation, analysis, and interpretation of data. EC, MACall, OD, MQG, JE, JMF, FP and AA made substantial contributions to the design of the work, generation, and interpretation of data. These authors were also participants of the study. FdAN and MACas drafted the manuscript. All the authors revised various iterations of the paper critically for important intellectual content, and approved the final submitted version (and any substantially modified version that involves the author’s contribution to the study). They have agreed both to be personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Fernando de Andrés-Nogales.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

FdAN, RDH and MACas are employees of PORIB, a consultancy firm specialized in health technology assessment, which received funding from AELMHU for the development of this project. RA is an employee of AELMHU. EC, MACall, OD, MQG, JE, JMF, FP and AA received fees for the participation in this project.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Andrés-Nogales, F., Cruz, E., Calleja, M.Á. et al. A multi-stakeholder multicriteria decision analysis for the reimbursement of orphan drugs (FinMHU-MCDA study). Orphanet J Rare Dis 16, 186 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: