As the differences in access to OMPs across EU settings become more and more visible, there is a growing understanding that new approaches should be implemented to ensure a more transparent and fair allocation of funds across all individuals who suffer from rare diseases. Despite some international initiatives, new P&R pathways tailored specifically to meet peculiarities of the value assessment of OMPs are still really scarce. The currently available HTA guidelines are mainly limited to the assessment of clinical and cost effectiveness as well as safety. In the field of rare diseases, the importance of the consideration of non-economic criteria in the P&R decision-making is especially raised. Therefore, it was interesting to investigate what kind of value attributes are considered in the HTA process when there are no guidelines designed for the assessment of OMPs.
The study proved that both clinical evidence and economic considerations (CEA, BIA, the cost of therapy) played an important role in the assessment of OMPs in Poland. The results were consistent across different MCDA methods, and both the AHP weights and the SLAM rankings lead to similar conclusions.
The Polish Appraisal Body tended to rank the clinical evidence as the most important factor in the decision-making process. Interestingly, the notion of cost of therapy constituted the second largest contribution to the HTA recommendation process followed by the safety aspects. The importance of these two criteria, however, was similar, as could be seen in the comparison of AHP weights and the results of the sensitivity analysis. Nevertheless, it should be noted that there were statistically significant differences between positive and negative HTA recommendations in terms of the deliberation of safety aspects, which received more attention in the latter group.
According to both SLAM and AHP methods, the existence of a comparator was the fourth most impactful factor in the recommendation process at the Polish HTA agency. This was more important than disease rarity and severity. Interestingly, the results indicated that the availability of a therapeutic alternative was also valued more than economic evaluations. The weights of both CEA and BIA were set to 9%. The sensitivity analysis provided more insight into the importance of economic evaluation. The maximum VIP results for the CEA were set to one, which are only the second criteria with such a high value. if a lower regret is set with CEA versus BIA, then the former was more influential in the recommendation process than the latter.
Even though the cost of therapy played such a significant role in the HTA appraisal process, the results of the budget impact analysis (BIA) did not contribute that heavily towards the HTA outcome. BIA’s maximal regret set above 0.5 indicates that there is a significant opportunity cost for its inclusion in the list of recommendation making criteria.
Oddly enough, HTA recommendations issued by other HTA agencies also played an important role in the Appraisal Body’s judgment. Their weight was almost as high as the one established for the economic criteria. In addition, it should be mentioned that the influence of other HTA agencies was greater for the negative rather than the positive HTA recommendations of the Polish Appraisal Body in a statistically significant manner.
Although the study produced some interesting results, it is not free from limitations, some of which are inherited within the MCDA methodology itself. Firstly, the consequences of the adaptation of the SLAM bring certain simplifications to the analysis that are not without an impact on the results. The SLAM assumes mutual preference independence, which means that preference for an outcome measured by one criterion is independent from the outcome related to another criterion. It can be envisaged, however, that the real value judgment of the HTA Appraisal Body may not be independent across different criteria in the same manner in which value attributes correlate with each other. SLAM introduces a complete compensatory rule that involves an offset mechanism where a bad performance on one criterion can be compensated by a good performance on others. In real life settings, it is, however, difficult to expect that the HTA Appraisal Body will be willing to trade the value of a given OMP between two different attributes. Secondly, the adaptation of AHP methods was applied without transforming the scale to Saaty’s integer values 1–9. There are a number of approaches, which allow for this kind of switch with the adoption of the geometric scale [19], balanced scale [20], power [21], logarithmic [22], and finally root square [21]. It has to be underscored, however, that the analytic hierarchy process (AHP) method was selected following great consideration as it allows a pair-wise comparison of attributes. The application of AHP in this setting allowed the identification of intransitivity of collected criteria in the recommendation making process as well.
In addition to the limitations inherited in the MCDA methodology itself, there are also some additional risks of bias in our results related to the study design. Firstly, although we have taken into consideration all drug indication pairs assessed by the HTA agency between 2011 and 2015, we have not accounted for any potential changes in the Appraisal Body’s preferences across the study period. Secondly, two reviewers subjectively conducted the assessment of attitudes towards different MCDA criteria. Thirdly, the study was limited to the HTA recommendations only. Some of the Appraisal Body’s considerations failed to be presented in these documents, and the assessment of other P&R decision makers were left outside of the scope of our analysis.
Despite these limitations, our study contributes to the growing body of literature focused on the adaptation of MCDA to the OMPs assessment. In contrast to other publications, it presents the application of MCDA to the analysis of preferences of decision makers towards the value attributes of OMPs. To our knowledge, none of previous studies had a similar objective. For instance, Schey et al. [23] adopted MCDA framework to the value OMPs from the UK perspective. Iskrov [24] developed an MCDA value model for the assessment of OMPs from the perspective of the Bulgarian payer. In the manuscript of Sussex J [25], the objective was to pilot the use of MCDA as a framework for a value assessment of OMPs.
Still, the current study could be regarded as a continuation of earlier research that verified the potential impact of the implementation of an MCDA approach on the value assessment of OMPs in the Polish context [14]. Despite different objectives and the scope of data analysed, there is some resemblance in the conclusions between both studies. Firstly, it was revealed that the economic arguments had an immense importance in the Polish HTA process. Secondly, it was indicated that other criteria beyond the set of value attributes defined in HTA guidelines, such as the cost of therapy, HTA outcomes from other jurisdictions, played a role in the appraisal process also in Poland.
In addition to MCDA studies, there are some publications that provide insight into the value attributes used by manufacturers in price setting for OMPs as well. Therefore, some comparisons can be made between the preferences of decision makers and producers. In this regard, at least three interesting observations should be noted. Firstly, the opposite valuation was found with respect to the criteria of the clinical effectiveness. Although the highest importance of data regarding efficacy in the recommendation process was clearly expressed, it has not been found among the predictors of the price of orphan drugs [26, 27]. Secondly, some similarities could be drawn between the preferences of manufacturers and decision makers with respect to the availability of comparator. While it was valued as the fourth most relevant criteria in the HTA appraisal process, some studies indicated its importance in the manufacturers’ setting of a pricing policy. For instance, the Belgian comparative analysis of prices of reimbursed OMPs revealed that total annual costs were lower for drugs with a comparator compared to those without alternative treatment options [28]. Finally, a contradiction could be noted with respect to the notion of disease rarity. As this was not listed amongst the key attributes of the HTA appraisal process, the opposite could be said with respect to pricing studies. This review of 75 OMPs, which were granted marketing authorization by the EMA until 2014, revealed a significant inverse relationship between disease prevalence and the annual cost of drug treatment [29]. However, analysis of 45 OMPs in the five biggest EU jurisdictions revealed that the number of available alternatives as well as the prevalence of disease was correlated with the price of the drug [30].
In order to ensure the contribution of our findings towards further improvements of access to treatments for rare diseases, some recommendations for future development of value framework of OMPs can be elicited. The first recommendation is dedicated to further enhancement of the broader engagement of different stakeholders’ groups. Given the nature and peculiarities of rare diseases, uncertainty regarding both clinical and economic consequences of the implementation of a given health technology to the routine practice is inevitable. Therefore, it is not surprising that the Polish Appraisal Body attributed the highest value to the clinical evidence and noted the issue with credibility of data so many times among the reasons for negative HTA recommendation. To address such challenges, the development of a value framework should take views of both all decision makers and patient advocacy groups (PAGs) into consideration. With respect to the first group, the need of collaboration between the HTA agencies and EMA must be highlighted. Given the scarcity of clinical and safety data for OMPs, early dialog can be of great value especially when it comes to common understanding of unmet medical needs and scientific approaches to real life data collection. In addition to the wide range of incentives designed for OMPs available for the regular approval process, there are a number of special pathways for the marketing authorization such as accelerated, conditional, and exceptional pathways that enable the most needed OMPs to reach the patients in timely fashion [31]. Early engagement of HTA bodies in such processes could potentially facilitate a generation of P&R outcomes as well. As far as the contribution of the second group is concerned, it is again the lack of sufficient clinical and safety data that makes the PAG’s role in the P&R decision making so valuable. Given limited evidence regarding particular health states, patient experience can provide a needed insight into the real burden of disease. Such individual stories cannot compete with the breadth of evidence available for common diseases. Nevertheless, as long as it is ensured that patients’ stories are collected in a systematic and transparent manner, it can certainly successfully address some of the uncertainties of the HTA process. The second recommendation calls for the introduction of new innovative pricing & reimbursement arrangements. The results of our study indicated that the notion of the cost of therapy was a key consideration during the recommendation process of OMPs. At the same time, the price of a drug and an unfavorable cost effectiveness ratio were the most frequent reasons for the negative HTA recommendations. Therefore, a new set of P&R rules should enable the implementation of a special form of innovative managed entry agreements. Alternatively, a financing mechanism that links reimbursement with drug performance can be introduced to validate cost-effectiveness claims. Following the preferences of expert payers, the second one should especially be taken into consideration in the CEE Region [32]. Temporary access to OMPs could be secured as long as an HTA agency envisages the opportunity for further real life data collection before a final recommendation has been completed. For example, this is a common approach in the Netherlands and Sweden where temporary coverage is granted conditional upon the conduct of an observational study.