Quantifying benefit-risk preferences for new medicines in rare disease patients and caregivers
Orphanet Journal of Rare Diseases volume 11, Article number: 70 (2016)
Rare disease patients and caregivers face uncommon, serious, debilitating conditions often characterised by poor prognosis and limited treatment options. This study aimed to explore what they consider of value when choosing between hypothetical therapeutic options and to quantify both their benefit-risk preferences and the influence of disease context.
A mixed-methods survey with patients and caregivers was conducted in the United Kingdom across a range of rare diseases. Discrete-choice experiments that compared hypothetical treatment profiles of benefits and risks were used to measure respondent preferences across a set of seven attributes related to health outcomes, safety, and process of care. Bespoke questions on current disease management and the joint use of the 12-item WHODAS 2.0 questionnaire and of two Likert scales capturing self- and proxy-assessed disease-induced threat to life and impairment were implemented to describe disease context. Additionally, qualitative insights on the definitions of value and risk were collected from respondents.
Final study sample included 721 patients and 152 informal caregivers, across 52 rare diseases. When choosing between hypothetical novel treatments for rare diseases, respondents attributed most importance to drug response, risk of serious side effects, and the ability to conduct usual activities while on treatment. In contrast, attributes related to treatment modalities were the least important. Respondents expressed a willingness to accept risks in hopes of finding some benefit, such as a higher chance of drug response or greater health improvement potential. Increasing disease severity, impairment or disability, and the lack of effective therapeutic options were shown to raise significantly the willingness to gain benefit through increased risk.
This is the first study performing a quantitative discrete choice experiment amongst patients and caregivers across 52 rare conditions. It enables a more detailed understanding of the relationship between disease context, treatment attributes and the degree of risk respondents are willing to take to gain a specific degree of benefit. Researchers of novel therapeutics for rare diseases should be encouraged to invest in preference elicitation studies to generate rigorous patient evidence and specific regulatory guidance should be issued to acknowledge their importance and their use in marketing authorisations.
Regulatory decisions are typically depicted as a balancing act between benefits and risks where both interpretations of hard clinical data and value judgements interplay. Traditionally, regulators have made assumptions, albeit implicit, about what patients actually think and prefer, including their willingness to trade benefits for harms [1, 2]. Notwithstanding that the ultimate raison d’être of any medicine is to benefit patients who have a unique knowledge about their disease and its current therapeutic environment, traditionally patient views and preferences were rarely explicitly sought.
Recent developments, however, have suggested a culture shift is taking place as health care systems and decision-makers have come to realise that the understanding of a disease and of the added value of new medicines could not be fully achieved without engaging with patients (and their caregivers). Roadmaps, processes and tools to enhance patient involvement in drug development [3, 4], medical decision-making [5–9], health technology assessment , or drug licensing decisions are now being debated. It still remains unclear, nonetheless, how the patient voice may formally be brought into decision-making beyond anecdotal approaches (such as including one patient on a committee or collecting a handful of testimonials) and how rigorous patient evidence may be generated to be readily accepted by decision-makers. Acknowledging ‘the added value of patients in benefit-risk considerations – in that they enrich regulatory decisions by complementing them with the views of those directly affected by regulatory decisions’ , the European Medicines Agency (EMA) and the U.S. Food and Drug Administration (FDA) have launched a number of initiatives to start capturing patient views, values and preferences to inform benefit-risk assessment [12–16]. Their underlying objective has been to better align acceptance of risks and uncertainties by regulators with that of patients and with the interests of public health more broadly. As the EMA stated, ‘an excessive focus on avoiding risks and uncertainties concerning new medicines might be against the interests of patients, delaying or reducing access to potentially life-saving treatments’ .
Taking account of patients’ interests is of particular significance when it comes to rare diseases which are serious, debilitating conditions often characterised by poor prognosis and limited treatment options. In this context, rare disease patients and their representatives have urged regulators to be more permissive and to allow for drugs with greater risk or side effects than traditionally accepted [2, 18–20]. For example, in its draft guidance to the FDA the patient advocacy group Parent Project Muscular Dystrophy stated that ‘in the absence of any approved treatment for Duchenne at all […], the community [of Duchenne patients and families] has expressed a willingness to accept a certain degree of uncertainty regarding both benefit and risk. Some in the community may be willing to take even greater risk – on account of accelerated rates of progression, or their proximity to loss of a vital function or death’ .
Adopting the perspective of the rare disease community (while acknowledging its wide diversity across 7,000 rare diseases), the main purpose of the present study was to contribute to the on-going discussion about what patients affected by a rare disease and their caregivers may consider ‘of value’ when considering new medicines and to quantify their benefit-risk preferences. Under the assumption that values and preferences are context-dependent, our secondary research objectives were to assess to what extent responder-reported assessments of unmet need, disease severity and impairment influence the value attached to key features (or attributes) of future hypothetical treatments. This research used a discrete-choice experiment to assess patient and caregiver preferences across a range of rare diseases.
Discrete-choice experiments (DCE) have been used increasingly in recent years to explore and quantify the relative importance of the benefits and risks of different treatments to patients and other stakeholders [21–30]. Well anchored in both psychology and economic theory DCEs offer several advantages over simple rating or ranking exercises, notably because they are shown to give more reliable results [31, 32]. The main postulate of DCEs is that treatments are composed of a set of features, or ‘attributes’ and that the relative value of a particular treatment to an individual is a function of these attributes [33–35]. DCEs can not only consider those treatment attributes specifically related to health outcomes such as efficacy and safety but also those that are process related such as treatment duration, care location or impact on caregivers. In a DCE, respondents are presented with a series of trade-offs in which they are forced to state their preferences by choosing a preferred alternative from a set of hypothetical treatment profiles. These treatment profiles vary by ‘levels’ of treatment attributes that may define the magnitude, severity, likelihood, or timing of each. Statistical analysis of the choices made reveals the implicit relative importance of the attributes of the treatment. The result is an estimate of the perceived value of a treatment as a weighted sum of the treatment attributes.
The design and implementation of this study relied on a partnership with a sample of UK-based patient organisations and support groups, whose activity is targeted at patients and families affected by rare diseases. Eligible study partners were identified and recruited through Orphanet UK’s network of patient organisations. Study participants were subsequently recruited from the study partners’ members list. Eligible study participants were at least 18 years of age, living in the United Kingdom, and were either individuals affected by a rare disease or individuals acting as informal caregivers to individuals affected by a rare disease. Respondents who did not meet the inclusion criteria were screened out. Respondents who did not complete the survey or who always picked the first or last alternative in a choice set were excluded from the sample. Full details on study recruitment flow are available in Additional file 1: Appendix A.
Patients and caregivers were informed that neither benefits nor risks were derived by participating in this study and provided informed consent. The information sheet, consent form, and study protocol were reviewed and approved by King’s College London’s Biomedical Sciences, Dentistry, Medicine and Natural and Mathematical Sciences Research Ethics Subcommittee. For each completed and valid survey, a donation of £2 was made to the participating patient organisations.
Survey instrument & experimental design
DCEs rely on survey instruments to elicit preferences. A web-based survey, tailored to the two study target audiences (i.e. patients, informal caregivers) was developed. While patients were invited to respond to this survey in light of their own experience of the disease and of their own values and preferences; caregivers, in contrast, were requested to adopt a ‘proxy-patient perspective’  whereby they, as proxies, were asked to respond as they think the patient they usually provide informal care to would respond. The survey had a similar structure for both study target audiences and consisted of three successive modules.
In the first module, respondents had to describe their experience with the disease (e.g. time since diagnosis, current disease management, satisfaction with current care etc.), and were subsequently invited to assess overall disease severity. For this latter purpose we chose to implement three alternative measurements: two Likert scales (range 0–10) capturing self- and proxy-assessed threat to life caused by the rare condition and disease-induced impairment; and the 12-item World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0). WHODAS 2.0 is a generic health and disability assessment tool designed according to the International Classification of Functioning, Disability, and Health (ICF) framework [37, 38] [see Additional file 2: Appendix B and Additional file 9: Appendix I].
The second survey module consisted of the DCE that comprised a series of treatment-choice questions defined by seven attributes: (1) ‘chance that the medicine will work’, (2) ‘expected health improvement’, (3) ‘risk of experiencing moderate side effects affecting quality of life’, (4) ‘additional risk of getting serious side effects leading to life-threatening consequences’, (5) ‘treatment duration’, (6) ‘burden of treatment’, and (7) ‘ability to conduct usual activities while on treatment’. Three levels were assigned to each attribute [see Additional file 3: Appendix C for details]. The choice of treatment attributes, definitions and levels were informed by peer-reviewed scientific literature, patient advocacy literature, and consultation with patients affected by a rare disease. Our design choices were presented to representatives of the participating patient organisations for validation. Clinical experts also reviewed the choice of attribute levels to ensure that they spanned the clinically relevant range of outcomes that has been seen or may likely be seen in clinical practice.
Attributes and levels were combined to create hypothetical medication profiles. A D-efficient design was generated with 40 choice sets in four orthogonal blocks, each block containing 10 pairs of hypothetical medication profiles. Participants were then randomly assigned to one of the four survey versions. Within each version, the order of treatment choice questions was randomised across participants. More information on the experimental design is included in Additional file 4: Appendix D. During the DCE task, survey participants were asked to point out which hypothetical medication profile in each pair they would choose if the two treatments shown were the only options available. Before completing it, participants were presented with a detailed description of all the attributes and levels by using pictures specifically designed for this purpose in a user-friendly format. Respondents were told to imagine that diagnosis had just happened, that two alternative treatments existed, and that they were in a position to influence the choice of treatment. However, we confirmed that these treatments do not currently exist and assured respondents that this ‘was not a test’ and that there were ‘no right or wrong answers’.
The third and last survey module included socio-demographic questions and an open-text comment field question where participants were invited to bring their own definition of the ‘value of a medicine’.
A pilot study (June 17th–July 5th 2014; n: 68) across patients, patient representatives and other healthy volunteers evaluated survey length, logic, skip patterns, trade-off complexity, and wording. Survey length, flow, and language were subsequently amended to address the feedback received. An optional paper version of the survey was made available to participating patient organisations where it was felt that the sole use of a web-based interface may introduce a selection bias as a result of uneven access to/use of Internet (e.g. lower income populations, elderly people).
Descriptive statistics were calculated for socio-demographic, disease, and care management history variables for all participants. The WHODAS total score was calculated as the sum of the 12-item scores according to the simple scoring instructions , with higher scores reflecting greater disability. Summary scores on the reported level of disease-induced impairment and threat to life were analysed. Scores were generated for the variables related to satisfaction with current care, current disease management and impact of care on daily life across the sub-group of participants who had reported on-going treatment for the rare condition. An exploratory unmet need score was derived by summing up the respective ratings of these three variables.
Based on the trade-offs generated through the DCE, a mixed logit model (or random parameter logit model) was estimated with Hierarchical Bayes methodology using the R-package bayesm. As highly flexible models, mixed logit models are considered to be the state-of-the-art discrete choice models [31, 32, 39–44]. They allow to model the variation in preferences among the respondents and yield not only estimates for the mean preference and preference heterogeneity, but also individual estimates representing the value that each respondent attaches to the different attribute levels. They are therefore best suited to investigate the relationship between the individual preferences and various covariates. The importance of each attribute was computed for each respondent as the difference between the individual part-worth estimate of the first (best) attribute level minus the individual part-worth estimate for the last (worst) attribute level (truncated below at zero if necessary) and then rescaled by the sum to obtain the individual relative importance value for each attribute. Tests for significance were performed on the individual part-worths and relative importance for each attribute and the robustness of these results was checked thoroughly as explained in Additional file 5: Appendix E.
To evaluate whether the relative importance values are influenced by the context of the disease – as captured by respondents’ reported assessments on disease-induced impairment, disability, threat to life, and overall unmet need – a series of regression analyses where run where the individual relative importance values for each attribute were related to these measures in the total sample and different subsamples to check the validity of the results.
Lastly, patients’ open-text comments on the ‘value of a medicine’ were reviewed: statements were categorised in terms of their specific focus and were then grouped into broad themes.
The survey was administered between August and November 2014 across 16 UK-based patient organisations – collectively representing over 80 rare/genetic conditions – which had agreed to partner this research project. At closure, the survey had yielded 1,160 responses and 893 participants qualified for the survey: 17.2 % had dropped out from the survey before answering any trade-off question and 5.8 % did not complete the trade-off module or survey. Twenty further observations were excluded from the dataset after internal validity checks showed that those respondents had always selected either the first or last alternative in each choice set [see Additional file 1: Appendix A and Additional file 5: Appendix E for further details]. The profiles of respondents who were included and excluded in the analysis were similar in terms of disease and socio-demographic characteristics.
After these exclusions, 873 respondents were available for the final analysis: 721 patients (82.6 % of total sample) and 152 informal caregivers (17.4 %), across a total of 52 rare diseases. Additional file 6: Appendix F presents summary statistics for the 873 respondents included in the final analysis. Just over two-thirds of participants were female (67.5 %), Caucasian (87.9 %) and half the participants were aged between 35 and 54 years. Overall, the three most represented groups of diseases were: ‘rare systemic or rheumatological diseases’ (49.6 %), ‘rare neurologic diseases’ (19.0 %), and ‘inborn errors of metabolism’ (17.9 %). Circa 70 % of respondents concentrated across 10 rare diseases, namely: sarcoidosis (n:418), Fabry disease (n:36), Duchenne muscular dystrophy (n:26), hypopituitarism (n:26), transverse myelitis (n:26), amyotrophic lateral sclerosis (n:24), Pompe disease (n:19), autosomal dominant polycystic kidney disease (n:18), McArdle’s disease (n:16), and mucopolysaccharidosis I (n:16).
Over a third of the surveyed patients (36.5 %) had been diagnosed over ten years ago and 70.7 % of patients (n:510) reported being on treatment for their condition. While 27.3 % of patients were employed full-time, a similar proportion of patients declared being unable to work due to disease-induced disability.
Caregivers were predominantly women (83.6 %) and mostly cared for patients affected by inborn errors of metabolism or rare neurologic diseases. 82.9 % (n:126) of the patients they provide informal care to were reportedly under treatment. Within our sample, one out of five caregivers had to stop working to provide informal care. Whilst most caregivers were providing care as a parent (63.8 %), they were almost equally split between caregivers to a child (48.7 %) and caregivers to an adult (51.3 %). Nearly 40 % of them had been providing informal care for over ten years, and spending over 60 h per week to perform that commitment.
The WHODAS 2.0 distribution of mean scores across our study sample suggests clinically significant disability. While the mean for the patient respondents was 16.6 (SD: 11.7, range 0–48) and that for the caregiver proxies was 26.2 (SD: 13.4), it is noteworthy that only 5 % of responding patients scored 0 – that is, reported no difficulty in any activity. In contrast, 31 % (n:223) scored above 24 and 6 % (n:46) above 36. Although there is no agreed upon cut-off point for identifying individuals with significant disability , scores ranging 24–35 and those above 36 are likely to represent moderate-to-severe and severe-to-extreme disability respectively. The fact that caregivers scored higher than patients seems logical since most caregivers from our study sample concentrate across some of the most debilitating conditions, such as Duchenne muscular dystrophy, amyotrophic lateral sclerosis, or Sanfilippo disease [see Additional file 6: Appendix F]. The other measurement tool used in our survey related to impairment assessment on a scale ranging 0–10 yielded a similar picture. Of note, correlation between these two scales was found to be 78.2 %.
When invited to assess their life expectancy compared to that of other individuals of the same age and gender but unaffected by the rare condition, 19 % of patients (n:134) reported they would expect to live as long as them; 11 % of patients (n:82), however, believed they would have a ‘much shorter life’. The opposite was observed from assessments from caregivers, as 51 % of them were of the opinion that the patients they provide informal care to would die earlier. The box-and-whisker plots available in Additional file 6: Appendix F provide further details on the distribution of the various assessment scores used in this study.
As shown in Table 4 in Additional file 6: Appendix F, ca. 45 % of respondents declared being satisfied with current care, with other 25 % being dissatisfied. Owing to the paucity of therapeutic alternatives to manage rare diseases, only 10.6 % of participants reported an improvement in health condition brought by current care or a cure from the rare disease. Current disease management was reported to result in: no improvement to health condition (10.8 % of respondents); moderate survival extension (8.3 %); improvements in quality of life (18.6 %); symptom management (30 %); disease stabilisation (21.7 %).
Relative importance scores, preference weights & risk-benefit trade-offs
Figure 1 presents for each attribute the mean relative importance value and corresponding two-standard-error interval. The most important attributes of a new medicine for patients and caregivers across our study sample were: ‘the chance that the medicine will work’, followed by ‘the risk of serious side effects’ and ‘the ability to conduct usual activities while on treatment’ (which were both valued equally high), and the ‘expected health improvement’ that the medicine may bring. The least important attributes proved to be ‘treatment duration’, followed by ‘burden of treatment’ – corresponding to two out of the three attributes relating to process of care.
Figure 2 shows the estimated mean preference weight (or part-worth) for each attribute level with the respective two-standard-error interval. The intervals did not overlap, inferring that mean estimates can be considered as statistically different between all the adjacent levels. In general, preferences for the levels under each treatment attribute were consistent with the a priori expectation that levels with better outcomes, lower toxicity, or lower care burden have higher preference weights (i.e. they are more preferred) than levels corresponding to worse outcomes, higher toxicity or greater care burden. Please note that the absolute scale of the part-worths is arbitrary; only relative differences across attributes and attribute levels are meaningful.
This graph conveys several messages.
First, the difference between adjacent importance weights indicates the relative importance of moving from one level of an attribute to an adjacent level of that attribute – the greater the distance, the greater the importance attached to moving from one level to the next. For instance, data suggested that, on average, respondents attached a greater importance to moving from a high risk to a moderate risk of serious side effects [see change from l to k] than to reducing that risk from a moderate to a low level [from k to j]. In contrast, respondents assigned the same value to the opportunity to shift from a monthly inpatient care setting to a weekly outpatient care setting [r to q], as to moving from weekly outpatient care to daily care delivery at home [q to p] and vice-versa.
Second, and most importantly, the difference between adjacent importance weights of one attribute can be compared with the difference between adjacent importance weights of another attribute – thus informing us on possible trade-offs. On average, respondents were willing to accept a 70 % increased risk of facing moderate side effects [g to i] in exchange for a 35 % increased chance to respond favourably to a new medicine [c to b]. Also, to get the opportunity to shift from a state of stabilised disease to actual gains in overall functioning and symptom alleviation [f to e], respondents showed a willingness to accept a 20 % increase in the risk of moderate side effects [g to h] or a 14 % increase in the risk of serious side effects [j to k]. Similarly, within the hypothesis that they would be able to move away from a state of improved functioning to a cure of the disease [e to d], respondents would be ready to give up all of their ability to conduct daily activities while on treatment [t to u], or to face an increase in the risk of serious side effects from 15 to 30 % [k to l].
In summary, when considering the opportunity of a new medicine targeting a rare condition, patients with rare diseases and their caregivers stressed that they attributed most importance to drug response, risk of serious side effects, and the ability to conduct usual activities while on treatment. However, our study suggested that they were prepared to trade a significant amount of risk associated with a new medicine for a higher chance of drug response, or greater health improvement potential.
Effect of respondent-reported disease burden assessment on attribute relative importance scores
A secondary objective of our study was to explore whether ‘disease context’, as measured by respondent-reported assessments on disease-induced impairment, disability, threat to life, satisfaction with current care, current disease management and unmet need had an influence on the relative importance of these attributes.
Figure 3 presents a summary of the regression analyses investigating the effect of disease context on the relative importance of the four attributes that were found earlier as the most important to respondents. The magnitude and direction of the estimated standardized effects are visualized either by an equals sign or by one or two arrows, representing respectively the absence of effect, a small effect, or a large effect. Additional file 5: Appendix E provides more technical details on the statistical methods followed and Additional file 7: Appendix G reports the standardized regression coefficients and related statistics.
Within our study sample across 52 rare diseases, the performed regression analyses described clear relationship patterns between context of disease and attribute relative importance. Data showed that as respondents reported increasing impairment or disability, the value attached to drug response and to health improvement potential increased; in contrast, both the risk of serious side effects and the ability to conduct usual activities became less important to patients or their caregivers. The same findings were observed as the rare disease becomes more life-threatening (with the exception of the ability to conduct daily activities where no relationship was found). Likewise, our data showed that expectations for health improvement brought by a new medicine increase as dissatisfaction with current treatment increases. Lastly, the analysis concluded that the less current disease management improves health condition, the more one is ready to face the risk of serious side effects associated with a new medicine (i.e. the projected value of this attribute decreases); and the more drug response potential becomes important.
An alternative analysis of the regression output – that looked at how often statistical significance and observed effect were found, instead of the observed size of the standardized effect as reported above – yielded similar conclusions. The outcome of this sensitivity analysis can be found in Additional file 7: Appendix G. Of note, the exploratory composite variable related to unmet need proved insensitive to change (whilst its individual components did) and is therefore not shown in Fig. 3. Details on its statistical performance are available in Additional file 5: Appendix E and Additional file 7: Appendix G.
Lastly, we also explored the presence of an effect of disease context on the other three attributes included in our study, namely ‘treatment duration’, ‘burden of treatment’ and ‘risk of moderate side effects’ – that were found the least important attributes for patients and their caregivers. Hardly any effect was found from our review, except to some extent on the attribute related to burden of treatment.
In conclusion, our analysis quantified the influence of disease context on rare disease patients’ and caregivers’ attitudes about benefit-risk. It provides evidence to support a widely shared assumption and a more detailed, robust, understanding of the benefit-risk trade-offs that respondents are ready to make, across different patient populations and respondent types.
Patients’ open-text comments on ‘the value of a medicine’
Circa a third of survey participants took the opportunity to comment at the end of the survey on their personal experience of living with a rare disease, on their own definition of value and on the survey itself. (Table 1) displays a few excerpts from the qualitative feedback collected from patients specifically. More quotations are available in Additional file 8: Appendix H. Overall, five main themes of comments emerged: hope, attitude to risk, healthcare professionals, steroids use, and introspection. All of these qualitative insights – stemming from patients who have learnt to live with a poorly understood, serious and progressing condition with limited treatment options – bring both real-life anchor and colour to the quantitative data that our study generated.
Confronted with the ever approaching outcome of disease progression, impairment or death, patients made a compelling statement of ‘hope’: hope for new medicines, hope for a halt to disease progression, hope for symptom alleviation or partial recovery. In the context of genetic diseases where a cure may reportedly be impossible (at least within their life span), very few patients stated being actually hopeful for a cure or full recovery. This pragmatic stance balancing hope with self-managed expectations may explain why survey respondents during the quantitative trade-off scenarios assigned a greater value to drug response potential than to the actual magnitude of health improvement brought by a new medicine.
A corollary to this ‘pragmatic hope’ is a willingness to take risks. When weighing uncertain benefits brought by a new medicine with unknown risks or outcomes, patients showed an articulate and informed attitude to risk that they modulate according to their personal circumstances of disease seriousness, disease stage, or current disease management. Whilst some patients declared having ‘nothing to lose but everything to gain’ and thus being ready to face serious side effects, unknown outcomes or even death in the hope that a treatment might offer them some benefit, others stated being more reserved. This correlation between risk readiness and disease context expressed across these patient statements echoes our quantitative findings shown in Figs. 2 and 3.
Patients also suggested an eagerness to be actively involved in their own care and to become joint decision-makers in treatment choices. They, however, report an ambivalent relationship with their treating doctors and other healthcare professionals. While ‘regular contact with [doctors] is invaluable’, the latter may prove paternalists and may not always listen to the patient’s feelings or treatment preferences. ‘Doctors need to give us the information so we can decide for ourselves’, some patients reported. The case of high-dose steroids use was mentioned by a number of patients, where they felt they had not been duly informed of their induced comorbidities at time of treatment decision.
Rare diseases are poorly understood, serious, debilitating conditions often characterised by poor prognosis and limited treatment options. Because patients and caregivers constantly navigate the health care system in search of disease management, they gradually gain an intimate knowledge of the rare condition and are thus in a position to identify when a treatment outcome becomes ‘meaningful’ to them and to subsequently articulate trade-offs.
Within the boundaries of our study, rare disease patients and their caregivers chose to adopt a realist view and attributed the highest importance to drug response potential, highlighting the importance of developing new targeted therapies, diagnostic tests and expanding the use of biomarkers [46–49] to better predict health benefit. We may also interpret this finding as a result of ‘pragmatic hope’, as discussed earlier. Next, the other two dimensions of highest value were the risk of serious side effects and the ability to conduct usual activities while on treatment. We consider the latter finding regarding patients’ activities compelling since it is usually overlooked by researchers and clinicians who tend to focus on the pathophysiological mechanisms and consequences of a disease rather than on how the patient feels and lives on a daily basis. In contrast, attributes related to treatment modalities (i.e. how long, where and how to take the medicine) were deemed the least important, although the latter tend to be the focus of many drug development programmes.
Additionally, our study data confirmed that patients and their caregivers were willing to accept greater risk or side effects associated with a new medicine, for instance, in the hope for some extra chance in drug response or greater health improvement potential. And, as demonstrated by the outcome from the regression analyses performed, attitudes about benefit-risk may change over time with disease progression or context of care. Our findings are consistent with the analysis by social scientists who defined risk as a ‘social construction’ rather than an objective and measurable function of the probability and magnitude of an event [50, 51]. As Douglas pointed out, ‘risk is not only the probability of an event but also the probable magnitude of its outcome, and everything depends on the value that is set on the outcome. The evaluation is a political, aesthetic and moral matter’ . We may wish to add the word ‘contextual’ to that definition to bring in the dimensions of unmet need and disease progression.
Overall, our research findings – which are reminiscent of the work by Kesselheim  or Peay  – highlight the need to systematically include patients in the process of identifying meaningful treatment outcomes that resonate with their experience, preferences, expectations and values. As stated by Dr. Janet Woodcock from the U.S. FDA ‘it is clear you have to start with an understanding of the impact of the disease on the people who have it, and what they value most in terms of alleviation before you set up a measurement and go forward with truly patient-focused drug development’ .
In that context, we believe that patients and caregivers should be increasingly involved as active research partners in the development of clinical outcomes assessments – including patient-reported outcomes (PRO) measures – that directly evaluate how the patient feels, functions or survives [56–58]. If a treatment effect is not meaningful to the patient, it is not a benefit to the patient. Additionally, we believe that patient preference data informing the development of those assessments should be treated with the same level of scientific rigour as currently with clinical data. Despite their different focus, if guided by a clear hypothesis-led strategy, both patient preference and PRO data offer complementary evidence to substantiate label claims of ‘significant benefit’ or ‘major contribution to patient care’. From that perspective, researchers of novel therapeutics for rare diseases should be encouraged to invest in the use of such methods and specific regulatory guidance should be issued to acknowledge their importance and to state where these methods may fit into drug development programmes and regulatory decision-making.
Patient-centred outcomes are key complements to traditional clinical data that can purposefully help detect meaningful and interpretable changes in health status and treatment benefit. Efforts from the research community to promote the development and use of patient-centred outcome measures in rare diseases, such as the current work by the International Rare Disease Research Consortium (IRDiRC) are, in that context, promising .
This study includes a number of limitations; the four main ones are discussed here. First, as a result of its ambition to review the preferences and values across over 50 rare conditions, the trade-offs module of our survey applied non-disease-specific, generic attributes. This limitation in the design of our study, however, was reviewed and validated by our partners from the patient organisations, and the survey was deemed appropriate for use across different disease profiles. Second, our analysis relied on observations across a subset of rare diseases – 52 of them – and an uneven number of patients by disease, in part reflecting differences in prevalence. Additionally, the diseases included in our study are very different from each other and variations within each disease are also observed. The reality of that limitation was acknowledged at time of study design and justified the use of the three alternative self- / proxy-reported assessment measurements to address it. In addition to the individual estimates yielded from our DCE model, individual differences in disease severity across the variety of diseases were thus duly taken into account in our regression analyses. Third, our study sample is limited to patients and caregivers living in the United Kingdom. Since patient autonomy has become the dominant principle shaping physician-patient relationships but remains culture-dependent, our study results may not apply as a proxy for patient preferences in another national healthcare setting. Fourth, our analysis relied on the pooled data of patients’ and caregivers’ preferences: comparative results between respondent types validated that choice.
To the best of the authors’ knowledge, this study – made possible thanks to the sustained commitment from sixteen UK-based patient organisations – is the first study performing a quantitative discrete choice experiment amongst patients and their caregivers across 52 rare conditions.
Conceptually anchored to the overarching goal of incorporating the patients’ perspectives into drug development and approval processes, our study aimed to better understand and quantify the preferences and values of rare disease patients and families and to explore to what extent the specific context of disease – such as disease-induced disability or the perceived medical added-value of currently available treatments – may alter these preferences and values. This study offers an alternative approach to generate robust evidence about patients’ preferences and values from a wide range of patients that can ultimately input to complete drug development and approval decision-making processes.
Eichler HG, Abadie E, Baker M, Rasi G. Fifty years after thalidomide; what role for drug regulators? Br J Clin Pharmacol. 2012;74:731–3.
Eichler HG, Bloechl-Daum B, Brasseur D, Breckenridge A, Leufkens H, Raine J, et al. The risks of risk aversion in drug regulation. Nat Rev Drug Discov. 2013;12:907–16.
Chakradhar S. Training on trials: patients taught the language of drug development. Nat Med. 2015;21:209–10.
Pushparajah DS, Geissler J, Westergaard N. EUPATI: collaboration between patients, academia and industry to champion the infromed patient in the research and development of medicines. J Med DevSci. 2015;1:74–80.
Barry MJ, Edgman-Levitan S. Shared decision making--pinnacle of patient-centered care. N Engl J Med. 2012;366:780–1.
Stacey D, Legare F, Col NF, Bennett CL, Barry MJ, Eden KB, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2014;1:CD001431.
Minvielle E, Waelli M, Sicotte C, Kimberly JR. Managing customization in health care: a framework derived from the services sector literature. Health Policy. 2014;117:216–27.
Eaton S, Collins A, Coulter A, Elwyn G, Grazin N, Roberts S. Putting patients first. BMJ. 2012;344:e2006.
Coulter A, Collins A. Making shared decision-making a reality: no decision about me, without me. London: The King’s Fund; 2011.
Health Technology Assessment international (HTAi). Completing a Patient Group Submission Template: Guidance for Patient Organisations for Health Technology Assessment and Appraisal of Medicines. 2015. Health Technology Assessment international (HTAi).
European Medicines Agency (EMA). Road map to 2015. The european medicines Agency’s contribution to science, medicines and health. London: European Medicines Agency (EMA); 2010.
European Medicines Agency (EMA). Incorporating patients’ views during evaluation of benefit-risk by the EMA scientific committees. EMA/413422/2013. London: European Medicines Agency (EMA); 2014.
European Medicines Agency (EMA). Pilot phase to involve patients in benefit/risk discussions at CHMP meetings. EMA/372554/2014. London: European Medicines Agency (EMA); 2014.
European Medicines Agency (EMA). Final CHMP work programme for 2011–2013 . EMA/CHMP/65166/2011. London: European Medicines Agency (EMA); 2011.
US Department of Health and Human Services FaDAF. Structured approach to benefit-risk assessment in drug regulatory decision-making. Draft PDUFA V implementation plan. Washington: Food and Drug Administration (FDA); 2013. 16-1-2015.
US Department of Health and Human Services FaDAF. Prescription drug user Fee Act patient-focused drug development; announcement of disease areas for meetings conducted in fiscal years 2013–2015. Vol. 78, No. 70. 11-4-2013. Washington: Federal Register; 2015.
European Medicines Agency (EMA). Regulatory and methodological standards to improve benefit-risk evaluation of medicines. Workshop report, EMA/141854/2014. London: European Medicines Agency (EMA); 2014.
Genetic Alliance UK. New medicines for serious conditions: weighing the risks and benefits. The verdict of a jury of patients. London: Genetic Alliance UK; 2012. 16-1-2015.
Parent Project Muscular Dystrophy. The Duchenne community imperatives for the guidance for industry on Duchenne muscular dystrophy: developing drugs for treatment over the spectrum of disease. Hackensak: Parent Project Muscular Dystrophy; 2014. 16-1-2015.
US Department of Health and Human Services FaDAF. Report: complex issues in developing drugs and biological products for rare diseases and accelerating the development of therapies for pediatric rare diseases, including strategic plan: accelerating the development of therapies for pediatric rare diseases. Washington: Food and Drug Administration (FDA); 2014.
de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21:145–72.
Bridges JF, Hauber AB, Marshall D, Lloyd A, Prosser LA, Regier DA, et al. Conjoint analysis applications in health--a checklist: a report of the ISPOR good research practices for conjoint analysis task force. Value Health. 2011;14:403–13.
Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy. 2003;2:55–64.
Marshall D, Bridges JF, Hauber B, Cameron R, Donnalley L, Fyie K, et al. Conjoint analysis applications in health - How are studies being designed and reported?: an update on current practice in the published literature between 2005 and 2008. Patient. 2010;3:249–56.
Ryan M, Farrar S. Using conjoint analysis to elicit preferences for health care. BMJ. 2000;320:1530–3.
Johnson FR. Why not ask?: measuring patient preferences for healthcare decision making. Patient. 2008;1:245–8.
Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making: a user’s guide. Pharmacoecon. 2008;26:661–77.
Bridges JF, Kinter ET, Kidane L, Heinzen RR, McCormick C. Things are looking up since we started listening to patients: trends in the application of conjoint analysis in health 1982–2007. Patient. 2008;1:273–82.
Harrison M, Rigby D, Vass C, Flynn T, Louviere J, Payne K. Risk as an attribute in discrete choice experiments: a systematic review of the literature. Patient. 2014;7:151–70.
Hauber AB, Fairchild AO, Reed J. Quantifying benefit-risk preferences for medical interventions: an overview of a growing empirical literature. Appl Health Econ Health Policy. 2013;11:319–29.
Train K. Discrete choice methods with simulation. Cambridge: Cambridge University Press; 2009.
Hensher DA, Greene WH. The Mixed Logit model: the state of practice. Transportation. 2003;30:133–76.
Ryan M, Gerard K, Amaya-Amaya M. Using discrete choice experiments to value health and health care. Dordrecht: Springer; 2008.
Louviere J, Hensher D, Swait J. Stated choice methods: analysis and applications. Cambridge: Cambridge University Press; 2000.
Hensher DA, Rose J, Greene W. Applied choice analysis: a primer. Cambridge: Cambridge University Press; 2005.
Pickard AS, Knight SJ. Proxy evaluation of health-related quality of life: a conceptual framework for understanding multiple proxy perspectives. Med Care. 2005;43:493–9.
Ustun TB, Chatterji S, Kostanjsek N, Rehm J, Kennedy C, Epping-Jordan J, et al. Developing the world health organization disability assessment schedule 2.0. Bull World Health Organ. 2010;88:815–23.
Ustun TB, Kastanjsek N, Chatterji S, Rehm J. Measuring health and disability. Manual for the WHO disability assessment schedule. WHODAS 2.0. Geneva: World Health Organisation; 2010.
Bliemer MCJ, Rose JM. Construction of experimental designs for mixed logit models allowing for correlation across choice observations. Transp Res B. 2010;44:720–34.
Kessels R, Goos P, Vandebroek M. A comparison of criteria to design efficient choice experiments. J Mark Res. 2006;43:419.
Orme J. Getting started with conjoint analysis: strategies for product design and pricing research. Madison: Research Publishers LLC; 2006.
Yu J, Goos P, Vandebroek M. Efficient conjoint choice designs in the presence of respondent heterogeneity. Mark Sci. 2009;28:122–35.
Yu J, Goos P, Vandebroek M. Individually adapted sequential Bayesian conjoint-choice designs in the presence of consumer heterogeneity. Int J Res Mark. 2011;28:378–88.
Yu J, Goos P, Vandebroek M. A comparison of different Bayesian design criteria for setting up stated preference studies. Transp Res B. 2012;46:789–807.
Andrews G, Kemp A, Sunderland M, Von KM, Ustun TB. Normative data for the 12 item WHO disability assessment schedule 2.0. PLoS One. 2009;4:e8343.
Kakkis ED, O’Donovan M, Cox G, Hayes M, Goodsaid F, Tandon PK, et al. Recommendations for the development of rare disease drugs using the accelerated approval pathway and for qualifying biomarkers as primary endpoints. Orphanet J Rare Dis. 2015;10:16.
Schork NJ. Personalized medicine: time for one-person trials. Nature. 2015;520:609–11.
US Department of Health and Human Services FaDAF. Paving the Way for Personalized Medicine. FDA’s Role in a New Era of Medical Product Development. Washington D.C., USA: U.S. Food and Drug Administration; 2013.
National Organization for Rare Disorders (NORD). Letter to the U.S. Senate HELP Committee in support of the Advancing Targeted Therapies for Rare Diseases Act. 2015.
Douglas M, Wildavsky A. Risk and culture: an essay on the selection of technical and environmental dangers. Berkeley: Berkeley: University of California Press; 1982.
Krimsky S, Golding D. Social theories of risk. Westport: Praeger-Greenwood; 1992.
Douglas M. Risk as a forensic resource. Daedalus Proc Am Acad Arts Sci. 1990;119:1–16.
Kesselheim AS, McGraw S, Thompson L, O’Keefe K, Gagne JJ. Development and use of new therapeutics for rare diseases: views from patients, caregivers, and advocates. Patient. 2015;8:75–84.
Peay HL, Hollin I, Fischer R, Bridges JF. A community-engaged approach to quantifying caregiver preferences for the benefits and risks of emerging therapies for Duchenne muscular dystrophy. Clin Ther. 2014;36:624–37.
Woodcock J. PDUFA V Clinical Outcome Assessments Public workshop. 2015.
Walton MK, Powers III JH, Hobart J, Patrick D, Marquis P, Vamvakas S, et al. Clinical outcome assessments: conceptual foundation-report of the ISPOR clinical outcomes assessment - emerging good practices for outcomes research task force. Value Health. 2015;18:741–52.
Frank L, Basch E, Selby JV. The PCORI perspective on patient-centered outcomes research. JAMA. 2014;312:1513–4.
Bartlett SJ, Barnes T, McIvor RA. Integrating patients into meaningful real-world research. Ann Am Thorac Soc. 2014;11 Suppl 2:S112–7.
International Rare Diseases Research Consortium (IRDiRC). Preparatory Document for Workshop on Patient-Centred Outcome Measures Initiatives in the Field of Rare Diseases. 2-10-2015.
We are very grateful to the sixteen participating UK-based patient organisations for their sustained support to this study, namely: Action Duchenne; the Association for Glycogen Storage Disease UK; the AKU Society; Ataxia UK; Behçet's Syndrome Society; the Brittle Bone Society; the Facioscapulohumeral Muscular Dystrophy Support Group (FSH-MD); the Myotonic Dystrophy Support Group; the Motor Neurone Disease Association (MNDA); the Society for Mucopolysaccharide Diseases; Myaware; the Polycystic Kidney Disease Charity; the Sickle Cell Society; the Sarcoidosis and Interstitial Lung Association (SILA); the Pituitary Foundation; the Transverse Myelitis Society.
We also address our special thanks to the following individuals for their advice and feedback on study design or manuscript draft: Barbara Babcock (Transverse Myelitis Society), Thomas Butt (University College London), Stefan Cano, Marc Dooms (Leuven University Hospital), Francesca Gatta, Sam Geuens (Leuven University Hospital), Josie Godfrey (NICE), Tristan Gonzalez, Julie Greenfield (Ataxia UK), Bernadette Hafkenscheid, Christine Lavery (MPS Society), The Orphanet Inserm US-14 team, Patricia Osborne (Brittle Bone Society), Menai Owen-Jones (The Pituitary Foundation), Samantha Price (MNDA), Angela Stringer (Action Duchenne), Durhane Wong-Rieger (Canadian Organization for Rare Disorders); the World Health Organization WHODAS 2.0 team.
We would like to thank Ryo Sakai for his help on the visuals included in this manuscript.
TM designed the study, in consultation with all authors (DC, MV, SA, MM, SS). TM, MM and SA organised study recruitment and ethics compliance. TM and MV designed the statistical model and analysis plan. TM managed data collection and liaison with the patient organisations. TM, MV and DC analysed the study data and interpreted the results, which were validated by the other authors. TM wrote the draft manuscript, which was then revised by all other authors. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Study Recruitment Flow & Study Sample. (DOCX 56 kb)
Self-/ Proxy- Reported Instruments used in the survey. (DOCX 67 kb)
Attributes and Levels used in the DCE. (DOCX 23 kb)
Experimental Design. (DOCX 24 kb)
Tests of Internal Validity & Statistical Analysis. (DOCX 23 kb)
Study Sample. (DOCX 133 kb)
Mixed logit estimates & regression output. (DOCX 186 kb)
Patients’ quotations on ‘the value of a medicine’. (DOCX 25 kb)
References used in supplementary material. (DOCX 21 kb)
About this article
Cite this article
Morel, T., Aymé, S., Cassiman, D. et al. Quantifying benefit-risk preferences for new medicines in rare disease patients and caregivers. Orphanet J Rare Dis 11, 70 (2016). https://doi.org/10.1186/s13023-016-0444-9
- Rare diseases
- Risk tolerance
- Discrete choice experiment
- Qualitative survey
- Patient-centered outcomes
- Patient-reported outcomes
- Drug development