Table 2 Study characteristics

Bendixen et al. [33]

USA

To investigate family-centred and physician-based attitudes and perceptions, to improve the future design of Duchenne Muscular Dystrophy clinical research protocols and improve participation in future clinical trials

Sites involved in the Cooperative International Neuromuscular Research Group, a clinical research academic consortium, and associated Muscular Dystrophy Association clinics. 5 geographically and demographically diverse sites (Pittsburgh, Pennsylvania; Washington, DC; Houston, Texas; Minneapolis, Minnesota; and Sacramento, California) with varying levels of DMD research recruitment and participation were selected to maximize variability. e.g. sites varied from being active and engaged in numerous clinical studies in DMD to having had limited research involvement. Recruitment was focused on parents (primary care- givers) of boys with DMD. The inclusion criteria for parents: were limited to having a child with DMD, the ability to understand and speak English, and a willingness to participate in a 1-time-only focus-group interview lasting up to 120 min

Institutional Review Board, University of Pittsburgh (14010024). Written informed consent was obtained from all parents and clinicians before participating in this study

28 parents and 33 clinicians completed the 13 focus group sessions

Focus-group sessions were conducted between July 2014 and October 2014 at each local site in central locations convenient to the participants. Parents in the DMD focus groups received $25 remuneration for participation and reimbursement for transportation. All participants received a free meal and the option of child care during the session

Thematic analysis was used to identify patterns of meaning across the datasets (Braun et al. 2006). Patterns within the datasets were identified through a rigorous approach of familiarization with the data through reading and rereading each transcription by 3 researchers (Shenton t al 2004). The dataset was then hand-coded line by line to identify important features of the data. Patterns within the data were synthesized and reduced to themes (Boeije 2010). Data relevant to each theme were reviewed and discussed at biweekly data-analysis meetings. Themes were refined and rephrased to describe the story being told by the parents and clinicians. We emailed each parent and clinician participant and provided a table of the thematic findings. Each participant was given the opportunity to provide feedback or comments

6 of 28 parents responded with affirmation and appreciation that their participation was beneficial

Information: There were issue of an over-abundance of information that was fragmented, difficult to obtain, or difficult to understand

The approach of using registries for recruitment requires individuals to log on to a website and provide their (child’s) medical history, typically without any contact with clinical staff. This passive strategy may be ideal for highly motivated or informed volunteers who are specifically interested in research

Recruitment opportunities for minorities are limited due to a belief that they will be non-compliant or unable to fulfil the research objectives

Conversation: The importance of regular communication, follow-up with potential participants, and accessibility to research staff was evident

Barriers: Burdensome travel commitments, especially with a disabled child in an unfamiliar city

Not meeting the inclusion criteria for enrolment

Incentives: Importance of providing rewards and incentives for the child participants. Acknowledgement that the participants are giving their time

Solutions: Use of peer support, social media, and educational outreach

Carroll et al. [7]

USA

To understand the motivations of patients with Pulmonary Arterial Hypertension for participating in RCTs so as to facilitate enrollment in future trials among patients with similar diseases

Participants recruited from the Pulmonary Vascular Disease Program at the University of Pennsylvania

Institutional Review Board of the University of Pennsylvania (810,120)

Semi-structured interviews with 26 participants, using a vignette of a hypothetical trial. Interviews (lasted 10–20 min)

Thematic data analysis and constant comparison techniques (Strauss and Corbin 1998; Ryan 2003)

Medical Benefits: Participants expressed hope that participation in RCTs of novel therapies for PAH would result in personal benefit

Medical risks of harm: Participants were concerned about the side effects of experimental drugs, and consequences of forgoing their usual treatments

Non-medical benefits: altruism; compensation/reimbursement

Non-medical burdens: travel to and from study appointments

Gaasterland et al. [42]

Europe

“We present the POWER-tool (an acronym of Patient participation in Outcome measure Weighting for Rare diseases).”

Asterix Patient Think Tank: 10 patients who had been educated about clinical research (patient representatives in the area of (rare)cancers, Duchenne Muscular Dystrophy (DMD), Mucopolysacchari-doses (MPS), Alkaptonuria (AKU), Hemophillia, Primary Sclerosing Cholangitis (PSC), Cystic Fibrosis, and Fragile X syndrome, as wellas a representative of EURORDIS)

Focus group: 25 patients with Spinal Muscular Atrophy, who had recently participated in a randomized clinical cross-over trial

Not reported

Patient Think Tank (n = 10), and a Focus Group (n = 25)

Quotes from Focus Group provided and three topics identified. MAXQDA used

Patient outcome measures: Patients were very comfortable in talking about the practical aspects and constraints of their disease, but it was more difficult for them to answer the question of how to measure these aspects

“We decided that the best results would be achieved when researchers translate the patient’s preferences in outcomes, that are formulated in the first meeting, into measurement instruments and a trial protocol which can then be evaluated again with patient representatives during the second meeting.”

Subsequent presentation of a model for involving patient representatives in choosing measures during rare disease clinical trials (The POWER tool)

Gaasterland et al. [43]

Europe

To investigate patients views on clinical trial design

10 educated rare disease patient representatives [as above]

The Medical Ethics Review Committee of the Academic Medical Center has confirmed that the Medical Research Involving Human Subjects Act does not apply for this study (W17–217 # 17.249)

Interviews were conducted (n = 10) according to a pre-set interview guide

Grounded theory using Thematic Analysis and MAXQDA. Topics were structured in chronological order

Involvement in trial design: Patient organizations in which participants are involved have initiated trials, whereas other participants felt that their patient organizations were not involved in the setup of a trial early enough. These patient organizations were only approached when the trial was already recruiting, as a source of participants. Participants wanted patient organizations to be involved in the trial at an early stage

Opinions on trial design: Participants feel they should be involved in the choice of clinically meaningful outcome measures, because patients know which outcomes are most relevant. Some thought trials that they were involved in were too short to show an effect of an intervention. Some wanted to decrease the chance of being allocated to the placebo arm of a trial. Participants do not always understand the consequences of random allocation

Trial participation: It should be more widely known that trials are performed in hospitals in order to generate evidence about treatments. Participants need to be clear about the consequences of participation and information does not mention the practical aspects which are relevant to patients

Participants make a risk–benefit analysis before taking part

Participants communicate with each other and discuss treatments. Some measures are burdensome and intrusive

After the trial: Results of a study should be clearly communicated to participants

Gengler [35]

USA

How families with a child with a rare disease accessed and negotiated care at a top 10 ranked university hospital

Strategic case selection at psuedonymised hospital. Cases reflected a variety of life-threatening childhood illnesses, and families from across geographical, educational racial, and class backgrounds

Not reported. Reported that families gave consent

Family centred ethnographic approach (Lareau 2003): Interviews and observations with 18 families, over 18 months

Grounded theory (Charmaz 2006); and ethnographic methods (Tavory and Timmermans 2009; Prus 1987; Schwarlbe et al. 2000; Flyvberg 2001)

Getting access: [With Cultural Health Capital] “Todd and Savannah Marin are a case in point. When their then 6-month-old son, Jacob, was diagnosed with Tay-Sachs—a rare, fatal, genetic, degenerative neurological disorder—Savannah, a white, 32-year-old, first-time mother with a bachelor’s degree in nursing, and Todd, a white, 35-year old contractor, were devastated to learn that the only option for Jacob in their West Coast home state was palliative hospice care. Todd recalled the diagnosing physician telling them, “Go look for clinical trials—‘you go do your homework and I’ll do mine.’” But at their next visit, Savannah reported, “He hadn’t done anything. Like... he had printed out another sheet from his database...and told us, ‘Oh, looks like he has 2–4 years to live.’” Unwilling to accept this outcome, the Marin’s “did their homework” and scoured FDA databases for clinical trials.”

Kesselheim et al. [36]

USA

To explore rare disease patients’, caregivers’, and advocates’ experiences with their conditions and the health care system, in addition to their perspectives on drug development

Participants recruited via the National Organization for Rare Diseases

Not reported

Three in-person focus groups, involving rare disease patients (n = 9), caregivers (n = 8), and advocates (n = 9)

Grounded theory (Bradley et al. 2007), using Atlas.ti

Concerns about clinical trial enrolment: An important barrier for those with rare diseases, was the perceived loss of an opportunity for treatment when assigned to a control arm in a randomized trial that compared a new drug with a placebo

Narrow eligibility criteria: limits to their access to clinical trials, due to exclusion due to co-morbidities or existing medications

Trial outcomes: Failure to account for quality of life or outcomes driven by patients and their families

Approvals: Participants were concerned that the development and testing of therapies should, as quickly as possible, yield effective treatments to advance their quality of life

Lopes et al. [44]

Brazil

To evaluate vulnerabilities and suggest approaches for rare disease (RD) diagnosis and treatment in Brazil based on the perceptions of those involved in the process: patients, caregivers, patient support groups, non-governmental organizations and primary and tertiary care professionals

Focus groups with 27 participants: Patients and primary care givers (n = 7), non-governmental associations and organizations (n = 9), primary care professionals (n = 4), physician specialists (n = 7)

Ethics Committee for Review of Research Projects (Comitê de Ética para Análise de Projetos de Pesquisa—CAPPesq) of Hospital das Clínicas of the FMUSP (268.417)

Non-random Sampling (n = 7). After the study objectives were presented, all the participants signed an informed consent form

Researchers involved in the study served as moderators and reporters in each group to ensure procedural homogeneity

A script containing guidelines was presented to each group before the session was initiated

Reports were recorded and transcribed in full and served as a basis for the thematic and categorical content analysis (Bomfim 2009, Oliveira 2008). Thematic units were coded and processed with NVivo 10 software, which allowed us to map their distribution among the different study groups (Bardin 2007, QRS 2013). For better reliability, data triangulation was used to achieve the highest degree of convergence among the researchers’ perceptions

Diagnosis: The patient’s journey does not end with the diagnosis of the disease; another significant obstacle faces health personnel and relatives after diagnosis: the challenge of searching for adequate treatment, which, at least in Brazil, is far from straightforward. The Brazilian public health system has been unable to meet the needs of RD patients who, due to the multiplicity of these diseases and their rarity, are frequently unable to organize a strong and representative support group

Treatment: Regarding medications for RDs or ‘‘orphan drugs’’, there is a lack of incentive for the national pharmaceutical industry to conduct research and testing related to the manufacture of these medications. Consequently, these drugs must be imported, which leads to greater public costs

Additional needs of RD patients could be met if appropriate technologies were directed toward research aimed at developing new, more accessible and affordable therapies

Li et al. [45]

China

To assess the unmet needs of rare disease patient organizations in China, and identify their unmet needs, providing essential information for the government and legislators

28 participants, representing 28 patient organizations for rare diseases

Institutional Ethics Committee of the Guangzhou Medical University, China. All interviewees signed the informed consent and agreed to participate in this study voluntarily

Participants (n = 28) were recruited through online advertisements or personal references. Interviews were conducted by phone and each participant asked 18 questions

Common themes from these transcripts were analysed in this study

Research: Patient organizations have not been able to establish registries or sponsor research due to lack of financial support. None of the participants in the study had been involved in research

Menon et al. [38]

Canada

The aims of this study were (1) to explore opportunities for patient involvement in reducing decision uncertainties throughout the lifecycle of orphan and ultra orphan drugs from the perspectives of patients within the Canadian rare disease community; and (2) to develop a policy framework for patient input that maximizes the impact of their involvement on decision uncertainties around orphan and ultra-orphan drugs

Two one-day conferences and four workshops involving patients and/or families from rare disease communities in Canada were held to discuss issues around orphan and ultra-orphan drug development, access, and coverage, and identify opportunities for patient input to reduce related decision uncertainties

The project was approved by the University of Alberta Health Research Ethics Board

Conference #1: Presentations were followed by small group sessions with participants from patient communities (n = 60), who were asked to discuss the goals of an ‘ideal’ process for managing the development and introduction of new therapies

Conference #2: Participants included patients and families, clinical specialists in rare diseases, and representatives from industry, Health Canada (federal regulatory body), and the provincial governments (payers). Of the patient participants (n = 69), most had attended the first conference

First 2 workshops:

Patients an family who had attended conference # 2 (n = 30), to explore roles specifically for patients, families, and patient organizations in reducing decision uncertainties throughout the technology lifecycle

Second 2 workshops: All patients and families attending the fora were invited to participate in the workshops (Toronto, n = 23; Vancouver, n = 18). None had been involved in the previous sessions

Transcripts from all four workshops were analysed qualitatively using a general inductive approach (Ritchie et al. 1994). 2 researchers first read through all of the transcripts and developed initial coding categories, which represented potential themes. Chunks of text were then assigned to one or more of these categories

Patient registries: Are seen as an important tool to monitor rare diseases and their treatment. Registries would be an efficient way of getting large enough numbers of patients for meaningful statistics, the lack of which is often an obstacle to getting an orphan drug funded

Reimbursement process: In Canada, coverage for orphan and ultra-orphan drugs is viewed by patients and families as one of the main issues affecting the rare disease community

Other complaints included a lack of an individual patient’s voice during reimbursement review meetings, the absence of a truly fair appeal process and opacity in the decision parameters that seem to be used by committees

Value Definition: As experts in their own disease, patients and families felt that they were best able to judge improvements in or worsening health. The choice of outcome measures in trials often reflects what is easy to measure, rather than what is of value to patients and families. As a result, the trials fail to capture the true value of a therapy

Clinical Trials: ‘‘Because we’re talking about all the problems that happen after clinical trials are designed by people who know the science and the industry, but don’t know the disease and that’s the problem. We’re dealing with the problems because we’re not included before the trial begins.’’

Patients and families were frustrated by the lack of opportunities for patient input into the design of trials, given that they are increasingly being asked to help identify potential patients or participate in the trials themselves

Benefit –Harm Assessment:

Throughout the technology

lifecycle, decisions around what constitutes acceptable harm in order to achieve a certain magnitude of benefit are often made with little input from patients. The existing paternalistic approach needs to be replaced with one that recognizes patients and families as empowered equal partners in such decisions

Early Studies and Drug Discovery (R&D): For many patients and families living with rare diseases, there are no treatments beyond supportive care. Therefore, the need for research that could lead to the development of new therapies was a dominant theme shared by all 4 workshops. Given that resources available for research are limited, patients and families felt that they should have a role in setting research funding priorities to ensure that decisions reflect a comprehensive understanding of the potential implications of different proposals

Managed Access: comprises a conditional funding option, which makes therapies available to eligible patients on the basis of an agreement. The terms of which may include collection of certain types of data or specific clinical or financial outcomes that must be achieved in order to receive continued funding

Peay et al. [37]

USA

Although our initial aim was to explore the experience of parents and clinician investigators involved in a clinical trial for a rare disorder, we were also able to explore participation in a trial that came to an abrupt, unexpected end

6 fathers and 6 mothers of 11 boys with Duchenne and Becker Muscular Dystrophy (including 1 mother–father pair) and 9 clinical investigators. All had participated in the phase II clinical trial. All participated at US study sites. Recruited through advocacy organisations and snowballing

Not reported

Semi structured telephone interviews with clinical investigators and parents of sons with DBMD who participated in the phase IIa or IIb ataluren clinical trial in the United States. The topics explored during the interviews – experiences in the trial, hopes, and expectations; perceptions of benefit; and relationships among stakeholders – were informed by the literature and clinical and anecdotal experience. Because these sources suggested that expectations and hopes for a clinical trial may differ, we asked participants to describe both their hopes and expectations

Using NVivo 8 QSR, the responses were analyzed by two independent investigators (T.F. and E.B.) to ensure coding consistency and high intercoder reliability

Discrepancies in the coding were discussed until reconciliation was achieved. All analyses were based on consensus codes. We conducted thematic analysis within and between the parent group and the clinician investigator group. Major themes that arose from the analysis and illustrative quotes are presented

Expectations and hopes for the clinical trial: Parents reported and demonstrated being well informed about the trial. All reported expecting some direct benefit of the drug, usually described as slowing or stabilizing progression of the disorder

Motivations and decision making: Parents’ primary motivation for enrolling was the potential for benefit. Less than 1/2 of the parents mentioned altruism. Most of the parents reported an easy decision or ‘non-decision’ to join the clinical trial

Pressures of a progressive disorder: Parents spoke about the pressures of a progressive, fatal disorder, and how these pressures played a role in decisions about and expectations of clinical trials

Perceptions of benefits: The parents delineated direct and indirect benefits of trial participation. All reported some degree of direct benefit for their boys, ranging from obvious improvements to subtle changes. These benefits included improved strength, endurance, and cognitive performance. A few parents described being unsure about whether there was benefit until they noted declines following the sudden end of access to the drug

Reactions to trial ending: Parents reported anger, shock, and distress when the trial was stopped. Several parents noted the need to better prepare participants for the possibility of a trial ending abruptly

Tingley et al. [39]

Canada

To integrate perspectives from published literature and key rare disease stakeholders to better understand the perceived challenges and proposed methodological approaches to research on clinical interventions for rare diseases

Recruitment invitations were distributed by email to physician members of the Garrod Association (a professional association whose members are involved in caring for patients with inherited metabolic diseases), to policy advisors by a member of their professional network (using publicly available contact information), and to patients/ caregivers attending the Canadian MPS Society’s 2017 Annual Family Meeting

Individuals interested in participating were instructed to contact a member of the research team (KT), and eligible respondents were asked to provide signed, informed consent to participate in the study

Approved by the Ottawa Health Science Network Research Ethics Board and the Children’s Hospital of Eastern Ontario Research Ethics Board (physicians and policy advisors), and the University of Ottawa Health Sciences and Sciences Research Ethics Board (patients/caregivers)

Literature review, plus: Focus group interviews were conducted by telephone with the physicians (n = 6) and policy advisors (n = 3), and in-person with the patients/caregivers (n = 4) in conjunction with the Canadian MPS Society’s 2017 Annual Family Meeting held in Montreal, QC, Canada

Each focus group transcript was analyzed using a qualitative descriptive approach that is aimed at “obtaining straight and largely unadorned (i.e., minimally theorized or otherwise transformed or spun) answers to questions of special relevance to practitioners and policy makers” (Sandalowski 2000). Four members of the study team (KT, BP, DC, IG) met to identify the key concepts and themes that were present in the focus group data. These concepts/ themes were organized into a coding system that was applied by one study team member (KT) using NVivo 10 Software (QSR International Pty Ltd.) and reviewed by a second member (BP) for credibility and trustworthiness (Shenton 2004)

Explanatory evidence generation: Participants in our focus groups highlighted the limited feasibility of conventional RCTs because of small sample sizes, but there was little emphasis on specific strategies that might be used to overcome this challenge

There can be a lack of patient/ family/ clinician acceptance of the possibility of being randomized to a control group, particularly for placebo controlled studies of treatments for rare diseases where few treatment alternatives exist. Therefore, study designs that make participation more appealing by maximizing time spent on- or guaranteeing provision of- the active treatment have been suggested

Studies designed to evaluate the efficacy of an intervention typically limit enrolment to a very homogenous group of participants, which strengthens the robustness of the causal interpretation of the findings, but at the expense of a reduction in the external validity or generalizability of study results

Comparative effectiveness/ pragmatic evidence generation: clinical heterogeneity is often not accounted for in conventional RCTs, and has raised concern among stakeholders about the applicability of study results to patients with clinical manifestations different from those included in RCTs

Several focus group participants have advocated for study designs that may compromise internal validity to some extent, by shifting away from the explanatory RCT, in order to address real-world effectiveness

Participants questioned the suitability of explanatory RCTs for establishing effectiveness of clinical interventions for rare diseases, but little of the discussion focused on specific solutions to overcome this challenge

Patient-oriented evidence generation: One of the main criticisms, by focus group participants, is of highly internally valid, explanatory study designs is their tendency to rely on short term, and often surrogate, outcomes that are not necessarily clinically meaningful. Many outcome measures, including patient-oriented outcome measures, have not been validated or standardized for the population of interest, leading to questions about the applicability of study results

Some participants expressed concern about balancing subjective outcomes (e.g., patient-reported quality of life) with more objective outcomes (e.g., biomarkers of disease progression) because of possible placebo effects with patient-reported outcomes

Tingley et al. [40]

Canada

To understand why and how patients and families with rare metabolic diseases, specialist metabolic physicians, and health policy advisors choose whether to participate in studies and how they use and value research

We distributed recruitment invitations by email to physician members of the Garrod Association (a professional organization whose members are involved in caring for children with IMD), to policy advisors by a member of their professional network using publicly available contact information, and to patients/caregivers attending the Canadian MPS’s Society’s 2017 annual Family Meeting

Individuals interested in participating in the focus groups contacted the lead author (KT) for more information and to confirm eligibility. Eligible respondents were asked to provide signed informed consent to participate in the study

Approved by the Ottawa Health Science Network Research Ethics Board, the Children’s Hospital of Eastern Ontario Research Ethics Board, and the University of Ottawa Health Sciences and Sciences Research Ethics Board. Informed consent was received from all participants in this study

Focus group were held separately for each stakeholder group; the patient and family focus group was conducted in person in conjunction with the Canadian MPS Society’s 2017 Annual Family Meeting, We reasoned an in-person focus group may be important for patients and families

We developed a semi-structured interview guide for each focus group that included questions related to the generation and evaluation of evidence for clinical interventions for rare diseases. Topics included: general perspectives on rare disease research, reasons for participating in research activities, outcomes used in clinical studies, and challenges in establishing treatment efficacy and effectiveness

One team member (KT) conducted all three focus group interviews, with at least 1 additional team member attending as an observer

We completed three focus group interviews with a total of 13 participants (physicians n = 6; policy advisors n = 3; patients/caregivers n = 4). Focus group interviews lasted between 45 and 75 min

Each interview was audio-recorded with participants’ consent and transcribed for data analysis. The transcripts were analyzed sequentially using thematic analysis (Braun et al. 2008), which involved generating a set of initial codes based on interesting features of the data and then organizing those codes into key themes related to the research topic. To do this, a series of research team meetings were held to review the transcripts and inductively identify emerging concepts from each interview. Key concepts that were identified in the focus group data were organized into a coding system that was applied by one member of the study team (KT) using NVivo 10 software (QSR International Pty Ltd.) across the entire data set. Coded transcripts were reviewed and verified by a second team member (BP) to confirm all codes had been applied appropriately and that no themes had been overlooked. We used several strategies to ensure credibility and trustworthiness of our data (Shenton et al. 2004), including: debriefing sessions after each focus group to identify key perspectives, multiple

Making choices about participating in research: Patients and caregivers did not explicitly mention choosing to participate based on gaining access to treatment. Participants viewed involvement in research activities as a form of advancing science and an act of altruism of potential benefit to the next generation of individuals affected by the disease, understanding that they or their family members may be unlikely to personally benefit

Patients and/or caregivers also described approaching research as an opportunity to share their own experiences and in turn to benefit from research findings that describe a broad range of experiences of other patients and families, to help inform decision-making

Not all patients and caregivers found it easy to make the decision to participate in a research study or to try a new therapy. Participants described being fearful of making the wrong decision in choosing whether to participate

Patients and their caregivers also reported difficulty with weighing the risks versus benefits of trying a new therapy and spoke about uncertainty about whether it would be “worth it”

Despite the difficulties and uncertainty in making decisions to participate in clinical research or to try an experimental treatment, one caregiver highlighted the importance of being persistent and continuing to ask questions and do research

Patients or caregivers expressed a desire for research to be more accessible and noted that sharing findings from research in which they’ve directly participated may help encourage further research engagement

Perspectives on the value of research: one concern that was raised in all three groups with respect to the quality of research was the difficulty of conducting high quality rare disease studies due to limited resources. In addition, participants across groups were concerned about potential bias in studies that are solely funded by pharmaceutical companies

Young et al. [41]

Canada

To explore ways in which Canadian patients with rare diseases and their families would like to be involved in the lifecycle of therapies and identify their priorities for involvement

Patients with rare diseases and their families were recruited to participate in two deliberative sessions, during which concepts related to decision-making uncertainty and the technology lifecycle were introduced before eliciting input around ways in which they could be involved. This was followed by a webinar, which was used to further identify opportunities for involvement

Approval from the University of Alberta Research Ethics Board, project name ‘Patient Preferences around Therapies for Rare Disease’, no. MS1_Pro00029603, 3 September 2014. All procedures performed in studies involving human participants were in accordance with the ethical standards of the University of Alberta Research Ethics Board and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study

Pragmatic qualitative research methods were used in which the methods selected for data collection and analysis were those most likely to provide insights into the research question, without adherence to any specific research approach (Savin-Baden 2013)

Patients and families (participants) were recruited through two national events of the Canadian Organization for Rare Disorders (CORD), Canada’s national network for organizations representing those with rare disorders

Deliberative session 1: Total participants n = 118, including health professionals, industry and government (patients, and their family members, and/or patient representatives (n = 46)

Deliberative session 2: Participants are all patients, and their family members, and/or patient representatives (n = 14)

Webinar: Participants are all patients, and their family members, and/or patient representatives (n = 8)

One researcher (AY) thematically analyzed the audio recordings and field notes from the sessions using eclectic coding (Saldana 2012). A second researcher (TS) analyzed an overlap of 25% of the recordings and field notes using the same methods to ensure validity of coding. Descriptive and process coding were used to identify the topic (e.g. coverage decision making) and the activity (e.g. providing input), respectively. This yielded a list of activities that patients and families felt they could be involved in

However, additional information on these activities was obtained using values coding (reflecting perspectives, values, attitudes or beliefs about the specific type of engagement), evaluation coding (assigning judgments about the merit, worth, or significance of programs or policy), and emotion coding (labeling the emotions recalled and/or experienced by the patients/families, or inferred by the researcher about the patients/families). The activities were further grouped into goals that patients and families hoped to achieve by participating in those activities identified

Patients or family members should provide input into clinical trial design, including identifying and selecting meaningful outcome measures: Some issues that occur later on in the lifecycle could be avoided by having patients, who are experts in their diseases, involved in the design of the trial to ensure that relevant data is collected

It is frustrating that there appears to be no consideration of the endpoints that will be meaningful to reimbursement decision-makers earlier on

It is not feasible to bring every patient or family member to the table to select meaningful outcome measures, but it is still necessary to have some input

These endpoints need to be well-defined

Patients should be involved in interpreting the meaningfulness of the data collected: The value that patients place on the benefits that they experience in a trial will be different than the value others (e.g., payers; society; etc.) will place on those benefits

It’s important that their input be used to capture the meaningfulness of the outcomes collected in a trial

Patients should participate in clinical trials: Orphan drugs often do not have a strong evidence base but patients are willing to participate in trials regardless

Patients should submit patient-reported outcome measures (PROMs) during clinical trials: In many clinical trials, the clinical outcomes that data were collected on did not capture the positive benefits that they experienced on a new drug. This is frustrating, as the data that is then considered by reimbursement decision-makers is incomplete

Having the ability to report on these benefits provides important data for decision-making

Patients should adhere to the treatment protocol: This has been an issue in the past where patients were less compliant with more burdensome treatments, negatively affecting their outcomes