A post pandemic roadmap toward remote assessment for neuromuscular disorders: limitations and opportunities
Orphanet Journal of Rare Diseases volume 17, Article number: 5 (2022)
Recent advances in technology and expanding therapeutic opportunities in neuromuscular disorders has resulted in greater interest in and development of remote assessments. Over the past year, the rapid and abrupt COVID-19 shutdowns and stay-at-home orders imposed challenges to routine clinical management and clinical trials. As in-person services were severely limited, clinicians turned to remote assessments through telehealth to allow for continued care. Typically, disease-specific clinical outcome assessments (COAs) for neuromuscular disorders (NMD) are developed over many years through rigorous and iterative processes to fully understand their psychometric properties. While efforts were underway towards developing remote assessments for NMD before the pandemic, few if any were fully developed or validated. These included assessments of strength, respiratory function and patient-reported outcomes, as well as wearable technology and other devices to quantify physical activity and function. Without many choices, clinicians modified COAs for a virtual environment recognizing it was not yet known how they compared to standard in-person administration. Despite being able to quickly adapt to the demands of the COVID-19 pandemic, these experiences with remote assessments uncovered limitations and opportunities. It became clear that existing COAs required modifications for use in a virtual environment limiting the interpretation of the information gathered. Still, the opportunity for real-world evaluation and reduced patient burden were clear benefits to remote assessment and may provide a more robust understanding and characterization of disease impact in NMD. Hence, we propose a roadmap navigating an informed post-pandemic path toward development and implementation of safe and successful use of remote assessments for patients with NMD.
Advances in technology over the past two decades have led clinicians to reconsider how clinical care is administered and research is conducted. The ability to collect information on patients or research participants remotely offers several benefits particularly in rare conditions, such as neuromuscular disorders (NMD). Travel to tertiary care centers for expert care or extended clinical trial visits are often challenging because of accessibility, need for assistance, and prolonged time away from home or work. Incorporating remote assessments in patient management and study design allows for data to be collected more frequently and in a person’s natural or home environment, while reducing the burden of the number of in-person visits.
With the expansion of therapeutic opportunities in NMD, there is a common need to describe the evolving natural history and responses to treatment. Outcome measures that quantify impairments and functional abilities are paramount to understanding the natural history and evaluating treatments. Measures of strength and function have been the cornerstone for assessments in NMD as the prevailing symptoms are often weakness, altered motor development or impaired functional abilities. The psychometric properties of the most commonly used clinical outcome assessments (COAs) for NMD are well-established through iterative processes and rigorous examination. These established COAs may inform the development of remote assessments.
The COVID-19 pandemic catapulted the utilization of telehealth and remote assessment, with limited experience amongst clinicians . Multidisciplinary teams scrambled to find ways to monitor disease progression, evaluate response to treatment and assess the needs of patients as traditional COAs were no longer possible. At the time, a limited number of remote assessments in the early stages of development were available, several requiring equipment or devices, and none were well established. To maintain continuity of care and adapt to a remote environment, interim strategies for assessment using COAs, despite lack of validation for remote settings, were implemented during NMD clinic telehealth visits. As in-person care resumes and we move back to the clinic, efforts to further develop and validate remote assessments, informed by this pandemic experience, are warranted. The benefits of real-world evaluation and reduced patient burden using remote assessment is clear. A systematic and thoughtful approach to developing remote assessment in NMD is critical.
Pre-pandemic approach to remote assessment
Clinically, providers have been interested in developing methods to assess individuals with neuromuscular conditions via telehealth for more than a decade. Several modes of providing telehealth are utilized but the terminology used to describe it varies (Table 1). Telehealth, encompasses all aspects of health-related activities through internet or other electronic means (not in person) including clinical, administrative and educational services [2, 3]. Increasing access to healthcare is one of the reasons that telehealth was first used as early as 2008 . Improved access to providers for individuals living in rural areas was an early benefit. In addition, decreasing the time spent traveling to the clinic, expenses accumulated while traveling and time out of work could be seen as a benefit for families, especially for those who live a great distance from their providers . While available, telehealth was rarely implemented by most clinicians in tertiary and or urban settings.
Recent efforts, initiated prior to the pandemic, towards developing remote assessments in NMD have focused on strength, respiratory function and patient-reported outcomes [13,14,15]. Activity monitoring with commercial or research grade devices have been used to evaluate gait and physical activity [13, 16,17,18,19,20]. However, few of these tools have been validated and there is little information on feasibility or barriers to use. Digital health technology such as customized phone apps developed to assess upper limb function and mobility are in early exploratory phases [13, 21]. Validation of these technologies and refinement of the study methodology is still needed to reduce variability and enable wide adoption for clinical trial purposes .
There is inconclusive and conflicting evidence supporting the reliability of remote assessment. Several studies in disorders other than NMD, demonstrate adequate inter-rater agreement between telehealth and face-to-face clinical assessments [23,24,25,26]. In contrast, mobility parameters obtained in different settings with participants with neurologic disorders, including Parkinson’s disease, differed significantly, with differences much larger than effects measured after an actual intervention . To our knowledge, there are no published studies evaluating established in-person COAs administered remotely in individuals with NMD.
Transformation of remote assessment during the pandemic
The rapid and abrupt transition due to COVID-19 shutdowns and stay-at-home orders imposed challenges to clinical management and on-going clinical trials for the NMD community. Existing standards of care supported the assumption that in-person assessments remained critical for differential diagnosis, assessment of motor function, and reimbursement from third-party payers for the provision of therapeutic interventions. However, local travel and hospital mandates and personal risk led to an upsurge in telehealth. As in-person services were severely limited, telehealth was rapidly implemented, despite a lack of prior experience at most institutions, to allow for continued care. Providers immediately adjusted the delivery of care, adapted to new regulatory constraints, and adopted new technology and terminology. Because of the urgency of the moment, disease specific standardized outcome measures were performed remotely with ad hoc adaptations in attempt to get the best possible representation of a patient’s status. The usual structured in-clinic procedures to administering assessments were superseded by desperate need for some measure of disease status.
Initial strategies for telehealth and remote assessments involving individuals with NMD focused on establishing a safe and feasible environment with adequate technological support. In this environment, challenges of adequate space, digital access and literacy, and equipment for testing were revealed [28, 29]. Connectivity via broadband was limited for some, particularly for underserved populations, introducing unwanted bias . Videoconferencing was challenging for those who were less technologically savvy or those who relied on eye-gaze, text to voice apps or caregiver support to connect. These issues impacted ability to complete and interpret most physical exam items and COAs that were attempted in the remote setting.
Younger as well as more impaired individuals with limited motor function relied on caregivers for accessing and participating in virtual visits. Higher functioning individuals had fewer physical restrictions but required more guidance during challenging functional assessments to ensure safety. Initial successful telehealth visits included subjective reports provided by the patient and caregivers and a brief physical exam, including active range of motion and functional observation. As the pandemic and in-person visit restrictions continued, the need to better assess disease progression or response to interventions increased. This led clinicians to informally explore the potential feasibility of the administration of disease-specific standardized functional assessments (COAs), as well as impairment-based assessments that traditionally require an in-person hands-on approach. Attempts at timed function tests like "floor to stand” and 10-m walk/run revealed lag time across internet servers. Completion of functional COAs were often limited by an individual’s ability, environmental constraints, or suboptimal camera views resulting in missing items and partial scores. Therefore, extrapolating test scores obtained during telehealth did not always translate to the scores obtained during in-clinic assessments.
Despite the challenges imposed by the pandemic, it is evident that a shift towards formally incorporating telehealth and remote assessment into NMD patient care and clinical trials is imminent and necessary. A new battery of COAs adapted for the virtual environment needs to be developed and/or validated. There are clear benefits to telehealth and remote assessments. The ability to observe an individual perform functional mobility tasks and self-care (ie. climbing stairs, transferring, getting dressed) in their natural environment is meaningful and invaluable.
Proposed roadmap for post-pandemic remote assessment
Despite being able to quickly adapt to the demands of the COVID-19 pandemic, telehealth experiences have uncovered limitations in and opportunities for remote assessments. Clinicians, researchers, and other stakeholders committed to exploring the boundaries of remote assessments of individuals with neuromuscular conditions should understand the foundational groundwork necessary to develop and validate assessments with the same rigor that has been used to develop currently accepted COAs. Understanding the feasibility of remote assessments to evaluate strength and function is fundamental to this work. Feasibility varies by type of assessments. Surveys and patient reported outcomes have higher feasibility and will require less time to establish efficacy in remote settings, whereas clinician administered motor function remote assessment and digital monitoring will require additional work and time to establish reliability and validity. Disease specific outcome measures are optimal in assessment of NMD patients particularly considering the therapeutic landscape and should be prioritized in the development of remote assessment.
Expert opinion and consensus-based recommendations are important contributions to identifying feasibility of remote assessments for different populations. COAs for use in the remote environment are in various stages of development (Fig. 1). Ideally, COAs identified as ready-to-use will have clearly established relationships with traditional in-clinic assessments. Identification of safe and practical assessments, appropriate for remote administration, is also necessary. In the meantime, patient-reported outcome measures whether adapted for remote collection or clinician administered can be implemented without additional adaptation or with very minimal development. Many of these measures have been developed as computer adaptive instruments or have online versions ready for use [31, 32]. Patients and caregivers can be instructed on active and passive range of motion activities to be assessed by the clinician remotely. Digital technology, smart-phone based apps and wearable technology is currently available with some evidence for feasibility, and performance in NMD can be incorporated. Selecting technology for remote assessments that features “usability” and easy set-up to reduce burden on the user needs to be considered. In some cases patients and caregivers might be able to leverage their own devices and resources (iPhone, iPad), however, researchers (and sponsors) will need to provide adequate support for remote visits in clinical trials when families experience issues with any technology that is implemented.
Remote assessments, whether supervised or derived from digital health technology, require established validity, reliability, and sensitivity to change. Existing measures should not simply be retrofit to the virtual environment presuming similar attributes established during in-person visits. However, the path for each remote assessment will depend on its stage of development (Fig. 1). Moreover, the purpose of the remote assessment should be well-defined a priori. For example, some assessments may serve as interim assessments to COAs performed in clinic and others may provide complementary evaluation of real-world performance and experiences. It is critical to examine the reliability, validity, and sensitivity to change with treatment and over time of these remote assessments in an iterative process (Fig. 2). Utilizing a methodological approach that meets established guidelines (https://www.fda.gov/about-fda/center-drug-evaluation-and-research-cder/division-clinical-outcome-assessment-dcoa) and incorporating regulatory and patient feedback is essential. Some remote assessments have undergone feasibility and psychometric testing, permitting an abbreviated timeline entering the roadmap at later stages. Gathering expert consensus and stakeholder feedback can be facilitated through virtual interactions, perhaps under-utilized prior to the pandemic, which would further expedite the process. Regardless, until this iterative process occurs, the ability to compare in-person data to remote data will be difficult, and the capacity to interpret change across both remote and in-person settings will remain limited.
Careful consideration to develop and deliver valid and reliable COAs for remote use is critical. As a result, remote assessments that are psychometrically sound, meaningful and sensitive to change will be available for use in NMD. Incorporating the lessons learned during the pandemic and in an attempt to circumvent failure and promote success, we propose recommendations for the development and delivery of remote assessments for individuals with NMD. Moving forward, remote assessment can expand upon an in-person assessment and provide a more comprehensive evaluation of the patient experience. Clinical trials could be adapted to include remote assessments to reduce burden on patients and clinical sites without gaps in monitoring. Additionally, remote assessment could foster access to participants outside of a clinical site’s geographical region. This would create opportunities to reach patients with limited clinical services, expertise and resources.
Successful translation of COAs to the virtual environment is dependent on the type of assessment, as well as the patient’s age, disease, phenotype, and environment. The roadmap (Fig. 2) to timely and effective adoption of remote outcome measures for use in NMDs should include: (1) identifying appropriate patient-centered and clinically relevant measures, (2) establishing recommendations to assess readiness for use of promising remote assessments, (3) determining the psychometric properties of the remote assessments, (4) comparing in-person data to remote data to understand change across both remote and in-person settings for clinician administered and patient reported outcomes (5) integrating the remote assessments into study design and clinical practice, (6) encouraging engagement of and feedback from user groups (clinicians and patients), and (7) insuring access to the internet, technology and equipment necessary to complete the remote assessments. This suggested road map will lead to improved remote assessments of individuals with NMD, address regulatory requirements for use in clinical trials, and enhance health care outcomes. Remote assessments should be considered exploratory until administration standards and psychometric properties are well-defined. In the meantime, clinicians and researchers should rely on well-established in-person assessments to evaluate disease status in patient management and clinical trials.
Advances in technology initiated interest in remote assessments of individuals with NMD both for clinical care and research endeavors. The COVID-19 pandemic created an environment that resulted in rapid transformation of how clinical care was delivered and research was conducted leading to an increased understanding of the benefits that remote assessment can bring to the care of those with NMDs. Across the lifespan of individuals with NMD, there are opportunities to capture objective and meaningful information regarding a person’s strength, function, and daily performance using remote assessments. The addition of real-world assessments may provide a more robust understanding and characterization of disease impact in NMD. The combined use of in-person and remote assessments may provide a more holistic view of the clinical manifestations and patient experience. Clinical trials could be adapted to include remote assessment to expand recruitment, facilitate retention, and reduce patient burden. Clinicians and researchers need to proceed with caution, in order to ensure safe, accurate and meaningful use of this modality. Risks of utilizing COAs established for in-person use through telehealth prematurely may lead to inaccurate characterization of clinical status and change. This roadmap provides a starting point for a measure’s readiness for use and a way to determine what measurement attributes need further work prior to use in a telehealth setting. Steps to guide the path to clinically valid and reliable remote COAs should mimic previous approaches utilized to achieve already well- established in person COAs.
Availability of data and materials
Clinical outcome assessment
Koonin LM, Hoots B, Tsang CA, et al. Trends in the Use of Telehealth During the Emergence of the COVID-19 Pandemic — United States, January–March 2020. MMWR Morb Mortal Wkly Rep. 2020;69:1595–9.
American Physical Therapy Association. Telehealth Modalities PTs and PTAs Can Use During the Public Health Emergency. APTA Web site. https://www.apta.org/your-practice/practice-models-and-settings/telehealth-practice/telehealth-modalities-that-pts-and-ptas-can-use-during-the-public-health-emergency. Updated: December 16, 2020. Accessed: March A
Health IT. What is telehealth? How is telehealth different from telemedicine? HealthIT.gov Web site. https://www.healthit.gov/faq/what-telehealth-how-telehealth-different-telemedicine) Updated: October 17, 2019. Accessed: March 11, 2021.
Theodoros D, Russell T. Telerehabilitation: current perspectives. Stud Health Technol Inform. 2008;131:191–209.
Rabatin AE, Lynch ME, Severson MC, et al. Pediatric telerehabilitation medicine: Making your virtual visits efficient, effective and fun. J Pediatr Rehabil Med. 2020;13:355–70.
World Health Organization. Digital Health. WHO Web site. https://www.who.int/health-topics/digital-health. Accessed: March 11, 2021.
Centers for Disease Control and Prevention. Telehealth and Telemedicine: A Research Anthology of Law and Policy Resources. 2019 CDC Web site: https://www.cdc.gov/phlp/publications/topic/anthologies/anthologies-telehealth.html. Accessed March 11, 2021.
Prvu Bettger J, Resnik LJ. Telerehabilitation in the age of COVID-19: an opportunity for learning health system research. Phys Ther. 2020;100:1913–6.
U.S. Food & Drug Administration. Clinical Outcome Assessment: Frequently Asked Questions. FDA website. https://www.fda.gov/medical-devices/digital-health-center-excellence/what-digital-health. Accessed: April 20, 2021.
Digital Biomarkers Journal. https://www.karger.com/Journal/Details/271954. Accessed: April 24, 2021.
Waller M, Stotler C. Telemedicine: a primer. Curr Allergy Asthma Rep. 2018;18:54.
Izmailova ES, Wagner JA, Perakslis ED. Wearable devices in clinical trials: hype and hypothesis. Clin Pharmacol Ther. 2018;104:42–52.
Rutkove SB, Qi K, Shelton K, et al. ALS longitudinal studies with frequent data collection at home: study design and baseline data. Amyotroph Lateral Scler Frontotemporal Degener. 2019;20:61–7.
Kaufmann P, Levy G, Montes J, et al. Excellent inter-rater, intra-rater, and telephone-administered reliability of the ALSFRS-R in a multicenter clinical trial. Amyotroph Lateral Scler. 2007;8:42–6.
Geronimo A, Simmons Z. Evaluation of remote pulmonary function testing in motor neuron disease. Amyotroph Lateral Scler Frontotemporal Degener. 2019;20:348–55.
Jimenez-Moreno AC, Charman SJ, Nikolenko N, et al. Analyzing walking speeds with ankle and wrist worn accelerometers in a cohort with myotonic dystrophy. Disability Rehabil 2018:1–7.
Block VA, Pitsch E, Tahir P, et al. Remote physical activity monitoring in neurological disease: a systematic review. PLoS ONE. 2016;11:e0154335.
Montes J, Garber CE, Kramer SS, et al. Single-blind, randomized, controlled clinical trial of exercise in ambulatory spinal muscular atrophy: why are the results negative? J Neuromuscular Dis. 2015;2:463–70.
Knak KL, Sheikh AM, Witting N, et al. Physical activity in myotonic dystrophy type 1. J Neurolo 2020.
van Eijk RPA, Bakers JNE, Bunte TM, et al. Accelerometry for remote monitoring of physical activity in amyotrophic lateral sclerosis: a longitudinal cohort study. J Neurol. 2019;266:2387–95.
Berry JD, Paganoni S, Carlson K, et al. Design and results of a smartphone-based digital phenotyping study to quantify ALS progression. Ann Clin Transl Neurol. 2019;6:873–81.
Govindarajan R, Berry JD, Paganoni S, et al. Optimizing telemedicine to facilitate amyotrophic lateral sclerosis clinical trials. Muscle Nerve. 2020;62:321–6.
Palacín-Marín F, Esteban-Moreno B, Olea N, et al. Agreement between telerehabilitation and face-to-face clinical outcome assessments for low back pain in primary care. Spine. 2013;38:947–52.
Cabana F, Boissy P, Tousignant M, et al. Interrater agreement between telerehabilitation and face-to-face clinical outcome measurements for total knee arthroplasty. Telemed J E Health. 2010;16:293–8.
Palsbo SE, Dawson SJ, Savard L, et al. Televideo assessment using Functional Reach Test and European Stroke Scale. J Rehabil Res Dev. 2007;44:659–64.
Russell T, Truter P, Blumke R, et al. The diagnostic accuracy of telerehabilitation for nonarticular lower-limb musculoskeletal disorders. Telemed J E Health. 2010;16:585–94.
Warmerdam E, Hausdorff JM, Atrsaei A, et al. Long-term unsupervised mobility assessment in movement disorders. Lancet Neurol. 2020;19:462–70.
Jennings SC, Manning KM, Bettger JP, et al. Rapid transition to telehealth group exercise and functional assessments in response to COVID-19. Gerontol Geriatr Med. 2020;6:2333721420980313.
Hwang R, Mandrusiak A, Morris NR, et al. Exploring patient experiences and perspectives of a heart failure telerehabilitation program: a mixed methods approach. Heart Lung. 2017;46:320–7.
EveryCRSReport. Broadband Internet Access and the Digital Divide: Federal Assistance Programs EveryCRSReport Web site. https://www.everycrsreport.com/reports/RL30719.html. Accessed: March 11, 2021.
Fragala-Pinkham M, Pasternak A, McDermott MP, et al. Psychometric properties of the PEDI-CAT for children and youth with spinal muscular atrophy. J Pediatr Rehabil Med. 2021;14(3):451–61.
Rodday AM, Graham RJ, Weidner RA, et al. Leveraging pediatric PROMIS item banks to assess physical functioning in children at risk for severe functional loss. J Patient Rep Outcomes. 2017;1(1):10.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests relevant to this manuscript.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Montes, J., Eichinger, K.J., Pasternak, A. et al. A post pandemic roadmap toward remote assessment for neuromuscular disorders: limitations and opportunities. Orphanet J Rare Dis 17, 5 (2022). https://doi.org/10.1186/s13023-021-02165-w
- Neuromuscular disorders
- Outcome measure
- Remote assessment
- Clinical trials
- Clinical outcome assessments (COA)