Evaluation of Diagnostic Accuracy and Patient Satisfaction in Telemedicine VS. IN-Person Examinations: A Non-Inferiority Study in Adult Ophthalmology

Abstract

Background: Primary objective of our study is to evaluate the effectiveness of telemedicine in providing accurate diagnoses and management of ocular diseases, and continuous patient care in an adult ophthalmology clinic. This aims to determine whether telemedicine can achieve diagnostic accuracy comparable to in-person examinations. Secondary objectives are the evaluation of any telemedicine visits interrupted due to technical limitations, thus assessing the feasibility of conducting complete visits, and the evaluation of patient satisfaction levels with telemedicine.

Methods: This single-center non-inferiority study includes 54 adult patients. Each patient was examined in two different modalities. One examination modality (A) involved tests performed by an orthoptist using digital equipment and streamed live to an ophthalmologist, who discussed diagnosis and therapy with the patients via webcam; the other telemedicine examination modality (B) involved tests performed by an orthoptist using digital equipment and evaluated in person by an ophthalmologist. The average time interval between the two visits (A and B) was 5 days.

Results: The main outcome measures are discrepancies in management plans or diagnoses between telemedicine and in-person examinations, with non-inferiority thresholds set at less than 1.5% for management plan discrepancies and less than 15% for diagnosis discrepancies.

Conclusion: Our study demonstrates the non-inferiority of real-time telemedicine compared to in-person examinations for diagnosing and managing ophthalmic conditions.

Keywords: Telemedicine, Ophthalmology, Diagnosis, Satisfaction

Background

The term "tele" originates from the Greek word for distance. In the realm of healthcare, telemedicine refers to the revolutionary practice of providing medical services remotely [1,2]. This approach relies on cutting-edge telecommunication tools like smartphones, wireless devices, and remote video connections [3]. By leveraging these technologies, healthcare professionals can deliver care from afar, breaking down geographical barriers [4].

When it comes to embracing technological advancements, ophthalmology stands out. This field has harnessed its unique ability to directly visualize neural tissue, connective tissue, and blood vessels within the eye. With advanced microsurgical techniques, ophthalmologists can even intervene safely. One remarkable example is Optical Coherence Tomography (OCT), enabling ophthalmologists to non-invasively capture detailed cross-sectional images of the retina, akin to histological slices.

And at the forefront of telemedicine is tele-ophthalmology. This field leads the way by integrating novel devices infused with Artificial Intelligence (AI), empowering remote patient evaluation and screening. The combination of advanced technology and medical expertise is transforming the way eye care is delivered, making it more accessible and efficient [3,5].

Global studies have unequivocally demonstrated the effectiveness and benefits of tele-ophthalmology screening [6-8]. Consider a study conducted in Nepal, where a state-of-the-art ophthalmic camera system attached to a 6th generation iPod Touch accurately identified patients needing further examination. In fact, 97% of the referrals were deemed appropriate by the ophthalmologist [9,10]. Meanwhile, groundbreaking research published in JAMA Ophthalmology found that remote diagnostic imaging using an FDA-approved non-contact retinal imaging device was just as effective as a traditional examination by a retinal specialist in detecting critical macular degeneration cases [8].

In this study, our objective is to examine the concordance between telemedicine and in-person examinations in the diagnosis and management of eye conditions.

Methods

Between November 2022 and March 2023, a single-center non-inferiority study was conducted to analyze the concordance between telemedicine and conventional in-person ophthalmic visits. The study included 54 adult participants, regardless of gender, aged between 19 and 74 years. The study was conducted in a private ophthalmology center in Palermo. The study was conducted in accordance with the principles of the Helsinki Declaration. All study participants provided informed consent for the treatment of personal data. No patient received compensation or paid for the service.

Each patient was examined in two different modalities. One examination modality(A) involved tests performed by an orthoptist using digital equipment and streamed live to an ophthalmologist, who discussed diagnosis and therapy with the patients via webcam; the other telemedicine examination modality (B) involved tests performed by an orthoptist using digital equipment and evaluated and discussed by an ophthalmologist in person. Each patient, both for visit A and visit B, underwent diagnostic tests performed by a professional orthoptist: medical history, Best Corrected Visual Acuity (BCVA) assessment, rebound Tonometry (iCare IC100), corneal Topography (Antares, CSO), endothelial microscopy (Perseus, CSO), Optical Coherence Tomography (OCT) examination of the macula and nerve fibers (Topcon), and anterior segment assessment using a video image with a slit lamp camera (SL9800,CSO). Fundus examination with dilation (tropicamide 1% 'Visumidriatic') was conducted using indirect ophthalmoscopy with a Volk 90D superfield lens during in-person visits (B), while telemedicine visits (A) used the Eidon Ultra-WideField retinal camera. The average time interval between the two visits (A and B) was 5 days.

To eliminate potential confirmation bias, each patient was examined on separate days by two different doctors, one in person and one via telemedicine. The professional orthoptist was the same across different visits.

The following platforms were used: MODI as electronic medical record management software, TeamViewer for online connection to the clinic’s computer and viewing the patient’s medical records and instrumental images, and Microsoft Teams for webcam communication with the patient. Patients were classified based on the ocular localization of the diagnosis. Specifically, we described conjunctival, corneal, lens-related, and vitreoretinal pathologies.

Subsequently, a comparison was made between the diagnoses made via telemedicine and those made in person to identify any discrepancies. The different visits of each patient were reviewed by a third external ophthalmologist. Finally, the average duration of telemedicine visits was compared with in-person visits.

Statistical Analysis

Continuous variables are presented as mean, Standard Deviation (SD), and range (minimum and maximum); while categorical variables are presented as values and percentages.

To analyze paired nominal data and determine if telemedicine visits could achieve diagnoses comparable to in-person visits, the non-parametric McNemar test (p-value < 0.05) was used. Association analysis was performed using the Chi-square test, and a p-value less than 0.05 was considered significant.

Patient satisfaction levels were also evaluated through three anonymous online survey questions (Table 1).

Table 1: Questionnaires to assess patient satisfaction levels.

Results

The study included 54 patients, comprising 30 males and 24 females. The average age was 47.56 ± 15.04 years (range 19-74).

The reasons for the visits were categorized into five categories: visual acuity problems, corneal pathology, conjunctival pathology, lens pathology, and vitreoretinal pathology (Table 2, Figure 1).

Figure 1: Classification of patients based on ophthalmologic pathology.

Table 2: Classification of patients based on ophthalmologic pathology.

Most patients accessed the study for visual acuity problems and vitreoretinal pathologies.

No telemedicine visits had to be interrupted.

Out of the total 54 patients who were evaluated both in-person and via telemedicine, correct diagnoses were made in 48 patients using both modalities, in 1 patient only through telemedicine, and in 5 patients only through in-person visits.

By conducting a McNemar test with a 95% confidence interval, the chi-squared value (χ²) is 1.5 (χ² > 3.841). Therefore, aiming to achieve a correct diagnosis, no statistically significant difference was found between in-person and telemedicine visits (Table 3). Thus, the two modes of visits can be considered equivalent (Table 3).

Table 3: Diagnostic difference between different modes of consultation: in-person visit and telemedicine visit.

The level of patient satisfaction was very high, as 84.6% of patients expressed their willingness to repeat the telemedicine visit experience. Among the patients, 38.5% initially had a somewhat low opinion of telemedicine visits, but after undergoing the visit, only 7% expressed a negative judgment regarding telemedicine visits.

The average durations of the visits are presented in the (Table 4).

Table 4: Average duration of telemedicine and in-person visits.

Conclusion

Telemedicine offers a promising solution to the increasing demand for ophthalmology care, which often leads to long waiting times for appointments. This technology can mitigate workforce shortages, enhance collaboration between specialists and primary care providers, and expedite access to care. Despite growing confidence among eye care providers, a significant portion remains uncertain about the efficacy of remote screening, highlighting the need for clinical validation and examination of patient experiences. Our study demonstrates the non-inferiority of real-time telemedicine compared to in-person examinations for diagnosing and managing ophthalmic conditions. Notably, telemedicine facilitated the diagnosis of active diabetic retinopathy in a patient where in-person examination had failed, likely due to the superior retinal imaging capabilities of digital equipment like the Eidon. However, in five cases, in-person examinations proved superior, particularly for early-stage ocular surface disorders that were not easily detected via telemedicine. A notable limitation of telemedicine is the cost of advanced diagnostic equipment, which exceeds that of standard tools used in traditional visits. Moreover, the time required for telemedicine consultations tends to be longer. Our study aimed to perform the best possible telemedicine evaluations, even at the expense of time and cost, to build ophthalmologists' trust in these new technologies. Future research should focus on optimizing telemedicine workflows to reduce costs and time, exploring the integration of specific tools for tear film disorders, and examining the efficacy of telemedicine under different pupil conditions to enhance the efficiency of retinal screenings. Telemedicine offers numerous benefits, including accessibility for remote patients, reduced wait times, streamlined screening for chronic conditions like diabetic retinopathy, and the ability to provide specialist consultations in emergency settings. However, the learning curve and need for specialized training for providers, such as pediatric optometrists, remain challenges. Addressing these limitations will be crucial for the widespread adoption of telemedicine in ophthalmology.

Statement of Ethics

This manuscript did not require ethical approval in accordance with local/national guidelines. This published case report complies with the guidelines for human studies in accordance with the World Medical Association Declaration of Helsinki. The patient provided written informed consent for publication.

Conflict of Interests

All authors declare no conflict of interest.

Funding Sources

No funding or grant support was received for publication of this manuscript.

Author Contributions

A.P was Head of the Unit Board. G. LG revised the manuscript. M.M, C.B. and G.T. wrote original draft of this manuscript.

Data Availability Statement

All data generated or analyzed during this study are included in this article. Further inquiries can be directed to the corresponding author.

Acknowledgement

None.

References

• Labiris G, Panagiotopoulou EK, Kozobolis VP (2018) A systematic review of teleophthalmological studies in Europe. Int J Ophthalmol 11(2): 314-325.
• Dorsey ER, Topol EJ (2016) State of Telehealth. N Engl J Med 375(2): 154-161.
• Armstrong GW, Lorch AC (2020) A(eye): a review of current applications of artificial intelligence and machine learning in ophthalmology. Int Ophthalmol Clin 60(1): 57-71.
• Huang D, Swanson EA, Lin CP (1991) Optical coherence tomography. Science 254(5035): 1178-1181.
• Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC (2018) Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 28(1): 39.
• Grau E, Horn F, Nixdorff U, Michelson G (2019) OCT and IOP findings in a healthy worker cohort: results from a teleophthalmic study in occupational medicine. Graefes Arch Clin Exp Ophthalmol 257(11): 2571-2578.
• Boucher MC, Desroches G, Garcia Salinas R, Amin Kherani, David Maberley, et al., (2008) Teleophthalmology screening for diabetic retinopathy through mobile imaging units within Canada. Can J Ophthalmol 43(6): 658-668.
• Hadziahmetovic M, Nicholas P, Jindal S, Mettu PS, Cousins SW (2019) Evaluation of a remote diagnosis imaging model vs dilated eye examination in referable macular degeneration. JAMA Ophthalmol 137(7): 802-808.
• Hong K, Collon S, Chang D, Sunil Thakalli, John Welling, et al., Teleophthalmology through handheld mobile devices: a pilot study in rural Nepal. JMTM 8(1): 1-10.
• De Lott LB, Newman Casey PA, Lee PP, Dena Ballouz, Lyna Azzouz, et al., (2021) Change in ophthalmic clinicians’ attitudes toward telemedicine during the coronavirus 2019 pandemic. Telemed J E Health 27(2): 231-235.