If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Develop a novel methodology to identify lapses in diabetic retinopathy care in electronic health records and evaluate health disparities by race and ethnicity.
Design
Retrospective cohort study.
Subjects, Participants, and/or Controls
Adult patients with diabetes mellitus who were evaluated at the Wilmer Eye Institute from 1/1/2013 to 4/2/2022.
Methods, Intervention or Testing
The methodology to identify lapses in care first identified diabetic retinopathy screening or treatment visits, and then compared the providers’ recommended follow-up timeframe with the patient’s actual time to next encounter. The association of race and ethnicity with odds of lapses in care was evaluated using a mixed effects logistic regression model controlling for age, sex, insurance, severity of diabetic retinopathy, presence of other retinal disorders, and glaucoma.
Main Outcome Measures
Lapses in diabetic retinopathy care
Results
The methodology to identify diabetic retinopathy-related visits had a 95.0% (95% confidence interval (CI) 93.0 to 96.6) sensitivity and 98.8% (98.1 to 99.3) specificity as compared to a gold standard grader. The methodology resulted in a 97.3% (96.2 to 98.4) sensitivity and 98.1% (97.3 to 98.9) specificity for detecting a follow-up recommendation, with an average error of -0.05 (-0.31 to 0.21) weeks in extracting the precise timeframe. A total of 39,561 patients with 91,104 office visits were included in the analysis. The average age was 61.4 years. More than three in four (77.6%) patients had a lapse in care. In multivariable analysis, non-Hispanic Black patients had 1.24 (1.19 to 1.30) odds and Hispanic patients had 1.26 (1.13 to 1.40) odds of ever having a lapse in care compared to non-Hispanic White patients, (p<0.001, respectively).
Conclusions
We have developed a reliable methodology for identifying lapses in diabetic retinopathy care that is tailored to a provider’s recommended follow-up. Using this approach, we find that three in four patients experience a lapse in diabetic retinopathy care and that these rates are higher among non-Hispanic Black and Hispanic patients. Deploying this methodology in the EHR is one potential means by which to identify and mitigate lapses in critical ophthalmic care in patients with diabetes.
Vision loss from diabetes mellitus can result from lapses in diabetic retinopathy care which can lead to delays or interruptions in ophthalmic treatment.
Treatment compliance and adherence among patients with diabetic retinopathy and age-related macular degeneration treated by anti-vascular endothelial growth factor under universal health coverage.
Outcomes of Eyes Lost to Follow-up with Proliferative Diabetic Retinopathy That Received Panretinal Photocoagulation versus Intravitreal Anti-Vascular Endothelial Growth Factor.
Racial and ethnic minoritized groups, including Black and Hispanic populations, have higher rates of vision loss from diabetes when compared to White patients.
Obeid, et al. found higher rates of lapses in diabetic retinopathy care among Black and Hispanic populations as compared to White patients in a large retrospective cohort study.
In the context of a clinical trial, non-White individuals, including Black, Hispanic, and other race populations had a higher proportion of lapses in care compared to White participants, but race and ethnicity was no longer statistically significantly associated with care lapses after adjusting for other baseline demographic and ophthalmic characteristics.
Differences in conclusions between these studies are likely due to variations in the context of the analysis, whether a clinical trial or observational study, and the specific definition of lapse in care. Most observational studies in clinical settings apply a singular definition for lapses in care for diabetic retinopathy (e.g., not returning in 6 months or 1 year).
Treatment compliance and adherence among patients with diabetic retinopathy and age-related macular degeneration treated by anti-vascular endothelial growth factor under universal health coverage.
Outcomes of Eyes Lost to Follow-up with Proliferative Diabetic Retinopathy That Received Panretinal Photocoagulation versus Intravitreal Anti-Vascular Endothelial Growth Factor.
However, the typical clinical trial uses an adaptive definition that accommodates the providers’ recommended follow-up (e.g., not returning in 10 weeks if the provider recommended an 8-week follow-up, or 20 weeks for a 16-week follow-up).
Tailoring the definition of a lapse in care to the providers’ recommendation has several advantages, including that it considers the underlying urgency of follow-up and potential risk for vision loss.
Since the risk of vision loss can be proportional to the number of missed days, lapses in care should be identified earlier in those with more urgent follow-up needs.
Methods are needed to study lapses in care in the clinical practice setting that incorporate the providers’ recommended follow-up. Therefore, there were two purposes of this study. The first was to develop a methodology for identifying lapses in diabetic retinopathy care in the electronic health record (EHR) that incorporates providers’ recommendations and is scalable across the range of diabetic eye disease complications. The second was to use this methodology to evaluate racial and ethnic disparities in lapses in diabetic retinopathy care in the practice of ophthalmology.
Methods:
Study Design / Patient Selection:
This was a retrospective cohort study of adult patients over the age of 18 years with diabetes mellitus seen at the Wilmer Eye Institute at Johns Hopkins Hospital from 1/1/2013 to 4/2/2022. Data were collected and patients were screened for eligibility. Diabetes mellitus was defined as having a qualifying International Classification of Disease (ICD) code (Supplemental Table 1) at any hospital encounter or a hemoglobin A1c value of ≥6.5%. This sensitive definition of diabetes mellitus was used to include as many patients as possible in this first step. Patients were included if they had at least one completed office visit with a Wilmer provider either for diabetic retinopathy screening or treatment with the first one occurring between 1/1/2013 to 4/2/2020. Patients were excluded if they had missing sociodemographic data or ophthalmic diagnoses (N=6). (Figure 1) Office visits from 1/1/2013 to 4/2/2022 in the first two years after patient’s entry into the cohort were analyzed to standardize the observation period and opportunity for lapse in care. Office visits were excluded if providers’ recommended follow-up timeframe was missing or occurred after 52 weeks (see Step B of the pipeline for identifying lapses in diabetic retinopathy care). This study was approved by the Institutional Review Board of the Johns Hopkins University School of Medicine and adhered to the tenets of the Declaration of Helsinki.
Figure 1Flow diagram for identification of adult patients with diabetes, and the diabetes related office visits in the electronic health record.
Methodology for identifying lapses in diabetic retinopathy care in the EHR:
Defining lapses in diabetic retinopathy care in the electronic health record (EHR) involved three major components (Figure 2): A) Office visits related to the screening or treatment of diabetic retinopathy were identified and isolated; B) The providers’ recommended follow-up timeframe was extracted; and C) The providers’ recommended follow-up timeframe was compared to the patient’s actual time to the next office visit. A patient was defined as having a lapse in care following an office visit if they returned beyond the preset threshold. (Supplemental Table 2). This multi-stage approach was adopted due to the complexities of utilizing EHR data and the specialization of eye care in our ophthalmology department. Some providers only address and follow specific ophthalmic issues (e.g., glaucoma, cataracts). Thus, to define a lapse in diabetic retinopathy care in a multidisciplinary ophthalmology department, we had to isolate the relevant visits first and identify the follow-up specific to diabetic retinopathy care.
Figure 2Methodology for identifying lapses in diabetic retinopathy care in the electronic health record (EHR). After the initial cohort selection, the ophthalmology office visits related to diabetic retinopathy screening and treatment were isolated (Step A), the providers’ recommended follow-up timeframe was identified (Step B), the time to the patient’s next office visit was calculated (Step C), and the thresholds for defining lapses in diabetic retinopathy care were applied (Final). A sample patient’s timeline of different office visits in the EHR is shown on the right with examples of which visits were considered to have lapses in diabetic retinopathy care.
A) Office visits related to the screening or treatment of diabetic retinopathy were identified using a combination of ICD codes and a rule-based natural language processing (NLP) algorithm. This hybrid approach was taken because the sensitivity of ICD codes to identify clinical phenotypes (such as diabetic retinopathy) can be limited; 29.0% of visits analyzed in our study were identified only using the NLP algorithm.
Office visits were included if there was an encounter ICD code related to diabetes mellitus (Supplemental Table 1). The rule-based NLP algorithm was applied on the clinical progress notes and used to identify additional office visits. (Supplemental NLP Algorithm Material) The re Python package was used to search for a list of target terms in the provider notes to produce a binary output regarding the presence of diabetic retinopathy screening or treatment. The list of target terms was fine-tuned through an iterative process, qualitatively reviewing outputs on a development sample of data and then updating the target term list until a sufficient level of performance was achieved. Quantitative validation of the NLP pipeline was performed on a separate held-out sample of test data when the fine-tuning process was complete.
B) The providers’ recommended follow-up timeframe was also extracted using a hybrid approach combining structured data available in the check-out section of the EHR and a custom rule-based NLP algorithm applied to the unstructured provider notes. Similar to the aforementioned extraction task, this hybrid approach was needed because most office visits (77.6%) did not contain the necessary follow-up in the structured check-out section, and this information was only available in unstructured provider notes. The algorithm leverages a dictionary of target terms and character patterns to flag the portion of the note that is likely to be immediately preceded or followed by the recommended return time interval. Additional logic was introduced to search through the provider notes in a specific sequence based on prior clinical knowledge regarding the location of follow-up recommendations in a provider note. When available, the recommended follow-up timeframe was extracted and numerically standardized. The list of target terms, character patterns, and processing logic were iteratively fine-tuned by qualitative review and evaluation of the algorithm’s performance on the same development set as above (Supplemental NLP Algorithm Material). The NLP pipeline was quantitatively validated using a held-out sample of test data.
C) The providers’ recommended follow-up timeframe was compared to the patients’ time to the next diabetic retinopathy screening or treatment office visit. A lapse in care was defined as not returning within certain timeframes that varied based on provider recommendations. (Supplemental Table 2) These thresholds generally correspond to ∼25% of the provider’s recommended timeframe (e.g., threshold of 2 weeks for an 8 week recommended follow-up, 4 weeks for a 20 week follow-up, 8 weeks for a 32 week follow-up, and 12 weeks for a 52 week follow-up) and were developed based on the existing literature and expert guidance (CXC).
If the threshold to return fell beyond the two-year observation window, then that encounter was categorized as being unknown for lapse in care. (Figure 2)
Validation of Rule-Based NLP algorithms:
Task 1: Automated detection of diabetic retinopathy screening or treatment from provider notes (Step A of the pipeline)
The NLP algorithm was evaluated using a held-out test set of 2000 randomly selected notes sampled from the initial set of patients with diabetes mellitus. The notes were annotated by 2 expert graders with extensive clinical experience (CXC, a fully trained vitreoretinal specialist; and TT, a post-graduate year 4 ophthalmology resident). The graders assessed whether the note text and ICD codes addressed diabetic retinopathy screening or treatment and annotated the document with a binary label. Performance of the NLP algorithm was compared to the gold standard grader (CXC) and sensitivity and specificity were calculated. The 95% confidence interval for both metrics were estimated via the bootstrap algorithm in which the test set was resampled 1000 times with replacement, and the sensitivity and specificity calculated and cached for each sample.
Agreement between the two graders were assessed using the Cohen’s kappa statistic. Kappa statistic <0.00 indicates poor agreement, 0.00−0.20 slight, 0.21−0.40 fair, 0.41−0.60 moderate, 0.61−0.80 substantial, 0.81−1.00 almost perfect agreement.
Task 2: Automated detection of providers’ recommended follow-up timeframe (Step B of the pipeline)
The NLP algorithm for this task was evaluated on the same test set of 2000 notes annotated by the same graders. The graders first annotated the document with a binary label indicating whether a provider recommended follow-up. When present, the graders extracted the timeframe of the providers’ recommended follow-up from the note text. The timeframe was normalized into week-based units for ease of comparison (e.g., 1 day = 0.14 weeks, 1 week = 7 days, 1 month = 4 weeks, 1 year = 52 weeks). If a range was provided in the note (e.g., the provider wanted the patient to return in 4-6 weeks), then the larger value was used in the analysis (e.g., 6 weeks).
The algorithm was evaluated based on coverage (i.e., correctly identifying the presence of a recommended follow-up), and precision (i.e., correctly identifying the follow-up timeframe when present). For coverage, the sensitivity and specificity of the NLP algorithm was compared to the gold standard grader and the 95% confidence interval calculated using bootstrapping with 1000 resamples.
The precision of the algorithm was evaluated using the average error (e.g., gold standard grader minus the NLP algorithm or second grader) and average absolute error. Agreement on precision between the two graders and the NLP algorithm was also assessed qualitatively using the Bland-Altman plot.
Statistical Analysis
Differences in sociodemographic characteristics were assessed between patients who ever had a lapse in diabetic retinopathy care and those who did not. These sociodemographic characteristics included: age, sex, race and ethnicity, and insurance. Continuous variables were compared using student’s t-test, and categorical variables using the Person’s chi-squared test. A mixed effects logistic regression model with random intercepts at the person level was constructed to evaluate the association of race and ethnicity on lapses in diabetic retinopathy care, controlling for the other sociodemographic characteristics, severity of diabetic retinopathy (divided into no diabetic retinopathy, non-proliferative diabetic retinopathy, and proliferative diabetic retinopathy on the basis of ICD codes), other retinal disorders (including age related macular degeneration, vein occlusion, artery occlusion), and glaucoma.
(Supplemental Table 3) The mixed effects model with a random intercept for person effect was chosen to account for multiple office visits per patient and differences in the number of available office visits per patient. Conditional odds ratios were transformed into marginal odds ratios for reporting.
Statistical significance was set at p ≤ 0.05. All analyses were conducted in Python (Python Software Foundation, Python Language Reference, version 3.8.9) and Stata statistical software (Version 17.0 for Windows; StataCorp LLC, College Station, TX, USA).
Sensitivity analyses
A sensitivity analysis was performed on the sub-cohort of patients with diabetes mellitus initially evaluated in the ophthalmology department for diabetic retinopathy screening and treatment from 1/1/2016 to 12/31/2017, with the end of the observation window occurring on 12/31/2019. This analysis was done to avoid any potential effects due to the transition from ICD-9 to the more complex ICD-10 which occurred in October 2015, and the significant interruptions in ophthalmic care that resulted from the COVID-19 pandemic.
Task 1: Automated detection of diabetic retinopathy screening or treatment from provider notes: The sensitivity of the NLP algorithm for detecting diabetic retinopathy screening or treatment was estimated to be 95.0% (95% confidence interval (CI) 93.0 to 96.6); and specificity was estimated to be 98.8% (95% CI 98.1 to 99.3). The kappa statistic for agreement between the two graders was 0.973 (SE 0.022, p<0.001).
Task 2: Automated detection of providers’ recommended follow-up timeframe: A total of 810 out of 2000 notes contained a follow-up timeframe identified by either the NLP algorithm or gold standard grader. The estimated sensitivity of the algorithm was 97.3% (95% CI 96.2 to 98.4) and specificity 98.1 (95% CI 97.3 to 98.9) for detecting the presence of a follow-up timeframe. 807 notes were identified as containing a follow-up timeframe by either of the two graders. The kappa statistic for agreement between the two graders was 0.97 (SE 0.02, p<0.001). In the 767 notes in which both the NLP algorithm and gold standard grader detected a follow-up timeframe, the average error (AE) was -0.05 (95% CI -0.31 to 0.21) weeks, while the average absolute error (AAE) was 0.42 (95% CI 0.20 to 0.72) weeks. In the set of 782 notes in which the two graders detected a follow-up timeframe, the AE was -0.09 (95% CI -0.23 to 0.08) weeks and AAE 0.32 (95% CI 0.17 to 0.52) weeks. Bland-Altman plots comparing the NLP algorithm and gold standard grader, and between the two graders are shown in Supplemental Figure 1. Overall, most differences fell between the 95% limits of agreement suggesting that there were no systematic biases in the NLP algorithm.
Lapses in Care and Racial and Ethnic Disparities Results:
A total of 39,561 patients with 91,104 office visits related to diabetic retinopathy screening and treatment were included in this study. (Figure 1) Overall, 46.9% of patients were non-Hispanic White, 36.7% non-Hispanic Black, 4.8% Hispanic, and 11.6% other race and ethnicity. The majority of patients were between ages 45 to 65 (average 61.4, range 18 to 100). (Table 1) Most patients were female, had private or Medicare insurance, and did not have diabetic retinopathy, other retinal diseases, or glaucoma.
Table 1Demographic characteristics of patients with diabetes mellitus seen in the ophthalmology department for diabetic retinopathy screening or treatment stratified by race and ethnicity (N = 39,561).
The proportion of office visits with subsequent lapses in care varied by provider recommended follow-up timeframe. (Table 2) When the recommended follow-up was less than 8 weeks, 29.9% of office visits were followed by a lapse in care; with 32-52 weeks recommended, the lapse rate was 42.7%.
Table 2Proportion of office visits followed by lapses in diabetic retinopathy care.
On average, patients had 2.3 office visits in the two-year observation window. More than 3 in 4 (77.6%) of patients had a lapse in their diabetic retinopathy care at least once during this time. (Table 3) When stratified by the providers’ recommended follow-up, the rate of any lapse in care decreased with the scheduled time until the next visit. For example, when the recommendation was less than or equal to 8 weeks, we estimate a lapse in care rate of 96.0%; when between 32 to 52 weeks to the recommended visit, 71.5% had a lapse in care. (Table 3)
Table 3Proportion of patients with diabetes mellitus who have ever had a lapse in diabetic retinopathy care and proportion of office visits per patient categorized as a lapse in care. Both outcomes are stratified by median provider recommend follow-up timeframe and total number of visits in the two-year observation window.
Recommended follow-up timeframe by the provider
Number of Office Visits
Proportion of patients who ever had a lapse in care
Average proportion of office visits with lapses in care
On the patient-level, the proportion of office visits for each patient categorized as a lapse in care averaged 60.1%. (Table 3) This value did not vary much when stratified by the providers’ follow-up timeframe, but differed significantly depending on the total number of office visits in the two-year observation window. By definition, 100.0% of office visits among patients who had only the index visit during the two year window were categorized as lapses in care. The lapses in care decreased to 31.8% among those with 2 visits, and 19.8% among those with 3 or more visits. (Table 3) Patients who only had 1 office visit in the two-year observation period, compared with those who had two or more office visits, were younger (mean 60.1 years compared to 62.5 years, p<0.001), more frequently non-Hispanic Black or Hispanic persons (43.6% compared to 39.8%, p<0.001), and had Medicaid or no insurance (15.9% compared to 11.5%, p<0.001). (Supplemental Table 4)
In multivariable analysis, after controlling for age, sex, severity of diabetic retinopathy, other retinal disorders, and glaucoma, non-Hispanic Black patients were estimated to have a 24% increased odds of ever having a lapse in care compared to non-Hispanic White patients. Hispanic patients had an estimated 26% increased odds, and patients of other race/ethnicity had an estimated 38% increased odds (p<0.001, respectively), as compared to non-Hispanic White patients. (Table 4) non-Hispanic Black patients had similar odds for having a lapse in care compared to Hispanic patients, but decreased odds compared to those of other race/ethnicity (p<0.05). Hispanic patients had similar odds of having a lapse in care compared to patients with other race/ethnicity.
Table 4Adjusted odds ratio (OR) of having a lapse in diabetic retinopathy care, results from the multivariable mixed effects logistic regression model with a random person effect (N = 39,561).
Older patients, those >45 to ≤65 and >65 years of age, had lower odds of ever having a lapse in care compared to those ≤20 years of age. Sex was not associated with lapses in care. Having Medicaid or no insurance was associated with higher odds of lapses in care, and having other insurance was associated with lower odds. Patients with diagnosed diabetic retinopathy had lower odds of having a lapse in care—those with non-proliferative diabetic retinopathy were estimated to have a 53% decreased odds, and those with proliferative diabetic retinopathy had an estimated 69% decreased odds (p<0.001, respectively) compared to patients without diabetic retinopathy. A diagnosis of other retinal disorders, including age-related macular degeneration, vein occlusion, and artery occlusion, was also associated with an estimated 24% lower odds of ever having a lapse in care. Finally, having glaucoma was associated with a 18% increased odds for a lapse in diabetic retinopathy care.
Sensitivity Analysis:
Similar results were seen in the cohort of patients (N=9,560) who were seen after the ICD-9 to ICD-10 transition and prior to the COVID-19 pandemic from 1/1/2016 to 12/30/2019. (Supplemental Table 5, Supplemental Table 6) non-Hispanic Black patients and patients with other race/ethnicity had increased odds for ever having a lapse in care compared to non-Hispanic White patients. Hispanic patients also had increased odds for a lapse in care that was not statistically significant. (Supplemental Table 6)
Discussion:
In a large clinical sample, we found that rates of lapses in diabetic retinopathy care were very high. Overall, we estimated that more than three in four patients had at least one office visit followed by a lapse in care in the two-year observation window. The overwhelming majority patients who were recommended to frequent follow-up (≤8 weeks, or >8 to ≤20 weeks) had a lapse in care. Those who were recommended to less frequent follow-up (>20 to ≤32 weeks, or >32 to ≤52 weeks), had fewer lapses. Patients who had more overall office visits (2 or 3+) in the two-year period had lower rates of lapses in care compared to those who only had one office visit. More than half (60.1%) of any given patient’s office visits were followed by a lapse in care. Non-Hispanic Black patients, Hispanic patients, and those of other race/ethnicity were more likely to have lapses in diabetic retinopathy care than otherwise similar non-Hispanic White patients.
The rates of lapses in diabetic retinopathy care reported in this study are largely consistent with those in the published literature. When using a fixed interval of time to define a lapse in care, other observational studies in a clinical setting have found rates ranging from 16% to 61%.
Treatment compliance and adherence among patients with diabetic retinopathy and age-related macular degeneration treated by anti-vascular endothelial growth factor under universal health coverage.
Outcomes of Eyes Lost to Follow-up with Proliferative Diabetic Retinopathy That Received Panretinal Photocoagulation versus Intravitreal Anti-Vascular Endothelial Growth Factor.
In this study, we found 44.6% had a lapse in diabetic retinopathy care when applying a similar definition of having an interval of ≥6 months between visits (data not shown). Rates of lapses in diabetic retinopathy care in routine clinical practice are higher than those found in clinical trials. Using our novel methodology for identifying lapses in care, we were able to apply an adaptive definition of lapses like the clinical trial studies.
In the context of the clinical trial, 55.3% of patients had one or more long lapse in care, but in our study, rates were much higher at 77.6%. Not surprisingly, most patients recommended for frequent follow-up had a lapse in care due to their having more opportunity to do so. The higher rates of lapses in care found in this study are also driven by a large proportion (45.2%) of patients who only had one office visit for diabetic retinopathy screening or treatment in the two-year observation window. Most prior studies using retrospective data excluded this group of patients by restricting the analysis to patients who have presented at least twice.
Treatment compliance and adherence among patients with diabetic retinopathy and age-related macular degeneration treated by anti-vascular endothelial growth factor under universal health coverage.
Outcomes of Eyes Lost to Follow-up with Proliferative Diabetic Retinopathy That Received Panretinal Photocoagulation versus Intravitreal Anti-Vascular Endothelial Growth Factor.
In our cohort, patients who only had one office visit had similar characteristics as those with lapses in care—they were younger, more frequently of non-Hispanic Black or Hispanic race and ethnicity, and with Medicaid or no insurance. Future interventions to decrease lapses in care should identify and assist this population.
The finding that non-Hispanic Black and Hispanic patients had a higher odds of a lapse in diabetic retinopathy care is consistent with other reports of disparities in diabetic eye care. These same populations bear a disproportionate burden of diabetes, present with more severe diabetic retinopathy and diabetic macular edema, and sustain more vision loss from diabetic eye disease.
Is the risk of diabetic retinopathy greater in non-Hispanic blacks and Mexican Americans than in non-Hispanic whites with type 2 diabetes? A U.S. population study.
The root causes underlying racial and ethnic disparities in diabetic retinopathy care likely can be traced to structural inequities that have led to the maldistribution of social determinants of health (SDoH)--the social, economic, and physical conditions in which people live, work, and play that affect health outcomes.
Organization WH.A conceptual framework for action on the social determinants of health. https://www.who.int/sdhconference/resources/ConceptualframeworkforactiononSDH_eng.pdf ; 2010 Accessed 1/2/2021.
Future work to better understand the SDoH that underlie higher rates of lapses in diabetic retinopathy care can be a means by which to address and eliminate this health disparity.
Our findings of other variables associated with lapses in care are in line with prior reports. Older patients are less likely to have a lapse in diabetic retinopathy care, consistent with other studies finding that the older population is more likely to have high eye care utilization.
Consistent with other studies, we found that having Medicaid or no insurance is associated with higher odds of having lapses in care. Having other insurance types was associated with lower odds of care lapses in our study. This is likely because this category included insurances such as Tricare, worker’s compensation, and international coverage. Patients with worker’s compensation or international coverage could be specifically seeking care in ophthalmology due to an acute issue in addition to their more long-term diabetic retinopathy care. Having a diagnosed retinal disorder, whether diabetic retinopathy, or another retinal disorder decreased the odds for a lapse in care, which is consistent with other studies.
However, having another major eye disease, glaucoma, increased the odds for a lapse in care. It could be that patients were busy with glaucoma treatment and thus delayed care for their diabetic retinopathy.
This study has several limitations. 1) The proposed pipeline relies on NLP algorithms that do not have perfect sensitivity and specificity, but the performance is in line with other algorithms targeted towards follow-up.
The NLP algorithm to identify office visits related to diabetic retinopathy screening and treatment behaves conservatively with higher specificity than sensitivity. As such, we are more likely to overestimate than underestimate lapses in diabetic retinopathy care. The NLP algorithm to identify a providers’ recommended follow-up timeframe tended to identify longer follow-up as compared to the gold standard grader (on average 0.05 weeks). Identifying a longer follow-up timeframe would cause the algorithm to underestimate lapses in diabetic retinopathy care. It is possible that these two components of the pipeline (one that overestimates and one that underestimates lapses in care) balance each other out. 2) Because we are relying on data from one institution, we do not know whether the patients sought diabetic retinopathy care elsewhere. Clinical experience and other studies suggest that most patients tend to stay at the same practice for longitudinal care.
However, the degree to which this is true needs to be studied more directly in a future study. 3) We were unable to evaluate the presence of other medical comorbidities that might impact a patient’s ability to follow-up for eye care. Most of this population only had office visits at the Wilmer Eye Institute and no available diagnosis codes for other systemic medical conditions (e.g., hypertension). Future NLP based studies are needed to extract systemic medical conditions from ophthalmology progress notes to further refine risk factors for lapses in care. 4) We also do not account for patient death in this analysis because the exact date of death is not reliably captured in the EHR. We presume this affects a small proportion of our cohort since the average age of this population is 61.4 years, we observe them for only 2 years, the average life expectancy in the United States is 77 years, and only 14% of our patients over the age of 75.
Arias E, Tejada-Vera B, Ahmad F, Kochanek KD. Provisional life expectancy estimates for 2020. Hyattsville, MD: National Center for Health Statistics; 2021.
5) Lastly, because we are relying on ICD codes for diagnoses, we do not know the extent to which diagnosis codes represent reality. There could be patients with vision threatening diabetic retinopathy who are not coded as such in the EHR. Future studies should be aimed at understanding the clinical significance of lapses in care and should offer additional resources and support for the population(s) at highest risk for vision loss and lapses in care to offer additional resources and support.
Our novel methodology of adaptively defining lapses in diabetic retinopathy care based on the providers’ recommended follow-up timeframe in the EHR confers numerous advantages. The more sensitive definition allows the detection of a lapse in care before a pre-set timeframe, such as 6 months, has elapsed. This definition also allows for identification of patients who only have one office visit followed by a lapse in care. This methodology could be deployed in real-time in the EHR to identify lapses in diabetic retinopathy care and create clinical decision support systems.
For example, once a lapse has been identified, the system could alert administrative staff to call and follow-up with patients. The clinical decision support system could also be deployed at the point of care and assist clinicians. For example, the system could show the treating clinician a patient’s history of lapses in care and future risk for lapses. Knowing the patient’s risk for a lapse in care can help guide choice of ophthalmic treatment, for example choosing between anti-vascular endothelial growth factor or panretinal photocoagulation for proliferative diabetic retinopathy, or offering social work support to address SDoH. This methodology can also be expanded to monitoring other eye diseases that require routine follow-up to prevent blindness, for example glaucoma.
The approach presented in this study can be implemented at other institutions. The NLP tools used to identify diabetic retinopathy screening or treatment visits, and the ones to extract a providers’ recommended follow-up timeframe are available on GitHub.
Deploying these tools on unstructured provider notes in combination with structured data elements in the EHR will allow researchers at other institutions to extract the two components needed to identify lapses in diabetic retinopathy care. However, the performance of these tools at other institutions, or external validity, is currently unknown. Rule-based NLP algorithms can have limited generalizability outside the text used to develop it, thus modifications of the search terms and rules will likely be needed. Validation studies should be performed, and the NLP tools should be tailored to optimize the output for the unique documentation styles at other institutions. Furthermore, the NLP algorithms developed were specific to the task of identifying lapses in diabetic retinopathy care. Researchers who wish to adapt the code to identify lapses in other types of ophthalmic care will also likely need to modify the rules to include target terms specific to a subspecialty (e.g., intraocular pressure for glaucoma care).
In conclusion, we have developed a novel methodology for identifying lapses in diabetic retinopathy care in the EHR and demonstrate high rates of lapses in diabetic retinopathy care in the clinical practice setting. Furthermore, we have identified health disparities whereby non-Hispanic Black, Hispanic, and other race/ethnicity populations have higher odds for lapses in diabetic retinopathy care compared to non-Hispanic White patients. This methodology is the first step towards EHR-based solutions such as clinical decision support systems that could be leveraged to eliminate disparities in diabetic retinopathy care.
Treatment compliance and adherence among patients with diabetic retinopathy and age-related macular degeneration treated by anti-vascular endothelial growth factor under universal health coverage.
Outcomes of Eyes Lost to Follow-up with Proliferative Diabetic Retinopathy That Received Panretinal Photocoagulation versus Intravitreal Anti-Vascular Endothelial Growth Factor.
Is the risk of diabetic retinopathy greater in non-Hispanic blacks and Mexican Americans than in non-Hispanic whites with type 2 diabetes? A U.S. population study.
Organization WH.A conceptual framework for action on the social determinants of health. https://www.who.int/sdhconference/resources/ConceptualframeworkforactiononSDH_eng.pdf ; 2010 Accessed 1/2/2021.
Arias E, Tejada-Vera B, Ahmad F, Kochanek KD. Provisional life expectancy estimates for 2020. Hyattsville, MD: National Center for Health Statistics; 2021.
This project was supported by a Career Development Award from the Research to Prevent Blindness (CXC), National Center for Advancing Translational Sciences funded Clinical and Translational Science Award Grant KL2TR003099 (CXC), K23 award from the NIH/NEI (award number K23EY033440) (CXC), and an unrestricted grant from Research to Prevent Blindness (Wilmer Eye Institute). Dr. Cai is the Jonathan and Marcia Javitt Rising Professor of Ophthalmology. The sponsor or funding organizations had no role in the design or conduct of this research.
Conflict of Interest:
No conflicting relationship exists for any author.
Precis:
Our novel methodology to identify lapses in diabetic retinopathy care in the electronic health record finds high rates of lapses in the clinical practice setting, particularly among non-Hispanic Black and Hispanic patients.
00027-1&id=gr1.jpg){kind=link}
00027-1&id=gr2.jpg){kind=link}