If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Department of Ophthalmology and Visual Sciences, John Moran Eye Center, University of Utah, Salt Lake City, UtahDepartment of Population Health Sciences, University of Utah, Salt Lake City, Utah
Department of Ophthalmology and Visual Sciences, Center for Eye Policy & Innovation, Kellogg Eye Center, University of Michigan, Ann Arbor, MichiganInstitute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, MichiganDepartment of Health Management and Policy, University of Michigan School of Public Health, Ann Arbor, Michigan
Department of Population Health Sciences, University of Utah, Salt Lake City, UtahDepartment of Internal Medicine, University of Utah, Salt Lake City, Utah
To rigorously develop a prototype clinical decision support (CDS) system to help clinicians determine the appropriate timing for follow-up visual field testing for patients with glaucoma and to identify themes regarding the context of use for glaucoma CDS systems, design requirements, and design solutions to meet these requirements.
Design
Semistructured qualitative interviews and iterative design cycles.
Participants
Clinicians who care for patients with glaucoma, purposefully sampled to ensure a representation of a range of clinical specialties (glaucoma specialist, general ophthalmologist, optometrist) and years in clinical practice.
Methods
Using the established User-Centered Design Process framework, we conducted semistructured interviews with 5 clinicians that addressed the context of use and design requirements for a glaucoma CDS system. We analyzed the interviews using inductive thematic analysis and grounded theory to generate themes regarding the context of use and design requirements. We created design solutions to address these requirements and used iterative design cycles with the clinicians to refine the CDS prototype.
Main Outcome Measures
Themes regarding decision support for determining the timing of visual field testing for patients with glaucoma, CDS design requirements, and CDS design features.
Results
We identified 9 themes that addressed the context of use for the CDS system, 9 design requirements for the prototype CDS system, and 9 design features intended to address these design requirements. Key design requirements included the preservation of clinician autonomy, incorporation of currently used heuristics, compilation of data, and increasing and communicating the level of certainty regarding the decision. After completing 3 iterative design cycles using this preliminary CDS system design solution, the design was satisfactory to the clinicians and was accepted as our prototype glaucoma CDS system.
Conclusions
We used a systematic design process based on the established User-Centered Design Process to rigorously develop a prototype glaucoma CDS system, which will be used as a starting point for a future, large-scale iterative refinement and implementation process. Clinicians who care for patients with glaucoma need CDS systems that preserve clinician autonomy, compile and present data, incorporate currently used heuristics, and increase and communicate the level of certainty regarding the decision.
Financial Disclosure(s)
Proprietary or commercial disclosure may be found after the references.
Modern clinical decision support (CDS) systems are computer-based tools designed to improve clinician decision making for individual patients at the point the decisions are made.
A systematic review of the effectiveness of interruptive medication prescribing alerts in hospital CPOE systems to change prescriber behavior and improve patient safety.
Glaucoma is a complex chronic disease. Clinicians who care for patients with glaucoma must make hundreds of critical clinical decisions efficiently in the midst of busy clinics.
Determining the timing of follow-up visual field testing is a clinical decision that is particularly challenging for clinicians who care for patients with glaucoma.
Visual field testing frequency should be tailored according to individual patient characteristics, including disease stage and test-retest variability.
In a United States nationwide cohort of patients with glaucoma, nearly three quarters of patients did not undergo visual field testing at the guideline-recommended frequency.
Clinical decision support systems could help clinicians determine the appropriate timing for visual field testing for individual patients with glaucoma.
The objective of this study was to use the User-Centered Design Process framework to rigorously develop a prototype CDS system to help clinicians determine the appropriate timing for follow-up visual field testing for patients with glaucoma. This prototype will be used as a starting point for a larger scale iterative refinement and electronic health record (EHR) integration process. The focus of this study was on the design of the CDS system itself rather than the recommendations it provides. Systematically using the User-Centered Design Process framework allowed us to identify themes regarding the context of use for the CDS system, design requirements, and design solutions to meet these requirements. These findings serve as a starting point for the future use of the User-Centered Design Process framework to design other informatics solutions and CDS systems to improve glaucoma care and decision making.
Methods
This qualitative study used deidentified data from semistructured telephone interviews with 5 clinicians who care for patients with glaucoma. These clinicians were purposefully sampled from members of the Utah Ophthalmology Society and the Utah Optometric Association to ensure a representation of a range of clinical specialties (glaucoma specialist, general ophthalmologist, optometrist), practice settings (academic medical center, private practice, and large multispecialty clinic), and years in clinical practice. A sample size of 5 was chosen a priori as a reasonable number for development of a prototype design based on published literature, with a plan to interview additional clinicians if thematic saturation was not achieved.
This is an iterative process that involves 4 phases: specifying the context of use, specifying design requirements, creating design solutions, and evaluating and refining designs. The goal of this framework is to ensure design solutions meet the needs of those who will be using the product.
Step 1: Specifying the Context of Use
The semistructured interviews were conducted over telephone by a single researcher (B.C.S.) in May 2021. The interview guide covered 4 major questions: (1) How do you determine the appropriate timing for the next visual field test for a patient with glaucoma? (2) What factors influence this decision? What might make you do it more often? Less often? (3) What factors make this decision more challenging? Easier? (4) What would help you make this decision? Before conducting the interviews, we piloted and refined these questions with 2 user-centered design researchers and 1 ophthalmologist who were not involved with the project to evaluate for clarity and comprehension. During the interviews, the interviewer recorded clinician responses to the questions by making notes. These notes were completely deidentified and no information that would link the notes to the clinicians was retained.
Two researchers (B.C.S. and B.T.) reviewed the notes from the interviews to identify distinct statements. The 2 researchers independently analyzed the statements using inductive thematic analysis and grounded theory.
The researchers evaluated the statements looking for themes that addressed Step 1 (specifying the context of use) and Step 2 (specifying design requirements). The 2 researchers then met to develop a consensus codebook of themes. A third reviewer was available to adjudicate disagreements (D.B.). After the codebook of themes was established, the 2 researchers independently recoded all the statements using the final coding scheme. The 2 researchers again met to reach consensus on the coding of the statements, with the third researcher again available to resolve disagreements. The 2 researchers reviewed the themes addressing the context of use for the prototype CDS system and developed a brief statement to describe the context of use for the system, with the third researcher available to adjudicate disagreements.
Step 2: Specifying Design Requirements
Two researchers (B.C.S. and B.T.) reviewed the statements and themes to identify design requirements. As the first step, the 2 researchers mapped the themes to the fundamental human need framework.
In this framework, the fundamental human needs domains are subsistence, security, affection, understanding, participation, idleness, creation, identity, and freedom. The 2 researchers then developed design requirements based on the themes and fundamental human needs. A third reviewer (D.B.) adjudicated disagreements.
Step 3: Creating Design Solutions
Based on the interviews and themes identified above, 2 researchers (B.C.S. and B.T.) developed user stories that depicted typical situations where a user would benefit from the CDS system. The research team then developed a CDS system design solution to meet these needs using Figma software (Figma Inc).
Step 4: Evaluating and Refining Designs
We showed the design solution to the 5 clinicians who care for patients with glaucoma described above and 3 informatics and design researchers. After interacting with the design solution, the clinicians and researchers provided feedback, which was incorporated in a second iterative design solution. We followed this iterative design loop for 2 additional iterations. The third iteration was satisfactory to the clinicians and researchers and was accepted as our prototype glaucoma CDS system for determining the appropriate timing of follow-up visual field testing. Throughout this process, design suggestions from the clinicians and researchers were recorded.
Results
We conducted semistructured telephone interviews with 5 clinicians who care for patients with glaucoma. There was a range of clinical specialties (2 glaucoma specialists, 2 general ophthalmologists, and 1 optometrist), practice settings (3 from academic medical centers, 1 from private practice, and 1 from a large multispecialty clinic), and years in clinical practice (2 with > 15 years and 3 with < 15 years in clinical practice). During the interviews, the clinicians made a total of 41 distinct statements.
Step 1: Specifying the Context of Use
We grouped the clinician statements into 9 themes that addressed the context of use for the CDS system. It appeared that thematic saturation was attained as each theme was discussed by > 1 clinician. The themes and representative statements for these themes are listed in Table 1. Four of the themes described how the clinicians currently make decisions regarding the timing of visual field testing, 1 theme described the importance of clinician experience in this decision, and 4 themes described clinician desires that would improve their ability to make this decision. Clinicians described making the decision regarding the timing of visual field testing based on intuition, broad heuristics, patient data and findings, and patient preferences. Clinicians desire compiled and easy-to-use data, more certainty when making this decision, help communicating with patients, and help keeping track of the published literature. Clinicians with less glaucoma experience reported more difficulty with the decision.
Table 1Themes Addressing the Context of Use for a Clinical Decision Support System to Improve Decision Making Regarding the Timing of Follow-Up Visual Field Testing for Patients with Glaucoma
Theme
Illustrative Statements
How clinicians currently make decisions regarding the timing of visual field testing
Timing adjustments are based on intuition
“Overall, I rely on my feeling more than on actual data.” “My decision about timing is pretty arbitrary.”
Broad heuristics are used to adjust timing
“Using standard timings makes it easier for me to keep track of.” “For mild or moderate glaucoma, visual field every year.”
Patient data and findings are used to adjust timing
“I do fields more frequently if I see progression.” “It’s different if they have a larger cup to disc.”
Patient preferences are used to adjust timing
“Patient convenience influences my decision.”
Clinicians with less glaucoma experience reported more difficulty determining the timing of visual field testing
Clinicians with less glaucoma experience have more difficulty
“I don’t see very much glaucoma, so it’s hard for me.” “It’s hard because my patients with glaucoma are mixed in with patients with many other conditions.”
Clinicians desire help with determining the timing of visual field testing
Clinician desires compiled and easy to use data
“Gathering the data takes longer than evaluating it.” “It’s difficult to keep track of all of the patient data.”
Clinician desires more certainty when determining timing
“I wish I could have more certainty about my recommendations.” “Sometimes, I’m not sure if the testing is necessary.”
Clinician desires help communicating with patients regarding timing
“It’s hard to explain to patients why it’s important.” “It’s difficult for me to convince patients.”
Clinician desires help keeping track of published literature regarding timing
“There is a lot of different published data, it’s difficult to incorporate into practice.” “The recommendations now are different than when I was in residency.”
Clinician illustrative statements are representative of the other statements categorized in each theme.
From these themes the researchers developed a statement describing the context of use for the prototype CDS system: design a CDS system to help clinicians who care for patients with glaucoma determine the timing for a follow-up visual field test for an individual patient with glaucoma because determining a personalized timing for follow-up testing must be done quickly in the middle of a busy clinic and requires the synthesis of a large amount of data that is scattered in different places in the EHR.
Step 2: Specifying Design Requirements
The 9 themes that addressed the context of use for the CDS system mapped to 7 domains of the fundamental human need framework—security, freedom, creation, participation, idleness, protection, and understanding (see Table 2), with 2 themes matching to each of security and participation. From the themes and their corresponding fundamental human needs, the researchers developed 9 design requirements for the prototype CDS system to help clinicians who care for patients with glaucoma (see Table 2). Key design requirements included the preservation of clinician autonomy, incorporation of currently used heuristics, compilation of data, and increasing and communicating the level of certainty regarding the decision.
Table 2Clinician Themes Regarding Decision Support for Determining the Timing of Visual Field Testing for Patients with Glaucoma Mapped to Fundamental Human Needs Domains, Design Requirements, and Design Features
Theme
Fundamental Human Need Domain
Clinical Decision Support Design Requirement
Clinical Decision Support Design Feature
Timing adjustments are based on intuition
Creation
Design should allow for clinician autonomy
Sliding widgets to allow clinician inputs for assumptions
Broad heuristics are used to adjust timing
Security
Design should incorporate currently used heuristics
Presentation of data commonly used to guide heuristics such as number of medications and central corneal thickness
Patient data and findings are used to adjust timing
Freedom
Design should incorporate patient data and findings
Presentation of patient data and findings such as eye pressures and prior visual fields
Patient preferences are used to adjust timing
Participation
Design should recognize patient preferences
Sliding widgets to allow patient input for assumptions
Clinicians with less glaucoma experience have more difficulty
Security
Design should be accessible to non-experts who care for glaucoma
Option to click on different domains in the design for additional detail and explanation
Clinician desires compiled and easy to use data
Idleness
Design should compile and present data in an efficient, easy to interpret format
Most pertinent data visible in a single initial user interface
Clinician desires more certainty when determining timing
Protection
Design should improve certainty and communicate the level of certainty
Clear recommendations provided with statistical information to show level of certainty
Clinician desires help communicating with patients regarding timing
Participation
Design should improve communication with patients
Display of data in graphical form to facilitate description to patients
Clinician desires help keeping track of published literature regarding timing
Understanding
Design should incorporate published literature and inform users
Option to click on different domains for additional explanation and links to supporting published literature
Based on the interviews and themes identified above, the researchers (B.C.S. and B.T.) developed 3 user stories that depicted typical situations where a user would benefit from the CDS system.
(1)
A glaucoma specialist who wants a recommendation for follow-up that is nuanced, so she is able to see the impact of alternative options.
(2)
A general ophthalmologist who wants a recommendation for follow-up that is presented efficiently, so he makes good decisions while seeing a lot of patients.
(3)
An optometrist who wants a recommendation for follow-up that she trusts, so she knows her patient is getting appropriate care.
With these user stories in mind, the research team then developed an interactive preliminary CDS system design solution to meet these needs (see Fig 1A).
After completing 3 iterative design cycles using this preliminary CDS system design solution with 5 clinicians who care for patients with glaucoma and 3 informatics and design researchers, the design was satisfactory to the clinicians and was accepted as our prototype glaucoma CDS system for determining the appropriate timing of follow-up visual field testing (see Fig 1B). During this iterative design process, we identified 9 design features intended to address the 9 CDS design requirements described in Step 2 above (Table 2). For example, clinicians expressed their desire to retain autonomy. The design feature to address this requirement was to use sliding widgets that allowed clinicians to see the impact of their decisions under varying assumptions. During the iterative design process, clinicians felt the sliding widgets allowed for both autonomy and data-driven guidance. Clinicians also desired complied and easy to use data. The design feature to address this requirement was to make pertinent data (such as prior eye surgeries, medications, and prior eye pressure measures) visible in a single initial user interface.
Discussion
Using the User-Centered Design Process framework, we developed a prototype CDS system to help clinicians determine the appropriate timing for follow-up visual field testing for patients with glaucoma. During this systematic design process, we identified design requirements for this CDS system and iteratively developed design solutions to address these needs. We learned that clinicians who care for patients with glaucoma need CDS systems that preserve clinician autonomy, compile and present data, incorporate currently used heuristics, and increase and communicate the level of certainty regarding the decision. The CDS prototype developed in this study will be used as a starting point for a future large-scale iterative refinement and EHR integration process.
During the qualitative interviews, clinicians reported that the effort and cognitive load from data gathering and visualization were key challenges in deciding the appropriate timing for follow-up visual field testing. This correlates with prior research showing that data gathering increases cognitive load and emotional distress and contributes to clinician burnout.
Clinicians caring for patients with glaucoma must incorporate large amounts of longitudinal data from many sources accurately and efficiently to make decisions.
During our iterative design process, clinicians strongly preferred design solutions that presented most of the pertinent data in a single user interface. Future CDS systems and informatics interventions for glaucoma should also use design solutions that address the challenge of data gathering and visualization. To make this possible, vendors will need to ensure that imaging and testing devices that monitor glaucoma use data standards that are interoperable and allow for data sharing. There is a good business case for vendors to improve interoperability as integrated data display is something that clinicians strongly desire and has the potential to improve patient care.
Preserving cognitive autonomy was important to the clinicians during both the cognitive interviews and iterative design process. This desire for cognitive autonomy is in line with prior work in the field of CDS as systems that preserve cognitive autonomy are more likely to be used.
The clinicians also desired data-driven guidance to increase certainty when determining the appropriate timing for follow-up visual field testing. Our design solution to allow for both cognitive autonomy and data-driven guidance was to use sliding widgets that allowed clinicians to see the impact of their decisions under varying circumstances. As we work to design CDS systems and informatics interventions to improve glaucoma care, we should remember the importance of supporting clinician autonomy.
Ophthalmologists and optometrists see many patients during clinic and experience serious time constraints.
In a prior study, nearly 90% of ophthalmologists and optometrists who care for patients with glaucoma felt that “minimal time required to use tool” would be an important characteristic of CDS systems for glaucoma.
In the present study, clinicians also felt that having data in an efficient, easy to interpret format was important. In future work, we plan to iteratively refine the prototype CDS system described in this paper in a large-scale workflow and EHR integration process. One of the major focuses of this integration project will be to ensure that the CDS system is efficient and requires minimal time for the clinicians to use. During this workflow integration, we will evaluate the appropriate timing in the workflow for CDS presentation and the appropriate role, if any, of prompts in the EHR to promote use of the CDS system.
Though not the focus of this manuscript, understanding the recommendations provided by this prototype CDS system is important. The recommendations will be provided by a statistical calculator based on prior work evaluating the variability of visual field tests.
This statistical calculator uses a power calculation to determine the frequency of visual field tests spread over a set period of time necessary to estimate a defined rate of visual field loss given a patient’s test variability with specified statistical power. Though this calculator is based on an accepted understanding of visual field test variability and statistical power, it will be important in future work to develop a strong evidence base for its effectiveness, as we found that clinicians desire confidence and certainty in the recommendations provided by CDS systems for glaucoma.
Our study had limitations. We had an a priori small sample size of 5 clinicians, though we appeared to reach thematic saturation and this may be a reasonable sample size for development of a prototype design based on published literature.
The CDS prototype developed in this study will be further developed and evaluated in a future large-scale iterative refinement and implementation process. Our study used a purposeful rather than a representative sample. Because of this, our findings are not intended to be statistically representative of all clinicians who care for patients with glaucoma. However, because we used a purposeful sample that included an appropriate range of clinical specialties and years in clinical practice, our findings provide an inclusive starting point for understanding the design requirements for a CDS system to help clinicians determine the appropriate timing for follow-up visual field testing for patients with glaucoma. Our study was not designed to address the recommendations provided by the prototype CDS system. The focus of this study was on the user-centered design of this prototype and understanding the design requirements for glaucoma CDS. This study was not quantitative and thus did not allow for statistical evaluation and hypothesis testing. The prototype glaucoma CDS system will be evaluated quantitatively in the future. As a preliminary design project, we employed telephone interviews and interactions with the prototype CDS system for our iterative design process. In future work, we plan to use cognitive task analyses, think-aloud methods, and direct observation of clinicians using the CDS system. These methods should provide a more nuanced understanding of how the users interact with the EHR and CDS system.
In this study, we used a systematic design process based on the established User-Centered Design Process framework to rigorously develop a prototype CDS system to help clinicians determine the appropriate timing for follow-up visual field testing for patients with glaucoma. The prototype CDS system we developed will be used as a starting point for a future, large-scale iterative refinement and EHR integration process. The systematic design process we used serves as a pattern for the future development of user-centered informatics solutions and CDS systems to improve glaucoma care. During the systematic design process, we identified design requirements for this CDS system and iteratively developed design solutions to address these needs. Clinicians who care for patients with glaucoma need CDS systems that preserve clinician autonomy, compile and present data, incorporate currently used heuristics, and increase and communicate the level of certainty regarding the decision.
References
Tcheng J.E.
National Academy of Medicine (U.S.)
Optimizing Strategies for Clinical Decision Support: Summary of a Meeting Series.
A systematic review of the effectiveness of interruptive medication prescribing alerts in hospital CPOE systems to change prescriber behavior and improve patient safety.
All authors have completed and submitted the ICMJE disclosures form.
The authors made the following disclosures:
F.A.M.: Grants – Google, Inc, Carl-Zeiss Meditec, National Eye Institute, during the conduct of the study; Personal fees – Aerie Pharmaceuticals, Allergan, Novartis, Biogen, Galimedix, Annexon, Heidelberg Engineering, Stuart Therapeutics, IDx, Reichert. J.D.S.: Research grant – Abbvie, outside the submitted work. The other authors have no conflicts related to the proposed work to report.
Financial Support: This work was supported by National Institutes of Health (K23EY032577 to B.C.S., R01EY029885 to F.A.M., R01EY026641 to J.D.S.), an Unrestricted Grant from Research to Prevent Blindness, New York, NY, to the Department of Ophthalmology & Visual Sciences, University of Utah (B.C.S.) and the Department of Ophthalmology & Visual Sciences, University of Utah (J.D.S), and a Mentoring for the Advancement of Physician Scientists (M.A.P.S) grant from the American Glaucoma Society, San Francisco, California (B.C.S). The funding organizations had no role in the design or conduct of this research.
HUMAN SUBJECTS: This study was evaluated by the University of Utah Institutional Review Board and deemed exempt. The study adhered to the Declaration of Helsinki. This is a qualitative study using de-identified subject details.
00011-8&id=gr1.jpg){kind=link}