Advertisement

DDLSNet: A novel deep learning-based system for grading funduscopic images for glaucomatous damage

Open AccessPublished:November 11, 2022DOI:https://doi.org/10.1016/j.xops.2022.100255

      Abstract

      Purpose

      To report an image analysis pipeline, DDLSNet, consisting of a rim segmentation (RimNet) branch and a disc size classification (DiscNet) branch to automate estimation of the disc damage likelihood scale (DDLS).

      Design

      Retrospective observational.

      Participants

      RimNet and DiscNet were developed with 1208 and 11536 optic disc photographs (ODP), respectively. DDLSNet performance was evaluated on 120 ODPs from the RimNet test set, for which the DDLS scores were graded by clinicians. Reproducibility was evaluated on a group of 781 eyes, each with two ODPs taken within four years apart.

      Methods

      DDLS calculation requires estimation of optic disc size, provided by DiscNet (VGG19 network), and the minimum rim-to-disc ratio (mRDR) or absent rim width (ARW), provided by RimNet (InceptionV3/LinkNet segmentation model). To build RimNet’s dataset, glaucoma specialists marked optic disc rim and cup boundaries on ODPs. The ‘ground truth’ mRDR or ARW was calculated. For DiscNet’s dataset, corresponding OCT images provided ‘ground truth’ disc size. ODPs were split into 80/10/10 for training, validation, and testing, respectively for RimNet and DiscNet. DDLSNet estimation was tested against manual grading of DDLS by clinicians with the average score used as ‘ground truth’. Reproducibility of DDLSNet grading was evaluated by repeating DDLS estimation on a dataset of non-progressing paired ODPs taken at separate times.

      Main Outcome Measures

      The main outcome measure was a weighted kappa score between clinicians and the DDLSNet pipeline with agreement defined as ±1 DDLS score difference.

      Results

      RimNet achieved an mRDR mean absolute error (MAE) of 0.04 (±0.03) and an ARW MAE of 48.9 (±35.9) degrees when compared to clinician segmentations. DiscNet achieved 73% (95% CI: 70%, 75%) classification accuracy. DDLSNet achieved an average weighted kappa agreement of 0.54 (95% CI: 0.40, 0.68) compared to clinicians. Average inter-clinician agreement was 0.52 (95% CI: 0.49, 0.56). Reproducibility testing demonstrated that 96% of ODP pairs had a difference of 1 DDLS score.

      Conclusions

      DDLSNet achieved moderate agreement with clinicians for DDLS grading. This novel approach illustrates the feasibility of automated ODP grading for assessing glaucoma severity. Further improvements may be achieved by increasing the number of incomplete rims sample size, expanding the hyperparameter search, and increasing the agreement of clinicians grading ODPs.

      Key Words

      Abbreviations/Acronyms:

      RimNet (rim segmentation model), DiscNet (disc size classification model), DDLS (disc damage likelihood scale), mRDR (minimum rim-to-disc ratio), ARW (Absent Rim Width), MAE (mean average error), ODP (optic disc photograph), OCT (optical coherence tomography), RimIoU (intersection over union of the optic disc rim), IoU (intersection over union)

      Introduction

      Glaucoma is the leading cause of irreversible blindness worldwide with an estimated 80 million people affected in 2020 and a projected rise to 111.8 million people by 2040

      Giangiacomo A, Coleman AL. The Epidemiology of Glaucoma. In: Glaucoma. Berlin, Heidelberg: Springer Berlin Heidelberg; :13–21.

      ,
      • Tham Y.C.
      • Li X.
      • Wong T.Y.
      • et al.
      Global Prevalence of Glaucoma and Projections of Glaucoma Burden through 2040: A Systematic Review and Meta-Analysis.
      . Glaucoma is asymptomatic in the early stages; untested individuals often remain undiagnosed until advanced symptoms are present. In developed countries, up to 70% of patients with glaucoma are undiagnosed, a number that rises in areas with less access to screening
      • Tan N.Y.Q.
      • Friedman D.S.
      • Stalmans I.
      • et al.
      Glaucoma screening: Where are we and where do we need to go?.
      . While patients with mild glaucoma have a quality of life comparable to that of healthy patients, the quality of life drastically decreases with more advanced glaucoma

      Goldberg I, Clement CI, Chiang TH, et al. Assessing Quality of Life in Patients With Glaucoma Using the Glaucoma Quality of Life-15 (GQL-15) Questionnaire. 2009. Available at: http://journals.lww.com/glaucomajournal.

      . Early diagnosis and treatment allow for preservation of patient quality of life and are at the forefront of strategies for reducing disease burden

      Cristina Leske M, Heijl A, Hussein M, et al. Factors for Glaucoma Progression and the Effect of Treatment The Early Manifest Glaucoma Trial. 2003. Available at: https://jamanetwork.com/.

      .
      Glaucoma diagnostic methods can be grouped into two categories: techniques that evaluate structural changes in the eye and techniques that evaluate functional changes in vision. Among those assessing structural changes, optical coherence tomography (OCT) and fundus photography are most often used in clinical practice. While OCT has been shown to have a high sensitivity for detection of structural glaucomatous changes, the high cost of the technique often restricts the device to large eye clinics or centers

      Shelton RL, Jung W, Sayegh SI, et al. Optical coherence tomography for advanced screening in the primary care office. J Biophotonics 2014;7:525–533. Available at: https://onlinelibrary.wiley.com/doi/full/10.1002/jbio.201200243 [Accessed March 12, 2022].

      ,

      Kim S, Crose M, Eldridge WJ, et al. Design and implementation of a low-cost, portable OCT system. Biomedical Optics Express, Vol 9, Issue 3, pp 1232-1243 2018;9:1232–1243. Available at: https://opg.optica.org/viewmedia.cfm?uri=boe-9-3-1232&seq=0&html=true [Accessed March 13, 2022].

      . This is especially problematic given that developing regions have the highest rates of undiagnosed glaucoma
      • Tan N.Y.Q.
      • Friedman D.S.
      • Stalmans I.
      • et al.
      Glaucoma screening: Where are we and where do we need to go?.
      . Moreover, the World Glaucoma Association considers the largest barrier to glaucoma screening to be cost

      Robert N. Weinreb PRH and FT. Glaucoma Screening: The 5th Consensus Report of the World Glaucoma Association. 2008. Available at: [Accessed March 13, 2022].

      . In contrast to OCT, fundus photography stands as a lower cost option; new advances such as telemedicine screening and smartphone fundoscopy have made fundus photography a feasible and financially viable option even in remote locations

      Panwar N, Huang P, Lee J, et al. Fundus Photography in the 21st Century—A Review of Recent Technological Advances and Their Implications for Worldwide Healthcare. Telemedicine Journal and e-Health 2016;22:198. Available at: /pmc/articles/PMC4790203/ [Accessed March 12, 2022].

      ,

      Nazari Khanamiri H, Nakatsuka A, El-Annan J. Smartphone Fundus Photography. JoVE (Journal of Visualized Experiments) 2017:e55958. Available at: https://www.jove.com/v/55958/smartphone-fundus-photography [Accessed March 12, 2022].

      .
      While OCT and fundus photography allow for the structural findings to be captured, a mechanism is needed to classify such changes and correlate them with functional glaucomatous damage. The Disc Damage Likelihood Scale (DDLS) is one such approach. DDLS is a well-established grading scale to correlate glaucomatous damage with progression of fundus photographs
      • Tan N.Y.Q.
      • Friedman D.S.
      • Stalmans I.
      • et al.
      Glaucoma screening: Where are we and where do we need to go?.
      ,

      Spaeth GL, Henderer J, Liu C, et al. The disc damage likelihood scale: reproducibility of a new method of estimating the amount of optic nerve damage caused by glaucoma. Trans Am Ophthalmol Soc 2002;100:181. Available at: /pmc/articles/PMC1358961/?report=abstract [Accessed March 12, 2022].

      ,
      • Formichella P.
      • Annoh R.
      • Zeri F.
      • Tatham A.J.
      The role of the disc damage likelihood scale in glaucoma detection by community optometrists.
      . DDLS has been incorporated into the eye health professional guidelines for optometrists and ophthalmologists
      • Formichella P.
      • Annoh R.
      • Zeri F.
      • Tatham A.J.
      The role of the disc damage likelihood scale in glaucoma detection by community optometrists.
      . 13The interobserver agreement of DDLS even among glaucoma specialists can vary from 85% based on optic disc photographs to 70% based on clinical exam although intraobserver reliability is high

      Henderer JD, Liu C, Kesen M, et al. Reliability of the Disk Damage Likelihood Scale. 2003.

      . This is especially troubling as DDLS scores can be used as the basis for referral by a variety of eye health professionals, and improper grading may result in missed opportunities for early intervention
      • Formichella P.
      • Annoh R.
      • Zeri F.
      • Tatham A.J.
      The role of the disc damage likelihood scale in glaucoma detection by community optometrists.
      .
      An ideal screening tool would be high-throughput, accurate, and reliable with high specificity. With the advent of neural network models and an increase in image processing capabilities, high specificity with acceptable sensitivity, together with high throughput, may be achieved with a neural network-based pipeline

      Ardila D, Kiraly AP, Bharadwaj S, et al. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nature Medicine 2019 25:6 2019;25:954–961. Available at: https://www.nature.com/articles/s41591-019-0447-x [Accessed March 13, 2022].

      . In this paper, we present DDLSNet, a neural network pipeline which aims to accurately grade DDLS based on optic disc photographs with a combination of a rim segmentation neural network (RimNet) and a disc size classification network (DiscNet).

      Methods

      The DDLS grading criteria was created by Spaeth et al

      Spaeth GL, Henderer J, Liu C, et al. The disc damage likelihood scale: reproducibility of a new method of estimating the amount of optic nerve damage caused by glaucoma. Trans Am Ophthalmol Soc 2002;100:181. Available at: /pmc/articles/PMC1358961/?report=abstract [Accessed March 12, 2022].

      . Their original grading schema is shown as Supplemental Figure 1. The DDLS score is determined by two features of the optic disc: the disc size and the narrowest rim width. Progression of glaucomatous damage is seen as enlargement of the optic disc cup and subsequent thinning of the optic disc rim. This thinning can be measured by the rim-to-disc ratio (mRDR) in intact rims. However, in severe glaucoma, the rim can be completely absent in certain areas. In these cases, the angle for which the rim is completely lost is measured. We call this the ‘absent rim width’ (ARW) and we call these rims ‘incomplete’. These three features, mRDR, the absent rim width, and disc size, are the metrics needed to calculate DDLS. The latter is crucial as the significance of mRDR or ARW varies depending on disc size

      Spaeth GL, Henderer J, Liu C, et al. The disc damage likelihood scale: reproducibility of a new method of estimating the amount of optic nerve damage caused by glaucoma. Trans Am Ophthalmol Soc 2002;100:181. Available at: /pmc/articles/PMC1358961/?report=abstract [Accessed March 12, 2022].

      . Therefore, the DDLSNet pipeline consists of two components: RimNet, which performs rim and cup segmentation and calculates mRDR or absent rim width, and DiscNet, which classifies the size of the optic disc into small, average, and large.

      Database

      Our image database was based on a collection of all the optic disc photographs (ODPs) available in the UCLA Stein Eye Glaucoma Division. For the RimNet database, three glaucoma specialists manually created a mask of the optic disc rim and optic disc cup for each funduscopic image using the image editing program GIMP. These masks were used as the ground truth. The RimNet dataset had two inclusion criteria. The images had to show signs of glaucomatous damage and the images had to be in focus and with discernable posterior pole and vasculature details, both as deemed by two board-certified glaucoma specialists. The exclusion criteria was concurrent non-glaucoma disease including optic neuritis, optic disc neovascularization, and vitreous hemorrhage that would impair visualization of the posterior pole. The demographic information for RimNet dataset is presented in Table 1. Table 2 presents the glaucoma diagnoses for the RimNet dataset. These requirements result in a database that displays the full range of glaucomatous changes to the optic disc rim, ranging from mild optic disc rim narrowing in early-stage glaucoma to absent optic disc rim in severe glaucoma.
      Table 1Demographic Data for Dataset. Lists the gender distribution, age distribution, and racial distribution by camera type.
      Slide ImagesDigital Camera 1Digital Camera 2Digital Camera 3
      Gender DistributionF4071195512
      M302854411
      Age DistributionMean60.7267.1372.8066.92
      SD13.4817.4312.7517.71
      Median61.8771.0673.9172.37
      IQR15.7916.3312.8622.54
      Min9.366.9216.1917.48
      Max90.0596.1094.4186.17
      Race DistributionAsian9034242
      Black632281
      Hispanic6620166
      White3661004512
      Other53530
      Unknown712232
      Table 2RimNet Database Diagnosis. Listed below are the diagnosis of the 1208 ODPs used in RimNet training, validation, and testing in an 80/10/10 split respectively.
      DiagnosisCount
      Primary Open-Angle Glaucoma530
      Glaucoma Suspect403
      Chronic Angle-Closure Glaucoma71
      Low-Tension Glaucoma47
      Secondary Open-Angle Glaucoma35
      Capsular glaucoma with psuedoexfoliation33
      Anatomical Narrow Angle27
      Glaucoma secondary to Eye Infection24
      Pigmentary Glaucoma15
      Secondary Angle Closure11
      Congenital glaucoma7
      Juvenile Glaucoma3
      Acute angle-closure glaucoma2
      The DiscNet database consisted of optic disc photographs with available corresponding Cirrus high-definition OCT Optic Disc Cubes (200x200). The size of the Bruch’s membrane as measured by Cirrus OCT was used as a proxy for disc area and was used to categorize the disc size into small, average, or large optic discs. The optic disc photographs had to be of ‘good’ quality—in focus with an unobstructed view of the posterior pole—as determined by a board-certified glaucoma specialist. The OCT images were required to have a good quality (signal strength >6) and be free of artifacts based on the review of printouts. To examine reliability, a database of non-progressing glaucomatous eyes was created. Each eye had two optic disc photographs available taken less than four years apart, which were deemed stable as confirmed by a glaucoma specialist. The time restriction was imposed to increase the population included but decrease the chance of glaucoma progression between the two photos.

      RimNet

      RimNet consists of a pre-processing step of contrast enhancement, an optic disc rim and cup segmentation model, and an image analysis step to calculate the mRDR for intact rims and ARW for incomplete rims. This latter case occurs in eyes with severe glaucomatous damage. The model was optimized by submitting it to a hyperparameter search with rim intersection over union as the metric. The included hyperparameters were the neural network structure, learning rate, loss function, and optimizer

      Mannor S, Peleg B, Rubinstein R. The cross entropy method for classification. ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning 2005:561–568. Available at: [Accessed June 10, 2022].

      Ruder S. An overview of gradient descent optimization algorithms. 2016. Available at: https://arxiv.org/abs/1609.04747v2 [Accessed June 10, 2022].

      Kingma DP, Ba JL. Adam: A Method for Stochastic Optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings 2014. Available at: https://arxiv.org/abs/1412.6980v9 [Accessed June 10, 2022].

      Zhao H, Shi J, Qi X, et al. Pyramid Scene Parsing Network. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 2016;2017-January:6230–6239. Available at: https://arxiv.org/abs/1612.01105v2 [Accessed June 10, 2022].

      Lin T-Y, Dollár P, Girshick R, et al. Feature Pyramid Networks for Object Detection. 2016. Available at: https://arxiv.org/abs/1612.03144v2 [Accessed June 10, 2022].

      Weng W, Zhu X. U-Net: Convolutional Networks for Biomedical Image Segmentation. IEEE Access 2015;9:16591–16603. Available at: https://arxiv.org/abs/1505.04597v1 [Accessed June 10, 2022].

      Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings 2014. Available at: https://arxiv.org/abs/1409.1556v6 [Accessed June 10, 2022].

      Tan M, Le Q v. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. 36th International Conference on Machine Learning, ICML 2019 2019;2019-June:10691–10700. Available at: https://arxiv.org/abs/1905.11946v5 [Accessed June 10, 2022].

      He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2015;2016-December:770–778. Available at: https://arxiv.org/abs/1512.03385v1 [Accessed June 10, 2022].

      Sandler M, Howard A, Zhu M, et al. MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2018:4510–4520. Available at: https://arxiv.org/abs/1801.04381v4 [Accessed June 10, 2022].

      Chaurasia A, Culurciello E. LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation. 2017 IEEE Visual Communications and Image Processing, VCIP 2017 2017;2018-January:1–4. Available at: http://arxiv.org/abs/1707.03718 [Accessed March 13, 2022].

      Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2015;2016-December:2818–2826. Available at: https://arxiv.org/abs/1512.00567v3 [Accessed March 13, 2022].

      Tan M, Le Q v. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. 36th International Conference on Machine Learning, ICML 2019 2019;2019-June:10691–10700. Available at: https://arxiv.org/abs/1905.11946v5 [Accessed September 24, 2022].

      . Table 2 lists the hyperparameter search space. Fifty total hyperparameter combinations were trained with the Keras Tuner Library with rim segmentation proficiency, measured as an intersection-over-union. The rim segmentation model was trained, validated, and tested on a database of images from the UCLA Stein Eye Glaucoma Division with an 80/10/10 split.

      DiscNet

      DiscNet is a deep neural network developed to assign disc size as small, average, or large as an essential process in DDLS grading. The disc photographs included scanned digitized slides and digital photographs. Disc size information taken from paired OCT data was used as the ground truth. While the original DDLS grading defined small, average, and large discs as diameters of <1.50 mm, between 1.50 mm and 2.00 mm, and >2.00 mm respectively

      Spaeth GL, Henderer J, Liu C, et al. The disc damage likelihood scale: reproducibility of a new method of estimating the amount of optic nerve damage caused by glaucoma. Trans Am Ophthalmol Soc 2002;100:181. Available at: /pmc/articles/PMC1358961/?report=abstract [Accessed March 12, 2022].

      , we modified the cutoffs slightly to ≤1.44mm, 1.44mm to 2.28mm, and ≥2.28 so that the three disc size categories had more evenly distributed sample sizes. This sorted our available data into a 15/70/15 split for small, average, and large discs.
      DiscNet was first built with transfer learning with model weights from ImageNet before submitting it to training

      Deng J, Dong W, Socher R, et al. ImageNet: A large-scale hierarchical image database. 2010:248–255. Available at: [Accessed March 13, 2022].

      . In transfer learning, a subsection of the model’s layers are ‘unlocked’ to hone the transferred model performance for a specified task. Often, only the last layer of the model is unlocked but more can be unlocked if needed. The proportion of the model layers allowed to be updated is termed the ‘tuning fraction’ of the model. DiscNet was trained in two phases, each with a unique learning rate. The first phase allowed for only the last layer of the model to be trained, while the second phase allowed the weights in a subsection of the model layers, the tuning fraction, to be updated.
      A hyperparameter search was completed to select the optimal learning rates in both phases, the tuning fraction, the optimizer, and the network architecture. Table 3 lists the hyperparameter search space, from which each hyperparameter was selected from. Thirty total hyperparameter combinations were trained with the Keras Tuner library with classification accuracy as the optimized metric
      • O’Malley Tom
      • Bursztein Elie
      • Long
      • et al.
      .
      Table 3Hyperparameter Search Space for RimNet. An InceptionV3/LinkNet architecture, binary cross-entropy loss function, learning rate of 10-3, and Adam optimizer were selected. RimIoU was the optimized metric.
      Hyperparameters
      EncodersMobileNetV2, ResNet34, EfficientnetB0, InceptionV3, ResNet101, VGG16, ResNet50
      DecodersU-Net, FPN, LinkNet, PSPnet
      Loss FunctionBinary_Crossentropy, Binary_Focal_Los
      Learning Rate10-3, 10-4, 10-5, 10-6
      OptimizerAdam, SGD

      DDLSNet Pipeline

      The mRDR and ARW from RimNet and the disc size from DiscNet were used to calculate the DDLS score. A full diagram of our pipeline is shown in Figure 1. DDLSNet was evaluated against a ground truth database of optic disc photographs, which three glaucoma specialists had graded with DDLS. The weighted kappa agreement ±1 DDLS grade between the DDLSNet’s output and the average of the grades of three glaucoma specialists was measured. The average of the interobserver agreement for clinicians was also measured.
      Figure thumbnail gr1
      Figure 1DDLSNet pipeline showcasing both the RimNet and DiscNet results. Once optic disc photographs are submitted to the pipeline, RimNet calculates the mRDR for intact rims or ARW for incomplete rims while DiscNet estimates the disc size. The disc size and mRDR or ARW are then used to calculate the DDLS score.
      Figure thumbnail gr2
      Figure 2DDLSNet Results on sample optic disc photographs. The white indicates the physician or DDLNet rim segmentation. The blue line on the rim indicates the shortest rim width detected. The green line indicates the disc diameter. Left-most column: raw optic disc photographs. Middle column: delineation of the disc and cup margin by clinicians. Right-most column: DDLSNet grading. The ARW is calculated in eyes with complete rim loss in certain regions. The mRDR is calculated in eyes with intact optic disc rim. The annotations above the photographs on the middle and right-most columns represent the disc size, ARW or RDR, and DDLS grade.
      Evaluating DDLSNet reliability is necessary, as physician intraobserver accuracy for DDLS grading should be matched by our proposed system for it to be clinically useful. A database of pairs of funduscopic photos of 781 non-progressing glaucomatous eyes taken within four years was used to test DDLSNet reliability. Each image was graded via DDLSNet, and the difference between the two images for each eye was recorded. Glaucoma specialists verified that the eyes were non-progressing, based on evaluation of the disc photos and the visual fields.

      Evaluation Criteria

      The main evaluation criterion was the weighted kappa agreement between DDLSNet and physicians with the ground truth database. Interobserver and intraobserver agreement was also measured as secondary evaluation criteria.

      Results

      RimNet was trained, validated, and tested on 1208 optic disc photographs with an 80/10/10 split respectively. The mean age was 63.7 (±14.9) years with a male:female ratio of 43:57. DiscNet was trained, validated, and tested on a database of 11,536 eyes in an 80/10/10 split. The mean age was 67.6 (±14.5) and had a male:female ratio of 58:42. DDLSNet was tested on 120 optic disc photographs from the RimNet test set manually graded based on DDLS by three glaucoma specialists. Reproducibility of DDLSNet was evaluated on 781 eyes, each with two optic disc photographs available (mean age=73.8 (±11.4) years, male:female ratio=43:57). The eyes were all classified as non-progressing by a glaucoma specialist based on review of the optic disc photographs. The demographic data for the 4 cohorts are presented in Table 4. The code used to train, run, and evaluate DDLSNet can be found on our public repository at https://github.com/TylerADavis/GlaucomaML.
      Table 4Hyperparameter Search Space for DiscNet. A VGG19 architecture, phase one learning rate of 10-4, phase two learning rate of 10-5, tuning fraction of 0.5, and Adam optimizer were selected. Classification accuracy was the optimized metric.
      Hyperparameters
      BackbonesInceptionV3, EfficientNetB4, EfficientNetB0, ResNet101v2, VGG16, VGG19
      Phase One Learning Rate10-4,10-5
      Phase Two Learning Rate10-5, 10-6, 10-7
      Tune Fraction0.1, 0.2, 0.5
      OptimizerAdam, sgd, rmsprop

      Model Architecture and Hyperparameter Search

      After exploring thirty different combinations of hyperparameters through random search, the following hyperparameters were identified as providing the highest classification accuracy for DiscNet: VGG19 architecture, phase one learning rate of 1-4, phase two learning rate of 1-5, tuning fraction of 0.5, and Adam optimizer

      Kingma DP, Ba JL. Adam: A Method for Stochastic Optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings 2014. Available at: https://arxiv.org/abs/1412.6980v9 [Accessed June 10, 2022].

      ,

      Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings 2014. Available at: https://arxiv.org/abs/1409.1556v6 [Accessed June 10, 2022].

      Tan M, Le Q v. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. 36th International Conference on Machine Learning, ICML 2019 2019;2019-June:10691–10700. Available at: https://arxiv.org/abs/1905.11946v5 [Accessed June 10, 2022].

      He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2015;2016-December:770–778. Available at: https://arxiv.org/abs/1512.03385v1 [Accessed June 10, 2022].

      ,

      Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2015;2016-December:2818–2826. Available at: https://arxiv.org/abs/1512.00567v3 [Accessed March 13, 2022].

      ,

      Hinton G, Srivastava N, Swersky K. Neural Networks for Machine Learning Lecture 6a Overview of mini--batch gradient descent. Available at: [Accessed June 14, 2022].

      ,

      Sutskever I, Martens J, Dahl G, Hinton G. On the importance of initialization and momentum in deep learning. 2013. Available at: [Accessed June 14, 2022].

      . VGG19 is a 19-layer convolutional neural network published in 2015 that has previously been used in medical image analysis

      Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings 2014. Available at: https://arxiv.org/abs/1409.1556v6 [Accessed June 10, 2022].

      ,

      Ahuja S, Panigrahi BK, Gandhi T. Transfer Learning Based Brain Tumor Detection and Segmentation using Superpixel Technique. 2020 International Conference on Contemporary Computing and Applications, IC3A 2020 2020:244–249. Available at: [Accessed September 18, 2022].

      ,

      Mateen M, Wen J, Nasrullah, et al. Fundus Image Classification Using VGG-19 Architecture with PCA and SVD. Symmetry 2019, Vol 11, Page 1 2018;11:1. Available at: https://www.mdpi.com/2073-8994/11/1/1/htm [Accessed September 18, 2022].

      . For RimNet, fifty different combinations were examined through a random search, which resulted as follows: InceptionV3/LinkNet architecture, binary cross-entropy loss function, learning rate of 10-3, and Adam optimizer

      Mannor S, Peleg B, Rubinstein R. The cross entropy method for classification. ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning 2005:561–568. Available at: [Accessed June 10, 2022].

      Ruder S. An overview of gradient descent optimization algorithms. 2016. Available at: https://arxiv.org/abs/1609.04747v2 [Accessed June 10, 2022].

      Kingma DP, Ba JL. Adam: A Method for Stochastic Optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings 2014. Available at: https://arxiv.org/abs/1412.6980v9 [Accessed June 10, 2022].

      Zhao H, Shi J, Qi X, et al. Pyramid Scene Parsing Network. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 2016;2017-January:6230–6239. Available at: https://arxiv.org/abs/1612.01105v2 [Accessed June 10, 2022].

      Lin T-Y, Dollár P, Girshick R, et al. Feature Pyramid Networks for Object Detection. 2016. Available at: https://arxiv.org/abs/1612.03144v2 [Accessed June 10, 2022].

      Weng W, Zhu X. U-Net: Convolutional Networks for Biomedical Image Segmentation. IEEE Access 2015;9:16591–16603. Available at: https://arxiv.org/abs/1505.04597v1 [Accessed June 10, 2022].

      Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings 2014. Available at: https://arxiv.org/abs/1409.1556v6 [Accessed June 10, 2022].

      Tan M, Le Q v. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. 36th International Conference on Machine Learning, ICML 2019 2019;2019-June:10691–10700. Available at: https://arxiv.org/abs/1905.11946v5 [Accessed June 10, 2022].

      He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2015;2016-December:770–778. Available at: https://arxiv.org/abs/1512.03385v1 [Accessed June 10, 2022].

      Sandler M, Howard A, Zhu M, et al. MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2018:4510–4520. Available at: https://arxiv.org/abs/1801.04381v4 [Accessed June 10, 2022].

      Chaurasia A, Culurciello E. LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation. 2017 IEEE Visual Communications and Image Processing, VCIP 2017 2017;2018-January:1–4. Available at: http://arxiv.org/abs/1707.03718 [Accessed March 13, 2022].

      Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2015;2016-December:2818–2826. Available at: https://arxiv.org/abs/1512.00567v3 [Accessed March 13, 2022].

      Tan M, Le Q v. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. 36th International Conference on Machine Learning, ICML 2019 2019;2019-June:10691–10700. Available at: https://arxiv.org/abs/1905.11946v5 [Accessed September 24, 2022].

      . InceptionV3 was first published in 2015, outperforming popular encoders at the time with a fraction of the computation costs

      Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2015;2016-December:2818–2826. Available at: https://arxiv.org/abs/1512.00567v3 [Accessed March 13, 2022].

      . It has been previously used in medical segmentation
      • Shoaib M.
      • Sayed N.
      YOLO Object Detector and Inception-V3 Convolutional Neural Network for Improved Brain Tumor Segmentation.
      ,
      • Salama W.M.
      • Aly M.H.
      Deep learning in mammography images segmentation and classification: Automated CNN approach.
      . LinkNet is a lightweight decoder first published in 2017

      Chaurasia A, Culurciello E. LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation. 2017 IEEE Visual Communications and Image Processing, VCIP 2017 2017;2018-January:1–4. Available at: http://arxiv.org/abs/1707.03718 [Accessed March 13, 2022].

      . Given our computational restrictions of our workstation, which uses NVIDIA RTX 2080 Ti graphics cards, these were appropriate choices.

      RimNet

      The RimNet evaluation criteria were the mean absolute error (MAE) for mRDR for intact rims and the MAE for ARW for incomplete rims between physician grading and RimNet grading with a secondary evaluation criterion of the rim intersection over union (RimIoU). The intersection over union (IoU) is a commonly used measure for segmentation accuracy. RimNet achieved an mRDR MAE of 0.04 (±0.03), a ARW MAE of 48.9 (±35.9), and a RimIoU of 0.68.

      DiscNet

      DiscNet raw classification accuracy was found to be 73% (95% CI: 70, 75) across a test set of 1,137 images, which included both scanned slides and digitally acquired optic disc photographs. Broken down by category, DiscNet had a classification accuracy of 62% (95% CI: 55, 70) for small discs, 77% (95% CI: 74, 80) for average discs, and 60% (95% CI: 52, 68) for large discs. Notably, only three small discs out of 234 (1.2%) were mistakenly classified as large and only two large discs out of 146 (1.3%) were mistakenly classified as small.

      DDLSNet

      DDLSNet was evaluated on a testing database of 120 optic disc photographs. Three glaucoma specialists also graded the same 120 funduscopic images with DDLS. The weighted kappa agreement between the average grading of the three glaucoma specialists and DDLSNet was 0.54 (95% CI: 0.4, 0.68). A full breakdown of the results can be found in Table 5. The model matched the kappa scores between physicians, which included 0.49, 0.52, and 0.56, averaged at 0.52. DDLSNet reproducibility was measured by evaluating pairs of non-progressing optic disc photographs. Of the 781 pairs of eyes, 485 (62%) had DDLS difference of 0, 267 (34%) had a DDLS difference of 1, 28 (4%) had a DDLS difference of 2, and 1 (0.1%) had a DDLS difference of 3 (Table 6).
      Table 5Demographics characteristics for the datasets used for RimNet, DiscNet, DDLSNet, and the DDLSNet reliability. The DDLS distribution for our test set of 120 images, graded by glaucoma specialists, is shown.
      DDLSNet Test SetRimNetDiscNetDDLSNet Reliability
      Total No. of Images120120811,5361562
      Total No. of Eyes10910215213781
      Gender: Male/Female45:5543:5758:4243:57
      Age: Mean (SD)65.9 (±14.8)63.7 (±14.9)67.6 (±14.5)73.8 (±11.4)
      DDLS Grading by Physician
      10
      212
      329
      430
      512
      619
      712
      84
      92
      100
      Table 6Kappa Agreement between DDLSNet and glaucoma specialist grading. A 95% confidence interval is also presented.
      GradersKappa (95% CI)
      Grader 1 vs. Grader 20.52 (0.32, 0.72)
      Grader 1 vs. Grader 30.56 (0.35, 0.77)
      Grader 2 vs. Grader 30.49 (0.29, 0.7)
      Grader Average vs. DDLSNet0.54 (0.4, 0.68)
      Table 7Difference in DDLSNet grading between paired images of non-progressing optic disc photographs. All photographs were taken within 4 years of each other. A total of 781 images were used.
      DDLS DifferenceNumber of Images
      0481
      1267
      228
      31

      Discussion

      We present an automated pipeline for estimating the DDLS score with optic disc photographs in patients with suspected or established glaucoma to facilitate detection and monitoring of the disease. The DDLSNet weighted kappa agreement of 0.54 (95% CI 0.40-0.68) demonstrated moderate agreement with clinician grading and matching inter-clinician agreement. Moreover, the DDLSNet reproducibility was high with 96% of 781 non-progressing eyes found to have ±1 DDLS grade difference on stable pairs of optic disc photographs.
      Automated glaucoma grading with optic disc photographs has been evolving. Most experimental approaches focus on accurate detection of the cup-to-disc ratio with techniques ranging from thresholding to level setting to artificial intelligence models
      • Thakur N.
      • Juneja M.
      Survey on segmentation and classification approaches of optic cup and optic disc for diagnosis of glaucoma.
      . As early as 2001, Chrástek et al. offered an automated method of optic disc segmentation with filtering and edge detection, which achieved a segmentation accuracy of 71% with accuracy defined subjectively as ‘good’ or ‘very good’
      • Chrástek R.
      • Wolf M.
      • Donath K.
      • et al.
      Automated segmentation of the optic nerve head for diagnosis of glaucoma.
      . More recently, Kumar and Bindu used U-Net
      • Ronneberger O.
      • Fischer P.
      • Brox T.
      U-Net: Convolutional Networks for Biomedical Image Segmentation.
      , a segmentation neural network architecture, to achieve an intersection-over-union (IoU) of 87.9% in optic disc segmentation
      • Kumar E.S.
      • Bindu C.S.
      Two-stage framework for optic disc segmentation and estimation of cup-to-disc ratio using deep learning technique.
      . Our algorithm for measuring mRDR, RimNet, combines both the image processing techniques used in older segmentation studies and the artificial intelligence of newer studies to achieve a high-efficacy segmentation on a variety of optic disc photographs.
      Cup-to-disc ratio has been repeatedly shown to be inferior to DDLS in grading glaucomatous damage

      Cheng KKW, Tatham AJ. Spotlight on the Disc-Damage Likelihood Scale (DDLS). Clin Ophthalmol 2021;15:4059. Available at: /pmc/articles/PMC8504474/ [Accessed March 12, 2022].

      . Several papers addressed detection of the minimum optic disc rim width, an important component of calculating the DDLS score
      • Haleem M.S.
      • Han L.
      • van Hemert J.
      • Li B.
      Automatic extraction of retinal features from colour retinal images for glaucoma diagnosis: a review.
      • Choudhary K.
      • Tiwari S.
      ANN Glaucoma Detection using Cup-to-Disk Ratio and Neuroretinal Rim.
      • Issac A.
      • Partha Sarathi M.
      • Dutta M.K.
      An adaptive threshold based image processing technique for improved glaucoma detection and classification.
      . However, few have used automated DDLS calculation due to the complexity of the challenge. Two studies examined the results of proprietary software built into a 3D stereographic camera (Kowa Nonmyd WX 3D, Kowa, Tokyo, Japan)
      • Han J.W.
      • Cho S.Y.
      • Kang K.D.
      Correlation between optic nerve parameters obtained using 3D nonmydriatic retinal camera and optical coherence tomography: Interobserver agreement on the disc damage likelihood scale.
      ,

      Gnaneswaran P, Devi S, Balu R, et al. Agreement between Clinical versus Automated Disc Damage Likelihood Scalw (DDLS) Staging in Asian Indian eyes. Invest Ophthalmol Vis Sci 2013;54:4806–4806. Available at: [Accessed September 25, 2022].

      . The camera automatically displays the DDLS grade in its final report. The study by Han et al. showed moderate agreement (weighted kappa value, 0.59) with one glaucoma specialist
      • Han J.W.
      • Cho S.Y.
      • Kang K.D.
      Correlation between optic nerve parameters obtained using 3D nonmydriatic retinal camera and optical coherence tomography: Interobserver agreement on the disc damage likelihood scale.
      . This study has two limitations compared to our study. First, the study only evaluates the camera against one glaucoma specialist rather than the three in our study. Second, such camera-specific software does not offer the generalizability of DDLSNet. While functional on certain cameras, such software would not offer the generalizability of DDLSNet. A third study provided clinical validation for RIA-G, an automated cloud-based optic nerve head analysis software that has been reported to be able to grade optic disc photographs based on DDLS
      • Singh Di
      • Gunasekaran S.
      • Hada M.
      • Gogia V.
      Clinical validation of RIA-G, an automated optic nerve head analysis software.
      . This study showed a moderate Kappa agreement of 0.62 (0.55, 0.69) between three glaucoma specialists and the software. However, the validation set favored photographs of mild glaucoma (average DDLS grade 3, DDLS 1-7 included) and required fundus photographs with a 30-degree field of view
      • Singh Di
      • Gunasekaran S.
      • Hada M.
      • Gogia V.
      Clinical validation of RIA-G, an automated optic nerve head analysis software.
      . Our validation set has a wider spectrum of glaucomatous damage (average DDLS grade 4.5, DDLS 2-9) and DDLSNet does not require a 30-degree field of view. Moreover, the RIA-G optic disc cup and disc detection software operates based on contrast detection which would be impaired in photographs with bright artifacts and abnormal pathology
      • Singh Di
      • Gunasekaran S.
      • Hada M.
      • Gogia V.
      Clinical validation of RIA-G, an automated optic nerve head analysis software.
      . A fourth study implemented a partial-DDLS grading using active discs, where a circular disc shape was assumed and DDLS grades were grouped into normal, moderate, and severe categories

      Kumar JRH, Seelamantula CS, Kamath YS, Jampala R. Rim-to-Disc Ratio Outperforms Cup-to-Disc Ratio for Glaucoma Prescreening. Scientific Reports 2019 9:1 2019;9:1–9. Available at: https://www.nature.com/articles/s41598-019-43385-2 [Accessed March 13, 2022].

      . The model achieved a category accuracy of 89%

      Kumar JRH, Seelamantula CS, Kamath YS, Jampala R. Rim-to-Disc Ratio Outperforms Cup-to-Disc Ratio for Glaucoma Prescreening. Scientific Reports 2019 9:1 2019;9:1–9. Available at: https://www.nature.com/articles/s41598-019-43385-2 [Accessed March 13, 2022].

      . DDLSNet improves upon this study by directly comparing ten DDLS grades rather than three categories. Additionally, our network accounts for disc size variations through DiscNet and intact and incomplete rims through RimNet. It is unclear if and to what extent the above studies included optic discs with areas of absent optic disc rim widths, which constitute the most severe DDLS grades.
      DDLSNet is the most accurate and generalizable approach developed to date for several reasons. First, it was validated on optic disc photographs with a wide breadth of glaucomatous damage. This included optic disc photographs with areas of absent optic disc rims. Second, it makes no assumptions of the size or shape of the optic disc when grading size. Third, it is built on a neural network model rather than thresholding or contrast-based algorithms which are limited in learning capacity. Finally, it is not restricted to specific fundus cameras, making it more amenable for use in mobile settings where smartphones or portable fundus cameras can be used for fundus photography.
      The shortcomings of our study need to be considered. Expanding the dataset could improve performance of both RimNet and DiscNet. The models will also have to be trained on images with significant concurrent pathologies, such as severe diabetic retinopathy and macular degeneration. The hyperparameter search was limited by the processing power and memory constraints of our NVIDIA RTX 2080 Ti graphics cards, which were used to train the model. A more extensive hyperparameter search can be done using larger architectures such as ResNet152 with more powerful computing hardware. Following the hyperparameter search, the selected DiscNet model and RimNet model had the highest accuracy and rim intersection over union respectively. However, their loss functions had evidence of possible overfitting. This would need to be addressed in future study. Finally, the number of physicians grading and segmenting funduscopic images could be increased to allow DDLSNet to learn a wider consensus of gradings.
      In conclusion, DDLSNet offers a unique, high-efficacy, high-throughput, reliable DDLS grading system, which is well-suited to perform as a screening, diagnostic, and prognostic tool for identifying and classifying glaucomatous damage and monitoring disease progression. DDLSNet is also well-suited for mobile applications in a variety of settings, including use by individuals without extensive ophthalmological training such as a neurology resident using a phone camera attachment or optometrists seeking to better evaluate their patients’ funduscopic images. Future study directions include increasing the number of physician graders and examining the implementation in remote areas with limited access. With powerful computing technology, glaucoma screening could be enhanced and widely disseminated, improving clinical outcomes for patients.

      Uncited reference

      Kara-José AC, Melo LAS, Esporcatte BLB, et al. The disc damage likelihood scale: Diagnostic accuracy and correlations with cup-to-disc ratio, structural tests and standard automated perimetry. PLoS One 2017;12:e0181428. Available at: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0181428 [Accessed March 12, 2022].

      .

      References

      1. Giangiacomo A, Coleman AL. The Epidemiology of Glaucoma. In: Glaucoma. Berlin, Heidelberg: Springer Berlin Heidelberg; :13–21.

        • Tham Y.C.
        • Li X.
        • Wong T.Y.
        • et al.
        Global Prevalence of Glaucoma and Projections of Glaucoma Burden through 2040: A Systematic Review and Meta-Analysis.
        Ophthalmology. 2014; 121 (Available at: [Accessed March 12, 2022]): 2081-2090
        • Tan N.Y.Q.
        • Friedman D.S.
        • Stalmans I.
        • et al.
        Glaucoma screening: Where are we and where do we need to go?.
        Curr Opin Ophthalmol. 2020; 31: 91-100
      2. Goldberg I, Clement CI, Chiang TH, et al. Assessing Quality of Life in Patients With Glaucoma Using the Glaucoma Quality of Life-15 (GQL-15) Questionnaire. 2009. Available at: http://journals.lww.com/glaucomajournal.

      3. Cristina Leske M, Heijl A, Hussein M, et al. Factors for Glaucoma Progression and the Effect of Treatment The Early Manifest Glaucoma Trial. 2003. Available at: https://jamanetwork.com/.

      4. Shelton RL, Jung W, Sayegh SI, et al. Optical coherence tomography for advanced screening in the primary care office. J Biophotonics 2014;7:525–533. Available at: https://onlinelibrary.wiley.com/doi/full/10.1002/jbio.201200243 [Accessed March 12, 2022].

      5. Kim S, Crose M, Eldridge WJ, et al. Design and implementation of a low-cost, portable OCT system. Biomedical Optics Express, Vol 9, Issue 3, pp 1232-1243 2018;9:1232–1243. Available at: https://opg.optica.org/viewmedia.cfm?uri=boe-9-3-1232&seq=0&html=true [Accessed March 13, 2022].

      6. Robert N. Weinreb PRH and FT. Glaucoma Screening: The 5th Consensus Report of the World Glaucoma Association. 2008. Available at: [Accessed March 13, 2022].

      7. Panwar N, Huang P, Lee J, et al. Fundus Photography in the 21st Century—A Review of Recent Technological Advances and Their Implications for Worldwide Healthcare. Telemedicine Journal and e-Health 2016;22:198. Available at: /pmc/articles/PMC4790203/ [Accessed March 12, 2022].

      8. Nazari Khanamiri H, Nakatsuka A, El-Annan J. Smartphone Fundus Photography. JoVE (Journal of Visualized Experiments) 2017:e55958. Available at: https://www.jove.com/v/55958/smartphone-fundus-photography [Accessed March 12, 2022].

      9. Spaeth GL, Henderer J, Liu C, et al. The disc damage likelihood scale: reproducibility of a new method of estimating the amount of optic nerve damage caused by glaucoma. Trans Am Ophthalmol Soc 2002;100:181. Available at: /pmc/articles/PMC1358961/?report=abstract [Accessed March 12, 2022].

        • Formichella P.
        • Annoh R.
        • Zeri F.
        • Tatham A.J.
        The role of the disc damage likelihood scale in glaucoma detection by community optometrists.
        Ophthalmic and Physiological Optics. 2020; 40: 752-759
      10. Kara-José AC, Melo LAS, Esporcatte BLB, et al. The disc damage likelihood scale: Diagnostic accuracy and correlations with cup-to-disc ratio, structural tests and standard automated perimetry. PLoS One 2017;12:e0181428. Available at: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0181428 [Accessed March 12, 2022].

      11. Henderer JD, Liu C, Kesen M, et al. Reliability of the Disk Damage Likelihood Scale. 2003.

      12. Ardila D, Kiraly AP, Bharadwaj S, et al. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nature Medicine 2019 25:6 2019;25:954–961. Available at: https://www.nature.com/articles/s41591-019-0447-x [Accessed March 13, 2022].

      13. Mannor S, Peleg B, Rubinstein R. The cross entropy method for classification. ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning 2005:561–568. Available at: [Accessed June 10, 2022].

      14. Ruder S. An overview of gradient descent optimization algorithms. 2016. Available at: https://arxiv.org/abs/1609.04747v2 [Accessed June 10, 2022].

      15. Kingma DP, Ba JL. Adam: A Method for Stochastic Optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings 2014. Available at: https://arxiv.org/abs/1412.6980v9 [Accessed June 10, 2022].

      16. Zhao H, Shi J, Qi X, et al. Pyramid Scene Parsing Network. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 2016;2017-January:6230–6239. Available at: https://arxiv.org/abs/1612.01105v2 [Accessed June 10, 2022].

      17. Lin T-Y, Dollár P, Girshick R, et al. Feature Pyramid Networks for Object Detection. 2016. Available at: https://arxiv.org/abs/1612.03144v2 [Accessed June 10, 2022].

      18. Weng W, Zhu X. U-Net: Convolutional Networks for Biomedical Image Segmentation. IEEE Access 2015;9:16591–16603. Available at: https://arxiv.org/abs/1505.04597v1 [Accessed June 10, 2022].

      19. Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings 2014. Available at: https://arxiv.org/abs/1409.1556v6 [Accessed June 10, 2022].

      20. Tan M, Le Q v. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. 36th International Conference on Machine Learning, ICML 2019 2019;2019-June:10691–10700. Available at: https://arxiv.org/abs/1905.11946v5 [Accessed June 10, 2022].

      21. He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2015;2016-December:770–778. Available at: https://arxiv.org/abs/1512.03385v1 [Accessed June 10, 2022].

      22. Sandler M, Howard A, Zhu M, et al. MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2018:4510–4520. Available at: https://arxiv.org/abs/1801.04381v4 [Accessed June 10, 2022].

      23. Chaurasia A, Culurciello E. LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation. 2017 IEEE Visual Communications and Image Processing, VCIP 2017 2017;2018-January:1–4. Available at: http://arxiv.org/abs/1707.03718 [Accessed March 13, 2022].

      24. Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2015;2016-December:2818–2826. Available at: https://arxiv.org/abs/1512.00567v3 [Accessed March 13, 2022].

      25. Tan M, Le Q v. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. 36th International Conference on Machine Learning, ICML 2019 2019;2019-June:10691–10700. Available at: https://arxiv.org/abs/1905.11946v5 [Accessed September 24, 2022].

      26. Deng J, Dong W, Socher R, et al. ImageNet: A large-scale hierarchical image database. 2010:248–255. Available at: [Accessed March 13, 2022].

        • O’Malley Tom
        • Bursztein Elie
        • Long
        • et al.
        Keras Tuner. 2019;
      27. Hinton G, Srivastava N, Swersky K. Neural Networks for Machine Learning Lecture 6a Overview of mini--batch gradient descent. Available at: [Accessed June 14, 2022].

      28. Sutskever I, Martens J, Dahl G, Hinton G. On the importance of initialization and momentum in deep learning. 2013. Available at: [Accessed June 14, 2022].

      29. Ahuja S, Panigrahi BK, Gandhi T. Transfer Learning Based Brain Tumor Detection and Segmentation using Superpixel Technique. 2020 International Conference on Contemporary Computing and Applications, IC3A 2020 2020:244–249. Available at: [Accessed September 18, 2022].

      30. Mateen M, Wen J, Nasrullah, et al. Fundus Image Classification Using VGG-19 Architecture with PCA and SVD. Symmetry 2019, Vol 11, Page 1 2018;11:1. Available at: https://www.mdpi.com/2073-8994/11/1/1/htm [Accessed September 18, 2022].

        • Shoaib M.
        • Sayed N.
        YOLO Object Detector and Inception-V3 Convolutional Neural Network for Improved Brain Tumor Segmentation.
        Traitement du Signal. 2022; 39 (Available at: [Accessed September 4, 2022]): 371-380
        • Salama W.M.
        • Aly M.H.
        Deep learning in mammography images segmentation and classification: Automated CNN approach.
        Alexandria Engineering Journal. 2021; 60 (Available at: [Accessed September 4, 2022]): 4701-4709
        • Thakur N.
        • Juneja M.
        Survey on segmentation and classification approaches of optic cup and optic disc for diagnosis of glaucoma.
        Biomed Signal Process Control. 2018; 42 (Available at: [Accessed March 12, 2022]): 162-189
        • Chrástek R.
        • Wolf M.
        • Donath K.
        • et al.
        Automated segmentation of the optic nerve head for diagnosis of glaucoma.
        Med Image Anal. 2005; 9 (Available at:) (Accessed): 297-314
        https://pubmed.ncbi.nlm.nih.gov/15950894/
        Date accessed: March 12, 2022
        • Ronneberger O.
        • Fischer P.
        • Brox T.
        U-Net: Convolutional Networks for Biomedical Image Segmentation.
        Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2015; 9351 (Available at:) (Accessed): 234-241
        https://arxiv.org/abs/1505.04597v1
        Date accessed: March 12, 2022
        • Kumar E.S.
        • Bindu C.S.
        Two-stage framework for optic disc segmentation and estimation of cup-to-disc ratio using deep learning technique.
        J Ambient Intell Humaniz Comput. 2021; (Available at: [Accessed March 12, 2022])
      31. Cheng KKW, Tatham AJ. Spotlight on the Disc-Damage Likelihood Scale (DDLS). Clin Ophthalmol 2021;15:4059. Available at: /pmc/articles/PMC8504474/ [Accessed March 12, 2022].

        • Haleem M.S.
        • Han L.
        • van Hemert J.
        • Li B.
        Automatic extraction of retinal features from colour retinal images for glaucoma diagnosis: a review.
        Comput Med Imaging Graph. 2013; 37 (Available at:) (Accessed): 581-596
        https://pubmed.ncbi.nlm.nih.gov/24139134/
        Date accessed: March 12, 2022
        • Choudhary K.
        • Tiwari S.
        ANN Glaucoma Detection using Cup-to-Disk Ratio and Neuroretinal Rim.
        Int J Comput Appl. 2015; 111 (Available at: [Accessed March 12, 2022]): 975-8887
        • Issac A.
        • Partha Sarathi M.
        • Dutta M.K.
        An adaptive threshold based image processing technique for improved glaucoma detection and classification.
        Comput Methods Programs Biomed. 2015; 122 (Available at:) (Accessed): 229-244
        https://pubmed.ncbi.nlm.nih.gov/26321351/
        Date accessed: March 12, 2022
        • Han J.W.
        • Cho S.Y.
        • Kang K.D.
        Correlation between optic nerve parameters obtained using 3D nonmydriatic retinal camera and optical coherence tomography: Interobserver agreement on the disc damage likelihood scale.
        J Ophthalmol. 2014; 2014
      32. Gnaneswaran P, Devi S, Balu R, et al. Agreement between Clinical versus Automated Disc Damage Likelihood Scalw (DDLS) Staging in Asian Indian eyes. Invest Ophthalmol Vis Sci 2013;54:4806–4806. Available at: [Accessed September 25, 2022].

        • Singh Di
        • Gunasekaran S.
        • Hada M.
        • Gogia V.
        Clinical validation of RIA-G, an automated optic nerve head analysis software.
        Indian J Ophthalmol. 2019; 67: 1089-1094
      33. Kumar JRH, Seelamantula CS, Kamath YS, Jampala R. Rim-to-Disc Ratio Outperforms Cup-to-Disc Ratio for Glaucoma Prescreening. Scientific Reports 2019 9:1 2019;9:1–9. Available at: https://www.nature.com/articles/s41598-019-43385-2 [Accessed March 13, 2022].