Advertisement

Radiomics-based assessment of Optical Coherence Tomography Angiography images for Diabetic Retinopathy diagnosis

  • Laura Carrera-Escalé
    Affiliations
    Intelligent Data Science and Artificial Intelligence (IDEAI) Research Center

    Department of Computer Science, Facultat d’Informàtica de Barcelona (FIB), Universitat Politècnica de Catalunya (UPC), Barcelona, Spain
    Search for articles by this author
  • Anass Benali
    Affiliations
    Intelligent Data Science and Artificial Intelligence (IDEAI) Research Center

    Department of Computer Science, Facultat d’Informàtica de Barcelona (FIB), Universitat Politècnica de Catalunya (UPC), Barcelona, Spain
    Search for articles by this author
  • Ann-Christin Rathert
    Affiliations
    Intelligent Data Science and Artificial Intelligence (IDEAI) Research Center

    Department of Computer Science, Facultat d’Informàtica de Barcelona (FIB), Universitat Politècnica de Catalunya (UPC), Barcelona, Spain
    Search for articles by this author
  • Ruben Martín-Pinardel
    Affiliations
    Intelligent Data Science and Artificial Intelligence (IDEAI) Research Center

    Department of Computer Science, Facultat d’Informàtica de Barcelona (FIB), Universitat Politècnica de Catalunya (UPC), Barcelona, Spain

    August Pi i Sunyer Biomedical Research Institute (IDIBAPS), Barcelona, Spain
    Search for articles by this author
  • Carolina Bernal-Morales
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain
    Search for articles by this author
  • Anibal Alé-Chilet
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain
    Search for articles by this author
  • Marina Barraso
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain
    Search for articles by this author
  • Sara Marín-Martinez
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain
    Search for articles by this author
  • Silvia Feu-Basilio
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain
    Search for articles by this author
  • Josep Rosinés-Fonoll
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain
    Search for articles by this author
  • Teresa Hernandez
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain

    August Pi i Sunyer Biomedical Research Institute (IDIBAPS), Barcelona, Spain
    Search for articles by this author
  • Irene Vilá
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain

    August Pi i Sunyer Biomedical Research Institute (IDIBAPS), Barcelona, Spain
    Search for articles by this author
  • Rafael Castro-Dominguez
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain
    Search for articles by this author
  • Cristian Oliva
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain

    August Pi i Sunyer Biomedical Research Institute (IDIBAPS), Barcelona, Spain
    Search for articles by this author
  • Irene Vinagre
    Affiliations
    Diabetes Unit, Hospital Clínic de Barcelona, Spain

    Institut Clínic de Malalties Digestives i Metaboliques (ICMDM), Hospital Clínic de Barcelona, Spain

    August Pi i Sunyer Biomedical Research Institute (IDIBAPS), Barcelona, Spain
    Search for articles by this author
  • Emilio Ortega
    Affiliations
    Diabetes Unit, Hospital Clínic de Barcelona, Spain

    Institut Clínic de Malalties Digestives i Metaboliques (ICMDM), Hospital Clínic de Barcelona, Spain

    August Pi i Sunyer Biomedical Research Institute (IDIBAPS), Barcelona, Spain
    Search for articles by this author
  • Marga Gimenez
    Affiliations
    Diabetes Unit, Hospital Clínic de Barcelona, Spain

    Institut Clínic de Malalties Digestives i Metaboliques (ICMDM), Hospital Clínic de Barcelona, Spain

    August Pi i Sunyer Biomedical Research Institute (IDIBAPS), Barcelona, Spain
    Search for articles by this author
  • Alfredo Vellido
    Affiliations
    Intelligent Data Science and Artificial Intelligence (IDEAI) Research Center

    Department of Computer Science, Facultat d’Informàtica de Barcelona (FIB), Universitat Politècnica de Catalunya (UPC), Barcelona, Spain
    Search for articles by this author
  • Enrique Romero
    Affiliations
    Intelligent Data Science and Artificial Intelligence (IDEAI) Research Center

    Department of Computer Science, Facultat d’Informàtica de Barcelona (FIB), Universitat Politècnica de Catalunya (UPC), Barcelona, Spain
    Search for articles by this author
  • Javier Zarranz-Ventura
    Correspondence
    Corresponding author & study coordinator: Javier Zarranz-Ventura MD PhD MSc FEBO, Hospital Clínic of Barcelona, C/ Sabino Arana 1, Barcelona, Spain 08028
    Affiliations
    Institut Clínic d´Oftalmología (ICOF), Hospital Clínic de Barcelona, Barcelona, Spain

    Diabetes Unit, Hospital Clínic de Barcelona, Spain

    August Pi i Sunyer Biomedical Research Institute (IDIBAPS), Barcelona, Spain

    School of Medicine, Universitat de Barcelona, Spain
    Search for articles by this author
Open AccessPublished:November 18, 2022DOI:https://doi.org/10.1016/j.xops.2022.100259

      ABSTRACT

      Purpose

      To evaluate the diagnostic accuracy of machine learning (ML) techniques applied to Radiomic features extracted from optical coherence tomography (OCT) and OCT angiography (OCTA) images for diabetes mellitus (DM), diabetic retinopathy (DR) and referable DR (R-DR) diagnosis.

      Design

      Cross sectional analysis of a retinal image dataset from a previous prospective OCTA study (ClinicalTrials.gov NCT03422965).

      Participants

      Type 1 DM patients and controls included in the progenitor study.

      Methods

      Radiomic features were extracted from fundus retinographies (FR), OCT and OCTA images in each study eye. Logistic regression (LR), linear discriminant analysis (LDA), support vector classifier (SVC)-linear, SVC-rbf and random forest (RF) models were created to evaluate their diagnostic accuracy for DM, DR and R-DR diagnosis in all images type.

      Main outcome measures

      Area under the receiver operating characteristic curve (AUC) mean and standard deviation (SD) for each ML model and each individual and combined image types.

      Results

      A dataset of 726 eyes (439 individuals) were included. For DM diagnosis, the greatest AUC was observed for OCT (0.82, 0.03). For DR detection the greatest AUC was observed for OCTA (0.77, 0.03), especially in the 3x3 mm superficial capillary plexus OCTA scan (AUC 0.76, 0.04). For R-DR diagnosis, the greatest AUC was observed for OCTA (0.87, 0.12) and the deep capillary plexus OCTA scan (0.86, 0.08). The addition of clinical variables (age, sex, etc.) improved most models AUC for both DM and DR diagnosis. The performance of the models was similar in unilateral and bilateral eyes image datasets.

      Conclusions

      Radiomics extracted from OCT and OCTA images allow identification of DM, DR and R-DR patients using standard ML classifiers. OCT was the best test for DM diagnosis, OCTA for DR and R-DR diagnosis and the addition of clinical variables improved most models. This pioneer study demonstrates that radiomics-based ML techniques applied to OCT and OCTA images may be an option for DR screening in type 1 DM patients.

      Keywords

      INTRODUCTION

      Artificial intelligence (AI) applications in Ophthalmology have exponentially grown in the last decade.
      • Topol E.J.
      High-performance medicine: the convergence of human and artificial intelligence.
      • Ting D.S.W.
      • Pasquale L.R.
      • Peng L.
      • et al.
      Artificial intelligence and deep learning in ophthalmology.
      • De Fauw J.
      • Ledsam J.R.
      • Romera-Paredes B.
      • et al.
      Clinically applicable deep learning for diagnosis and referral in retinal disease.
      • Zarranz-Ventura J.
      • Bernal-Morales C.
      • Saenz de Viteri M.
      • et al.
      Artificial intelligence and ophthalmology: Current status.
      AI algorithms are used in multiple tasks such as image recognition, image processing
      • Maloca P.M.
      • Lee A.Y.
      • De Carvalho E.R.
      • et al.
      Validation of automated artificial intelligence segmentation of optical coherence tomography images.
      or clinical data association and allow characterization of patient profiles, being able to provide evolution predictions
      • Yim J.
      • Chopra R.
      • Spitz T.
      • et al.
      Predicting conversion to wet age-related macular degeneration using deep learning.
      , treatment response predictions
      • Bogunovic H.
      • Waldstein S.M.
      • Schlegl T.
      • et al.
      Prediction of Anti-VEGF Treatment Requirements in Neovascular AMD Using a Machine Learning Approach.
      and associations with systemic data of interest.
      • Wagner S.K.
      • Fu D.J.
      • Faes L.
      • et al.
      Insights into Systemic Disease through Retinal Imaging-Based Oculomics.
      In the computer vision and image recognition field, the main applications have been focused in the retina subspecialty area, and in particular the study of the retinal complications of diabetes mellitus (DM), such as diabetic retinopathy (DR) or diabetic macular edema (DME), as well as age-related macular degeneration and retinopathy of prematurity.
      • Burlina P.M.
      • Joshi N.
      • Pekala M.
      • et al.
      Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks.
      • Wang Y.
      • Zhang Y.
      • Yao Z.
      • et al.
      Machine learning based detection of age-related macular degeneration (AMD) and diabetic macular edema (DME) from optical coherence tomography (OCT) images.
      • ElTanboly A.
      • Ismail M.
      • Shalaby A.
      • et al.
      A computer-aided diagnostic system for detecting diabetic retinopathy in optical coherence tomography images.
      • Brown J.M.
      • Campbell J.P.
      • Beers A.
      • et al.
      Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks.
      Fundus retinography (FR) has been the most common type of retinal image employed in these analysis, frequently using large datasets from existing DR screening programs.
      • Poplin R.
      • Varadarajan A.V.
      • Blumer K.
      • et al.
      Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning.
      • Gulshan V.
      • Peng L.
      • Coram M.
      • et al.
      Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.
      • Xie Y.
      • Nguyen Q.D.
      • Hamzah H.
      • et al.
      Artificial intelligence for teleophthalmology-based diabetic retinopathy screening in a national programme: an economic analysis modelling study.
      • van der Heijden A.A.
      • Abramoff M.D.
      • Verbraak F.
      • et al.
      Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System.
      • Grzybowski A.
      • Brona P.
      • Lim G.
      • et al.
      Artificial intelligence for diabetic retinopathy screening: a review.
      Recently, the diagnostic potential of AI applied to optical coherence tomography (OCT) images has also been investigated.
      • De Fauw J.
      • Ledsam J.R.
      • Romera-Paredes B.
      • et al.
      Clinically applicable deep learning for diagnosis and referral in retinal disease.
      While most previous work done leverages FR or OCT images, scarce efforts have been implemented on the rich granular data afforded by OCT angiography (OCTA) images
      • Hua C.H.
      • Kim K.
      • Huynh-The T.
      • et al.
      Convolutional Network with Twofold Feature Augmentation for Diabetic Retinopathy Recognition from Multi-Modal Images.
      ,

      Le D, Alam M, Yao C, et al. Transfer learning for automated OCTA detection of diabetic retinopathy. arXiv 2019:1–20.

      , as not many pre-existing large datasets are available. OCTA is a newly developed, non-invasive, retinal imaging technique that depicts details of the perifoveal vascular network with an unprecedented level of resolution, allowing the objective quantification of parameters such as vessel density or flow impairment areas.
      • Spaide R.F.
      • Fujimoto J.G.
      • Waheed N.K.
      • et al.
      Optical coherence tomography angiography.
      ,
      • Kalra G.
      • Zarranz-Ventura J.
      • Chahal R.
      • et al.
      Optical computed tomography (OCT) angiolytics: A review of OCT angiography quantiative biomarkers.
      Since this technique allows direct in vivo visualization of the microvascular circulation, it is sensible to think that the detection of subtle microvascular changes may be improved compared to other retinal techniques, especially in vascular diseases as DM.
      Radiomics is a specific methodology that refers to the extraction and analysis of a large number of quantitative features from medical images using computer vision and image processing techniques.
      • Kumar V.
      • Gu Y.
      • Basu S.
      • et al.
      Radiomics: The process and the challenges.
      The main interest of this approach is that these mathematical features offer potential to discover disease characteristics beyond human perception capacity, which can offer clinicians additional support for decision making in diagnostic and therapeutic scenarios.
      • Rizzo S.
      • Botta F.
      • Raimondi S.
      • et al.
      Radiomics: the facts and the challenges of image analysis.
      Furthermore, replacing an image by its radiomic features enables the use of more standard classification models. These techniques have been applied in multiple medical imaging areas, such as tumour phenotyping in brain
      • Wu G.
      • Chen Y.
      • Wang Y.
      • et al.
      Sparse Representation-Based Radiomics for the Diagnosis of Brain Tumors.
      , breast
      • Braman N.M.
      • Etesami M.
      • Prasanna P.
      • et al.
      Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response to neoadjuvant chemotherapy based on breast DCE-MRI.
      , lung
      • Thawani R.
      • McLane M.
      • Beig N.
      • et al.
      Radiomics and radiogenomics in lung cancer: A review for the clinician.
      and prostate
      • Penzias G.
      • Singanamalli A.
      • Elliott R.
      • et al.
      Identifying the morphologic basis for radiomic features in distinguishing different Gleason grades of prostate cancer on MRI: Preliminary findings.
      cancers, surgical planification
      • Huang Y.Q.
      • Liang C.H.
      • He L.
      • et al.
      Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer.
      and treatment response predictions
      • Núñez L.M.
      • Romero E.
      • Julià-Sapé M.
      • et al.
      Unraveling response to temozolomide in preclinical GL261 glioblastoma with MRI/MRSI using radiomics and signal source extraction.
      ,
      • Prasanna P.
      • Patel J.
      • Partovi S.
      • et al.
      Radiomic features from the peritumoral brain parenchyma on treatment-naïve multi-parametric MR imaging predict long versus short-term survival in glioblastoma multiforme: Preliminary findings.
      mainly employing magnetic resonance imaging and computerized tomography (CT) scan image datasets. In DM, radiomics has been applied to abdominal CT scans to assess the risk of DM
      • Lu C.Q.
      • Wang Y.C.
      • Meng X.P.
      • et al.
      Diabetes risk assessment with imaging: a radiomics study of abdominal CT.
      , for early detection of diabetic kidney disease

      Deng Y, Yang B ran, Luo J wen, et al. DTI-based radiomics signature for the detection of early diabetic kidney damage. Abdom Radiol 2020;45:2526–2531.

      , and to evaluate pancreas lesions including DM.
      • Abunahel B.M.
      • Pontre B.
      • Kumar H.
      • Petrov M.S.
      Pancreas image mining: a systematic review of radiomics.
      In Ophthalmology, recent reports have investigated their potential to predict intravitreal treatments durability in retinal vein occlusion and DME cases using wide field fluorescein angiography
      • Moosavi A.
      • Figueiredo N.
      • Prasanna P.
      • et al.
      Imaging Features of Vessels and Leakage Patterns Predict Extended Interval Aflibercept Dosing Using Ultra-Widefield Angiography in Retinal Vascular Disease: Findings from the PERMEATE Study.
      ,
      • Prasanna P.
      • Bobba V.
      • Figueiredo N.
      • et al.
      Radiomics-based assessment of ultra-widefield leakage patterns and vessel network architecture in the PERMEATE study: Insights into treatment durability.
      and OCT images
      • Sil Kar S.
      • Sevgi D.D.
      • Dong V.
      • et al.
      Multi-Compartment Spatially-Derived Radiomics from Optical Coherence Tomography Predict Anti-VEGF Treatment Durability in Macular Edema Secondary to Retinal Vascular Disease: Preliminary Findings.
      and for diagnostic purposes in myopic maculopathy using FR images.
      • Du Y.
      • Chen Q.
      • Fan Y.
      • et al.
      Automatic identification of myopic maculopathy related imaging features in optic disc region via machine learning methods.
      To present date, we are not aware of any study directed to investigate the potential of radiomics applied to retinal images for DM or DR diagnosis.
      The aim of this pioneer study is to evaluate the potential of radiomics techniques for DM and DR diagnostic classification, applied to a dataset of OCT and OCTA images from a previous prospective trial.
      • Zarranz-Ventura J.
      • Barraso M.
      • Alé-Chilet A.
      • et al.
      Evaluation of microvascular changes in the perifoveal vascular network using optical coherence tomography angiography (OCTA) in type I diabetes mellitus: a large scale prospective trial.
      Radiomic features will be extracted from retinal images and a standard set of classification models will be trained to evaluate the diagnostic accuracy of each image type and the combination of them to identify the optimal strategy for DM, DR and referable DR (R-DR) diagnosis. Secondary aims will be to evaluate the influence of adding clinical data and to explore the use of unilateral and bilateral image datasets for these predictions, to study further the external validity of our findings in future studies.

      METHODS:

      Dataset description

      This study was approved by the Institutional Review Board (IRB) (HCB/2021/0350) and adhered to the Declaration of Helsinki. The retinal images dataset was collected in a prospective OCTA trial (ClinicalTrials.gov NCT03422965) with a study protocol described elsewhere.
      • Zarranz-Ventura J.
      • Barraso M.
      • Alé-Chilet A.
      • et al.
      Evaluation of microvascular changes in the perifoveal vascular network using optical coherence tomography angiography (OCTA) in type I diabetes mellitus: a large scale prospective trial.
      Written informed consent was obtained for all participants. A total of 593 individuals with retinal images corresponding to 1186 eyes and ocular and systemic data were recruited, 478 type 1 DM patients (n= 956 eyes) and 115 healthy controls (n= 230 eyes). Systemic, ocular and retinal images quality exclusion criteria were applied as described elsewhere
      • Barraso M.
      • Alé-Chilet A.
      • Hernández T.
      • et al.
      Optical coherence tomography angiography in type 1 diabetes mellitus. Report 1: Diabetic retinopathy.
      ,
      • Bernal-Morales C.
      • Alé-Chilet A.
      • Martín-Pinardel R.
      • et al.
      Optical Coherence Tomography Angiography in Type 1 Diabetes Mellitus. Report 4: Glycated Haemoglobin.
      and 439 individuals (n= 726 eyes) with a complete battery of all retinal images constituted the primary dataset for analysis. To evaluate the influence of a potential bilaterality bias a secondary analysis was conducted selecting randomly only one eye per patient (1 eye/ 1 patient) that included 439 patients (n= 439 eyes).

      Retinal Images characteristics

      The battery of retinal images captured for each study eye included FR, OCT and OCTA images (Figure 1). FR images were obtained with a 45º field of view (Topcon DRI-Triton, Topcon Corp, Japan). OCT images included the central horizontal B-scan centered in fovea using a Macular Cube 6x6 mm (512x128 pixels) scanning protocol (Cirrus 5000, Carl Zeiss Meditec, Dublin, CA). OCTA scanning protocols included 3x3mm and 6x6mm OCTA macular cubes. Two-dimensional en-face OCTA images corresponding to the superficial and deep capillary plexus (SCP and DCP) automatically segmented by the built-in software (Angioplex Zeiss) for both scan sizes were analysed. OCT and OCTA images with artifacts or a signal strength index (SSI) <7 were excluded from analysis.
      Figure thumbnail gr1
      Figure 1Retinal images collected for each individual eye included in the study dataset. A: Fundus retinography (FR); B: Structural optical coherence tomography (OCT) macular scan; C: OCT angiography (OCTA) 3x3 mm Superficial Capillary Plexus (SCP); D: OCTA 3x3 mm Deep Capillary Plexus (DCP); E: OCTA 6x6 mm Superficial Capillary Plexus (SCP); F: OCTA 6x6 mm Deep Capillary Plexus (DCP). (FR: Topcon DRI-Triton, Topcon Corp, Japan; OCT and OCTA: Cirrus 5000, Carl Zeiss Meditec, Dublin, CA).

      Radiomics and clinical features

      Two types of data were analysed in this study, namely the clinical data collected during medical examination and the radiomic features derived from each type of retinal images.
      • -
        Clinical data: Systemic data included age, sex, body mass index, smoking status, and DM duration, as described elsewhere.
        • Zarranz-Ventura J.
        • Barraso M.
        • Alé-Chilet A.
        • et al.
        Evaluation of microvascular changes in the perifoveal vascular network using optical coherence tomography angiography (OCTA) in type I diabetes mellitus: a large scale prospective trial.
        No additional ocular data were included for this study (i.e. axial length, refractive status, visual acuity, etc.).
      • -
        Radiomics data: Radiomic features were extracted from all images in each study eye: FR, OCT, OCTA 3x3 mm SCP, OCTA 3x3 mm DCP, OCTA 6x6 mm SCP and OCTA 6x6 mm DCP. A special category was created for all OCTA images subtypes combined (“OCTA - All”), created with the average radiomic features of each OCTA image in each case. A total of 91 radiomic features were extracted related to the statistics distribution in each individual image, such as the 10th percentile, 90th percentile, energy, interquartile range, kurtosis, maximum, mean, mean absolute deviation, median, minimum, range, robust mean absolute deviation, robust mean squared, root mean squared, skewness, total energy and variance.
      Missing values were imputed, one-hot encoding was applied on categorical data and data was normalized (mean=0, standard deviation=1).

      Machine learning models

      A set of standard Machine Learning (ML) and related statistical techniques were used for classification purposes. Logistic Regression (LR), Linear Discriminant Analysis (LDA), Support Vector Classifiers (SVC) using linear (SVC-linear) and radial basis function (SVC-rbf) kernel and Random Forest (RF) models were selected as statistical (LR, LDA) and ML classifiers (SVC-linear, SVC-rbf, RF) based on their performance and adaptability to different environments. The dataset was split into training, validation and test cohorts with cross-validation for each classification problem, model and features subset, as described below.

      Feature selection, model optimization and performance

      Feature selection was performed using mutual information (MI) to rank each individual feature (10-fold cross-validation) and a wrapper backward elimination feature selection process, starting from the 60 features with highest MI. This process aimed to identify a parsimonious while predictive feature subset and the most appropiate model parameters. A 5-4 double cross-validation process was performed to train, validate and test each feature subset and to select the model parameters in each step of the backward elimination procedure. From the resulting models, the area under the curve (AUC) was extracted from each double cross-validation split and receiver operating curves (ROC) were constructed to graphically present the performance of the models.

      RESULTS:

      The demographics and baseline characteristics of included patients and eyes in the three classification tasks are described in Table 1. The diagnostic accuracy of the ML models constructed with the radiomic features extracted from each retinal imaging technique was quantified by their AUC performance and graphically presented in the form of box-whiskers plots and ROC-curves, as detailed below.
      Table 1Demographics and baseline characteristics of study patients and eyes for all classification tasks: Diabetes mellitus diagnosis, diabetic retinopathy diagnosis and referable diabetic retinopathy diagnosis. (DM= Diabetes mellitus, DR= Diabetic retinopathy, R-DR: Referable DR).
      VariableStatisticsDM diagnosisDR DiagnosisReferable DR Diagnosis
      ControlsType 1 DMp-valueNo DRDRp-valueNon R-DRR-DRp-value
      General characteristicsn=65n=374n=249n=125n=356n=18
      Age (years)Mean (SD)43.81 (14.53)38.85 (11.50)<0.0537.62 (11.78)41.29 (10.53)<0.0538.56 (11.47)44.47 (10.88)<0.05
      Median (Q1, Q3)40.30 (31.10,57.10)37.65 (29.83,46.68)36.40 (27.40,45.40)40.50 (34.30,48.30)37.40 (29.40,46.23)43.95 (36.67,52.00)
      Sex, femalen (%)41 (63.1%)195 (52.1%)0.134130 (52.2%)65 (52.0%)0.943189 (53.1%)6 (33.3%)0.146
      Smoking habits0.0690.305
      Non smokern (%)43 (70.5%)242 (64.9%)0.477166 (66.9%)76 (60.8%)0.291231 (65.1%)11 (61.1%)0.802
      Actual smokern (%)5 (8.2%)74 (19.8%)<0.0549 (19.8%)25 (20.0%)0.93473 (20.6%)1 (5.6%)0.220
      Ex-smokern (%)13 (21.3%)57 (15.3%)0.31833 (13.3%)24 (19.2%)0.18051 (14.4%)6 (33.3%)<0.05
      Hypertensionn (%)6 (9.8%)32 (8.6%)0.93317 (6.8%)15 (12.0%)0.13629 (8.1%)3 (16.7%)0.192
      BMI (kg/m2)Mean (SD)23.66 (3.77)24.68 (3.65)<0.0524.40 (3.55)25.21 (3.79)<0.0524.57 (3.62)26.73 (3.81)<0.05
      Median (Q1, Q3)22.89 (21.05,25.46)24.16 (22.16,26.84)23.81 (21.89,26.48)24.69 (22.50,27.28)24.09 (22.12,26.60)26.90 (23.55,29.68)
      DM- characterisitics
      DM duration (years)Mean (SD)0.00 (0.00)19.62 (10.70)-16.27 (9.96)26.31 (8.85)<0.0519.03 (10.46)31.90 (8.14)<0.05
      Median (Q1, Q3)0.00 (0.00,0.00)19.25 (10.50,27.05)15.40 (8.28,22.20)25.95 (20.38,32.90)19.00 (10.20,26.30)30.10 (25.50,36.00)
      HbA1c (2017)Mean (SD)5.30 (0.33)7.43 (0.977)<0.057.37 (1.01)7.56 (0.89)<0.057.43 (0.99)7.57 (0.74)0.301
      Median (Q1, Q3)5.30 (5.10, 5.50)7.40 (6.80, 7.90)7.30 (6.70, 7.80)7.40 (6.98, 8.10)7.40 (6.80, 7.90)7.60 (6.90, 8.20)
      Ocular Measuresn=102n=621n=436n=185n=594n=27
      Visual AcuityMean (SD)0.96 (0.07)0.96 (0.37)<0.050.97 (0.44)0.94 (0.08)0.1550.97 (0.38)0.91 (0.12)<0.05
      Median (Q1, Q3)1.00 (0.95,1.00)0.95 (0.95,1.00)0.95 (0.95,1.00)0.95 (0.95,1.00)0.95 (0.95,1.00)0.95 (0.90,0.97)
      Axial LengthMean (SD)23.64 (1.08)23.62 (1.22)0.79823.74 (1.18)23.36 (1.26)<0.0523.66 (1.22)22.89 (0.86)<0.05
      Median (Q1, Q3)23.53 (22.94,24.37)23.52 (22.82,24.39)23.65 (22.90,24.54)23.22 (22.68,23.91)23.54 (22.84,24.42)23.19 (22.43,23.38)
      OCTAn=103n=623n=438n=185n=596n=27
      Vessel Density (mm-1)Mean (SD)20.87 (1.25)20.14 (1.67)<0.0520.52 (1.52)19.24 (1.67)<0.0520.22 (1.64)18.30 (1.26)<0.05
      Median (Q1, Q3)21.00 (20.20,21.80)20.30 (19.10,21.40)20.70 (19.50,21.60)19.40 (18.20,20.30)20.40 (19.20,21.40)18.40 (17.60,18.95)
      Perfusion DensityMean (SD)0.374 (0.022)0.367 (0.026)<0.050.371 (0.025)0.358 (0.026)<0.050.368 (0.025)0.348 (0.021)<0.05
      Median (Q1, Q3)0.377 (0.365,0.389)0.370 (0.352,0.385)0.375 (0.355,0.390)0.363 (0.343,0.377)0.371 (0.353,0.386)0.350 (0.340,0.363)
      FAZ Area (mm2)Mean (SD)0.240 (0.080)0.239 (0.098)0.3600.236 (0.096)0.249 (0.102)0.1250.237 (0.097)0.290 (0.104)<0.05
      Median (Q1, Q3)0.250 (0.190,0.290)0.230 (0.170,0.300)0.230 (0.170,0.290)0.240 (0.180,0.310)0.230 (0.170,0.292)0.300 (0.255,0.370)
      FAZ Perimeter (mm)Mean (SD)2.070 (0.433)2.095 (0.488)0.9992.055 (0.465)2.189 (0.528)<0.052.077 (0.479)2.479 (0.535)<0.05
      Median (Q1, Q3)2.150 (1.860,2.350)2.090 (1.790,2.415)2.060 (1.790,2.370)2.190 (1.820,2.570)2.070 (1.788,2.400)2.540 (2.220,2.815)
      FAZ CircularityMean (SD)0.674 (0.080)0.655 (0.087)<0.050.666 (0.081)0.626 (0.094)<0.050.658 (0.085)0.581 (0.092)<0.05
      Median (Q1, Q3)0.680 (0.630,0.730)0.660 (0.600,0.720)0.670 (0.620,0.728)0.640 (0.560,0.700)0.670 (0.610,0.720)0.560 (0.525,0.650)
      Structural OCTn=103n=621n=436n=185n=594n=27
      Central Macular Thickness (μm)Mean (SD)258.5 (18.9)262.0 (20.5)<0.05261.1 (20.0)264.3 (21.6)0.076261.7 (20.22)270.3 (26.2)0.065
      Median (Q1, Q3)256.0 (245.0,272.0)262.0 (250.0,276.0)261.0 (249.7,275.0)266.0 (250.0,279.0)262.0 (250.0,276.0)272.0 (251.0,286.0)
      Macular VolumeMean (SD)10.28 (0.49)10.31 (0.69)0.77310.31 (0.61)10.30 (0.86)0.10310.304 (0.695)10.430 (0.538)0.143
      Median (Q1, Q3)10.20 (9.90,10.60)10.30 (10.00,10.60)10.30 (10.00,10.60)10.20 (9.90,10.50)10.30 (10.0,10.5)10.40 (10.10,10.75)
      Macular Thickness Average (μm)Mean (SD)285.31 (13.77)284.54 (18.28)0.874284.60 (20.14)284.40 (12.95)0.108284.323 (18.397)289.259 (15.009)0.169
      Median (Q1, Q3)284.0 (276.0,294.5)285.0 (278.0,293.0)286.0 (278.0,293.0)284.0 (276.0,292.0)285.000 (277.250,292.000)288.000 (279.000,298.500)

      Classification task 1: Diabetes mellitus diagnosis

      The performance of the models for the classification of DM patients is shown in AUC box plots (Figure 2) and ROC curves (Figure 3). OCT showed the best performance to predict a DM diagnosis in all the models (Figure 3, left column). In LR, the highest mean AUC was observed for OCT (AUC 0.82, SD 0.08) compared to FR (AUC 0.63, 0.08) or OCTA (AUC 0.74, 0.04) (Figure 2, top-left). Interestingly, the combination of all retinal imaging techniques showed a slightly superior mean AUC (0.87, 0.03). These results were also consistent in the LDA, SVC-linear and SVC-rbf trained models.
      Figure thumbnail gr2
      Figure 2Diabetes Mellitus (DM) diagnosis: Machine learning models performance for DM classification based on radiomic features of retinal images. Box-and-whiskers plots representing AUC values for each retinal image type. Top-left: radiomics extracted from each retinal image type. Top-right: radiomics extracted from each retinal image type + clinical variables. Bottom-left: radiomics, extracted from each OCTA type. Bottom-right: radiomics, extracted from each OCTA type + clinical variables (FR: Fundus retinography; OCT: optical coherence tomography; OCTA: optical coherence tomography angiography; LDA: Linear discriminant analysis; SVC: support vector classifier).
      Figure thumbnail gr3
      Figure 3Receiver operating characteristic (ROC) curves of models performance for Diabetes Mellitus (DM), Diabetic Retinopathy (DR) and Referable DR (R-DR) diagnosis. Left column: ROC curves for DM diagnosis detailed by models. Central column: ROC curves for DR diagnosis detailed by models. Right column: ROC curves for R-DR diagnosis detailed by models. Top row: example of a linear model (Linear discriminant analysis, LDA). Bottom row: example of a non-linear model (Support Vector Classifier – RBF).

      Classification task 2: Diabetic retinopathy diagnosis

      For DR diagnosis, the best AUC performance was observed for OCTA (Figure 3, central column, and Figure 4). In the LR model, OCTA achieved the best performance (AUC 0.77, 0.03), compared to FR (0.56, 0.09) and OCT (0.54, 0.03), and similar results were consistently achieved with LDA, SVC-linear, SVC-rbf and RF models (Figure 4, top-left). The combination of all techniques did not achieve a superior performance (AUC 0.77, 0.02). The best performance was observed for 3x3mm SCP (AUC 0.76, 0.04) compared to 3x3mm DCP (AUC 0.73, 0.03), 6x6mm SCP (AUC 0.69, 0.06) or 6x6mm DCP (AUC 0.74, 0.04). The combination of all OCTA scanning protocol images improved slightly the performance in all the models (Figure 4, bottom-left).
      Figure thumbnail gr4
      Figure 4Diabetic Retinopathy (DR) diagnosis: Machine learning models performance for DR classification based on radiomic features of retinal images. Box-and-whiskers plots representing AUC values for each retinal image type. Top-left: radiomics extracted from each retinal image type. Top-right: radiomics extracted from each retinal image type + clinical variables. Bottom-left: radiomics, extracted from each OCTA type. Bottom-right: radiomics, extracted from each OCTA type + clinical variables (FR: Fundus retinography; OCT: optical coherence tomography; OCTA: optical coherence tomography angiography; LDA: Linear discriminant analysis; SVC: support vector classifier).

      Classification task 3: Referable diabetic retinopathy diagnosis

      For R-DR diagnosis, OCTA images achieved again the best AUC performance (Figure 3, right column, and Figure 5). In the LR model, OCTA achieved better performance (AUC 0.87, 0.12), than FR (0.42, 0.18) and OCT (0.60, 0.11); with similar results achieved with LDA, SVC-linear, SVC-rbf and RF models (Figure 5, top-left). The combination of all retinal imaging techniques did not achieve a superior performance (AUC 0.87, 0.10). The best performance was observed for 3x3mm DCP (AUC 0.86, 0.08) compared to 3x3mm SCP (AUC 0.83, 0.15), 6x6mm SCP (AUC 0.74, 0.18) or 6x6mm DCP (AUC 0.82, 0.11). The combination of all OCTA scanning protocol images did not improve the performance of the models (Figure 5, bottom-left).
      Figure thumbnail gr5
      Figure 5Referable Diabetic Retinopathy (R-DR) diagnosis: Machine learning models performance for R-DR classification based on radiomic features of retinal images. Box-and-whiskers plots representing AUC values for each retinal image type. Top-left: radiomics extracted from each retinal image type. Top-right: radiomics extracted from each retinal image type + clinical variables. Bottom-left: radiomics, extracted from each OCTA type. Bottom-right: radiomics, extracted from each OCTA type + clinical variables (FR: Fundus retinography; OCT: optical coherence tomography; OCTA: optical coherence tomography angiography; LDA: Linear discriminant analysis; SVC: support vector classifier).

      Impact of clinical variables data addition to the predictive models

      The impact of the clinical variables (detailed in the Methods section) on the models' prediction for the three tasks was also analyzed, as presented in Table 2 and Table 3, Figure 2, Figure 4 and Figure 5. For DM diagnosis, the addition of clinical data variables improved the performance of the models in all the individual and combined retinal image techniques (Figure 2, top-right) as well as the OCTA scanning protocols (Figure 2, bottom-right). Consistently, similar improvements were observed for DR and R-DR diagnosis with all images (Figure 4 and Figure 5, top right) and OCTA scanning protocols (Figure 4 and Figure 5, bottom-right).
      Table 2Models performance disclosed by retinal imaging type and OCTA scanning protocol (n = 726). Models performance described by mean AUC (standard deviation). In bold, greatest AUCs by model and single image type. In bold italic, combination of all retinal images techniques. (DM: Diabetes mellitus; DR: Diabetic retinopathy; LR: Logistic regression; LDA: Linear discriminant analisys; SVC: Support vector classifier; RF: Random forest).
      TaskDataModelsFROCTOCTA (All)OCTA (3x3 SCP)OCTA (3x3 DCP)OCTA (6x6 SCP)OCTA (6x6 DCP)FR + OCT + OCTA
      DM diagnosisImagesLR0.63 (0.08)0.82 (0.03)0.74 (0.04)0.70 (0.06)0.63 (0.05)0.67 (0.06)0.65 (0.06)0.87 (0.03)
      LDA0.64 (0.10)0.82 (0.04)0.74 (0.06)0.71 (0.04)0.64 (0.05)0.67 (0.04)0.66 (0.04)0.87 (0.03)
      SVC-Linear0.53 (0.07)0.67 (0.08)0.64 (0.09)0.59 (0.05)0.53 (0.06)0.62 (0.08)0.50 (0.08)0.85 (0.03)
      SVC-RBF0.57 (0.06)0.78 (0.04)0.66 (0.04)0.65 (0.05)0.59 (0.10)0.53 (0.13)0.55 (0.08)0.86 (0.03)
      RF0.65 (0.07)0.78 (0.04)0.66 (0.05)0.62 (0.06)0.58 (0.07)0.58 (0.05)0.61 (0.05)0.86 (0.05)
      + clinical variablesLR0.67 (0.08)0.86 (0.03)0.78 (0.04)0.77 (0.05)0.71 (0.06)0.72 (0.05)0.74 (0.05)0.89 (0.03)
      LDA0.67 (0.06)0.86 (0.03)0.77 (0.05)0.76 (0.06)0.73 (0.04)0.72 (0.05)0.75 (0.05)0.89 (0.04)
      SVC-Linear0.57 (0.09)0.79 (0.07)0.77 (0.04)0.69 (0.09)0.61 (0.05)0.69 (0.09)0.56 (0.08)0.88 (0.03)
      SVC-RBF0.66 (0.06)0.85 (0.05)0.76 (0.06)0.69 (0.06)0.66 (0.04)0.69 (0.06)0.68 (0.04)0.87 (0.04)
      RF0.67 (0.05)0.86 (0.03)0.70 (0.06)0.72 (0.04)0.66 (0.05)0.65 (0.06)0.72 (0.05)0.87 (0.05)
      DR diagnosisImagesLR0.56 (0.09)0.54 (0.03)0.77 (0.03)0.76 (0.04)0.73 (0.03)0.69 (0.06)0.74 (0.04)0.77 (0.02)
      LDA0.55 (0.07)0.54 (0.04)0.77 (0.04)0.76 (0.04)0.74 (0.03)0.70 (0.08)0.73 (0.04)0.77 (0.03)
      SVC-Linear0.47 (0.07)0.50 (0.05)0.77 (0.04)0.77 (0.05)0.74 (0.03)0.68 (0.06)0.73 (0.04)0.77 (0.03)
      SVC-RBF0.49 (0.09)0.49 (0.05)0.78 (0.03)0.77 (0.04)0.73 (0.02)0.70 (0.07)0.73 (0.03)0.77 (0.03)
      RF0.54 (0.04)0.47 (0.03)0.75 (0.03)0.72 (0.05)0.71 (0.03)0.63 (0.06)0.69 (0.05)0.76 (0.04)
      + clinical variablesLR0.78 (0.04)0.78 (0.04)0.82 (0.04)0.81 (0.04)0.80 (0.03)0.79 (0.05)0.80 (0.05)0.81 (0.04)
      LDA0.78 (0.04)0.78 (0.04)0.82 (0.03)0.82 (0.04)0.80 (0.04)0.79 (0.05)0.79 (0.05)0.82 (0.03)
      SVC-Linear0.77 (0.04)0.77 (0.04)0.82 (0.04)0.81 (0.04)0.80 (0.04)0.79 (0.05)0.80 (0.05)0.81 (0.05)
      SVC-RBF0.77 (0.04)0.78 (0.04)0.82 (0.04)0.81 (0.05)0.80 (0.04)0.79 (0.05)0.80 (0.04)0.81 (0.04)
      RF0.74 (0.03)0.77 (0.04)0.81 (0.04)0.80 (0.05)0.79 (0.04)0.79 (0.05)0.80 (0.06)0.81 (0.04)
      Referable DR diagnosisImagesLR0.42 (0.18)0.60 (0.11)0.87 (0.12)0.83 (0.15)0.86 (0.08)0.74 (0.18)0.82 (0.11)0.87 (0.10)
      LDA0.55 (0.19)0.45 (0.11)0.87 (0.08)0.86 (0.11)0.86 (0.10)0.70 (0.20)0.80 (0.13)0.87 (0.08)
      SVC-Linear0.49 (0.19)0.49 (0.13)0.85 (0.10)0.76 (0.14)0.73 (0.14)0.76 (0.14)0.83 (0.09)0.83 (0.08)
      SVC-RBF0.52 (0.20)0.58 (0.11)0.73 (0.11)0.70 (0.18)0.77 (0.09)0.69 (0.14)0.77 (0.10)0.73 (0.13)
      RF0.57 (0.12)0.53 (0.12)0.77 (0.08)0.79 (0.08)0.83 (0.07)0.71 (0.10)0.72 (0.10)0.73 (0.09)
      + clinical variablesLR0.82 (0.11)0.82 (0.11)0.92 (0.02)0.89 (0.08)0.90 (0.03)0.84 (0.09)0.90 (0.03)0.92 (0.02)
      LDA0.75 (0.14)0.81 (0.10)0.91 (0.06)0.90 (0.06)0.90 (0.04)0.87 (0.04)0.88 (0.06)0.87 (0.07)
      SVC-Linear0.74 (0.14)0.66 (0.08)0.90 (0.04)0.86 (0.05)0.84 (0.07)0.76 (0.10)0.79 (0.11)0.85 (0.06)
      SVC-RBF0.70 (0.16)0.63 (0.11)0.91 (0.05)0.88 (0.06)0.86 (0.06)0.70 (0.13)0.83 (0.07)0.89 (0.04)
      RF0.68 (0.10)0.74 (0.10)0.79 (0.08)0.84 (0.08)0.87 (0.06)0.78 (0.10)0.70 (0.10)0.79 (0.07)
      Table 3Influence of bilaterality on models performance. Comparison using 1 eye per patient (n = 439) or 2 eyes per patient (n = 726). Models performance described by mean AUC (standard deviation). In bold, greatest AUCs obtained in 1 and 2 eyes cohorts (if different), disclosed by model and image type. (DM: Diabetes mellitus; DR: Diabetic retinopathy; LR: Logistic regression; LDA: Linear discriminant analisys; SVC: Support vector classifier; RF: Random forest].
      TaskDataModelsFROCTOCTA (All)FR + OCT + OCTA
      1 eye2 eyes1 eye2 eyes1 eye2 eyes1 eye2 eyes
      DM diagnosisImagesLR0.56 (0.13)0.63 (0.08)0.82 (0.04)0.82 (0.03)0.71 (0.06)0.74 (0.04)0.85 (0.03)0.87 (0.03)
      LDA0.50 (0.09)0.64 (0.10)0.81 (0.04)0.82 (0.04)0.72 (0.07)0.74 (0.06)0.80 (0.04)0.87 (0.03)
      SVC-Linear0.51 (0.08)0.53 (0.07)0.71 (0.07)0.67 (0.08)0.62 (0.12)0.64 (0.09)0.83 (0.03)0.85 (0.03)
      SVC-RBF0.49 (0.09)0.57 (0.06)0.80 (0.04)0.78 (0.04)0.64 (0.09)0.66 (0.04)0.82 (0.03)0.86 (0.03)
      RF0.57 (0.09)0.65 (0.07)0.78 (0.06)0.78 (0.04)0.62 (0.07)0.66 (0.05)0.85 (0.03)0.86 (0.05)
      + clinical variablesLR0.67 (0.09)0.67 (0.08)0.87 (0.03)0.86 (0.03)0.80 (0.06)0.78 (0.04)0.89 (0.03)0.89 (0.03)
      LDA0.65 (0.09)0.67 (0.06)0.86 (0.03)0.86 (0.03)0.75 (0.08)0.77 (0.05)0.88 (0.04)0.89 (0.04)
      SVC-Linear0.49 (0.15)0.57 (0.09)0.83 (0.04)0.79 (0.07)0.77 (0.08)0.77 (0.04)0.88 (0.03)0.88 (0.03)
      SVC-RBF0.61 (0.11)0.66 (0.06)0.85 (0.03)0.85 (0.05)0.77 (0.08)0.76 (0.06)0.88 (0.03)0.87 (0.04)
      RF0.69 (0.06)0.67 (0.05)0.87 (0.05)0.86 (0.03)0.72 (0.08)0.70 (0.06)0.86 (0.05)0.87 (0.05)
      DR diagnosisImagesLR0.54 (0.08)0.56 (0.09)0.52 (0.08)0.54 (0.03)0.75 (0.05)0.77 (0.03)0.75 (0.04)0.77 (0.02)
      LDA0.58 (0.09)0.55 (0.07)0.52 (0.08)0.54 (0.04)0.74 (0.05)0.77 (0.04)0.74 (0.05)0.77 (0.03)
      SVC-Linear0.48 (0.08)0.47 (0.07)0.52 (0.07)0.50 (0.05)0.76 (0.05)0.77 (0.04)0.74 (0.05)0.77 (0.03)
      SVC-RBF0.52 (0.11)0.49 (0.09)0.52 (0.08)0.49 (0.05)0.74 (0.06)0.78 (0.03)0.75 (0.05)0.77 (0.03)
      RF0.55 (0.06)0.54 (0.04)0.50 (0.07)0.47 (0.03)0.72 (0.04)0.75 (0.03)0.68 (0.04)0.76 (0.04)
      + clinical variablesLR0.76 (0.03)0.78 (0.04)0.76 (0.03)0.78 (0.04)0.80 (0.04)0.82 (0.04)0.80 (0.04)0.81 (0.04)
      LDA0.76 (0.03)0.78 (0.04)0.76 (0.03)0.78 (0.04)0.80 (0.04)0.82 (0.03)0.79 (0.03)0.82 (0.03)
      SVC-Linear0.77 (0.02)0.77 (0.04)0.77 (0.02)0.77 (0.04)0.79 (0.04)0.82 (0.04)0.79 (0.04)0.81 (0.05)
      SVC-RBF0.77 (0.02)0.77 (0.04)0.77 (0.02)0.78 (0.04)0.80 (0.05)0.82 (0.04)0.78 (0.03)0.81 (0.04)
      RF0.73 (0.03)0.74 (0.03)0.76 (0.02)0.77 (0.04)0.78 (0.04)0.81 (0.04)0.78 (0.03)0.81 (0.04)
      Referable DR diagnosisImagesLR0.41 (0.14)0.42 (0.18)0.52 (0.11)0.60 (0.11)0.87 (0.14)0.87 (0.12)0.88 (0.12)0.87 (0.10)
      LDA0.39 (0.11)0.55 (0.19)0.48 (0.10)0.45 (0.11)0.90 (0.09)0.87 (0.08)0.88 (0.11)0.87 (0.08)
      SVC-Linear0.49 (0.20)0.49 (0.19)0.47 (0.14)0.49 (0.13)0.79 (0.20)0.85 (0.10)0.81 (0.13)0.83 (0.08)
      SVC-RBF0.49 (0.15)0.52 (0.20)0.52 (0.14)0.58 (0.11)0.81 (0.12)0.73 (0.11)0.82 (0.14)0.73 (0.13)
      RF0.59 (0.17)0.57 (0.12)0.73 (0.13)0.53 (0.12)0.80 (0.11)0.77 (0.08)0.78 (0.13)0.73 (0.09)
      + clinical variablesLR0.82 (0.08)0.82 (0.11)0.82 (0.10)0.82 (0.11)0.90 (0.08)0.92 (0.02)0.90 (0.09)0.92 (0.02)
      LDA0.81 (0.09)0.75 (0.14)0.73 (0.11)0.81 (0.10)0.91 (0.07)0.91 (0.06)0.81 (0.11)0.87 (0.07)
      SVC-Linear0.62 (0.24)0.74 (0.14)0.52 (0.15)0.66 (0.08)0.90 (0.07)0.90 (0.04)0.89 (0.06)0.85 (0.06)
      SVC-RBF0.56 (0.25)0.70 (0.16)0.69 (0.09)0.63 (0.11)0.84 (0.11)0.91 (0.05)0.86 (0.08)0.89 (0.04)
      RF0.74 (0.10)0.68 (0.10)0.72 (0.14)0.74 (0.10)0.80 (0.12)0.79 (0.08)0.82 (0.09)0.79 (0.07)

      Influence of unilateral vs bilateral image datasets in models performance

      For DM diagnosis, models' prediction in bilateral eyes image sets were not overall superior to the use of unilateral eyes set of images and favoured 1 eye for OCT and 2 eyes for FR, OCTA and the combination of images (Table 3, Figure 6). When clinical variables were included, these differences were reduced. For DR diagnosis, AUC values were greater for 1 eye in FR and OCT and for 2 eyes in OCTA and the combination of techniques in all ML models and image types. For R-DR, AUC values were similar for FR, OCT, OCTA and the combination of techniques. The addition of clinical data variables improved the performance of all the individual and combined retinal image sets.
      Figure thumbnail gr6
      Figure 6Comparison of models performance for Diabetes Mellitus (DM), Diabetic Retinopathy (DR) and Referable Diabetic Retinopathy (R-DR) diagnosis using unilateral and bilateral datasets. Left column: Unilateral dataset (1 eye per patient). Right column: Bilateral dataset (2 eyes per patient). Top-Top: Logistic regression receiver operating characteristic (ROC) curves for DM diagnosis for each retinal image type using 1 eye (left) and 2 eyes (right) per patient. Top-Bottom: Box-and-whiskers plots representing AUC values for DM diagnosis for each model and retinal image type, using 1 eye (left) and 2 eyes (right) per patient. Middle-top: Logistic regression ROC curves for DR diagnosis for each retinal image type using 1 eye (left) and 2 eyes (right) per patient. Middle-Bottom: Box-and-whiskers plots representing AUC values for DR diagnosis for each model and retinal image type, using 1 eye (left) and 2 eyes (right) per patient. Bottom-top: Logistic regression ROC curves for R-DR diagnosis for each retinal image type using 1 eye (left) and 2 eyes (right) per patient. Bottom-Bottom: Box-and-whiskers plots representing AUC values for R-DR diagnosis for each model and retinal image type, using 1 eye (left) and 2 eyes (right) per patient. (FR: Fundus retinography; OCT: optical coherence tomography; OCTA: optical coherence tomography angiography; LDA: Linear discriminant analysis; SVC: support vector classifier).

      DISCUSSION:

      This study demonstrates the diagnostic potential of ML classifiers applied to the radiomic features extracted from OCT and OCTA images for DM, DR and R-DR diagnosis. The best performance of the ML models is achieved in OCT images for DM diagnosis, and OCTA images for DR and R-DR diagnosis. Interestingly, the best OCTA scanning protocol to identify DR is the 3x3mm SCP scan, meanwhile for R-DR is the 3x3mm DCP scan. We demonstrate that the addition of clinical variables improves the performance of all ML models and types of retinal images. The relevance of this pioneer project is that it demonstrates that radiomic-based ML techniques may be applied to different types of retinal images as an effective method to identify DM and DR cases, as a step forward to allow the automatic detection of these conditions in larger scale cohorts.
      Radiomics is an emerging translational field of research. The main interest of this methodology lies in its ability to extract mineable multivariate quantitative data from medical images using computer vision techniques, with the potential to discover disease characteristics beyond human perception capacity.
      • Kumar V.
      • Gu Y.
      • Basu S.
      • et al.
      Radiomics: The process and the challenges.
      To the best of our knowledge, this is the first study directed to evaluate the potential of radiomics applied to OCTA images, a recently developed non-invasive retinal imaging technique, and also the first one directed to assess this potential in a multimodal series of retinal imaging techniques for DM, DR and R-DR diagnosis.
      The results reported for DM diagnosis suggest that our radiomics-based approach its able to discriminate between controls and DM patients, particularly in OCT images. Most of the AI studies in this field of ophthalmology have been conducted in existing image datasets from DR screening programs with DM patients but no healthy controls. For this reason, AI for DM diagnosis has been applied to clinical data in nutritional surveys
      • Han L.
      • Luo S.
      • Yu J.
      • et al.
      Rule extraction from support vector machines using ensemble learning approach: An application for diagnosis of diabetes.
      and hospital data,
      • Shankaracharya Odedra D.
      • Samanta S.
      • Vidyarthi A.S.
      Computational intelligence-based diagnosis tool for the detection of prediabetes and type 2 diabetes in India.
      but no consistent data are available for retinal images for this task. There is an exponential growth in the number of OCT scans performed daily in the community setting, and our results suggest that this approach would potentially be implemented as a general screening tool for type 1 DM diagnosis, the less frequent type of DM, in the general population. In this scenario, due to the marginal improvement observed with the combination of imaging techniques (AUC 0.82 vs 0.87), the addition of clinical variables (AUC 0.82 vs 0.86) or both (AUC 0.82 vs 0.89) cost-benefit studies will be required to investigate further whether these items should be included in potential strategies designed for DM diagnosis in the general population in the future.
      OCTA was the imaging technique that achieved the best performance for DR diagnosis, and consistently showed superior performance in all models, particularly in the 3x3 SCP scanning protocol images. This is an important point, as some previous studies have suggested that the DCP is affected earlier in DR development. Nevertheless, the SCP is less prone to have projection artifacts, being this a factor that could explain, at least in part, the results observed. Conversely to these findings, our results for DR and R-DR diagnosis suggest that the combination of techniques does not improve the performance of the models. Unfortunately, no previous studies have been conducted to evaluate specifically radiomics features in OCTA datasets. To benchmark our results, we have analysed separately DR and R-DR diagnosis, as most of the previous ML studies conducted in FR from DR screening programs frequently describe detection rates only for R-DR cases.
      • Gulshan V.
      • Peng L.
      • Coram M.
      • et al.
      Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs.
      ,
      • Ting D.S.W.
      • Cheung C.Y.-L.L.
      • Lim G.
      • et al.
      Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes.
      Future comparative studies will reveal were radiomics applied to OCTA images fit in the diagnostic algorithm of DR and R-DR detection.
      With regards to technical considerations, the performance of the models was overall consistent when experiments were conducted in both datasets, with bilateral eyes and unilateral eyes. This is relevant, as most of the AI applied to ophthalmology studies include both eyes of patients, with a potential risk of correlation bias that could lead to model overfitting. We have observed no differences for DM diagnosis and only slightly superior results for DR and R-DR diagnosis using bilateral eyes, especially in OCTA images. We do believe that this may be related to the greater number of images analysed for each individual eye due to the different scanning protocols employed. Also, an important point is that, in our dataset, linear models seem to achieve better and more robust results, suggesting that these simpler models may be more appropriate for these type of classification tasks, possibly in relationship with the data type or the number of instances evaluated.
      This study presents a series of strengths and limitations. First, the study cohort includes type 1 DM patients, when most of the DR screening programs only include type 2 DM or do not differentiate between both DM types. The results of this study may not be applicable to type 2 DM patients, as multiple differences in presenting age, ocular and systemic comorbidities prevent this comparison. However, this also confers high internal validity, as it is allegedly the largest type 1 DM retinal imaging dataset of the literature.
      • Zarranz-Ventura J.
      • Barraso M.
      • Alé-Chilet A.
      • et al.
      Evaluation of microvascular changes in the perifoveal vascular network using optical coherence tomography angiography (OCTA) in type I diabetes mellitus: a large scale prospective trial.
      • Barraso M.
      • Alé-Chilet A.
      • Hernández T.
      • et al.
      Optical coherence tomography angiography in type 1 diabetes mellitus. Report 1: Diabetic retinopathy.
      • Bernal-Morales C.
      • Alé-Chilet A.
      • Martín-Pinardel R.
      • et al.
      Optical Coherence Tomography Angiography in Type 1 Diabetes Mellitus. Report 4: Glycated Haemoglobin.
      Second, the control and DM groups differed in some baseline characteristics such as age, smoking status and BMI, which may have influenced the results. Third, for OCT images only a single b-scan centred in the fovea has been used to extract the radiomics features, when each macular cube scan consists on 128 consecutive b-scans, which could potentially under or overestimate the results reported either way. Fourth, the subgroup “OCTA all” included 4 images instead of 1, however no superior results were observed compared to single 3x3 OCTA scans, suggesting that this may have not influenced the results. And finally, although most of the clinical variables included in the models are just demographics (i.e. age, sex, etc.), adding this information may require data collection beyond the processing of retinal images, limiting the potential deployment of this strategy in the community setting.
      In conclusion, this proof-of-concept study demonstrates that the radiomic features extracted from OCT and OCTA images permit the identification of DM and DR patients with a standard set of ML classifiers. For each task evaluated, recommendations about specific imaging tests (OCT for DM diagnosis, OCTA for DR and R-DR diagnosis) and protocols (OCTA 3x3mm SCP and DCP scans) for best performance of the models are provided. Moreover, the value of adding clinical data and the impact of using images from one or both eyes is also evaluated, to provide readers with relevant information for planning future strategies for each classification task. Given the expected DM prevalence increase in the next decades due to population aging, the market size of the clinical need, and the predicted economic cost associated with DM in the next future, all the efforts directed to find the optimal diagnostic tool should be a priority for the health system. In this scenario, the exploration of the potential of AI combined with radiomics in state-of-the-art non-invasive retinal image techniques such as OCT and OCTA appears to be a promising avenue of research.

      REFERENCES:

        • Topol E.J.
        High-performance medicine: the convergence of human and artificial intelligence.
        Nat Med. 2019;
        • Ting D.S.W.
        • Pasquale L.R.
        • Peng L.
        • et al.
        Artificial intelligence and deep learning in ophthalmology.
        Br J Ophthalmol. 2019; 103: 167-175
        • De Fauw J.
        • Ledsam J.R.
        • Romera-Paredes B.
        • et al.
        Clinically applicable deep learning for diagnosis and referral in retinal disease.
        Nat Med. 2018; 24: 1342-1350
        • Zarranz-Ventura J.
        • Bernal-Morales C.
        • Saenz de Viteri M.
        • et al.
        Artificial intelligence and ophthalmology: Current status.
        Arch la Soc Española Oftalmol (English Ed. 2021; 96: 399-400
        • Maloca P.M.
        • Lee A.Y.
        • De Carvalho E.R.
        • et al.
        Validation of automated artificial intelligence segmentation of optical coherence tomography images.
        PLoS One. 2019; 14
        • Yim J.
        • Chopra R.
        • Spitz T.
        • et al.
        Predicting conversion to wet age-related macular degeneration using deep learning.
        Nat Med. 2020;
        • Bogunovic H.
        • Waldstein S.M.
        • Schlegl T.
        • et al.
        Prediction of Anti-VEGF Treatment Requirements in Neovascular AMD Using a Machine Learning Approach.
        Investig Opthalmology Vis Sci. 2017; 58: 3240
        • Wagner S.K.
        • Fu D.J.
        • Faes L.
        • et al.
        Insights into Systemic Disease through Retinal Imaging-Based Oculomics.
        Transl Vis Sci Technol. 2020; 9: 6
        • Burlina P.M.
        • Joshi N.
        • Pekala M.
        • et al.
        Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks.
        JAMA Ophthalmol. 2017;
        • Wang Y.
        • Zhang Y.
        • Yao Z.
        • et al.
        Machine learning based detection of age-related macular degeneration (AMD) and diabetic macular edema (DME) from optical coherence tomography (OCT) images.
        Biomed Opt Express. 2016;
        • ElTanboly A.
        • Ismail M.
        • Shalaby A.
        • et al.
        A computer-aided diagnostic system for detecting diabetic retinopathy in optical coherence tomography images.
        Med Phys. 2017;
        • Brown J.M.
        • Campbell J.P.
        • Beers A.
        • et al.
        Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks.
        JAMA Ophthalmology. 2018;
        • Poplin R.
        • Varadarajan A.V.
        • Blumer K.
        • et al.
        Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning.
        Nat Biomed Eng. 2018; 2: 158-164
        • Gulshan V.
        • Peng L.
        • Coram M.
        • et al.
        Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.
        JAMA - J Am Med Assoc. 2016; 316: 2402-2410
        • Xie Y.
        • Nguyen Q.D.
        • Hamzah H.
        • et al.
        Artificial intelligence for teleophthalmology-based diabetic retinopathy screening in a national programme: an economic analysis modelling study.
        Lancet Digit Heal. 2020; 2: e240-e249
        • van der Heijden A.A.
        • Abramoff M.D.
        • Verbraak F.
        • et al.
        Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System.
        Acta Ophthalmol. 2018; 96: 63-68
        • Grzybowski A.
        • Brona P.
        • Lim G.
        • et al.
        Artificial intelligence for diabetic retinopathy screening: a review.
        Eye. 2020; 34: 451-460
        • Hua C.H.
        • Kim K.
        • Huynh-The T.
        • et al.
        Convolutional Network with Twofold Feature Augmentation for Diabetic Retinopathy Recognition from Multi-Modal Images.
        IEEE J Biomed Heal Informatics. 2021; 25: 2686-2697
      1. Le D, Alam M, Yao C, et al. Transfer learning for automated OCTA detection of diabetic retinopathy. arXiv 2019:1–20.

        • Spaide R.F.
        • Fujimoto J.G.
        • Waheed N.K.
        • et al.
        Optical coherence tomography angiography.
        Prog Retin Eye Res. 2018; 64: 1-55
        • Kalra G.
        • Zarranz-Ventura J.
        • Chahal R.
        • et al.
        Optical computed tomography (OCT) angiolytics: A review of OCT angiography quantiative biomarkers.
        Surv Ophthalmol. 2021;
        • Kumar V.
        • Gu Y.
        • Basu S.
        • et al.
        Radiomics: The process and the challenges.
        Magn Reson Imaging. 2012; 30: 1234-1248
        • Rizzo S.
        • Botta F.
        • Raimondi S.
        • et al.
        Radiomics: the facts and the challenges of image analysis.
        Eur Radiol Exp. 2018; 2
        • Wu G.
        • Chen Y.
        • Wang Y.
        • et al.
        Sparse Representation-Based Radiomics for the Diagnosis of Brain Tumors.
        IEEE Trans Med Imaging. 2018; 37: 893-905
        • Braman N.M.
        • Etesami M.
        • Prasanna P.
        • et al.
        Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response to neoadjuvant chemotherapy based on breast DCE-MRI.
        Breast Cancer Res. 2017; 19: 1-14
        • Thawani R.
        • McLane M.
        • Beig N.
        • et al.
        Radiomics and radiogenomics in lung cancer: A review for the clinician.
        Lung Cancer. 2018; 115: 34-41
        • Penzias G.
        • Singanamalli A.
        • Elliott R.
        • et al.
        Identifying the morphologic basis for radiomic features in distinguishing different Gleason grades of prostate cancer on MRI: Preliminary findings.
        PLoS One. 2018; 13: 1-20
        • Huang Y.Q.
        • Liang C.H.
        • He L.
        • et al.
        Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer.
        J Clin Oncol. 2016; 34: 2157-2164
        • Núñez L.M.
        • Romero E.
        • Julià-Sapé M.
        • et al.
        Unraveling response to temozolomide in preclinical GL261 glioblastoma with MRI/MRSI using radiomics and signal source extraction.
        Sci Rep. 2020; 10: 1-13
        • Prasanna P.
        • Patel J.
        • Partovi S.
        • et al.
        Radiomic features from the peritumoral brain parenchyma on treatment-naïve multi-parametric MR imaging predict long versus short-term survival in glioblastoma multiforme: Preliminary findings.
        Eur Radiol. 2017; 27: 4188-4197
        • Lu C.Q.
        • Wang Y.C.
        • Meng X.P.
        • et al.
        Diabetes risk assessment with imaging: a radiomics study of abdominal CT.
        Eur Radiol. 2019; 29: 2233-2242
      2. Deng Y, Yang B ran, Luo J wen, et al. DTI-based radiomics signature for the detection of early diabetic kidney damage. Abdom Radiol 2020;45:2526–2531.

        • Abunahel B.M.
        • Pontre B.
        • Kumar H.
        • Petrov M.S.
        Pancreas image mining: a systematic review of radiomics.
        Eur Radiol. 2021; 31: 3447-3467
        • Moosavi A.
        • Figueiredo N.
        • Prasanna P.
        • et al.
        Imaging Features of Vessels and Leakage Patterns Predict Extended Interval Aflibercept Dosing Using Ultra-Widefield Angiography in Retinal Vascular Disease: Findings from the PERMEATE Study.
        IEEE Trans Biomed Eng. 2021; 68: 1777-1786
        • Prasanna P.
        • Bobba V.
        • Figueiredo N.
        • et al.
        Radiomics-based assessment of ultra-widefield leakage patterns and vessel network architecture in the PERMEATE study: Insights into treatment durability.
        Br J Ophthalmol. 2021; 105: 1155-1160
        • Sil Kar S.
        • Sevgi D.D.
        • Dong V.
        • et al.
        Multi-Compartment Spatially-Derived Radiomics from Optical Coherence Tomography Predict Anti-VEGF Treatment Durability in Macular Edema Secondary to Retinal Vascular Disease: Preliminary Findings.
        IEEE J Transl Eng Heal Med. 2021; 9
        • Du Y.
        • Chen Q.
        • Fan Y.
        • et al.
        Automatic identification of myopic maculopathy related imaging features in optic disc region via machine learning methods.
        J Transl Med. 2021; 19: 1-12
        • Zarranz-Ventura J.
        • Barraso M.
        • Alé-Chilet A.
        • et al.
        Evaluation of microvascular changes in the perifoveal vascular network using optical coherence tomography angiography (OCTA) in type I diabetes mellitus: a large scale prospective trial.
        BMC Med Imaging. 2019; 19: 91
        • Barraso M.
        • Alé-Chilet A.
        • Hernández T.
        • et al.
        Optical coherence tomography angiography in type 1 diabetes mellitus. Report 1: Diabetic retinopathy.
        Transl Vis Sci Technol. 2020; 9: 1-15
        • Bernal-Morales C.
        • Alé-Chilet A.
        • Martín-Pinardel R.
        • et al.
        Optical Coherence Tomography Angiography in Type 1 Diabetes Mellitus. Report 4: Glycated Haemoglobin.
        Diagnostics. 2021; 11: 1537
        • Han L.
        • Luo S.
        • Yu J.
        • et al.
        Rule extraction from support vector machines using ensemble learning approach: An application for diagnosis of diabetes.
        IEEE J Biomed Heal Informatics. 2015; 19: 728-734
        • Shankaracharya Odedra D.
        • Samanta S.
        • Vidyarthi A.S.
        Computational intelligence-based diagnosis tool for the detection of prediabetes and type 2 diabetes in India.
        Rev Diabet Stud. 2012; 9: 55-62
        • Gulshan V.
        • Peng L.
        • Coram M.
        • et al.
        Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs.
        JAMA. 2016; 316: 2402-2410
        • Ting D.S.W.
        • Cheung C.Y.-L.L.
        • Lim G.
        • et al.
        Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes.
        JAMA - J Am Med Assoc. 2017; 318: 2211-2223