Fully Convolutional Networks for Monocular Retinal Depth Estimation and Optic Disc-Cup Segmentation

monocular-retinal-depth-estimation-computational-imaging-lab-iit-madras

Abstract

Description

Method

  1. Pseudo-depth pretraining. Instead of the usual denoising-autoencoder pretraining, the network is first trained to reconstruct pseudo-depth maps from color fundus images ➯ a proxy task that better captures depth-relevant features and works around the scarcity of labelled retinal depth data. L2, L1, and reverse-Huber (berHu) losses were all evaluated for this stage.
  2. Dilated Residual Inception (DRI) blocks. A novel building block that extracts features at multiple receptive-field sizes simultaneously, with residual connections for gradient flow, improving model capacity ➯ especially for the harder optic cup boundary ➯ without extra computational overhead.
  3. Multimodal feature fusion. A dedicated fusion block extracts features separately from the color-image stream and the depth-map stream and learns how to combine them into a single representation for segmentation.
  4. Depth-guided segmentation. The fused representation constrains the optic disc/cup prediction to anatomically plausible regions, using the 3D structural cues in the depth map to sharpen the cup boundary in particular.
  5. CRF post-processing (explored, not adopted). Conditional Random Fields on image intensity, depth, and combined fields were tested as a refinement step but did not improve results, since retinal images lack the sharp intensity/depth transitions that CRFs exploit.

Results

monocular-retinal-depth-estimation-workflow-pipeline-animation-computational-imaging-lab-iit-madras

Animation: End-to-end pipeline ➯ a color fundus image is pseudo-depth pretrained, passed through the depth estimation module, fused with the color stream via multimodal feature fusion, and decoded into the final optic disc/cup segmentation.

monocular-retinal-depth-estimation-fully-convolutional-network-architecture-computational-imaging-lab-iit-madras

Figure. Fully convolutional network architecture ➯ the depth estimation module (RGB fundus image → encoder with DRI blocks → decoder → depth map) guides the multimodal segmentation network, which fuses the RGB image and depth map to predict OD & OC masks.

monocular-retinal-depth-estimation-quantitative-performance-computational-imaging-lab-iit-madras

Figure. Quantitative performance ➯ on ORIGA, PD DRIUnet reaches 0.051 disc / 0.212 cup overlap error; on RIMONE, disc error stays around 0.06 across variants; pseudo-depth pretraining (PD+L2) reaches the highest depth correlation (0.9629) and lowest RMSE (0.0059) among all tested methods.

monocular-retinal-depth-estimation-detailed-results-tables-computational-imaging-lab-iit-madras

Tables. Detailed benchmark results ➯ pseudo-depth pretraining with an L2 loss lowers RMSE from 0.0064 (denoising-autoencoder pretraining) to 0.0059 — an ~8% reduction — while also raising correlation from 0.9566 to 0.9629 on the INSPIRE-Stereo dataset (Table 1), and PD DRIUnet gives the best or most competitive disc/cup segmentation accuracy across ORIGA, RIMONE, and DRISHTI-GS (Table 2).

Key Findings

  1. Pseudo-depth pretraining outperformed denoising autoencoder pretraining for retinal depth estimation. It provides a significantly better proxy task for retinal depth estimation, directly addressing the shortage of labelled depth data.
  2. DRI blocks help most where it's hardest. They give consistent improvements for optic cup segmentation while matching standard residual blocks on the (easier) disc segmentation task.
  3. Depth guidance improves cup segmentation. Multimodal fusion of depth with color meaningfully improves cup accuracy, confirming that 3D structural information carries real signal for this task.
  4. Reverse-Huber loss doesn't help here. Unlike in natural-image depth estimation, berHu shows minimal improvement over L1 for retinal depth ➯ likely because retinal depth variations are smoother than natural scenes.
  5. CRF post-processing doesn't help either. Multimodal fusion networks outperform CRF refinement, since retinal images lack the sharp intensity/depth transitions CRFs are designed to exploit.

Datasets & Experimental Setup

  • ORIGA ➯ 650 retinal images with ground-truth depth maps; the primary evaluation dataset.
  • RIMONEr3 ➯ an extensive dataset with multiple annotations per image, used for segmentation validation.
  • DRISHTI-GS ➯ a multi-domain dataset used to test cross-dataset generalization.

Clinical Applications

  • Glaucoma screening: automated cup-to-disc ratio (CDR) calculation for early detection and longitudinal monitoring of glaucomatous changes.
  • Optic disc analysis: standardized, reproducible quantitative metrics that reduce inter-observer variability and scale to resource-limited screening settings.
  • Clinical decision support: risk stratification for referral and rapid analysis of large screening populations, alongside other diagnostic markers.

Future Directions

Publications