Fully Convolutional Networks for Monocular Retinal Depth Estimation and Optic Disc-Cup Segmentation
(IEEE Journal of Biomedical and Health Informatics, 2019)
Abstract
Description
Method
- Pseudo-depth pretraining. Instead of the usual denoising-autoencoder pretraining, the network is first trained to reconstruct pseudo-depth maps from color fundus images ➯ a proxy task that better captures depth-relevant features and works around the scarcity of labelled retinal depth data. L2, L1, and reverse-Huber (berHu) losses were all evaluated for this stage.
- Dilated Residual Inception (DRI) blocks. A novel building block that extracts features at multiple receptive-field sizes simultaneously, with residual connections for gradient flow, improving model capacity ➯ especially for the harder optic cup boundary ➯ without extra computational overhead.
- Multimodal feature fusion. A dedicated fusion block extracts features separately from the color-image stream and the depth-map stream and learns how to combine them into a single representation for segmentation.
- Depth-guided segmentation. The fused representation constrains the optic disc/cup prediction to anatomically plausible regions, using the 3D structural cues in the depth map to sharpen the cup boundary in particular.
- CRF post-processing (explored, not adopted). Conditional Random Fields on image intensity, depth, and combined fields were tested as a refinement step but did not improve results, since retinal images lack the sharp intensity/depth transitions that CRFs exploit.
Results
Animation: End-to-end pipeline ➯ a color fundus image is pseudo-depth pretrained, passed through the depth estimation module, fused with the color stream via multimodal feature fusion, and decoded into the final optic disc/cup segmentation.
Figure. Fully convolutional network architecture ➯ the depth estimation module (RGB fundus image → encoder with DRI blocks → decoder → depth map) guides the multimodal segmentation network, which fuses the RGB image and depth map to predict OD & OC masks.
Figure. Quantitative performance ➯ on ORIGA, PD DRIUnet reaches 0.051 disc / 0.212 cup overlap error; on RIMONE, disc error stays around 0.06 across variants; pseudo-depth pretraining (PD+L2) reaches the highest depth correlation (0.9629) and lowest RMSE (0.0059) among all tested methods.
Tables. Detailed benchmark results ➯ pseudo-depth pretraining with an L2 loss lowers RMSE from 0.0064 (denoising-autoencoder pretraining) to 0.0059 — an ~8% reduction — while also raising correlation from 0.9566 to 0.9629 on the INSPIRE-Stereo dataset (Table 1), and PD DRIUnet gives the best or most competitive disc/cup segmentation accuracy across ORIGA, RIMONE, and DRISHTI-GS (Table 2).
Key Findings
- Pseudo-depth pretraining outperformed denoising autoencoder pretraining for retinal depth estimation. It provides a significantly better proxy task for retinal depth estimation, directly addressing the shortage of labelled depth data.
- DRI blocks help most where it's hardest. They give consistent improvements for optic cup segmentation while matching standard residual blocks on the (easier) disc segmentation task.
- Depth guidance improves cup segmentation. Multimodal fusion of depth with color meaningfully improves cup accuracy, confirming that 3D structural information carries real signal for this task.
- Reverse-Huber loss doesn't help here. Unlike in natural-image depth estimation, berHu shows minimal improvement over L1 for retinal depth ➯ likely because retinal depth variations are smoother than natural scenes.
- CRF post-processing doesn't help either. Multimodal fusion networks outperform CRF refinement, since retinal images lack the sharp intensity/depth transitions CRFs are designed to exploit.
Datasets & Experimental Setup
- ORIGA ➯ 650 retinal images with ground-truth depth maps; the primary evaluation dataset.
- RIMONEr3 ➯ an extensive dataset with multiple annotations per image, used for segmentation validation.
- DRISHTI-GS ➯ a multi-domain dataset used to test cross-dataset generalization.
Clinical Applications
- Glaucoma screening: automated cup-to-disc ratio (CDR) calculation for early detection and longitudinal monitoring of glaucomatous changes.
- Optic disc analysis: standardized, reproducible quantitative metrics that reduce inter-observer variability and scale to resource-limited screening settings.
- Clinical decision support: risk stratification for referral and rapid analysis of large screening populations, alongside other diagnostic markers.
