Deep iterative vessel segmentation in OCT angiography

Theodoros Pissas; Theodoros Pissas; Theodoros Pissas; Edward Bloch; Edward Bloch; M. Jorge Cardoso; Blanca Flores; Odysseas Georgiadis; Sepehr Jalali; Claudio Ravasio; Claudio Ravasio; Danail Stoyanov; Lyndon Da Cruz; Lyndon Da Cruz; Lyndon Da Cruz; Christos Bergeles; Christos Bergeles; Christos Bergeles

doi:10.1364/BOE.384919

Biomedical Optics Express
Vol. 11,
Issue 5,
pp. 2490-2510
(2020)
•https://doi.org/10.1364/BOE.384919

Deep iterative vessel segmentation in OCT angiography

Theodoros Pissas, Edward Bloch, M. Jorge Cardoso, Blanca Flores, Odysseas Georgiadis, Sepehr Jalali, Claudio Ravasio, Danail Stoyanov, Lyndon Da Cruz, and Christos Bergeles

Open Access

Get PDF
Email
Share
Get Citation
Copy Citation Text
Theodoros Pissas, Edward Bloch, M. Jorge Cardoso, Blanca Flores, Odysseas Georgiadis, Sepehr Jalali, Claudio Ravasio, Danail Stoyanov, Lyndon Da Cruz, and Christos Bergeles, "Deep iterative vessel segmentation in OCT angiography," Biomed. Opt. Express 11, 2490-2510 (2020)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article
Editors' Pick

Check for updates

Abstract

This paper addresses retinal vessel segmentation on optical coherence tomography angiography (OCT-A) images of the human retina. Our approach is motivated by the need for high precision image-guided delivery of regenerative therapies in vitreo-retinal surgery. OCT-A visualizes macular vasculature, the main landmark of the surgically targeted area, at a level of detail and spatial extent unattainable by other imaging modalities. Thus, automatic extraction of detailed vessel maps can ultimately inform surgical planning. We address the task of delineation of the Superficial Vascular Plexus in 2D Maximum Intensity Projections (MIP) of OCT-A using convolutional neural networks that iteratively refine the quality of the produced vessel segmentations. We demonstrate that the proposed approach compares favourably to alternative network baselines and graph-based methodologies through extensive experimental analysis, using data collected from 50 subjects, including both individuals that underwent surgery for structural macular abnormalities and healthy subjects. Additionally, we demonstrate generalization to 3D segmentation and narrower field-of-view OCT-A. In the future, the extracted vessel maps will be leveraged for surgical planning and semi-automated intraoperative navigation in vitreo-retinal surgery.

Published by The Optical Society under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Full Article | PDF Article

More Like This

Automated diagnosis and segmentation of choroidal neovascularization in OCT angiography using deep learning

Jie Wang, Tristan T. Hormel, Liqin Gao, Pengxiao Zang, Yukun Guo, Xiaogang Wang, Steven T. Bailey, and Yali Jia
Biomed. Opt. Express 11(2) 927-944 (2020)

MF-AV-Net: an open-source deep learning network with multimodal fusion options for artery-vein segmentation in OCT angiography

Mansour Abtahi, David Le, Jennifer I. Lim, and Xincheng Yao
Biomed. Opt. Express 13(9) 4870-4888 (2022)

Improving foveal avascular zone segmentation in fluorescein angiograms by leveraging manual vessel labels from public color fundus pictures

Dominik Hofer, Ursula Schmidt-Erfurth, José Ignacio Orlando, Felix Goldbach, Bianca S. Gerendas, and Philipp Seeböck
Biomed. Opt. Express 13(5) 2566-2580 (2022)

Previous Article Next Article

Supplementary Material (1)

Name	Description
Visualization 1	This video presents several qualitative results of 3D OCT-A vessel segmentation. We obtain those results using a network trained on 2D maximum intensity projections of OCT-A, which we use to segment every axial slice of the 3D volume, without any tra

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.

Fig. 1. OCT-A vs Preoperative and Intraoperative Imaging: For several subjects with retinal pathology we present: (a) An Intraoperative Video Frame (IVF) captured during vitreo-retinal surgery (b) IVF affinely aligned to OCT-A (c) a preoperative CFP affinely aligned to OCT-A, (d) the OCT-A and (e) the vessel segmentation obtained by our vessel segmentation method. In all cases, OCT-A visualizes the maximum level of vasculature detail around the surgical RoI, the macula, where both CFPs and IVFs tend to provide blurry information and are susceptible to subretinal pathology, such as choroidal pigmentation (last column), that impacts retinal vessel visibility.

Download Full Size | PDF

Fig. 2. Intraoperative vs OCT-A Vessel Visibility: Vessel map annotations by two expert clinicians on intraperative video frames revealed that in the vicinity of the macula (outlined in red) they are unable to detect the level of vasculature details that can be reliably annotated on OCT-A.

Download Full Size | PDF

Fig. 3. OCT-A data overview: (a) Retina cross section: outlined in blue is the volume corresponding to the slices used in the dataset. They span the space from the retinal surface (upper limit of the blue line) to the start of the choroid (outlined in red) where the Superficial and Deep Vascular Plexuses are located. (b) The imaging device produces geometrically flattened slices that correspond to curved slices of the retina’s cross-section. (c) Maximum Intensity Projection is performed on the extracted stack of geometrically flattened slices along the axis vertical to the plane of the slices. Outlined in red is the (approximate) location of the macula around which the scans are centered. The zoomed-in patch depicts (in green) vessels that are considered by our models and areas (in orange) where microvessels are likely to be located, which however cannot be delineated reliably and the models learn to ignore. (d) The imaging device locates the limiting surface between the retinal layers (blue space) and the choroid (red space) allowing us to access geometrically flattened slices thus separating chroroidal and retinal layers.

Download Full Size | PDF

Fig. 4. Schematic representation of the architecture of the base-network as described in Sec. 2.3. The base-network follows the architecture paradigm of UNet. The number below any tensor denotes the number of feature maps at that stage of the network.

Download Full Size | PDF

Fig. 5. Considered CNN architectures: (a) UNet, (b) SHN, and (c) iUNet. For illustration purposes the SHN, and iUNet, are presented with

$2$

distinct UNet modules, and

$2$

iterations, respectively.

Download Full Size | PDF

Fig. 6. For all models, adding online data augmentation during training (described in 3.4) prevents overfitting by regularizing training while leading to higher validation Quality. The presented curves are computed when training and validating on the same cross-validation fold of the dataset, but this finding was consistent across folds.

Download Full Size | PDF

Fig. 7. Qualitative comparison of results from Imagenet pretrained and fine-tuned baseline DRIU [25] and i-unet-4-topo, trained from scratch. The latter was the top performing model/loss function combination. The two methods achieve similar recall. However, DRIU exhibits noisier predictions with a considerable amount of false positives. Columns

$4$

and

$6$

present centerline errors (dilated by one pixel to improve visibility) made by the two models, with false and true positives shown in red and green respectively, while missed segments are shown in blue.

Download Full Size | PDF

Fig. 8. Adding iterative refinement to unet-topo: outlined in red are some examples of fine details that are recovered only by i-unet-4-topo. The outermost column depicts zoomed-in regions of interest corresponding to the red bounding boxes, and aids with the comparison of the response of the two models. Columns

$3$

and

$5$

Download Full Size | PDF

Fig. 9. OCT-A

$3$

D segmentation: The 1^st row depicts the MIP associated with the raw

$3$

D volume which is per-slice segmented by shn-4-topo (the model that gave the best, based on visual inspection,

$3$

D results), with the resulting

$3$

D segmentation displayed below. The 3^rd row displays cross-sections of the segmentation (gray) overlayed on the OCT-A cross-section, the location of which is denoted by the red dashed line. Finally zoomed in cross-sectional details are shown (denoted in upper rows by red dots) which reveal the network mistakenly segments shadowing artefacts (1^st, 2^nd, 4^th columns) below bigger vessels which is normal due to it being unaware of

$3$

D context. A video demonstration of the

$3$

D segmentations is provided as supplementary material.

Download Full Size | PDF

Fig. 10. Generalizing to

$3\,$

$\times 3$

mm scans: Using i-unet-4-topo, we can produce plausible segmentations of the narrower FoV scans which reveal more details of the central part of the macula. The 1^st and 2^nd, and 3^rd and 4^th columns, demonstrate the correspondence between the two scans and the two segmentations respectively, while the 5^th and 6^th presents the ground truth centerline and errors with respect to it respectively, with false (red), true (green) positives and missed segments (blue).

Download Full Size | PDF

Fig. 11. Performance as a function of the number of trainable parameters: i-unet-4-topo constitutes the top performing while requiring the minimum number of parameters.

Download Full Size | PDF

Tables (8)

Table 1. Distribution of types of pathology in the dataset

View Table | View all tables in this article

Table 2. Model/loss-function comparisons using $4$ -folds cross validation. Mean of metrics on the test set across folds is reported and standard deviation is in parenthesis. Best of each metric in bold, Statistical Significance of difference in Quality between the two top competing methods is indicated.

View Table | View all tables in this article

Table 3. Effect of iterations (iUNet) and modules (SHN) on quality. Statistically significant differences between top performing model with iterative refinement and top performing model with iterative refinement and topological loss. Best of each model is in bold.

View Table | View all tables in this article

Table 4. Effect of base network depth on quality. Statistically significant differences between deeper and shallower base networks are indicated. Best of each model is in bold.

View Table | View all tables in this article

Table 5. Improvements through iterative refinement combined with topological loss.

View Table | View all tables in this article

Table 6. Quality metric when fractions of the full dataset are considered.

View Table | View all tables in this article

Table 7. Paired Wilcoxon significance tests for cross validated Quality metric.

View Table | View all tables in this article

Table 8. Ablation study for the VGG-feature loss on validation sets.

View Table | View all tables in this article

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

\begin{aligned} L_{b c e} = - β \sum_{i \in Y^{+}} \log (P (y_{i} = 1 ∣ X; θ) - (1 - β) \sum_{i \in Y^{-}} \log (P (y_{i} = 0 ∣ X; θ), \end{aligned}

L_{t o p o} = \sum_{n = 1}^{N} \frac{μ_{n}}{W_{n} H_{n} C_{n}} \sum_{c = 1}^{C_{n}} {‖ F_{n}^{c} (f (X; θ)) - F_{n}^{c} (Y) ‖}_{2}^{2},

L_{c o m b} = L_{b c e} + L_{t o p o}

L_{i U n e t} = \frac{2}{T (T + 1)} \sum_{t = 1}^{T} t L_{c o m b}^{(t)},

Completeness = \frac{μ_{\hat{Y}} (Y, τ)}{∣ Y ∣},

Correctness = \frac{μ_{Y} (\hat{Y}, τ)}{∣ \hat{Y} ∣},

Quality = \frac{μ_{Y} (\hat{Y}, τ)}{∣ \hat{Y} ∣ - μ_{\hat{Y}} (Y, τ) + ∣ Y ∣} .

p_{i j} = d_{i j} \frac{p_{i} l o g (p_{i}) + p_{j} (1 - l o g (p_{j}))}{p_{i} - p_{j}},

Model	Q $_{t e s t, τ = 2}$	Corr $_{t e s t, τ = 2}$	Comp $_{t e s t, τ = 2}$	PR $_{t e s t}$
Graph-based	0.7267	0.7884	0.7871	-
unet	0.8230 (0.0348)	0.8583 (0.0417)	0.9529 (0.0165)	0.8535
DRIU [25]	0.8360 (0.0248)	0.8838 (0.0290)	0.9400 (0.0196)	0.8328
i-unet-4	0.8334 (0.0345)	0.8694 (0.0404)	0.9532 (0.0171)	0.8572
shn-4	0.8464 (0.0263)	0.8877 (0.0333)	0.9484 (0.0209)	0.8588
unet-topo	0.8598 (0.0244)	0.9257 (0.0304)	0.9246 (0.0304)	0.8477
shn-4-topo	0.8624 (0.0227)	0.9301 (0.0278)	0.9227 (0.0227)	0.8552
i-unet-4-topo	${0.8671}^{*} (0.0226)$	0.9373 (0.0266)	0.9214 (0.0251)	0.8540

Model	k=2	k=3	k=4	k=5
i-unet-k	0.8302 (0.0335)	0.8322 (0.0330)	0.8334 (0.0340)	0.8312 (0.0317)
i-unet-k-topo	0.8626 (0.0232)	0.8631 (0.0236)	${0.8671}^{* * *}$ (0.0226)	0.8661 (0.0210)
shn-k	0.8310 (0.0264)	0.8314 (0.0268)	0.8464 (0.0263)	0.8452 (0.0274)
shn-k-topo	0.8598 (0.0222)	0.8616 (0.0235)	${0.8624}^{* * *}$ (0.0227)	0.8609 (0.0235)

Blocks	unet-topo	i-unet-4-topo	shn-4-topo
5	0.8484 (0.0235)	0.8548 (0.0254)	0.8510 (0.0235)
3	${0.8598}^{* *}$ (0.0244)	${0.8671}^{* * *}$ (0.0226)	${0.8624}^{* *}$ (0.0227)

Model	$Δ Q_{t e s t, τ = 2}^{2 - 1}$	$Δ Q_{t e s t, τ = 2}^{3 - 2}$	$Δ Q_{t e s t, τ = 2}^{4 - 3}$
i-unet-4-topo	0.01659	0.0016	0.0002
shn-4-topo	0.01749	0.0008	$< 0.0001$

Models	unet	DRIU	unet-topo	i-unet-4	i-unet-4-topo	shn-4	shn-4-topo
unet		$* *$	$* * *$	$* *$	$* * *$	$* * *$	$* * *$
DRIU	$* *$		$* * *$	$n s$	$* * *$	$* *$	$* * *$
unet-topo	$* * *$	$* * *$		$* * *$	$* *$	$* * *$	$n s$
i-unet-4	$* *$	$n s$	$* * *$		$* * *$	$* * *$	$* * *$
i-unet-4-topo	$* * *$	$* * *$	$* *$	$* * *$		$* * *$	$*$
shn-4	$* * *$	$* *$	$* * *$	$* * *$	$* * *$		$* * *$
shn-4-topo	$* * *$	$* * *$	$n s$	$* * *$	$*$	$* * *$
Q $_{t e s t, τ = 2}$	0.8230	0.8360	0.8598	0.8334	0.8671	0.8464	0.8624

Weighing	Feature Maps	Q $_{v a l, τ = 2}$	PR $_{v a l}$
$10^{- 1}, 10^{- 2}, 10^{- 3}$	all	0.8650	0.8520
$10^{- 2}, 10^{- 3}, 10^{- 4}$	all	0.8668	0.8538
$10^{- 2}$	all	0.8567	0.8445
$10^{- 2}$	only $c o n v_{12}$	0.8603	0.8534

Abstract

Supplementary Material (1)

Cited By

Figures (11)

Tables (8)

Equations (8)

Biomedical Optics Express