Collaborative semi-global stereo matching

Penghui Bu; Hong Zhao; Jiaxing Yan; Yusheng Jin

doi:10.1364/AO.435530

Applied Optics
Vol. 60,
Issue 31,
pp. 9757-9768
(2021)
•https://doi.org/10.1364/AO.435530

Collaborative semi-global stereo matching

Penghui Bu, Hong Zhao, Jiaxing Yan, and Yusheng Jin

Not Accessible

Your library or personal account may give you access

Get PDF
Email
Share
Get Citation
Copy Citation Text
Penghui Bu, Hong Zhao, Jiaxing Yan, and Yusheng Jin, "Collaborative semi-global stereo matching," Appl. Opt. 60, 9757-9768 (2021)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article

Check for updates

Related Topics
Table of Contents Category
- Instrumentation and Measurements
Optics & Photonics Topics
?

The topics in this list come from the Optics and Photonics Topics applied to this article.

About this Article
History
- Original Manuscript: June 30, 2021
- Revised Manuscript: September 17, 2021
- Manuscript Accepted: September 17, 2021
- Published: October 25, 2021

Abstract

Efficiency and accuracy of semi-global matching (SGM) make it outperform many stereo matching algorithms and is widely used under challenging occasions. However, SGM only incorporates information along a scanline in each pass and lacks interaction between scanlines, resulting in streak artifacts in the disparity image. We introduce a local edge-aware filtering method to SGM to enhance the interaction of neighboring scanlines, since streak artifacts can be avoided. We use bilateral weights based on intensity similarity and spatial affinity between pixels to build connections among scanlines. In each pass, we recursively estimate the aggregated cost of SGM and compute the weighted average of aggregated costs for pixels in the orthogonal direction to obtain the output of our method along each scanline. As one-dimensional bilateral filtering is used in our method, the extra computation is linear to image resolution and label space, which is a small fraction of that needed by SGM. We present ablation studies using stereo pairs under both constrained and natural conditions to verify the effectiveness of our method. Extensive experiments on Middlebury and Karlsruhe Institute of Technology and Toyota Technology Institute datasets demonstrate that our method removes all streak artifacts, improves the quality of the disparity image, and outperforms many other non-local cost aggregation approaches.

Full Article | PDF Article

More Like This

Four-direction global matching with cost volume update for stereovision

Yusheng Jin, Hong Zhao, and Penghui Bu
Appl. Opt. 60(19) 5471-5479 (2021)

Stereo matching based on multi-scale fusion and multi-type support regions

Haibin Li, Yakun Gao, Ziyue Huang, and Yakun Zhang
J. Opt. Soc. Am. A 36(9) 1523-1533 (2019)

3D cost aggregation with multiple minimum spanning trees for stereo matching

Lincheng Li, Xin Yu, Shunli Zhang, Xiaolin Zhao, and Li Zhang
Appl. Opt. 56(12) 3411-3420 (2017)

Previous Article Next Article

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (9)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (5)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (10)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Method	GF	NL	ST	MGM	SGM4	SGM8	CSGM4	CSGM8
Aloe	${5.29}_{7}$	${4.79}_{6}$	${4.79}_{6}$	${4.44}_{5}$	${4.32}_{3}$	${4.43}_{4}$	${3.14}_{1}$	${3.58}_{2}$
Baby1	${3.62}_{6}$	${7.47}_{8}$	${4.10}_{7}$	${2.65}_{4}$	${2.89}_{5}$	${2.62}_{3}$	${2.54}_{2}$	${2.42}_{1}$
Baby2	${3.39}_{5}$	${11.94}_{7}$	${13.36}_{8}$	${1.87}_{1}$	${4.24}_{6}$	${2.57}_{4}$	${3.14}_{3}$	${2.03}_{2}$
Baby3	${3.98}_{2}$	${5.00}_{7}$	${4.29}_{3}$	${4.50}_{4}$	${4.72}_{5}$	${4.81}_{6}$	${3.83}_{1}$	${4.29}_{3}$
Bowling1	${9.36}_{1}$	${19.49}_{8}$	${19.44}_{7}$	${11.20}_{3}$	${13.03}_{5}$	${14.34}_{6}$	${10.62}_{2}$	${12.35}_{4}$
Bowling2	${4.70}_{3}$	${8.21}_{6}$	${8.25}_{7}$	${8.38}_{8}$	${5.11}_{5}$	${5.00}_{4}$	${4.20}_{1}$	${4.48}_{2}$
Cloth1	${1.20}_{8}$	${0.62}_{3}$	${0.63}_{4}$	${0.96}_{7}$	${0.90}_{6}$	${0.69}_{5}$	${0.56}_{2}$	${0.45}_{1}$
Cloth2	${1.89}_{1}$	${3.87}_{8}$	${3.74}_{7}$	${2.25}_{3}$	${2.88}_{6}$	${2.59}_{5}$	${2.27}_{4}$	${2.22}_{2}$
Cloth3	${1.70}_{2}$	${2.39}_{7}$	${2.57}_{8}$	${1.92}_{4}$	${2.20}_{6}$	${2.13}_{5}$	${1.67}_{1}$	${1.72}_{3}$
Cloth4	${2.92}_{8}$	${1.81}_{4}$	${1.80}_{3}$	${1.97}_{5}$	${2.11}_{6}$	${2.12}_{7}$	${1.42}_{1}$	${1.60}_{2}$
Flowerpots	${7.52}_{2}$	${13.56}_{8}$	${10.13}_{3}$	${7.42}_{1}$	${12.14}_{7}$	${11.19}_{5}$	${11.38}_{6}$	${10.70}_{4}$
Lampshade1	${7.81}_{6}$	${8.64}_{7}$	${9.23}_{8}$	${6.29}_{2}$	${7.46}_{5}$	${6.65}_{4}$	${6.16}_{1}$	${6.30}_{3}$
Lampshade2	${16.42}_{8}$	${11.80}_{7}$	${11.56}_{6}$	${8.57}_{4}$	${9.90}_{5}$	${8.19}_{3}$	${6.94}_{2}$	${6.91}_{1}$
Rocks1	${2.75}_{8}$	${2.50}_{7}$	${2.42}_{6}$	${1.75}_{5}$	${1.71}_{4}$	${1.62}_{3}$	${1.33}_{1}$	${1.38}_{2}$
Rocks2	${1.21}_{1}$	${1.70}_{8}$	${1.67}_{7}$	${1.37}_{4}$	${1.65}_{6}$	${1.57}_{5}$	${1.29}_{2}$	${1.32}_{3}$
Wood1	${3.22}_{5}$	${8.49}_{8}$	${4.41}_{7}$	${4.38}_{6}$	${0.91}_{2}$	${1.43}_{3}$	${0.87}_{1}$	${1.90}_{4}$
Wood2	${1.38}_{4}$	${2.05}_{6}$	${2.57}_{7}$	${1.33}_{3}$	${1.10}_{2}$	${2.86}_{8}$	${0.79}_{1}$	${1.91}_{5}$
Art	${8.00}_{5}$	${9.13}_{8}$	${9.03}_{7}$	${8.79}_{6}$	${7.90}_{4}$	${7.87}_{3}$	${6.75}_{1}$	${7.20}_{2}$
Books	${8.80}_{5}$	${10.44}_{8}$	${10.01}_{7}$	${8.95}_{6}$	${8.06}_{3}$	${8.07}_{4}$	${6.63}_{1}$	${7.13}_{2}$
Cones	${2.33}_{1}$	${3.53}_{2}$	${3.83}_{4}$	${5.36}_{8}$	${4.76}_{6}$	${4.70}_{7}$	${3.59}_{3}$	${3.89}_{5}$
Dolls	${4.15}_{2}$	${4.97}_{6}$	${4.51}_{3}$	${3.94}_{1}$	${5.44}_{8}$	${5.38}_{7}$	${4.62}_{4}$	${4.87}_{5}$
Laundry	${11.31}_{3}$	${10.46}_{2}$	${10.42}_{1}$	${15.59}_{8}$	${15.51}_{7}$	${15.32}_{6}$	${12.81}_{4}$	${14.05}_{5}$
Moebius	${8.35}_{3}$	${7.52}_{2}$	${7.37}_{1}$	${8.76}_{4}$	${9.87}_{7}$	${10.03}_{8}$	${8.97}_{5}$	${9.64}_{6}$
Reindeer	${6.26}_{4}$	${9.50}_{8}$	${7.55}_{7}$	${5.41}_{2}$	${6.94}_{6}$	${6.67}_{5}$	${5.34}_{1}$	${5.81}_{3}$
Teddy	${6.35}_{4}$	${5.17}_{1}$	${5.64}_{2}$	${9.51}_{8}$	${7.48}_{7}$	${7.39}_{6}$	${6.03}_{3}$	${6.52}_{5}$
Tsukuba	${2.30}_{5}$	${1.76}_{1}$	${2.04}_{4}$	${2.53}_{7}$	${2.59}_{8}$	${2.34}_{6}$	${1.99}_{3}$	${1.87}_{2}$
Venus	${0.64}_{2}$	${0.61}_{1}$	${0.76}_{4}$	${1.27}_{8}$	${1.02}_{7}$	${0.85}_{6}$	${0.77}_{5}$	${0.71}_{3}$
Average	${5.07}_{4.1}$	${6.57}_{5.7}$	${6.15}_{5.3}$	${5.24}_{4.7}$	${5.44}_{5.4}$	${5.31}_{5.1}$	${4.43}_{2.3}$	${4.71}_{3.0}$

	Initial Disparity		Final Disparity
Method	${O u t}_{-} n$ (%)	${O u t}_{-} a$ (%)	${O u t}_{-} n$ (%)	${O u t}_{-} a$ (%)
GF [11]	5.07	16.68	5.22	11.51
NL [13]	6.57	17.20	6.64	12.67
ST [14]	6.15	17.29	6.08	12.11
SGM4 [23]	5.49	16.47	4.14	11.14
SGM8 [23]	5.31	15.49	4.21	11.24
MGM [39]	5.24	17.62	4.47	11.21
CSGM4	4.46	14.96	3.80	10.54
CSGM8	4.76	14.47	4.10	10.91

	Initial Disparity				Final Disparity
Method	${O u t}_{-} a$	${A v g}_{-} a$	${O u t}_{-} n$	${A v g}_{-} n$	${O u t}_{-} a$	${A v g}_{-} a$	${O u t}_{-} n$	${A v g}_{-} n$
NL [13]	13.87	3.80	11.67	2.45	11.57	1.97	10.26	1.75
ST [14]	15.66	3.86	13.50	2.53	13.44	2.15	12.16	1.93
MGM [39]	12.03	3.10	9.84	2.00	11.02	2.59	8.77	1.61
LRNL [15]	9.29	2.88	7.49	2.00	8.51	2.07	7.14	1.67
SGM4 [23]	9.81	2.77	7.53	1.67	7.01	1.75	5.42	1.23
SGM8 [23]	10.48	2.85	8.22	1.71	7.28	1.67	5.80	1.28
RSSGM [42]	/	/	/	/	10.05	/	8.71	/
RESGM [43]	/	/	/	/	7.60	/	6.66	/
LRSGM [44]	/	/	/	/	10.94	/	9.55	/
CSGM4	9.42	2.72	7.14	1.61	6.62	1.52	5.18	1.17
CSGM8	10.16	2.81	7.89	1.66	7.11	1.53	5.68	1.23

	Initial Disparity				Final Disparity
Method	${O u t}_{-} a$	${A v g}_{-} a$	${O u t}_{-} n$	${A v g}_{-} n$	${O u t}_{-} a$	${A v g}_{-} a$	${O u t}_{-} n$	${A v g}_{-} n$
NL [13]	11.30	2.57	10.09	2.11	8.91	1.66	8.61	1.64
ST [14]	12.57	2.68	11.37	2.22	9.97	1.74	9.68	1.72
MGM [39]	13.17	2.57	12.01	2.15	11.79	2.40	10.66	1.92
LRNL [15]	7.79	2.29	7.06	2.05	6.35	1.55	6.13	1.53
SGM4 [23]	9.50	2.19	8.28	1.78	6.42	1.38	6.06	1.32
SGM8 [23]	10.59	2.31	9.41	1.89	7.09	1.44	6.75	1.42
ROSGM [46]	/	/	/	/	7.27	/	6.58	/
ERSGM [47]	/	/	/	/	8.24	/	6.56	/
RESGM [43]	/	/	/	/	7.44	/	6.54	/
DRSGM [48]	/	/	/	/	11.40	/	10.50	/
LRSGM [44]	/	/	/	/	8.91	/	8.55	/
CSGM4	9.36	2.16	8.13	1.75	6.27	1.33	5.97	1.30
CSGM8	10.45	2.22	9.23	1.79	6.95	1.39	6.63	1.36

		Tsukuba	Teddy	Adirondack
Method	Complexity	R:384 * 288;D:16 (s)	R:450 * 375;D:60 (s)	R:720 * 497;D:71 (s)
MGM [39]	$O (2 M N D^{2})$	2.20	43.51	128.56
SGM [23]	$O (M N D^{2})$	1.33	24.84	73.51
CSGM	$O (M N D (D + (2 W + 1)))$	1.39	25.33	74.97

Abstract

Data Availability

Cited By

Figures (9)

Tables (5)

Equations (10)

Applied Optics