Foveated 3D range geometry compression via loss-tolerant variable precision depth encoding

Broderick S. Schwartz; Matthew G. Finley; Tyler Bell

doi:10.1364/AO.472356

Applied Optics
Vol. 61,
Issue 33,
pp. 9911-9925
(2022)
•https://doi.org/10.1364/AO.472356

Foveated 3D range geometry compression via loss-tolerant variable precision depth encoding

Broderick S. Schwartz, Matthew G. Finley, and Tyler Bell

Not Accessible

Your library or personal account may give you access

Get PDF
Email
Share
Get Citation
Copy Citation Text
Broderick S. Schwartz, Matthew G. Finley, and Tyler Bell, "Foveated 3D range geometry compression via loss-tolerant variable precision depth encoding," Appl. Opt. 61, 9911-9925 (2022)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article

Check for updates

Related Topics
Table of Contents Category
- Imaging Systems, Image Processing, and Displays
Optics & Photonics Topics
?

The topics in this list come from the Optics and Photonics Topics applied to this article.

About this Article
History
- Original Manuscript: August 4, 2022
- Revised Manuscript: October 10, 2022
- Manuscript Accepted: October 16, 2022
- Published: November 15, 2022

Abstract

The capacity of three-dimensional (3D) range geometry acquisition methods to capture high-precision scans at high frame rates increases every year. These improvements have influenced a broadening range of disciplines to implement 3D range geometry capture systems, including telepresence, medicine, the visual arts, and many others. However, its increased popularity, precision, and capture rates have caused mounting pressure on the storage and transmission of 3D range geometry, thus straining their capacities. Compression techniques seek to alleviate this pressure by offering reduced file sizes, while maintaining the levels of precision needed for particular applications. Several such compression methods use sinusoidal modulation approaches to encode floating-point 3D data into conventional 2D red, green, and blue (RGB) images. In some applications, such as telepresence, high precision may only be required in a particular region within a depth scan, thus allowing less important data to be compressed more aggressively. This paper proposes a feature-driven compression method that provides a way to encode regions of interest at higher levels of precision while encoding the remaining data less precisely to reduce file sizes. This method supports both lossless and lossy compression, enabling even greater file-size savings. For example, in the case of a depth scan of a bust, an algorithmically extracted bounding box of the face was used to create a foveated encoding distribution so that the facial region was encoded at higher precisions. When using JPEG 80, the RMS reconstruction error of this novel, to the best of our knowledge, encoding was 0.56 mm in the region of interest, compared to a globally fixed higher precision encoding where the error was 0.54 mm in the same region. However, the proposed encoding achieved a 26% reduction in overall compressed file size compared to the fixed, higher-precision encoding.

Full Article | PDF Article

More Like This

Downsampled depth encoding for enhanced 3D range geometry compression

Broderick S. Schwartz and Tyler Bell
Appl. Opt. 61(6) 1559-1568 (2022)

Variable precision depth encoding for 3D range geometry compression

Matthew G. Finley, Jacob Y. Nishimura, and Tyler Bell
Appl. Opt. 59(17) 5290-5299 (2020)

Two-channel depth encoding for 3D range geometry compression

Matthew G. Finley and Tyler Bell
Appl. Opt. 58(25) 6882-6890 (2019)

Previous Article Next Article

Supplementary Material (3)

Name	Description
Visualization 1	A 3D video was captured using a Microsoft Azure Kinect. The 22.83 second video consisted of 685 frames recorded at 30 frames per second. Each frame contained both a 16-bit depth map and a color texture with resolution 1536 × 2048 pixels. The equivale
Visualization 2	The proposed approach compared to two fixed precision encodings as applied to a 3D video sequence. (a) 3D rendering of the original depth information with depth range 1900 mm. (b) Detected bounding boxes marked on the texture image denoting regions o
Visualization 3	The proposed approach compared to two fixed precision encodings as applied to a 3D video sequence. (a) 3D rendering of the original depth information with depth range 1900 mm. (b) A potential eyetracking path marked on the texture image denoting the

Data availability

Data underlying the results presented in this paper are available from [20] for Figs. 6–9 and from [21] for Fig. 10.

20. R. B. Hayes, and Smithsonian Institution, 2021, https://3d.si.edu/object/3d/rutherford-b-hayes%3A1f3700e8-6d01-4488-8ab9-ea14031ef641.

21. NASA/JPL/University of Arizona, “Digital terrain map: crater,” https://www.uahirise.org/dtm/dtm.php?ID=ESP_027065_2405.

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (11)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (3)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (17)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

(a)
PNG Results	Global RMS Error/Accuracy	ROI RMS Error/Accuracy	File Size/Compression Ratio
Low constant $(n_{s t r} = 1)$	0.0364 mm/99.9636%	0.0362 mm/99.9638%	148.8 KB/7.05:1
Proposed $(n_{s t r} = [1, 6])$	0.0319 mm/99.9681%	0.0067 mm/99.9933%	185.1 KB/5.66:1
High constant $(n_{s t r} = 6)$	0.0061 mm/99.9939%	0.0061 mm/99.9939%	281.7 KB/3.72:1
(b)
JPEG 80 results	Global RMS Error/Accuracy	ROI RMS Error/Accuracy	File Size/Compression Ratio
Low constant $(n_{s t r} = 1)$	0.16 mm/99.84%	0.16 mm/99.84%	17.27 KB/60.72:1
Proposed $(n_{s t r} = [1, 6])$	0.14 mm/99.86%	0.06 mm/99.94%	23.45 KB/44.72:1
High constant $(n_{s t r} = 6)$	0.13 mm/99.87%	0.05 mm/99.95%	42.52 KB/24.66:1

(a)
PNG Results	Global RMS Error/Accuracy	ROI RMS Error/Accuracy	File Size/Compression Ratio
Low constant ( $n_{s t r} = 1$ )	0.173 mm/99.964%	0.174 mm/99.963%	246.7 KB/10.38:1
Proposed ( $n_{s t r} = [1, 6]$ )	0.111 mm/99.977%	0.030 mm/99.994%	321.7 KB/7.96:1
High constant ( $n_{s t r} = 6$ )	0.029 mm/99.994%	0.029 mm/99.994%	431.8 KB/5.93:1
(b)
JPEG 80 results	Global RMS Error/Accuracy	ROI RMS Error/Accuracy	File Size/Compression Ratio
Low constant ( $n_{s t r} = 1$ )	1.94 mm/99.56%	1.61 mm/99.66%	39.32 KB/65.15:1
Proposed ( $n_{s t r} = [1, 6]$ )	1.35 mm/99.72%	0.56 mm/99.88%	54.20 KB/47.27:1
High constant ( $n_{s t r} = 6$ )	0.64 mm/99.87%	0.54 mm/99.89%	73.47 KB/34.87:1

H.264 CRF-15 Results	Global RMS Error/Accuracy	ROI RMS Error/Accuracy	Bitrate (Mbps)
Low constant ( $n_{s t r} = 1$ )	3.84 mm/99.80%	4.23 mm/99.78%	9.54
Proposed ( $n_{s t r} = [1, 4]$ )	2.59 mm/99.86%	1.68 mm/99.91%	10.75
High constant ( $n_{s t r} = 4$ )	1.43 mm/99.92%	1.48 mm/99.92%	15.35

(a)
PNG Results	Global RMS Error/Accuracy	ROI RMS Error/Accuracy	File Size/Compression Ratio
Low constant $(n_{s t r} = 1)$	0.0364 mm/99.9636%	0.0362 mm/99.9638%	148.8 KB/7.05:1
Proposed $(n_{s t r} = [1, 6])$	0.0319 mm/99.9681%	0.0067 mm/99.9933%	185.1 KB/5.66:1
High constant $(n_{s t r} = 6)$	0.0061 mm/99.9939%	0.0061 mm/99.9939%	281.7 KB/3.72:1
(b)
JPEG 80 results	Global RMS Error/Accuracy	ROI RMS Error/Accuracy	File Size/Compression Ratio
Low constant $(n_{s t r} = 1)$	0.16 mm/99.84%	0.16 mm/99.84%	17.27 KB/60.72:1
Proposed $(n_{s t r} = [1, 6])$	0.14 mm/99.86%	0.06 mm/99.94%	23.45 KB/44.72:1
High constant $(n_{s t r} = 6)$	0.13 mm/99.87%	0.05 mm/99.95%	42.52 KB/24.66:1

(a)
PNG Results	Global RMS Error/Accuracy	ROI RMS Error/Accuracy	File Size/Compression Ratio
Low constant ( $n_{s t r} = 1$ )	0.173 mm/99.964%	0.174 mm/99.963%	246.7 KB/10.38:1
Proposed ( $n_{s t r} = [1, 6]$ )	0.111 mm/99.977%	0.030 mm/99.994%	321.7 KB/7.96:1
High constant ( $n_{s t r} = 6$ )	0.029 mm/99.994%	0.029 mm/99.994%	431.8 KB/5.93:1
(b)
JPEG 80 results	Global RMS Error/Accuracy	ROI RMS Error/Accuracy	File Size/Compression Ratio
Low constant ( $n_{s t r} = 1$ )	1.94 mm/99.56%	1.61 mm/99.66%	39.32 KB/65.15:1
Proposed ( $n_{s t r} = [1, 6]$ )	1.35 mm/99.72%	0.56 mm/99.88%	54.20 KB/47.27:1
High constant ( $n_{s t r} = 6$ )	0.64 mm/99.87%	0.54 mm/99.89%	73.47 KB/34.87:1

Abstract

Supplementary Material (3)

Data availability

Cited By

Figures (11)

Tables (3)

Equations (17)

Applied Optics