Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group
Journal Home About Issues in Progress Current Issue All Issues Early Posting

From picture to 3D hologram: end-to-end learning of real-time 3D photorealistic hologram generation from 2D image input

Chenliang Chang, Bo Dai, Dongchen Zhu, Jiamao Li, Jun Xia, Dawei Zhang, Lianping Hou, and Songlin Zhuang

Open Access Open Access

Abstract

In this Letter, we demonstrate a deep-learning-based method capable of synthesizing a photorealistic 3D hologram in real-time directly from the input of a single 2D image. We design a fully automatic pipeline to create large-scale datasets by converting any collection of real-life images into pairs of 2D images and corresponding 3D holograms and train our convolutional neural network (CNN) end-to-end in a supervised way. Our method is extremely computation-efficient and memory-efficient for 3D hologram generation merely from the knowledge of on-hand 2D image content. We experimentally demonstrate speckle-free and photorealistic holographic 3D displays from a variety of scene images, opening up a way of creating real-time 3D holography from everyday pictures. © 2023 Optical Society of America

© 2023 Optica Publishing Group

Full Article  |  PDF Article
More Like This
Deep-learning-based computer-generated hologram from a stereo image pair

Chenliang Chang, Di Wang, Dongchen Zhu, Jiamao Li, Jun Xia, and Xiaolin Zhang
Opt. Lett. 47(6) 1482-1485 (2022)

Three-dimensional computer holography enabled from a single 2D image

Chenliang Chang, Dongchen Zhu, Jiamao Li, Di Wang, Jun Xia, and Xiaolin Zhang
Opt. Lett. 47(9) 2202-2205 (2022)

Investigating learning-empowered hologram generation for holographic displays with ill-tuned hardware

Xinxing Xia, Furong Yang, Weisen Wang, Xinghua Shui, Frank Guan, Huadong Zheng, Yingjie Yu, and Yifan Peng
Opt. Lett. 48(6) 1478-1481 (2023)

View More...

References

  • View by:
  • Article Order
  • Year
  • Author
  • Publication
  1. Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
    [Crossref]
  2. T. Shimobaba, T. Kakue, and T. Ito, IEEE Trans. Ind. Inf. 12, 1611 (2016).
    [Crossref]
  3. M. H. Eybposh, N. W. Caira, M. Atisa, P. Chakravarthula, and N. C. Pégard, Opt. Express 28, 26636 (2020).
    [Crossref]
  4. R. Horisaki, Y. Nishizaki, K. Kitaguchi, M. Saito, and J. Tanida, Appl. Opt. 60, A323 (2021).
    [Crossref]
  5. J. Lee, J. Jeong, J. Cho, D. Yoo, B. Lee, and B. Lee, Opt. Express 28, 27137 (2020).
    [Crossref]
  6. Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, ACM Trans. Graph. 39, 1 (2020).
    [Crossref]
  7. S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, ACM Trans. Graph. 40, 1 (2021).
    [Crossref]
  8. L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, Nature 591, 234 (2021).
    [Crossref]
  9. A. G. Marrugo, F. Gao, and S. Zhang, J. Opt. Soc. Am. A 37, B60 (2020).
    [Crossref]
  10. D. Wang, C. Liu, C. Shen, Y. Xing, and Q.-H. Wang, PhotoniX 1, 6 (2020).
    [Crossref]
  11. C. Chang, D. Wang, D. Zhu, J. Li, J. Xia, and X. Zhang, Opt. Lett. 47, 1482 (2022).
    [Crossref]
  12. Z. He, X. Sui, and L. Cao, Appl. Sci. 11, 9889 (2021).
    [Crossref]
  13. Y. Ming, X. Meng, C. Fan, and H. Yu, Neurocomputing 438, 14 (2021).
    [Crossref]
  14. R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun, “Towards robust monocular depth estimation: mixing datasets for zero-shot cross-dataset transfer,” arXiv, (2019).
    [Crossref]
  15. J. Watson, O. M. Aodha, D. Turmukhambetov, G. J. Brostow, and M. Firman, Proc. of European Conference on Computer Vision, 722 (2020).
  16. T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).
  17. Y. Qi, C. Chang, and J. Xia, Opt. Express 24, 30368 (2016).
    [Crossref]
  18. N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, In Proceedings of the European Conference on Computer Vision (ECCV), pp.746–760, (2012).
  19. A. Saxena, M. Sun, and A. Y. Ng, IEEE Trans. Pattern Anal. Mach. Intell. 31, 824 (2009).
    [Crossref]
  20. N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A Dosovitskiy, and T. Brox, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4040–4048 (2016).
  21. www.bigbuckbunny.org
  22. S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’toole, and G. Wetzstein, ACM SIGGRAPH Conference Proceedings (2022).
  23. D. Yoo, Y. Jo, S.-W. Nam, C. Chen, and B. Lee, Opt. Lett. 46, 4769 (2021).
    [Crossref]

2022 (2)

Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
[Crossref]

C. Chang, D. Wang, D. Zhu, J. Li, J. Xia, and X. Zhang, Opt. Lett. 47, 1482 (2022).
[Crossref]

2021 (6)

Z. He, X. Sui, and L. Cao, Appl. Sci. 11, 9889 (2021).
[Crossref]

Y. Ming, X. Meng, C. Fan, and H. Yu, Neurocomputing 438, 14 (2021).
[Crossref]

D. Yoo, Y. Jo, S.-W. Nam, C. Chen, and B. Lee, Opt. Lett. 46, 4769 (2021).
[Crossref]

R. Horisaki, Y. Nishizaki, K. Kitaguchi, M. Saito, and J. Tanida, Appl. Opt. 60, A323 (2021).
[Crossref]

S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, ACM Trans. Graph. 40, 1 (2021).
[Crossref]

L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, Nature 591, 234 (2021).
[Crossref]

2020 (5)

2016 (2)

Y. Qi, C. Chang, and J. Xia, Opt. Express 24, 30368 (2016).
[Crossref]

T. Shimobaba, T. Kakue, and T. Ito, IEEE Trans. Ind. Inf. 12, 1611 (2016).
[Crossref]

2009 (1)

A. Saxena, M. Sun, and A. Y. Ng, IEEE Trans. Pattern Anal. Mach. Intell. 31, 824 (2009).
[Crossref]

Aodha, O. M.

J. Watson, O. M. Aodha, D. Turmukhambetov, G. J. Brostow, and M. Firman, Proc. of European Conference on Computer Vision, 722 (2020).

Atisa, M.

Belongie, S.

T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).

Brostow, G. J.

J. Watson, O. M. Aodha, D. Turmukhambetov, G. J. Brostow, and M. Firman, Proc. of European Conference on Computer Vision, 722 (2020).

Brox, T.

N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A Dosovitskiy, and T. Brox, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4040–4048 (2016).

Caira, N. W.

Cao, L.

Z. He, X. Sui, and L. Cao, Appl. Sci. 11, 9889 (2021).
[Crossref]

Chakravarthula, P.

Chang, C.

Chen, C.

Cho, J.

Choi, S.

S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, ACM Trans. Graph. 40, 1 (2021).
[Crossref]

Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, ACM Trans. Graph. 39, 1 (2020).
[Crossref]

S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’toole, and G. Wetzstein, ACM SIGGRAPH Conference Proceedings (2022).

Chu, F.

Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
[Crossref]

Cremers, D.

N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A Dosovitskiy, and T. Brox, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4040–4048 (2016).

Dollar, P.

T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).

Dosovitskiy, A

N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A Dosovitskiy, and T. Brox, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4040–4048 (2016).

Eybposh, M. H.

Fan, C.

Y. Ming, X. Meng, C. Fan, and H. Yu, Neurocomputing 438, 14 (2021).
[Crossref]

Fergus, R.

N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, In Proceedings of the European Conference on Computer Vision (ECCV), pp.746–760, (2012).

Firman, M.

J. Watson, O. M. Aodha, D. Turmukhambetov, G. J. Brostow, and M. Firman, Proc. of European Conference on Computer Vision, 722 (2020).

Fischer, P.

N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A Dosovitskiy, and T. Brox, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4040–4048 (2016).

Gao, F.

Gopakumar, M.

S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, ACM Trans. Graph. 40, 1 (2021).
[Crossref]

S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’toole, and G. Wetzstein, ACM SIGGRAPH Conference Proceedings (2022).

Hausser, P.

N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A Dosovitskiy, and T. Brox, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4040–4048 (2016).

Hays, J.

T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).

He, Z.

Z. He, X. Sui, and L. Cao, Appl. Sci. 11, 9889 (2021).
[Crossref]

Hoiem, D.

N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, In Proceedings of the European Conference on Computer Vision (ECCV), pp.746–760, (2012).

Horisaki, R.

Ilg, E.

N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A Dosovitskiy, and T. Brox, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4040–4048 (2016).

Ito, T.

T. Shimobaba, T. Kakue, and T. Ito, IEEE Trans. Ind. Inf. 12, 1611 (2016).
[Crossref]

Jeong, J.

Jo, Y.

Kakue, T.

T. Shimobaba, T. Kakue, and T. Ito, IEEE Trans. Ind. Inf. 12, 1611 (2016).
[Crossref]

Kellnhofer, P.

L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, Nature 591, 234 (2021).
[Crossref]

Kim, C.

L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, Nature 591, 234 (2021).
[Crossref]

Kim, J.

S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, ACM Trans. Graph. 40, 1 (2021).
[Crossref]

S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’toole, and G. Wetzstein, ACM SIGGRAPH Conference Proceedings (2022).

Kitaguchi, K.

Kohli, P.

N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, In Proceedings of the European Conference on Computer Vision (ECCV), pp.746–760, (2012).

Lee, B.

Lee, J.

Lee, S.-D.

Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
[Crossref]

Li, B.

L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, Nature 591, 234 (2021).
[Crossref]

Li, J.

Li, N.-N.

Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
[Crossref]

Li, Y.-L.

Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
[Crossref]

Lin, T. Y.

T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).

Liu, C.

D. Wang, C. Liu, C. Shen, Y. Xing, and Q.-H. Wang, PhotoniX 1, 6 (2020).
[Crossref]

Maire, M.

T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).

Marrugo, A. G.

Matusik, W.

L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, Nature 591, 234 (2021).
[Crossref]

Mayer, N.

N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A Dosovitskiy, and T. Brox, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4040–4048 (2016).

Meng, X.

Y. Ming, X. Meng, C. Fan, and H. Yu, Neurocomputing 438, 14 (2021).
[Crossref]

Ming, Y.

Y. Ming, X. Meng, C. Fan, and H. Yu, Neurocomputing 438, 14 (2021).
[Crossref]

Nam, S.-W.

Ng, A. Y.

A. Saxena, M. Sun, and A. Y. Ng, IEEE Trans. Pattern Anal. Mach. Intell. 31, 824 (2009).
[Crossref]

Nishizaki, Y.

O’toole, M.

S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’toole, and G. Wetzstein, ACM SIGGRAPH Conference Proceedings (2022).

Padmanaban, N.

Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, ACM Trans. Graph. 39, 1 (2020).
[Crossref]

Pégard, N. C.

Peng, Y.

S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, ACM Trans. Graph. 40, 1 (2021).
[Crossref]

Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, ACM Trans. Graph. 39, 1 (2020).
[Crossref]

S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’toole, and G. Wetzstein, ACM SIGGRAPH Conference Proceedings (2022).

Perona, P.

T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).

Qi, Y.

Ramanan, D.

T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).

Saito, M.

Saxena, A.

A. Saxena, M. Sun, and A. Y. Ng, IEEE Trans. Pattern Anal. Mach. Intell. 31, 824 (2009).
[Crossref]

Shen, C.

D. Wang, C. Liu, C. Shen, Y. Xing, and Q.-H. Wang, PhotoniX 1, 6 (2020).
[Crossref]

Shi, L.

L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, Nature 591, 234 (2021).
[Crossref]

Shimobaba, T.

T. Shimobaba, T. Kakue, and T. Ito, IEEE Trans. Ind. Inf. 12, 1611 (2016).
[Crossref]

Silberman, N.

N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, In Proceedings of the European Conference on Computer Vision (ECCV), pp.746–760, (2012).

Sui, X.

Z. He, X. Sui, and L. Cao, Appl. Sci. 11, 9889 (2021).
[Crossref]

Sun, M.

A. Saxena, M. Sun, and A. Y. Ng, IEEE Trans. Pattern Anal. Mach. Intell. 31, 824 (2009).
[Crossref]

Tanida, J.

Turmukhambetov, D.

J. Watson, O. M. Aodha, D. Turmukhambetov, G. J. Brostow, and M. Firman, Proc. of European Conference on Computer Vision, 722 (2020).

Wang, D.

C. Chang, D. Wang, D. Zhu, J. Li, J. Xia, and X. Zhang, Opt. Lett. 47, 1482 (2022).
[Crossref]

Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
[Crossref]

D. Wang, C. Liu, C. Shen, Y. Xing, and Q.-H. Wang, PhotoniX 1, 6 (2020).
[Crossref]

Wang, Q.-H.

Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
[Crossref]

D. Wang, C. Liu, C. Shen, Y. Xing, and Q.-H. Wang, PhotoniX 1, 6 (2020).
[Crossref]

Watson, J.

J. Watson, O. M. Aodha, D. Turmukhambetov, G. J. Brostow, and M. Firman, Proc. of European Conference on Computer Vision, 722 (2020).

Wetzstein, G.

S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, ACM Trans. Graph. 40, 1 (2021).
[Crossref]

Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, ACM Trans. Graph. 39, 1 (2020).
[Crossref]

S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’toole, and G. Wetzstein, ACM SIGGRAPH Conference Proceedings (2022).

Xia, J.

Xing, Y.

D. Wang, C. Liu, C. Shen, Y. Xing, and Q.-H. Wang, PhotoniX 1, 6 (2020).
[Crossref]

Yoo, D.

Yu, H.

Y. Ming, X. Meng, C. Fan, and H. Yu, Neurocomputing 438, 14 (2021).
[Crossref]

Zhang, S.

Zhang, X.

Zheng, Y.-W.

Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
[Crossref]

Zhu, D.

Zitnick, C. L.

T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).

ACM Trans. Graph. (2)

Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, ACM Trans. Graph. 39, 1 (2020).
[Crossref]

S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, ACM Trans. Graph. 40, 1 (2021).
[Crossref]

Appl. Opt. (1)

Appl. Sci. (1)

Z. He, X. Sui, and L. Cao, Appl. Sci. 11, 9889 (2021).
[Crossref]

IEEE Trans. Ind. Inf. (1)

T. Shimobaba, T. Kakue, and T. Ito, IEEE Trans. Ind. Inf. 12, 1611 (2016).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

A. Saxena, M. Sun, and A. Y. Ng, IEEE Trans. Pattern Anal. Mach. Intell. 31, 824 (2009).
[Crossref]

J. Opt. Soc. Am. A (1)

Light: Sci. Appl. (1)

Y.-L. Li, N.-N. Li, D. Wang, F. Chu, S.-D. Lee, Y.-W. Zheng, and Q.-H. Wang, Light: Sci. Appl. 11, 188 (2022).
[Crossref]

Nature (1)

L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, Nature 591, 234 (2021).
[Crossref]

Neurocomputing (1)

Y. Ming, X. Meng, C. Fan, and H. Yu, Neurocomputing 438, 14 (2021).
[Crossref]

Opt. Express (3)

Opt. Lett. (2)

PhotoniX (1)

D. Wang, C. Liu, C. Shen, Y. Xing, and Q.-H. Wang, PhotoniX 1, 6 (2020).
[Crossref]

Other (7)

N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, In Proceedings of the European Conference on Computer Vision (ECCV), pp.746–760, (2012).

R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun, “Towards robust monocular depth estimation: mixing datasets for zero-shot cross-dataset transfer,” arXiv, (2019).
[Crossref]

J. Watson, O. M. Aodha, D. Turmukhambetov, G. J. Brostow, and M. Firman, Proc. of European Conference on Computer Vision, 722 (2020).

T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, Proc. of European Conference on Computer Vision, 740 (2014).

N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A Dosovitskiy, and T. Brox, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 4040–4048 (2016).

www.bigbuckbunny.org

S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’toole, and G. Wetzstein, ACM SIGGRAPH Conference Proceedings (2022).

Supplementary Material (1)

NameDescription
Supplement 1       Test for CNN performance

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)
Fig. 1.
Fig. 1. Schematic of the end-to-end 3D photorealistic hologram generation network architecture. The network receives a single gray-level 2D image as input and rapidly synthesizes a complex hologram that records the corresponding 3D diffractive wavefront of the image content.

Download Full Size | PDF

Fig. 2.
Fig. 2. Workflow for learning 3D holograms from 2D images. We use the “MiDaS diffraction-based approach” to create a training dataset consisting of “2D image + ground truth 3D holograms.” All the synthesized large-scale data are fed to train the proposed CNN under supervision. ASM, angular spectrum method

Download Full Size | PDF

Fig. 3.
Fig. 3. Simulation results comparing the proposed CNN to the existing algorithm. (a) Generation of ground truth 3D complex hologram from original RGB-D data and the CNN predicted 3D complex hologram from 2D image input. (b) and (c) Simulated reconstructions of front and rear focuses from ground truth hologram. (d) and (e) Simulated reconstructions of front and rear focuses from CNN predicted hologram.

Download Full Size | PDF

Fig. 4.
Fig. 4. (a) Experimental setup for holographic 3D display. (b) Optical recordings of the 3D image at front focused distance. (c) Optical recordings of the 3D image at rear focused distance. SLM, spatial light modulator.

Download Full Size | PDF

Fig. 5.
Fig. 5. Holographic 3D display of various scenes from the proposed CNN-predicted holograms using 2D image input. The left column shows each input 2D image from various indoor and outdoor image datasets. The middle and right columns are the experimental recordings of 3D reconstructions at the front (z = 0.001 m from the hologram) and rear (z = 0.005 m from the hologram) focal distances.

Download Full Size | PDF

Equations (2)

Equations on this page are rendered with MathJax. Learn more.

$$\scalebox{0.83}{$\displaystyle H({{x_m},{y_n}} )= \sum\limits_i {A({{x_i},{y_i}} )\cdot } \exp \left[ {\frac{{i2\pi }}{\lambda }\sqrt {{{({{x_i} - {x_m}} )}^2} + {{({{y_i} - {y_n}} )}^2} + d{{({{x_i},{y_i}} )}^2}} + i{\varphi_0}} \right],$}$$
$$\begin{aligned} H({{x_m},{y_n}} )&= \sum\limits_{{d_i}}^{} {{\mathrm{{\cal F}}^{ - 1}}({\mathrm{{\cal F}}({A({{x_i},{y_i}} )\cdot {e^{i{\varphi_0}({{x_i},{y_i}} )}}} )\cdot {\mathrm{{\cal H}}_{{d_i}}}({{f_x},{f_y}} )} )} ,\\ {\mathrm{{\cal H}}_{{d_i}}}({{f_x},{f_y}} )&= \left\{ \begin{aligned} &{e^{i\frac{{2\pi }}{\lambda }\sqrt {1 - ({\lambda {f_x}^2} )- ({\lambda {f_x}^2} )} {d_i}}},\;\;\;\;\;\;\;\;\textrm{if}\sqrt {{f_x}^2 + {f_y}^2} < \frac{1}{\lambda },\\ &0\quad \quad \quad \quad \quad \quad \quad \quad \;\;\;\textrm{otherwise,} \end{aligned} \right. \end{aligned}$$

Metrics

Select as filters
Select Topics Cancel
Top
       
© Copyright 2023 | Optica Publishing Group. All Rights Reserved
Privacy | Terms of Use