Open Access
Add to BibSonomy
Abstract
In this paper, we proposed a new technique to realize a high-quality three-dimensional (3D) display by using binary holograms. First, we applied a localized random down-sampling (LRDS) mask to down-sample the object function and generated a binary CGH by direct sign-thresholding. Subsequently, we devised the display by adaptive intensity accumulation (AIA). In AIA, multiple CGHs of the same object are generated. However, selective sampling points of the same scene are removed according to the reconstructed image of previous binary CGHs as the second and more binary CGHs are generated. Finally, these holograms are sequentially displayed on a fast spatial light modulator, a digital micromirror device (DMD). Thus, a high-quality 3D image is reconstructed without artifacts and speckle noise.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Computer-generated hologram (CGH) is a promising technique for three-dimensional (3D) display [1]. Color holography [2], dynamic holography [3,4], and near-eye holographic display [5,6] have been demonstrated to prove the abilities of CGH. In the applications of holographic display, usually a phase-only spatial light modulator (SLM), such as a liquid-crystal on silicon (LCoS) device, is applied. Ideally, high-quality 3D image can be reconstructed by a phase-only hologram. Nevertheless, there is usually zeroth-order light and noise among the reconstructed image of the LCoS SLM, which is caused by the Fresnel reflection, modulation error, or phase flicker of the LCoS. An alternative display device is a binary-modulation SLM, such as the Ferroelectric liquid crystal SLM (binary phase) or digital micromirror device (DMD) (binary intensity). Because of the less degrees of freedom, the quality of image reconstructed by a binary SLM is worse than that by a phase-only SLM. On the upside, the modulation speed of binary SLM can be as fast as ${10^4}$ binary frames per second. The high frame rate provides additional degrees of freedom, including the manipulation of color and viewing directions [7–11].
In using a binary SLM, the complex hologram must be converted to a binary hologram. Typical complex-to-binary conversion methods include detour-phase method [12], down-sampling [13–17], error diffusion [18], direct binary search (DBS) [19–24], and Gerchberg–Saxton (GS) algorithm [25–27]. Particularly, DBS and GS are iteration-based methods and thus the computing speed is very slow. As a feedback, they can generate binary holograms with better quality. In using either of the above methods, the reconstructed image always contains significant artifact and speckle due to the binarization. There have been many studies on the speckle reduction methods for laser display. One kind of the methods are based on the decoherence of the light source [28–31], and thus it will also blur the reconstructed image in holographic display [32]. The other kind of speckle reduction methods is based on intensity accumulation. Intensity accumulation, in which a scene is reconstructed using multiple holograms, has been applied in the 3D display by phase-only CGHs [33–35]. Recently, intensity accumulation was also applied to the display by binary CGHs [27,36,37]. Although intensity accumulation can partially suppress the speckle, there is still residual speckle in the reconstructed image. It is time-consuming to further suppress the residual speckle because fifty GS binary holograms are demanded for the display of a single scene [27].
In this paper, we proposed a new method for high-quality 3D display by using a binary SLM with low computing load. First, the binary holograms are generated by random down-sampling, which is much faster than DBS or GS, and is compatible to both point-based and layer-based CGH algorithms. The other merit of random down-sampling is that it will not produce artifact due to regular-grid or regular-point down sampling, which is a significant problem in conventional down-sampling binary hologram. To suppress the residual artifact and speckle in the reconstructed image, we propose adaptive intensity accumulation (AIA). The concept of AIA is that multiple holograms of the scene are not independently generated. As the first binary hologram is generated, the artifact in the reconstructed image is measured and compensated in the second binary hologram, and so on. By this way, we can produce a high-quality reconstructed image with relatively low computing load. The remaining of this paper is organized as follows. In section 2, the principle of down-sampling-based binary CGH and display by intensity accumulation are explained. In section 3, simulations of various binary CGHs are provided and discussed. Some experimental results are shown and discussed in section 4. Finally, the concluding remarks are provided in section 5.
2. Principle
2.1 Localized random down-sampling
A computer-generated complex Fresnel hologram is calculated by
Fig. 1. Sampling grid (black lines) and the selected points (circles) for $k = 2$ down-sampling. The different colors of circles represent samples from different tiles. j denotes the index of generated mask. (a) RPDS, and (b) LRDS.
In LRDS, the object space is divided to many tiles, and each tile contains $k \times k$ pixels. Subsequently, one over the ${k^2}$ points in the tile is randomly selected as the down-sampling point. Therefore, the down-sampling rate is also $R = 1/{k^2}$. The LRDS mask can be modeled as
2.2 Display by adaptive intensity accumulation
In the display by intensity accumulation, multiple holograms of the same scene are fast displayed sequentially. As the refresh rate of the SLM is fast enough, the human vision only detects the intensity accumulation of all reconstructed images. Explicitly, the resulting image reconstructed from J holograms and detected by human eye can be modeled as
The flow chart of generating the binary holograms for AIA is shown in Fig. 3. First a binary hologram is generated by LRDS. It should be noted that for display by AIA, the down-sampled object function is obtained by
where $P(x,y)$ is an adaptive binary mask and initially $P(x,y) = 1$. As the first binary hologram is generated, it is reconstructed by applying $J = 1$ in Eq. (6). Subsequently, the normalized difference function is evaluated as where $d(x,y) = {|{O(x,y;z)} |^2} - I(x,y)$; ${d_{\max }}$ and ${d_{\min }}$ are the maximum and minimum of $d(x,y)$, respectively. A small value in $D(x,y)$ means that the reconstructed light in this location is stronger. Hence the region with smaller $D(x,y)$ can be ignored in generating the next hologram. Accordingly, a new adaptive mask for the next binary hologram is produced by3. Simulation
A series of simulations are performed to verify our method. The simulation parameters are as follows. The wavelength is $650\,\textrm{nm}$. The size of both the object and the hologram are $X \times Y = 512 \times 512$ pixels, and the pixel pitch is $7.56\,\mathrm{\mu}\textrm{m}$. The object distance is $190\,\textrm{mm}$. The target object is shown in Fig. 4(a), and a binary hologram generated by LRDS is shown in Fig. 4(b). The intensity of the full reconstructed field calculated by angular spectrum method [38] is shown in Fig. 4(c). Because there is always zeroth-order light and the conjugate term, only the first order in the lower right corner is selected as the region of interest for quality evaluation. The peak signal-to-noise ratio (PSNR) is adopted as the quality metric in this paper.
Fig. 4. (a) The original object (Image by 192635 from Pixabay). (b) Central $128 \times 128$ pixels of a binary hologram generated by LRDS. (c) The intensity of full reconstructed field of the LRDS binary hologram.
In the first set of simulation, various down-sampling factor $k$ is applied for generating the LRDS holograms. First, the display by DIA is evaluated. The dependence of image quality and the number of holograms is shown in Fig. 5(a). The curves of $k = 3\sim 5$ are similar, and the PSNR of reconstructed image is significantly improved from $7.80\,\textrm{dB}$ ($J = 1$) to $21.8\,\textrm{dB}$ ($J = 100$). As a comparison, by using RPDS together with DIA, the best PSNR is $15.7\,\textrm{dB}$, at $k = 9$ ($J = {k^2} = 81$). The dependence of the quality and the number of LRDS holograms for AIA is shown in Fig. 5(b). In comparison with Fig. 5(a), the image quality can be improved to $PSNR = 29.6\,\textrm{dB}$ ($J = 100$). We also performed simulations in various thresholding value T, as shown in Fig. 6. It is noted that the influence of T is larger than k and thus must be carefully selected. We have applied various target images to the simulations and concluded that good image quality can be always achieved by using $T = 0.6\sim 0.7$. Finally, it should be mentioned that the same set of LRDS masks are applied in all the above simulations. However, the resulting display quality possibly depends on the applied LRDS masks. Therefore, we generated ten different sets of LRDS masks for evaluating both the DIA and the AIA display. For the LRDS + DIA, the difference of the resulting PSNR is little; the standard deviation of the PSNR is always smaller than 0.1 dB. For the LRDS + AIA, the PSNR curve is slightly affected by the applied LRDS masks. This is shown in Fig. 7, in which the standard deviation bars are marked on the PSNR curve. The maximum standard deviation (0.35 dB) occurs at $J = 37$. Nevertheless, the standard deviation at $J = 100$ is only 0.18 dB, implying that the influence of LRDS masks can be mostly omitted, provided the number of accumulation is large enough.
Fig. 5. PSNR of the reconstructed image versus the number of holograms generated by various down-sampling factor ($k$). (a) Direct intensity accumulation; (b) Adaptive intensity accumulation ($T = 0.6$).
Fig. 6. PSNR of the reconstructed image versus the number of holograms generated by various thresholding value ($T$). The down-sampling factor is $k = 3$.
Fig. 7. PSNR of the reconstructed image versus the number of holograms generated by LRDS + AIA ($k = 3$, $T\textrm{ = }0.6$). The mean of ten simulations is plotted as the curve, on which the red bars mark ${\pm} 1$ standard deviations.
Selected reconstructed images of simulations are shown in Fig. 8. In Fig. 8(a), the reconstructed images of RPDS holograms with DIA display show that the gridding structure and speckle can be suppressed, but serious artifacts cannot. These artifacts can be completely suppressed by using the LRDS holograms with DIA display, as shown in Fig. 8(b). However, speckle is detected even though in the case that 100 intensities are accumulated. By using the LRDS holograms with AIA display, both the artifacts and the speckle can be significantly suppressed, as shown in Fig. 8(c). Finally, we also generated binary holograms by using modified iterative Fresnel ping-pong algorithm (MIFA) [27], and display by DIA. In MIFA, every binary hologram is generated by five iterations. The quality of the reconstructed image is a little better than that of LRDS + DIA [Fig. 8(b)], but still worse than that of LRDS + AIA. We have also measured the average diffraction efficiency, which is defined as the ratio of power of first-order image and total power. The diffraction efficiencies of LRDS + DIA, LRDS + AIA, and MIFA + DIA are 17.3%, 18.0%, and 19.7%, respectively. The computing times per hologram are 0.28 seconds (LRDS + DIA), 0.55 seconds (LRDS + AIA), and 3.86 seconds (MIFA), respectively. The computing is at the platform of Intel Core i7-870 @ 2.93 GHz and MATLAB 2018b.
Fig. 8. Reconstructed images. (a) Display by DIA of RPDS holograms. (b) Display by DIA of LRDS holograms. (c) Display by AIA of LRDS holograms. (d) Display by DIA of MIFA holograms. The red number under images denotes (J, PSNR).
4. Experiment and discussion
The experimental setup is depicted in Fig. 9. We applied a fiber-coupled diode laser with wavelength of $650\,\textrm{nm}$ as the light source. The SLM is a DMD (DLP6500FYE) by Texas Instruments with pixel pitch $7.56\,\mathrm{\mu}\textrm{m}$ [39]. The DMD was illuminated by a collimated laser beam with a diagonal tilt angle ($\theta = {24^ \circ }$) to locate the strongest diffraction order at the center of optical axis [40–42]. A lensless monochromatic CCD (FLIR FL3-GE-50S5M-C, $2448 \times 2048$ pixels with pitch $3.45\,\mathrm{\mu}\textrm{m}$) is set at the reconstruction plane ($190\,\textrm{mm}$ measured from the DMD) to acquire the reconstructed image. The exposure time of a single image is 10.5 ms. The DMD is synchronized with the CCD, and thus all holograms of the same object are displayed sequentially within the time of a single shot. For example, in the AIA of 100 holograms, the display time of each hologram is $\textrm{10}\textrm{.5}\,\textrm{ms/100 = 105}\,\mathrm{\mu}\textrm{s}$, which is the shortest display time of the DMD. The acquired reconstructed images of different holograms are shown in Fig. 10. The experimental images are very similar to the images in simulation (Fig. 8). However, the images are a little blurred due to the difficult alignment of blazed-grating condition of DMD. It is noted that Fig. 10(d) is slightly brighter than other images. This is the reason that the diffraction efficiency of MIFA + DIA is a little higher than that of other methods. It is also noted that in the best results of LRDS + AIA, there is a little speckle noise, which is different from simulation. The residual speckle noise may be due to the scattering light from the optical elements (e.g. the lens). This kind of unpredicted noise source cannot be compensated in the calculation of AIA.
Fig. 10. Experimentally reconstructed images of (a) RPDS holograms, (b) LRDS holograms, (c) LRDS holograms, (d) MIFA holograms, respectively. The red number under image denotes the accumulation number (J).
We have also conducted a simplified demonstration of 3D display by LRDS + AIA method. The object consists of two layers. The first layer is the letters “FCU” at $z = 190\,\textrm{mm}$, while the other layer is the letters “OIP” at $z = 210\,\textrm{mm}$. In generating the holograms for AIA, every hologram are reconstructed at both $z = 190\,\textrm{mm}$ and $z = 210\,\textrm{mm}$, respectively. Thus, the compensation thresholding is applied on both planes to generate the next hologram. By this way, 100 holograms are generated and displayed by AIA. The simulation results together with the experimental results are shown in Fig. 11. Both layers are successfully reconstructed with good quality.
Fig. 11. Reconstructed images of a hologram of a 2-layer object. (a) and (b) are simulation results, and (c) and (d) are experimental results. The reconstruction distances of the two layers are 190 mm [(a) and (c)] and 210 mm [(b) and (d)].
5. Conclusion
We have discussed two methods for 3D display by binary holograms. In the first method, multiple binary holograms based on localized random down-sampling (LRDS) are generated, and the display is accomplished by direct intensity accumulation (DIA) of reconstructed images of all holograms. LRDS + DIA is efficient to suppress the artifacts in the reconstructed image, and its computing time is the shortest. However, it still contains speckle noise. In the second method, in addition to LRDS, we propose the adaptive intensity accumulation (AIA) method. That is, selective object points are removed by an adaptive mask, which is generated according to the reconstructed image from previous holograms. The LRDS + AIA method suppresses not only the artifacts, but also the speckle noise. The PSNR of the reconstructed image can reach nearly 30 dB, which is high enough for most visual applications. The holographic display by either DIA or AIA demands high-frame rate SLM and high transfer-rate interface. The time for displaying a full-color 3D scene is $105\,\mathrm{\mu}\textrm{s} \times \textrm{100} \times \textrm{3} = \textrm{31}\textrm{.5}\,\textrm{ms}$, which is short enough for dynamic display. The transfer rate is ${{1920 \times 1080\,\textrm{bits}} / {105\,\mathrm{\mu}\textrm{s}}} = 19.7\,\textrm{Gbits/s}$, which is also achievable by current hardware framework. Therefore, the proposed LRDS + AIA method can be applied for dynamic color 3D display by using three lasers and time-multiplexing theoretically. In our demonstration, the proposed LRDS + AIA method is about seven times faster than the MIFA + DIA. However, the computing time for a scene is still too long to realize real-time holographic display. The hardware acceleration methods or algorithm optimization for realizing real-time calculation is worth further studying in the future.
Funding
Ministry of Science and Technology, Taiwan (106-2628-E-035-002-MY3, 109-2221-E-035-076-MY3); Research Grants Council, University Grants Committee (11200319).
Disclosures
The authors declare no conflicts of interest.
References
1. T.-C. Poon, ed., Digital Holography and Three-Dimensional Display (Springer, Berlin, 2006).
2. Y. Tsuchiyama and K. Matsushima, “Full-color large-scaled computer-generated holograms using RGB color filters,” Opt. Express 25(3), 2016–2030 (2017). [CrossRef]
3. J. S. Chen and D. P. Chu, “Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications,” Opt. Express 23(14), 18143–18155 (2015). [CrossRef]
4. J.-S. Chen and D. Chu, “Realization of real-time interactive 3D image holographic display [Invited],” Appl. Opt. 55(3), A127–A134 (2016). [CrossRef]
5. A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 1–16 (2017). [CrossRef]
6. Z. Zhang, J. Liu, Q. Gao, X. Duan, and X. Shi, “A full-color compact 3D see-through near-eye display system based on complex amplitude modulation,” Opt. Express 27(5), 7023–7035 (2019). [CrossRef]
7. Y. Takaki and N. Okada, “Hologram generation by horizontal scanning of a high-speed spatial light modulator,” Appl. Opt. 48(17), 3255–3260 (2009). [CrossRef]
8. Y. Takaki and N. Okada, “Reduction of image blurring of horizontally scanning holographic display,” Opt. Express 18(11), 11327–11334 (2010). [CrossRef]
9. Y. Takaki, Y. Matsumoto, and T. Nakajima, “Color image generation for screen-scanning holographic display,” Opt. Express 23(21), 26986–26998 (2015). [CrossRef]
10. J. Li, Q. Smithwick, and D. Chu, “Full bandwidth dynamic coarse integral holographic displays with large field of view using a large resonant scanner and a galvanometer scanner,” Opt. Express 26(13), 17459–17476 (2018). [CrossRef]
11. Y. Sando, D. Barada, and T. Yatagai, “Full-color holographic 3D display with horizontal full viewing zone by spatiotemporal-division multiplexing,” Appl. Opt. 57(26), 7622–7626 (2018). [CrossRef]
12. C.-H. Wu, C.-L. Chen, and M. A. Fiddy, “Iterative procedure for improved computer-generated-hologram reconstruction,” Appl. Opt. 32(26), 5135–5140 (1993). [CrossRef]
13. P. Tsang, T.-C. Poon, W. K. Cheung, and J.-P. Liu, “Computer generation of binary Fresnel holography,” Appl. Opt. 50(7), B88–B95 (2011). [CrossRef]
14. W. K. Cheung, P. Tsang, T.-C. Poon, and C. Zhou, “Enhanced method for the generation of binary Fresnel holograms based on grid-cross downsampling,” Chin. Opt. Lett. 9(12), 120005 (2011). [CrossRef]
15. P. Tsang, W. K. Cheung, T.-C. Poon, and J.-P. Liu, “An enhanced method for generation of binary Fresnel hologram based on adaptive and uniform grid-cross down-sampling,” Opt. Commun. 285(20), 4027–4032 (2012). [CrossRef]
16. P. W. M. Tsang, T.-C. Poon, and A. S. M. Jiao, “Embedding intensity image in grid-cross down-sampling (GCD) binary holograms based on block truncation coding,” Opt. Commun. 304, 62–70 (2013). [CrossRef]
17. P. W. M. Tsang and T.-C. Poon, “Generation of integrated binary Fresnel hologram for multiple images,” J. Opt. 16(10), 105403 (2014). [CrossRef]
18. G. Yang, S. Jiao, J.-P. Liu, T. Lei, and X. Yuan, “Error diffusion method with optimized weighting coefficients for binary hologram generation,” Appl. Opt. 58(20), 5547–5555 (2019). [CrossRef]
19. M. A. Seldowitz, J. P. Allebach, and D. W. Sweeney, “Synthesis of digital holograms by direct binary search,” Appl. Opt. 26(14), 2788–2798 (1987). [CrossRef]
20. B. K. Jennison, J. P. Allebach, and D. W. Sweeney, “Efficient design of direct-binary-search computer-generated holograms,” J. Opt. Soc. Am. A 8(4), 652–660 (1991). [CrossRef]
21. J. Y. Zhuang and O. K. Ersoy, “Fast decimation-in-frequency direct binary search algorithms for synthesis of computer-generated holograms,” J. Opt. Soc. Am. A 11(1), 135–143 (1994). [CrossRef]
22. B. B. Chhetri, S. Yang, and T. Shimomura, “Stochastic approach in the efficient design of the direct-binary-search algorithm for hologram synthesis,” Appl. Opt. 39(32), 5956–5964 (2000). [CrossRef]
23. T. Leportier and M. C. Park, “Generation of binary holograms for deep scenes captured with a camera and a depth sensor,” Opt. Eng. 56(1), 013107 (2017). [CrossRef]
24. J.-P. Liu, C.-Q. Yu, and P. W. M. Tsang, “Enhanced direct binary search algorithm for binary computer-generated Fresnel holograms,” Appl. Opt. 58(14), 3735–3741 (2019). [CrossRef]
25. F. Wyrowski and O. BryngdahI, “Iterative Fourier-transform algorithm applied to computer holography,” J. Opt. Soc. Am. A 5(7), 1058–1965 (1988). [CrossRef]
26. R. Piestun, B. Spektor, and J. Shamir, “On-axis binary-amplitude computer generated holograms,” Opt. Commun. 136(1-2), 85–92 (1997). [CrossRef]
27. K. Masuda, Y. Saita, R. Toritani, P. Xia, K. Nitta, and O. Matoba, “Improvement of Image Quality of 3D Displayby Using Optimized Binary Phase Modulation and Intensity Accumulation,” J. Disp. Technol. 12(5), 472–477 (2016). [CrossRef]
28. T.-K.-T. Tran, X. Chen, Ø. Svensen, and M. N. Akram, “Speckle reduction in laser projection using a dynamic deformable mirror,” Opt. Express 22(9), 11152–11166 (2014). [CrossRef]
29. S. Kubota and J. W. Goodman, “Very efficient speckle contrast reduction realized by moving diffuser device,” Appl. Opt. 49(23), 4385–4391 (2010). [CrossRef]
30. L. Wang, T. Tschudi, T. Halldórsson, and P. R. Pétursson, “Speckle reduction in laser projection systems by diffractive optical elements,” Appl. Opt. 37(10), 1770–1775 (1998). [CrossRef]
31. F. Shevlin, “Phase randomization for spatiotemporal averaging of unwanted interference effects arising from coherence,” Appl. Opt. 57(22), E6–E10 (2018). [CrossRef]
32. D. Lee, C. Jang, K. Bang, S. Moon, G. Li, and B. Lee, “Speckle Reduction for Holographic Display Using Optical Path Difference and Random Phase Generator,” IEEE Trans. Ind. Inf. 15(11), 6170–6178 (2019). [CrossRef]
33. W.-F. Hsu and C.-F. Yeh, “Speckle suppression in holographic projection displays using temporal integration of speckle images from diffractive optical elements,” Appl. Opt. 50(34), H50–H55 (2011). [CrossRef]
34. Y. Mori, T. Fukuoka, and T. Nomura, “Speckle reduction in holographic projection by random pixel separation with time multiplexing,” Appl. Opt. 53(35), 8182–8188 (2014). [CrossRef]
35. S.-J. Liu, D. Wang, S.-J. Li, and Q.-H. Wang, “Speckle noise suppression method in holographic display using time multiplexing,” Opt. Eng. 56(6), 063107 (2017). [CrossRef]
36. B. Lee, D. Yoo, J. Jeong, S. Lee, D. Lee, and B. Lee, “Wide-angle speckleless DMD holographic display using structured illumination with temporal multiplexing,” Opt. Lett. 45(8), 2148–2151 (2020). [CrossRef]
37. Y. Takaki and M. Yokouchi, “Speckle-free and grayscale hologram reconstruction using time-multiplexing technique,” Opt. Express 19(8), 7567–7579 (2011). [CrossRef]
38. T.-C. Poon and J.-P. Liu, Introduction to Modern Digital Holography with MATLAB® (Cambridge University Press, 2014).
39. https://www.ti.com/product/DLP6500FYE.
40. J.-Y. Son, B.-R. Lee, O. O. Chernyshov, K.-A. Moon, and H. Lee, “Holographic display based on a spatial DMD array,” Opt. Lett. 38(16), 3173–3176 (2013). [CrossRef]
41. M.-C. Park, B.-R. Lee, J.-Y. Son, and O. Chernyshov, “Properties of DMDs for holographic displays,” J. Mod. Opt. 62(19), 1600–1607 (2015). [CrossRef]
42. M. Chlipala and T. Kozacki, “Color LED DMD holographic display with high resolution across large depth,” Opt. Lett. 44(17), 4255–4258 (2019). [CrossRef]
