Open Access
Add to BibSonomy
Abstract
Multi-plane crosstalk is a key issue affecting the quality of holographic three-dimensional (3D) displays. The time-multiplexing stochastic gradient descent (TM-SGD) method has been applied to solve the inter-plane crosstalk problem in multi-plane reconstruction. However, the inter-plane crosstalk increases greatly as the inter-plane interval decreases, and the optimization time increases greatly as the number of planes increases. In this paper, we propose a double-constraint stochastic gradient descent method to suppress inter-plane crosstalk in multi-plane reconstruction. In the proposed method, we use the mask to make the optimization process focus more on the signal region and improve the reconstruction quality. Meanwhile, we adopt a constraint strategy of phase regularization to reduce the phase randomness of the signal region and suppress inter-plane crosstalk. Numerical simulation and optical experiment results confirm that our method can effectively suppress the inter-plane crosstalk and improve the quality of the reconstructed planes at a lower inter-plane interval. Moreover, the optimization time of our method is almost 4 times faster than that of TM-SGD. The proposed method can contribute to the realization of tomographic 3D visualization in the biomedical field, which requires the reconstruction of multiple tomographic images without inter-plane crosstalk.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Holographic display is viewed as a promising three-dimensional (3D) display technology for the upcoming generation of augmented reality (AR) and virtual reality (VR) devices [1–5]. Because it can reconstruct the entire optical wave field of a 3D scene and offers users both monocular and binocular depth cues without vergence-accommodation conflict. Computer-generated holography (CGH) utilizes computational simulations of the optical diffraction propagation process to generate holograms. CGH has the advantages of a simple system and dynamic real-time compared with optical holography. These holograms are subsequently modulated using spatial light modulators(SLMs) to enable holographic displays. Since the commercial SLMs cannot directly modulate complex amplitude incident light, the complex amplitude hologram must be encoded as either a phase-only hologram (POH) or an amplitude-only hologram. The POHs have been widely used in holographic displays due to their higher diffraction efficiency compared to amplitude-only holograms [6–8].
In recent years, many methods have been proposed to improve the reconstruction quality of POHs. Among them, iteration-based methods have the best performance for enhancing the reconstruction quality. Previous iterative methods such as the Gerchberg–Saxton (GS) method [9] optimize the hologram by iterative calculation between the object plane and the holographic plane. Other improved forms of the GS method such as the weighted constraint GS method [10] use weighted feedback to constrain the signal region amplitude to improve the reconstruction quality. However, the desired POHs cannot be obtained accurately by only replacing the amplitude of the object plane, and thus cannot provide a high-quality holographic display. Recently, the stochastic gradient descent (SGD) method [11,12] and its improved methods [13–15] have been proposed to optimize POHs, which have far better imaging quality than traditional iterative algorithms have. The SGD method sets the POH as an optimization variable and diffracts it to the object plane to obtain the reconstruction results. The loss function of the reconstructed field is calculated during the optimization process, and then the POH is updated by calculating the gradient of the loss function using automatic differentiation. To obtain high-quality reconstruction, the SGD method can be combined with the camera-in-the-loop method [16] to reduce the error between the ideal reconstruction and the optical reconstruction.
The SGD method significantly enhances the quality of single-plane holographic displays. To achieve a more immersive 3D display, multiple tomographic images need to be reconstructed at different depths in the same axial direction [17,18]. However, it brings the problem of inter-plane crosstalk in multi-plane reconstruction. Therefore, some improved methods of GS and SGD have been proposed to solve the inter-plane crosstalk problem. Velez-Zea et al. propose the sequential Gerchberg–Saxton (SGS) and global Gerchberg–Saxton (GGS) methods and combine them with a hybrid constraint strategy of amplitude in multi-plane reconstruction [19–22]. However, SGS and GGS methods tend to fall into local optimal solutions and cannot provide high-quality multi-plane reconstruction. Chen et al. improve the SGD method by using a complex loss function instead of an amplitude-only loss function, which greatly reduces the optimization time for holograms of complicated 3D objects [23]. However, this method only discusses 3D objects without overlapping parts in depth and does not consider 3D objects with overlapping parts in depth and the inter-plane crosstalk problem. To resolve the inter-plane crosstalk problem in multi-plane reconstruction, Wang et al. introduce the time-multiplexing strategy into the optimization process of the multi-plane SGD method to increase input information [24]. Since only the reconstructed amplitude is constrained in the time-multiplexing stochastic gradient descent (TM-SGD) method, the randomness of the reconstructed phase causes phase singularities and inter-plane crosstalk when the inter-plane interval decreases [25]. Phase singularities and inter-plane crosstalk cannot be effectively eliminated even if the number of optimizations is increased [26,27]. Therefore, as the inter-plane interval decreases or the number of planes increases, the problem of increasing inter-plane crosstalk and decreasing reconstruction quality is still not effectively solved.
In this paper, the double-constraint SGD (DC-SGD) method is proposed to solve the inter-plane crosstalk problem at a lower inter-plane interval. We constrain both the amplitude and phase of the signal region in the reconstructed plane to suppress inter-plane crosstalk. On the one hand, a mask is added to enhance the optimization of the signal region of the reconstructed amplitude, which can help to improve the reconstruction quality. On the other hand, the phase regularization strategy is used to reduce the randomness of the signal area of the reconstructed phase, which can eliminate contour information generated by other planes. Therefore, speckle noise and inter-plane crosstalk can be effectively suppressed. The experimental results confirm that our proposed method can effectively suppress the inter-plane crosstalk and improve the quality of the reconstructed planes at a lower inter-plane interval. Moreover, the optimization time of the DC-SGD is nearly 4 times faster than that of the TM-SGD method in 4-plane reconstruction.
2. Method
2.1 Multi-plane SGD and TM-SGD method
In the previous research [23,24], the effectiveness of the SGD method for multi-plane reconstruction has been demonstrated. The schematic diagram of the multi-plane SGD method is presented in Fig. 1. The multi-plane reconstructed amplitudes are obtained by setting different diffraction distances. The multi-plane SGD method calculates the loss value between the reconstructed amplitude and the target amplitude in each depth plane and sums them to get the total loss value. The loss function is shown in Eq. (1), $Amp_n$ and $Target_n$ are the reconstructed amplitude and target amplitude of the n-th plane, respectively. The initial POH is updated by calculating the gradient resulting from the loss function in each optimization step.
The mean square error (MSE) function is defined as the loss function of the multi-plane SGD method, which is expressed by:
where $R(m,n)$ and $T(m,n)$ denote the reconstructed image and the target image at pixel coordinates (m, n), respectively.To reduce the multi-plane reconstruction crosstalk, the TM-SGD optimization algorithm converts the quantitative conditions between the holograms and the object planes from one-to-many to many-to-many by using the time-multiplexing strategy. As shown in Fig. 2, the TM-SGD method controls the input-output relationship as N-to-N, the 4-plane reconstruction will optimally generate 4 sub-holograms. During the optimization process, the reconstructed amplitudes of each depth plane are synthesized from the reconstructed amplitudes of the individual sub-holograms. Therefore, the loss value for each depth plane is calculated from the synthesized amplitude and the target amplitude. Finally, the total loss value is obtained by summing the loss values of each depth plane, and the sub-holograms are updated by calculating the gradient of the total loss. Therefore, the TM-SGD method can obtain a crosstalk-free multi-plane reconstruction by sequentially displaying multiple sub-holograms. However, as the number of reconstructed planes increases and the inter-plane interval decreases, the total optimization time can be increased and the crosstalk-free effect would be weakened.
The numerical simulation and optical results of the 2-plane reconstructed by the TM-SGD method are shown in Fig. 3. The distances of plane 1 and plane 2 from SLM are 7cm and 11cm, respectively. As shown in Fig. 3(b), due to the use of the time-multiplexing strategy, the white pattern area in both planes reconstructed by the TM-SGD method is clear. However, because we choose a small inter-plane interval, the inter-plane crosstalk is not effectively suppressed. The presence of inter-plane crosstalk affects the reconstruction quality and the visual experience, which also causes speckle noise in the white pattern area. Therefore, when the inter-plane interval is small, although the TM-SGD method can improve the reconstruction quality in the white pattern region, the crosstalk in the black background region cannot be effectively suppressed.
Fig. 3. 2-plane numerical simulation results and optical reconstruction results of TM-SGD. Distances of plane 1 and plane 2 from SLM are 7 cm and 11 cm, respectively.
2.2 Proposed method
The inter-plane crosstalk is not only reflected in the reconstructed amplitude, but the reconstructed phase can better illustrate the effect of inter-plane crosstalk. Figure 4 shows the phase distribution of plane 1 and plane 2 reconstructed by the TM-SGD method. It can be clearly seen that the contour information of the pattern of plane 2 appears in the phase distribution of the black background area of plane 1, and the same is true for plane 2. In the optimization process of the multi-plane SGD method and TM-SGD method, the loss function only constrains the amplitude of the reconstruction, but not the phase of the reconstruction. Therefore, the randomness and inhomogeneity of the reconstructed phases lead to the interference of inter-plane phase information. On the one hand, the randomness of the unconstrained reconstructed phase will generate speckle noise in the reconstructed plane. On the other hand, the interference of the phase contour information will cause significant intensity fluctuations, and its resulting inhomogeneous phase will generate crosstalk in the reconstructed plane. In the TM-SGD method, even if the number of iterations is increased, the inter-plane phase interference is difficult to be eliminated by automatic optimization. In addition, the randomness of the reconstructed phase produces a wide range of phase singularities, which also generate speckle noise in the reconstructed field and affect the reconstruction quality [25].
To suppress the undesired phase of the current plane, we constrain the reconstructed phase of each plane to improve phase uniformity and reduce the randomness of the phase. The phase distribution of the reconstructed field can be effectively smoothed by using a constrained strategy of phase regularization, which will help suppress speckle noise in the reconstructed field [27,28]. Therefore, we propose a double-constraint SGD (DC-SGD) method to suppress inter-plane crosstalk and improve the reconstruction quality. The proposed method will constrain both the amplitude and phase of the signal region in the reconstructed plane during the optimization process. The schematic diagram of the proposed method is shown in Fig. 5. On the one hand, we use a mask to make the optimization process more focused on the signal region of the reconstruction plane. The red area in Fig. 5 is the signal domain, where the reconstructed amplitude and phase of the signal domain are the region of interest and are constrained. The gray area in Fig. 5 is the non-signal domain, where the reconstructed amplitude and phase of the non-signal domain are free. In the optimization process, introducing amplitude and phase freedom in the non-signal regions can help to improve the reconstruction quality in the signal regions. On the other hand, we add the constraint strategy of phase regularization to the optimization process. Using phase regularization can help to eliminate phase contour information caused by other planes and reduce the randomness of the reconstructed phase, thus suppressing inter-plane crosstalk and speckle noise.
The DC-SGD method employs the band-limited angular spectrum method (BLASM) as the diffraction calculation approach to diffract the POH into different depth planes separately to obtain the reconstructed image, as shown in Eq. (3):
where $U_0(x_0,y_0)$ and $U_1(x_1,y_1)$ denote the amplitude distribution of the input image and the complex amplitude distribution of the output plane, respectively. $FFT$ and $IFFT$ denote the fast Fourier transform and the inverse Fourier transform. $H_f(f_x,f_y)$ is the band-limited angular spectrum transfer function, the expression of $H_f(f_x,f_y)$ is provided in Eq. (4), where $k$ represents the wave number and is equal to $\frac {2\pi }{\lambda }$, $\lambda$ is the light wavelength. $z$ is the diffraction distance, and $f_x$, $f_y$ represent the spatial frequencies. $rect(\xi )$ is a rectangular function with width unity. $u_{limit}$ and $v_{limit}$ are bandwidth-limiting frequencies, the details of which can be found in [29].The total loss function is composed of both an amplitude constraint and a phase constraint in the signal region of the reconstruction plane, and it can be expressed as:
In the amplitude constraint $Loss_a$, $Amp_n$ and $Target_n$ are the reconstructed amplitude and target amplitude of the n-th plane, respectively. $mask\_sign_n$ is the binary mask of the signal region of the n-th plane, whose size is adjusted according to the signal region size of the plane. $s$ is an automatically optimizable scaling factor to adjust the energy difference between the reconstructed amplitude and the target amplitude. $\circ$ denotes element-by-element multiplication. $\vert \vert \cdot \vert \vert _2$ is to calculate the L2-norm between the reconstructed amplitude signal area and the target amplitude signal area at different depths separately and sum up to obtain the total $Loss_a$. In the phase constraint $Loss_P$, $Phase_n$ is the phase distribution of the n-th reconstructed plane. $\Delta$ is the Laplace operator that calculates the second derivative. $\Delta Phase_n$ is to calculate the second derivative of the reconstructed phase, which can measure the degree of fluctuation in the phase distribution. $\vert \vert \cdot \vert \vert _1$ is to calculate L1-norm of the second derivatives of the phase signal region for each reconstruction plane separately and sum up to obtain the total loss $Loss_P$. Thus phase regularization means reducing the degree of fluctuation and randomness of its phase during the optimization process, which helps to solve the inter-plane crosstalk problem. $\beta$ is a custom weight value to adjust the ratio between the loss of the reconstructed amplitude and the loss of the reconstructed phase. A larger $\beta$ indicates that the optimization process favors the reconstructed phase, and a smaller $\beta$ indicates that the optimization process favors the reconstructed amplitude. The mask $mask\_sign_n$ is used to distinguish the signal and non-signal domains in each plane. As shown in Eq. (6), the size of the signal area can be manually adjusted by setting the threshold value, where $Amp(m,n)$ is the normalized amplitude of the target complex amplitude at pixel coordinates (m, n).
Finally based on the new loss function, the DC-SGD method could automatically update the POH using the Adam optimizer during the optimization process [30]. The proposed method no longer requires the time-multiplexing strategy, so the optimization time is significantly shorter compared to the TM-SGD method, while the crosstalk problem between planes is effectively suppressed by simultaneously constraining the amplitude and phase of the reconstructed planes.
3. Result
3.1 Numerical simulation results
The numerical simulation and optical reconstruction results of the 2-plane by DC-SGD method are shown in Fig. 6. As shown in Fig. 6(a), it is evident that the crosstalk noise in the black background area of plane 1 and plane 2 is effectively suppressed. The MSE value in the yellow box area of plane 1 and plane 2 is reduced to 0.004. Compared with TM-SGD, the PSNR values of plane 1 and plane 2 are 2.97 dB and 7.68 dB higher, respectively. As shown in Fig. 6(b), optical reconstruction results are consistent with numerical simulation results, there is no crosstalk in the black background region of plane 1 and plane 2.
Fig. 6. 2-plane numerical simulation results and optical reconstruction results of DC-SGD. Distances of plane 1 and plane 2 from SLM are 7 cm and 11 cm, respectively.
To better compare the multi-plane SGD method, TM-SGD method, and DC-SGD method in multi-plane reconstruction, numerical simulation experiments are carried out with four planes. As shown in Fig. 7, the four planes A, B, C, and D are coaxial and set at different depths in the same Z-direction. The resolution and sampling interval of the POH is 1920$\times$1080 and 8 $\mu$m respectively. The resolution of the target image is 1600$\times$800. The wavelength of the light source in the simulation is set to 671 nm. The central processing unit (CPU) and graphics processing unit (GPU) used in the simulation are Intel Xeon E5-2650 v4 processor and NVIDIA TU102[TITAN RTX] with CUDA version 10.2, respectively. The experiments utilize PyTorch 1.12.1 and Python 3.8.8 to implement the optimization process. The number of optimizations for all three methods is set to 1000. The scale factor $\beta$ in the loss function is set to $10^{-6}$ to adjust the amplitude loss and phase loss to the same order of magnitude.
To more clearly and accurately compare the reconstruction quality of three different optimization methods, we use the peak signal-to-noise ratio (PSNR) as the evaluation index of reconstruction amplitude quality. The PSNR for the 8-bit gray-level image is calculated using the following equation:
The 4-plane simulation reconstruction results for three optimization methods are shown in Fig. 8, setting the distance between plane 1 and SLM to 5 cm and the inter-plane interval to 5 cm. As shown in Fig. 8(a), the reconstruction quality of the four planes reconstructed by multi-plane SGD is the worst, with a lot of speckle noise in the white letter region and more obvious inter-plane crosstalk in the black background region. As shown in Fig. 8(b), due to the increased input information of TM-SGD, the sharpness and contrast of the white letter area in the reconstruction plane are improved, but strong crosstalk still exists in the black background area. By contrast, it is obvious that the white letter area of the plane reconstructed by DC-SGD is more uniform and clear, and the strong crosstalk in the black background area of the reconstructed plane is effectively suppressed as shown in Fig. 8(c). Compared with the multi-plane SGD method, the PSNR of the four planes reconstructed by DC-SGD is improved by 9.52 dB, 3.65 dB, 5.56 dB, and 8.14 dB, respectively. Compared with the TM-SGD method, the PSNR of the four planes reconstructed by DC-SGD is improved by 8.55 dB, 3.94 dB, 6.1 dB, and 6.93 dB, respectively. The blue-boxed region and the yellow-boxed region are the areas where crosstalk may occur in the white pattern and the black background, and we select some of the reconstruction details to calculate their MSE values. We can find that the reconstructed blue-boxed area and yellow-boxed area by the DC-SGD method have the smallest MSE value, which is closest to the target area and the crosstalk noise is effectively suppressed.
Fig. 8. 4-plane numerical simulation results of (a) Multi-plane SGD, (b) TM-SGD, and (c) DC-SGD. Distance between plane 1 and SLM is 5 cm, and inter-plane interval is 5 cm.
The phase distributions of reconstructed plane 3 of three methods are shown in Fig. 9. It can be clearly seen that the phase distribution in Fig. 9(a) and Fig. 9(b) has clear contour information generated by other planes and the fluctuation of the phase is more undulating. In contrast, Fig. 9(c) shows that the constraint strategy of phase regularization can effectively eliminate contour information generated by other planes, which contributes to a more uniform phase distribution. Therefore, inter-plane crosstalk can be decreased effectively.
Fig. 9. Phase distribution of reconstructed plane 3 of (a) multi-plane SGD, (b) TM-SGD and (c) DC-SGD.
To compare the variation of reconstruction amplitudes more visually, Fig. 10 illustrates the variation of PSNR values with the number of iterations in each reconstruction plane for the three optimization methods. The PSNR curves of the four planes reconstructed by the multi-plane SGD method always converge fastest. The PSNR curves of the four planes reconstructed by TM-SGD start with a fast growth rate, but when the number of iterations reached 400, the growth rate gradually slowed down and the PSNR curves converged in the interval of 18 dB to 20 dB. However, the PSNR curves of our proposed method always maintain a high growth rate and only show a tendency to converge when the PSNR curves grow to 25 dB. Therefore, if the number of iterations continues to increase, the planes reconstructed by DC-SGD can further obtain higher reconstruction quality.
Fig. 10. PSNR of (a) plane1, (b) plane2, (c) plane3, (d) plane4 for multi-plane SGD, TM-SGD, and DC-SGD reconstructions with different number of iterations.
To more clearly compare the computation time of the three methods, we define the calculation time ratio between the DC-SGD method and the other methods. Figure 11(a) shows the variation in the computation time of the three optimization methods as the number of iterations increases. As the number of iterations increases, the computation time of all three optimization methods increases. Figure 11(b) and Table 1 illustrates the computation time ratio. Since the DC-SGD method does not apply a time-division multiplexing strategy, the $Ratio_{DM}$ value is approximately 1 and the $Ratio_{DT}$ value is approximately 4, the computation time of the DC-SGD is 4 times faster than that of the TM-SGD and is similar to that of the multi-plane SGD.
Table 1. Calculation times and time ratio
Based on the 4-plane binary image reconstruction, we then perform a comparison of the 4-plane grayscale image reconstruction. We choose four grayscale images as shown in Fig. 12(a), the reconstruction distance between plane 1 and SLM is 5 cm, and the inter-plane interval is 4 cm. Figure 12(b) shows that the four planes reconstructed by the multi-plane SGD method are influenced by strong inter-plane crosstalk, which causes speckle noise in the butterfly pattern of each plane and seriously affects the reconstruction quality. As shown in Fig. 12(c), the brightness and contrast of the four planes reconstructed by the TM-SGD method are improved because of the time-multiplexing strategy. However, the crosstalk noise in their black background region is still not effectively suppressed, and the crosstalk even increases in plane 2 and plane 3. By using the DC-SGD method, the crosstalk noise in the black background area of the four planes is effectively suppressed in Fig. 12(d), and the MSE values of the marked area are greatly reduced compared to the TM-SGD method. Due to the crosstalk noise suppression, the PSNR of the four reconstructed planes is improved by 8.08 dB, 2.92 dB, 2.76 dB, and 6.35 dB compared to TM-SGD. The simulation experiment shows that our method can also effectively suppress crosstalk in multi-plane grayscale images.
3.2 Optical experiments
In this section, we conduct optical experiments to compare the reconstruction results of the 4-plane binary and grayscale images for the three optimization methods. The distance between plane 1 and SLM is set to 5 cm, and the inter-plane interval is 4 cm. The optical experiment reconstruction system is illustrated in Fig. 13. The reflective phase-only 8-bit spatial light modulator (SLM) is used in the experiment, and the sampling interval and resolution of the SLM are 8 $\mu$m $\times$ 8 $\mu$m and 1920 $\times$ 1080. The frame rate and the phase modulation range of the SLM are 60 Hz and [0, 2$\pi$], respectively. A laser with a wavelength of 671 nm is employed as the light source in the experiment. The beam expander and collimating lens are used to expand and collimate the laser, thus illuminating the SLM. After loading the pre-calculated CGH on the SLM, we use the CCD (GS3-U3-23S6C-C) produced by Teledyne FLIR to capture the reconstruction results. The CCD can be connected to a computer to adjust the shooting parameters and control the shooting. Camera parameters such as gain, integration time, and gamma curve are kept constant in the experimental result shots. For the optical reconstruction of the TM-SGD method, we take 4 sub-holograms as a group and sequentially generate 75 groups totaling 300 holograms. Then the 300 holograms are made into a 5-second video with a frame rate of 60 fps. After loading the hologram video into the SLM and controlling the exposure time of the CCD to 66.667 ms, the reconstruction results of 4 sub-holograms can be obtained. Other than that, the exposure time is kept constant in all other experiments, while no post-treatment such as gain correction is done on the reconstructed images.
The results of the 4-plane binary image optical reconstruction are shown in Fig. 14. As shown in Fig. 14(a), since only the amplitude loss is constrained in the multi-plane SGD, obvious speckle noises appear in the letter region of the four planes. In the meantime, significant inter-plane crosstalk also appears in the black background region. Due to the increased input information of the TM-SGD method, the brightness and contrast of the letter area in the four reconstructed planes are improved in Fig. 14(b). However, the inter-plane crosstalk in the background area is not effectively suppressed and even causes speckle noise in the letter area. In contrast, Fig. 14(c) shows that the DC-SGD method significantly suppresses the crosstalk artifacts in the black background regions of the four planes by simultaneously constraining the amplitude and phase. The letter area of the reconstruction plane is also more uniform and smooth, but unlike the simulation results, a small amount of noise appears in the white letter area. Since the actual optical reconstruction system does not conform to the simulation environment and contains many disturbances, it cannot achieve the same level of accuracy as the simulated reconstruction. However, similar to the numerical simulation results, the optical experimental results can also be concluded that our proposed method can effectively suppress inter-plane crosstalk.
The optical reconstruction results of the 4-plane grayscale image are shown in Fig. 15. Figure 15(a) shows that the presence of extensive inter-plane crosstalk produces a lot of speckle noise in the four planes, which seriously affects the sharpness and reconstruction quality. As shown in Fig. 15(b), the TM-SGD method could improve the brightness and contrast of the reconstructed planes. However, the inter-plane crosstalk is also enhanced simultaneously, which severely affects the reconstruction details. Compared to the previous two results, the DC-SGD method effectively suppresses inter-plane crosstalk in the background region in Fig. 15(c). The suppression of inter-plane crosstalk effectively reduces speckle noise and improves the reconstruction quality. The optical experimental results agree with the simulation results showing that our method can effectively suppress the crosstalk between multi-plane grayscale images.
4. Discussion
In previous studies [24], the multi-plane reconstructed by TM-SGD all has an inter-plane interval greater than 10 cm, while in our proposed method, inter-plane crosstalk can be suppressed at a much lower inter-plane interval. Therefore, we performed the multi-plane reconstruction of the DC-SGD method at different inter-plane intervals as shown in Fig. 16. The numerical simulation experiment sets the distance between plane 1 and the SLM to 5 cm, and the inter-plane interval is set to 3 cm, 4 cm, and 5 cm respectively. We can see that as the inter-plane interval decreases, the black background area of four planes still maintains low plane crosstalk, and when the inter-plane interval is 3 cm, the maximum MSE value in the black background area of the yellow box is only 0.01. The reconstruction quality of the white letter area then decreases as the inter-plane interval decreases. When the inter-plane interval is 3 cm, speckle noise appears in the letter area of plane 2 and plane 3, while the reconstruction quality of the Letter area of plane 1 and plane 4 is relatively more uniform. To further validate the suppression of inter-plane crosstalk by the proposed method as the inter-plane interval decreases, we plot the curve of the MSE value of the dark region as the inter-plane interval varies. As shown in Fig. 17, when the inter-plane interval is greater than 3 cm, the MSE values in the dark region remain low and continue to decrease as the inter-plane interval increases. In contrast, when the inter-plane interval is less than 3 cm, the MSE value of the dark region increases dramatically with decreasing inter-plane interval. Therefore, compared to the TM-SGD method, the proposed method can effectively suppress the inter-plane crosstalk at a lower inter-plane interval.
Fig. 16. 4-plane numerical simulation reconstruction results of DC-SGD at inter-plane interval of (a) 3 cm, (b) 4 cm and (c) 5 cm.
5. Conclusion
In conclusion, the DC-SGD method is proposed to solve the inter-plane crosstalk problem at a lower inter-plane interval. The proposed method simultaneously constrains the amplitude and phase of the signal region in the reconstructed plane to suppress inter-plane crosstalk. A mask is added to make the optimization process more focused on the signal region of the reconstruction plane. Besides that, the phase regularization constraint strategy is applied to reduce the randomness of the reconstructed phase and suppresses inter-plane crosstalk. Finally, the POH can be updated by calculating the gradient resulting from the loss function in each optimization step. We analyze the 4-plane reconstruction results of binary maps and grayscale maps, numerical simulation and optical experiment results confirm that our proposed method can effectively suppress inter-plane crosstalk in multi-plane reconstruction at a smaller inter-plane interval compared to the multi-plane SGD and TM-SGD method. Since DC-SGD only needs to optimize one hologram, the optimization time of the proposed method is 4 times faster than that of the TM-SGD method. In the future, our proposed method could have wider applications in tomographic 3D visualization in the biomedical field where tomographic images need to be reconstructed.
Funding
National Natural Science Foundation of China (62275178); Chengdu Municipal Science and Technology Program (2022-GH02-00016-HZ); Open Fund of Jiangsu Engineering Research Center of Novel Optical Fiber Technology and Communication Network (SDGC2233).
Acknowledgments
The authors wish to thank the anonymous reviewers for their valuable suggestions.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
Data availability
Data underlying the results presented in this Letter are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. P. Yu, Y. Liu, Z. Wang, J. Liang, X. Liu, Y. Li, C. Qiu, and L. Gong, “Ultrahigh-density 3d holographic projection by scattering-assisted dynamic holography,” Optica 10(4), 481–490 (2023). [CrossRef]
2. A. H. Dorrah, P. Bordoloi, V. S. de Angelis, J. O. de Sarro, L. A. Ambrosio, M. Zamboni-Rached, and F. Capasso, “Light sheets for continuous-depth holography and three-dimensional volumetric displays,” Nat. Photonics 17(5), 427–434 (2023). [CrossRef]
3. D. Pi, J. Liu, and Y. Wang, “Review of computer-generated hologram algorithms for color dynamic holographic three-dimensional display,” Light: Sci. Appl. 11(1), 231 (2022). [CrossRef]
4. J. Xiong, E.-L. Hsiang, Z. He, T. Zhan, and S.-T. Wu, “Augmented reality and virtual reality displays: emerging technologies and future perspectives,” Light: Sci. Appl. 10(1), 216 (2021). [CrossRef]
5. C. Chang, K. Bang, G. Wetzstein, B. Lee, and L. Gao, “Toward the next-generation vr/ar optics: a review of holographic near-eye displays from a human-centric perspective,” Optica 7(11), 1563–1578 (2020). [CrossRef]
6. L. Shi, B. Li, and W. Matusik, “End-to-end learning of 3d phase-only holograms for holographic display,” Light: Sci. Appl. 11(1), 247 (2022). [CrossRef]
7. F. Wang, T. Ito, and T. Shimobaba, “High-speed rendering pipeline for polygon-based holograms,” Photonics Res. 11(2), 313–328 (2023). [CrossRef]
8. L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, “Towards real-time photorealistic 3d holography with deep neural networks,” Nature 591(7849), 234–239 (2021). [CrossRef]
9. R. Gerchberg and W. Saxton, “Practical algorithm for determination of phase from image and diffraction plane pictures,” Optik 35(2), 237–246 (1972).
10. L. Chen, H. Zhang, Z. He, X. Wang, L. Cao, and G. Jin, “Weighted constraint iterative algorithm for phase hologram generation,” Appl. Sci. Basel 10(10), 3652 (2020). [CrossRef]
11. Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, “Neural holography with camera-in-the-loop training,” ACM Trans. Graph. 39(6), 1–14 (2020). [CrossRef]
12. S. Choi, J. Kim, Y. Peng, and G. Wetzstein, “Optimizing image quality for holographic near-eye displays with michelson holography,” Optica 8(2), 143–146 (2021). [CrossRef]
13. J. Zhang, N. Pegard, J. Zhong, H. Adesnik, and L. Waller, “3d computer-generated holography by non-convex optimization,” Optica 4(10), 1306–1313 (2017). [CrossRef]
14. P. Chakravarthula, Y. Peng, J. Kollin, H. Fuchs, and F. Heide, “Wirtinger holography for near-eye displays,” ACM Trans. Graph. 38(6), 1–13 (2019). [CrossRef]
15. G. Kuo, L. Waller, R. Ng, and A. Maimone, “High resolution etendue expansion for holographic displays,” ACM Trans. Graph. 39(4), 1 (2020). [CrossRef]
16. P. Chakravarthula, E. Tseng, T. Srivastava, H. Fuchs, and F. Heide, “Learned hardware-in-the-loop phase retrieval for holographic near-eye displays,” ACM Trans. Graph. 39(6), 1–18 (2020). [CrossRef]
17. G. Makey, Ö. Yavuz, D. K. Kesim, A. Turnalı, P. Elahi, S. Ilday, O. Tokel, and F. Ö. Ilday, “Breaking crosstalk limits to dynamic holography using orthogonality of high-dimensional random vectors,” Nat. Photonics 13(4), 251–256 (2019). [CrossRef]
18. J. Neumüller, “Electron tomography?a tool for ultrastructural 3d visualization in cell biology and histology,” Wien. Med. Wochenschr. 168(11-12), 322–329 (2018). [CrossRef]
19. A. Velez-Zea, J. Fredy Barrera-Ramirez, and R. Torroba, “Improved phase hologram generation of multiple 3d objects,” Appl. Opt. 61(11), 3230–3239 (2022). [CrossRef]
20. A. Velez-Zea, J. F. Barrera-Ramirez, and R. Torroba, “Alternative constraints for improved multiplane hologram generation,” Appl. Opt. 61(5), B8–B16 (2022). [CrossRef]
21. A. Velez-Zea and R. Torroba, “Mixed constraint in global and sequential hologram generation,” Appl. Opt. 60(7), 1888–1895 (2021). [CrossRef]
22. A. Velez-Zea, “Iterative multiplane hologram generation with mixed constraint,” Appl. Opt. 60(2), 224–231 (2021). [CrossRef]
23. C. Chen, B. Lee, N.-N. Li, M. Chae, D. Wang, Q.-H. Wang, and B. Lee, “Multi-depth hologram generation using stochastic gradient descent algorithm with complex loss function,” Opt. Express 29(10), 15089–15103 (2021). [CrossRef]
24. Z. Wang, T. Chen, Q. Chen, K. Tu, Q. Feng, G. Lv, A. Wang, and H. Ming, “Reducing crosstalk of a multi-plane holographic display by the time-multiplexing stochastic gradient descent,” Opt. Express 31(5), 7413–7424 (2023). [CrossRef]
25. H. Aagedal, M. Schmid, S. Teiwes, and F. Wyrowski, “Theory of speckles in diffractive optics and its application to beam shaping,” J. Mod. Opt. 43(7), 1409–1421 (1996). Workshop on Diffractive Optics, PRAGUE, CZECH REPUBLIC, AUG, 1995. [CrossRef]
26. P. Senthilkumaran, F. Wyrowski, and H. Schimmel, “Vortex stagnation problem in iterative fourier transform algorithms,” Opt. Lasers Eng. 43(1), 43–56 (2005). [CrossRef]
27. L. Chen, R. Zhu, and H. Zhang, “Speckle-free compact holographic near-eye display using camera-in-the-loop optimization with phase constraint,” Opt. Express 30(26), 46649–46665 (2022). [CrossRef]
28. S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, “Neural 3d holography: Learning accurate wave propagation models for 3d holographic virtual and augmented reality displays,” ACM Trans. Graph. 40(6), 1–12 (2021). [CrossRef]
29. K. Matsushima and T. Shimobaba, “Band-limited angular spectrum method for numerical simulation of free-space propagation in far and near fields,” Opt. Express 17(22), 19662–19673 (2009). [CrossRef]
30. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv, arXiv:1412.6980 (2014). [CrossRef]
