Recent progress in near-eye displays aims to achieve a photorealistic image over a wide field of view for an immersive user experience [1,2]. Although a high numerical aperture eyepiece is necessary for a wide field of view, the optical aberration from the eyepiece gets severe as the numerical aperture increases. Spatially varying (SV) aberrations from non-ideal off-axis behavior of the high numerical aperture eyepiece hinder the realization of high-quality images over the wide field of view in near-eye displays. In this respect, holographic near-eye displays have been proposed, showing their capabilities in correcting optical aberrations [3–5]. Since the conventional point-wise integration method for SV aberration correction requires an intractable computational load, approximation methods such as spatially invariant (SI) correction, piece-wise correction, iterative optimization, and the deep learning-based method have been proposed [4–9]. However, a fast SV correction method with high reliability has not been introduced.

Here, we propose a low-rank approximation method that accelerates the calculation of a computer-generated hologram (CGH) for SV aberration correction. The proposed low-rank approximation method decomposes the sub-hologram into a small number of modes, approximating the point-wise integration as a summation of a few convolutions between the target image and the modes, as shown in Fig. 1. In addition, we present two sub-hologram decomposition methods: the Karhunen–Loeve transform (KLT) and a uniform sub-hologram decomposition (USHD). The KLT is a decomposition method widely used in computer vision and computational imaging [10–12]. We use the KLT as a baseline method since it offers an optimum solution with respect to the mean-squared error (MSE) of the sub-hologram reconstruction. Furthermore, we propose the USHD as a novel decomposition method that optimizes the modes to have identical reconstruction errors at every target point. The USHD prevents the mode from being biased to the frequent sub-holograms and delivers the image with uniform quality over the entire field of view. We experimentally demonstrate the proposed method with a bench-top holographic display and analyze the results with the image contrast and the computation time.

 

Fig. 1. Proposed low-rank approximation method. The phase values of the sub-holograms, modes, weights, and CGH are used for the visualization. (a) SV point spread functions and corresponding sub-holograms. (b) Sub-holograms decomposed into a small number of modes and weights. (c) CGH for SV correction obtained by a series of Hadamard products and convolutions.

Download Full Size | PDF

SV aberration correction in holographic displays is performed by finding the phase profile of the sub-hologram that reconstructs a sharp point in the space. The point-wise integration method sums up all the sub-holograms corresponding to every point of the target [4]. In previous research, the phase profile of the sub-hologram is expressed with Zernike polynomials that are obtained with a manual calibration or optical simulation [5,6]. Therefore, a complex amplitude of CGH $u_{\textrm {sv}}$ with a target amplitude $a_{\textrm {target}}$ is expressed as

SI correction uses a single sub-hologram for every target point and calculates the CGH with a convolution, neglecting the SV characteristic of the sub-holograms. Since the convolution can be replaced with a fast Fourier transform, the time complexity of the SI correction is $O\left (N_x N_y \log \left (N_x N_y\right )\right )$, with an assumption of $N_x, N_y > M_x, M_y$. Although the SI correction significantly shortens the computation time, it only corrects the region near the point where the sub-hologram is selected.

The proposed low-rank approximation method decomposes the SV sub-holograms into a small number of modes with corresponding weights. Therefore, we can estimate the sub-hologram $h_{\tilde {x},\tilde {y}}$ as a weighted sum of the modes,

A straightforward way for the low-rank decomposition is to minimize the total MSE of the reconstructed sub-holograms as

The USHD resolves this issue by forcing the reconstruction error to be identical regardless of the sub-hologram position. We optimize the modes to have identical reconstruction error $\bar {e}_{\textrm {KLT}}$ using the objective function

We demonstrate the proposed method with a bench-top holographic display, as shown in Fig. 2(a). The display system consists of a laser diode (Thorlabs, CPS635F), a spatial light modulator (Thorlabs, EXULUS-HD1), a linear polarizer LP, collimating lens L1, and a 4-$f$ system L2–L3. High-order and DC noises are eliminated with the spatial filter in the 4-$f$ system. In addition, we adopt a tilted and shifted lens L4 with a 30-mm focal length to introduce SV aberrations. The captured image of point spread functions without aberration correction is shown in Fig. 2(b). The phase profile of the sub-hologram for SV correction is measured with a manual calibration. The sub-hologram with a diameter of 450 pixels is expressed with six Zernike polynomials: 4th, 5th, 6th, 7th, 8th, and 11th in the Noll index [15]. Both KLT and USHD algorithms are implemented with PyTorch, and the CGH generation part is implemented with CUDA C on a Nvidia RTX 3080Ti GPU. Furthermore, the amplitude discard method is used to encode phase-only CGH from the complex amplitude. In the USHD, we optimize the modes for 50,000 iterations with 150 batches. The stochastic gradient descent is performed with an ADAM optimizer in PyTorch with a learning rate of 0.0001.

 

Fig. 2. (a) Diagram of the bench-top holographic display. SLM, spatial light modulator; L1, collimating lens; LP, linear polarizer; L2–L3, 4-$f$ filtering system; L4, tilted and shifted lens to introduce SV aberration. (b) A captured image of point spread functions without aberration correction.

Download Full Size | PDF

Figure 3 shows experimental results of the various aberration correction methods in our holographic display setup. We digitally combined the captured images reconstructed from 25 CGHs with different random phases to reduce the speckle noise and emphasize the impact of the aberration in the image. Although the runtime of the SI correction is the shortest, 0.02 seconds, it suffers from image quality degradation since the aberration still remains even after the correction. In contrast, the point-wise integration method has the longest runtime, 169 seconds, with the best image quality. The proposed low-rank approximation method shows the results that are in between the SI correction and the point-wise integration method. The runtime of the low-rank approximation methods is 0.4 seconds, which is approximately 420 times faster than the point-wise integration method, while the results are comparable. However, we can see from the result that KLT still shows image quality degradation in the peripheral region. The USHD with the identical number of modes successfully corrects the aberration in the peripheral region with a slight sacrifice of the image quality in the center.

 

Fig. 3. Experimentally captured results with various aberration correction methods. The numbers in the parentheses indicate computation time for a single CGH with corresponding methods. The results of SI correction demonstrate that our system has severe SV aberration, which is corrected by the point-wise integration method. We can see the presence of the aberration in the periphery of the KLT results while the USHD with the same number of modes corrects both central and peripheral aberration successfully.

Download Full Size | PDF

We compare the modulation transfer function (MTF) of the KLT and the USHD in Fig. 4 to quantitatively show that the USHD prevents the biasing of the modes. The MTF is obtained by the intensity of 200$\times$200 holographic images of binary gratings, cropped at the center and the bottom right. From the maximum intensity $I_{\textrm {max}}$ and minimum intensity $I_{\textrm {min}}$ of the cropped images, the contrast is calculated by $\left (I_{\textrm {max}}-I_{\textrm {min}}\right )/\left (I_{\textrm {max}}+I_{\textrm {min}}\right )$. Figure 4 shows the MTF curves at the center and the periphery with the various aberration correction methods. Both KLT and USHD show higher image contrast compared to the SI correction in the periphery. However, the MTF of KLT at the center and the periphery largely differ because of the mode biasing, as we discussed before. In contrast, USHD shows similar MTF curves for both the center and periphery, placed between two MTF curves of the KLT. We note here that the overall low image contrast is due to the lack of an optimization process for phase-only encoding during the CGH generation. It can be further improved by combining the proposed method with conventional CGH optimization methods [16,17].

 

Fig. 4. MTF curves measured at the center and the periphery of the image with the various aberration correction methods. The number of modes used in KLT and USHD is both 20.

Download Full Size | PDF

In the low-rank approximation method, the image quality increases and the calculation gets slower as the number of modes increases, and vice versa. The experimental result of the low-rank approximation method with a different number of modes is shown in Fig. 5. Both KLT and USHD correct the aberration better when more modes are used, while the runtime increases linearly with an increasing number of modes. MTF curves in Fig. 6(a), drawn with KLT and USHD with a different number of modes in the periphery, also show that the increased number of modes leads to a higher image contrast in both methods. Figure 6(b) shows the computation time of CGH versus the size of the target image, measured on the GPU. The diameter of the sub-hologram is 450 pixels, the same as in the experiment, and the width and the height of the target image are identical. Solid lines in the graph indicate the fitted curve with the corresponding time complexity; $O\left (N_x N_y M_x M_y\right )$ for the point-wise integration and $O\left (k N_x N_y \log \left (N_x N_y\right )\right )$ for the others. We can see that the computation time of the low-rank approximation method is linearly proportional to the number of modes and far shorter than the point-wise integration method.

 

Fig. 5. Experimentally captured holographic images with a different number of modes in KLT and USHD. CGH calculation with a large number of modes corrects the aberration better in both methods. The KLT results only get better in the periphery since the central region is already corrected with only 10 modes. USHD results show image quality improvement at both the center and periphery. The numbers in the parentheses indicate the computation time for a single CGH.

Download Full Size | PDF

 

Fig. 6. (a) MTF curves of KLT and USHD with 10 and 50 modes, all in the periphery. The contrast gets higher as the number of modes increases. (b) Computation time versus the size of the target image, measured on the GPU. The solid lines represent fitted curves with corresponding time complexity. The low-rank approximation significantly reduces the computation time compared to the point-wise integration method.

Download Full Size | PDF

In this work, we present the low-rank approximation method for SV aberration correction in holographic displays, which accelerates the conventional point-wise integration method by 420 times without noticeable artifacts in the image. Also, the proposed USHD method provides an additional option of improving the image quality in the periphery with a slight sacrifice in the center, compared to the KLT. Although we demonstrate our method with a monochromatic holographic display, our method can be adopted to a full-color display setup easily by treating each color channel separately. In addition, our results suffer from low contrast due to the amplitude discard method used in the phase-only encoding. Future works could improve the contrast of the image by plugging our method into iterative CGH optimization methods and replacing the wavefront propagation part during the CGH generation.