1. Introduction

Holographic three-dimensional (3D) displays have been evolving in the past few decades. A key device of holographic 3D displays is a spatial light modulator (SLM) which modulates light signal, generating the complex field of target 3D images. Digital micro-mirror device (DMD) is one of the most frequently used SLMs [1–5]. High refresh rate of the DMDs enable modulation of the light with high space-bandwidth product in a unit time, which makes the DMD attractive in time-multiplexing approaches. The modulation depth of the DMD, however, is only limited to binary, i.e. 1 bit. Therefore, the complex optical field of the 3D images must be encoded to the binary amplitude signal before it is loaded to the DMD.

Binarization with simple thresholding causes a large error between the original signal and the binarized one. In order to overcome this problem, various techniques have been proposed. One approach is a super-pixel technique. Multiple DMD pixels are illuminated with different initial phases to achieve the amplitude and phase modulation simultaneously [6]. This technique, however, has a limitation that it decreases spatial resolution by assigning multiple DMD pixels to a single signal data point of the complex field. Preconditioning the target image for better reconstruction quality of the binary hologram has also been proposed. W.K. Cheung et al. proposed generation of binary Fresnel hologram based on grid-cross down-sampling of the target image [7]. This technique was then further enhanced by P.W.M. Tsang et al. by using shape adaptive sampling which considers the geometry of the target 3D objects [8]. These approaches, however, generally do not preserve the original target image quality due to the down-sampling process. W.-B. Cao et al. proposed a discrete wavelet transform based preconditioning of the target image [9]. They strengthen the high frequency component of the target image to enhance reconstruction quality of the binary hologram. M. chilipala et al. proposed a histogram-based binarization technique for DMD with time-multiplexing [10]. The target image is divided into multiple sub-images based on the histogram which are then processed for the hologram generation and binarization separately. The reconstruction quality enhancement performance, however, could be limited because these techniques modify only the target images while the binarization happens for their complex holograms. There are also thresholding strategies including various global and local thresholding techniques [11,12]. But carrier wave used in the hologram synthesis and the resultant angular spectrum of the hologram complex field are not explicitly considered in those techniques, preventing systematic complex field synthesis and binarization of the hologram.

Error diffusion (ED), first suggested as an image halftoning technique by R. W. Floyd and L. Steinberg [13], is another approach of signal binarization. ED technique decreases the error between the original signal and binarized one by diffusing the error to neighboring pixels with corresponding weights. In their original proposal, Floyd and Steinberg empirically determined the diffusion weights based on a subjective test on the image halftoning.

The application of the ED technique to the hologram binarization has been studied since R. Hauck and O. Bryngdahl’s initial work [14]. Y. Matsumoto and Y. Takaki used the ED technique along with the look-up table to enhance the gray scale reconstruction in their holographic display [15]. P.W.M. Tsang and T.-C. Poon proposed a bi-directional ED technique for better reconstruction of the phase hologram [16]. S. Jiao et al. used the ED technique with the DMD super-pixel method [17]. These previous works, however, simply use the traditional Floyd and Steinberg’s weight coefficient without further optimization for the hologram. Recently, G. Yang et al. optimized the weight coefficients using a GA algorithm [18], but it requires the optimization for each hologram, which is time-consuming.

In another approach, theoretical study on the ED weights and their relationship with the hologram reconstruction has been conducted [19]. Based on the angular spectrum domain analysis, S. Weissbach and F. Wyrowski presented a systematic method to design the weights, which results in high quality images in Fourier holograms. F. Fetthauer et al. further expanded the study to Fresnel holograms [20]. They, however, were limited to two-dimensional (2D) image reconstructions in a fixed plane. Moreover, they sacrifice the image size in the reconstruction plane below the maximum size supported by the hologram or the SLM.

In this paper, we propose a novel ED hologram binarization technique considering the hologram carrier wave. The proposed technique designs the ED weights such that the binarization error is suppressed around the hologram carrier wave in the angular spectrum domain where most signal energy is concentrated. Since the proposed technique forms the error suppression area not in the reconstruction plane but in the angular spectrum domain, the proposed technique makes the full use of the reconstruction size and can be applied to 3D images. In the following sections, we briefly review the ED theory and explain the proposed method with experimental results, demonstrating holographic 3D image displays with enhanced quality.

2. Theory

2.1 Brief review of ED algorithm

Figure 1(a) shows the concept of the ED algorithm. The ED algorithm sequentially binarizes each data pixel of the signal. The binarization error at a current pixel is diffused to its not-yet-processed neighboring pixels and updates them with the corresponding weights. The updated pixel in the next position is then processed while diffusing the binarization error again to its neighbors. This process is repeated until the final pixel is reached.

 

Fig. 1. Illustration of ED algorithm. (a) Sequential processing and (b) the error-diffused pixel.

Download Full Size | PDF

Suppose f(x) is an original signal where x=(x,y) stands for the pixel position as defined in Fig. 1. Denoting pixel pitch along the x and y axes as p_x and p_y, respectively, a discrete signal f(x) is given at x=(x,y)=(m_xp_x, m_yp_y) for integer m_x and m_y. Note that in the coordinate definition used in this paper, the sequential ED processing is conducted from -x to + x direction and from + y to -y direction. The ED algorithm generates the binary signal g_c(x) by taking pixelwise binarization operator B[·] to the error diffused signal f_c(x), i.e. g_c(x)=B[f_c(x)]. As shown in Fig. 1(b), the error diffused signal f_c(x) used for the binarization can be written by

The effect of the ED weight w(r) to the final binarized signal g_c(x) can be understood in the angular spectrum domain [19]. Equation (1) can be rearranged to

2.2 Proposed method

The proposed technique uses the bandpass feature of the ED algorithm, considering the carrier wave of the hologram of 3D objects. The proposed technique consists of a few steps. First, we synthesize a hologram or the complex field with a plane carrier wave instead of random phase carrier wave. Second, the synthesized complex field is binarized using the ED algorithm with the diffusion weight w(r) designed to make W(u) = 1 or H(u) = 0 around the spatial frequency of the carrier wave. Third, in the optical reconstruction, a physical aperture is used in the Fourier plane of the 4-f system to pass only the spatial frequency band around the hologram carrier wave. Figure 2 shows the overall steps of the proposed technique.

 

Fig. 2. Proposed method.

Download Full Size | PDF

The first step of the proposed method is the synthesis of the hologram with a plane carrier wave. If an object is considered as a set of points, the hologram with the plane carrier wave can be written as

In the simulations and experiments of this paper, we use multi-layer objects instead of point cloud objects for simplicity. Denoting each two-dimensional object layer amplitude image at a depth z_l as L_l(x,y;z_l), the object is represented by

In the simulations and experiments, we use two layers (l=1, 2), giving different depths to left and right half of the scene.

In holographic displays, a random phase carrier wave is frequently used in the hologram synthesis. The random phase distribution assigned to a target amplitude 3D object expands the angular spectrum bandwidth of the signal, making better use of either SLM area in Fourier hologram geometry or viewing angle in Fresnel hologram geometry. However, the random phases in the reconstructed 3D image interfere in the observing system, i.e. eye, creating the speckle noise. The use of the plane carrier wave is effective in suppressing the speckle noise in the reconstruction [22]. Since the plane carrier wave gives linear phase distribution on the surface of the reconstructed 3D images, the local integration by the observing system over the reconstructed image does not generate unwanted intensity fluctuation, suppressing the speckle noise. Using the single plane carrier wave, however, has a limitation that the depth of focus (DOF) of the reconstruction is large because of the narrow spatial frequency bandwidth of the reconstructed complex field. Therefore, in the proposed method, we scan the direction of the plane carrier wave, i.e. (u_cx, u_cy), using the time-multiplexing. By temporal accumulation, larger spatial frequency band is covered and the DOF is reduced while suppressing the speckle noise.

The second step is the application of the ED algorithm with a designed diffusion weight. To obtain the desired diffusion weight, we first design the weight such that the low noise band, i.e. W(u) = 1 or H(u) = 0 is formed around the zero spatial frequency area, i.e. u=0 as the conventional ED algorithms. We then shift the low noise band to where the hologram carrier wave is located. The final diffusion weight w_shift(r) after the shift is obtained by

The designed diffusion weight w_shift(r_x,r_y) is applied to the ED binarization of the hologram f_complex(x) synthesized in the previous step. Following the single side band (SSB) encoding procedure, the real part of the synthesized hologram complex field is taken, and adjusted to be within [0, 1] range by adding a bias and applying the normalization. The resultant normalized real part f(x) is then binarized by the ED algorithm with the weight w_shift(r_x,r_y), giving the binary hologram g_c(x). Note that the diffusion weight w_shift(r_x,r_y) obtained by Eq. (9) is complex-valued. Although the original signal before the ED algorithm f(x) is real in our work, the error diffused signal f_c(x) becomes complex due to the complex valued weight w_shift(r_x,r_y). Therefore, we use a complex-to-binary operator B[f_c(x)] which gives 1 or 0 by comparing the real part of the complexed valued f_c(x) with a threshold 1/2.

The final step of the proposed method is the optical reconstruction. The binarized hologram g_c(x) is loaded to the DMD and reconstructed. We use the 4-f system with an aperture in the Fourier plane to pass only the spatial frequency band where the binarization noise is suppressed by the ED algorithm.

3. Numerical simulation

Simulations are conducted to verify the reconstruction quality enhancement by the proposed ED technique. Figure 3 shows the target image and its hologram used in the simulation. Two cat images shown in Fig. 3(a) are used as the objects located at 3 cm and 1 cm distances from the hologram plane, respectively. Figure 3(b) and 3(c) show the amplitude and phase of the hologram synthesized with a plane carrier wave. The hologram has 1920 × 1080 resolution with 7.6 µm pixel pitch and the wavelength is 532 nm. The spatial frequency of the plane carrier wave is (u_cx, u_cy)=(0, 5.27 × 10⁴) m⁻¹ which corresponds to 1.61 degree in y-axis from the hologram normal direction. Figure 3(d) shows the amplitude of the angular spectrum of the hologram. It can be observed that the most signal energy is concentrated around the spatial frequency of the plane carrier wave which is indicated by red circle in Fig. 3(d).

 

Fig. 3. (a) Target image with two depths. (b) Amplitude, (c) phase, and (d) angular spectrum of the synthesized complex field hologram with a single plane carrier wave. (in (d) the contrast is adjusted for better visibility).

Download Full Size | PDF

First, we verify the binarization noise suppression by the proposed ED algorithm. We use the ED weight w_shift(r) calculated by Eq. (9) with the w_{initial_vertical}(r_x,r_y) and (u_cx, u_cy)=(0, 5.27 × 10⁴) m⁻¹. Note that because u_cx=0 the w_shift(r) is still 1 for r=(r_x,r_y)=(p_x,0) and 0 otherwise, which is the same as the w_{initial_vertical}(r_x,r_y). This diffusion weight suppresses the noise along the low horizontal spatial frequency band as shown in Fig. 4. Note that the spatial frequency (u_cx,u_cy) of the carrier wave used in the hologram shown in Fig. 3(d) is included in the error suppression band of this ED weight.

 

Fig. 4. (a) |H(u)|=|W(u)−1| of the w_shift(r) used in the first simulation. Blue region means noise suppression band. (b), (c), and (d) are the |H(u)|, ∠W(u), and |W(u)|, respectively, at the cross-section along the dashed line A in (a).

Download Full Size | PDF

Figure 5 shows the angular spectrums of the original complex field, its real part after SSB filtering, amplitude hologram binarized by the simple thresholding, and the amplitude hologram binarized by the ED algorithm. It can be observed that in the case of the simple thresholding binarization, the angular spectrum includes the high order terms which are located at integer multiples of the spatial frequency of the carrier wave [23]. To the contrary, in the case of the ED algorithm, those unintended high order terms are suppressed leaving only the desired signal and its conjugate within the noise suppression vertical band. Note that the DC and the conjugate terms are un-avoidable as the binary amplitude hologram is a real-valued function. In the optical reconstruction, those DC and the conjugate terms are masked by the 4-f system to give clear images.

 

Fig. 5. The simulated results of proposed method in angular spectrum domain.

Download Full Size | PDF

Figure 6 shows the simulated reconstructions. In all cases, we assume that a rectangular aperture is used in the 4-f system to pass the spatial frequency range only around the carrier wave as indicated by the red dot and the black rectangle in the bottom-left inset of each figure. Figures 6(a) and 6(f) show the reconstructions of the original complex and continuous-valued field synthesized using a single plane carrier wave. Since the original complex field is used without any modifications, the reconstruction is ideal and clean. Figures 6(b)–6(e) and 6(g)–6(j) are the reconstructions of the binary amplitude holograms shown in Figs. 6(k)–6(n). When the random phase carrier wave is applied in the hologram synthesis and the resultant complex field is binarized by the simple thresholding, severe speckle noise is observed as shown in Fig. 6(b) which adds to the noise caused by the binarization and degrades the reconstruction quality significantly. When the single plane carrier wave is applied in the hologram synthesis but still the simple thresholding is used in the binarization, the speckle noise is much suppressed as shown in Fig. 6(c), but the gray scale is not reconstructed properly and the ripple artefacts are observed due to the noise caused by the simple thresholding binarization. When the ED binarization is used to the hologram synthesized with a single plane carrier wave, we generally have enhanced reconstruction quality as shown in Figs. 6(d) and 6(e). But when the conventional Floyd & Steinberg’s ED weights are used without the shift, i.e. w_shift(r)=w_{initial_FS}(r), the reconstruction quality is degraded as the spatial frequency of the plane carrier wave is deviated from the noise suppression band of the w_shift(r)=w_{initial_FS}(r). By locating the noise suppression band to the spatial frequency of the plane carrier wave as shown in Fig. 6(e), the best reconstruction quality is achieved comparable to the original complex field shown in Fig. 6(a). The weight used in Fig. 6(e) is w_shift(r)=w_{initial_vertical}(r) = 1 for r=(r_x,r_y)=(p_x,0). For quantitative evaluation, the peak signal noise ratio (PSNR) of the focused reconstructions of the binary holograms is also calculated by comparing them with the focused reconstructions of the complex continuous-valued hologram. For the comparison between the focused reconstructions, the left half of the reconstructions in the upper row of Figs. 6(a)–(e) and the right half in the lower row of Figs. 6(a)–(e) are used for the PSNR calculation. The calculated PSNR values agree well with the subjective evaluation, indicating the highest values 28.9 dB and 30.8 dB for Fig. 6(e).

 

Fig. 6. Simulated reconstructions of (a) complex field hologram with a single plane carrier wave, (b) binary hologram with random phase carrier wave with simple threholding binarization, (c) single plane carrier wave with simple threholding binarization, (d) single plane carrier wave with ED binarization using Floyd&Steinberg’s filter, and (e) the single plane carrier wave with ED binarization using the vertical error suppression band. (f)–(j) are magnified parts of the red boxed region of (a)-(e). In each figure, the reconstructions are focused at the left cat in the upper row and at the right cat in the lower row. (k)-(n) are the binarized holograms used for (b)-(e). The PSNR is calculated for the focused parts which are the left (in the upper row of (a)-(e)) or the right half region (in the lower row of (a)-(e)) of the reconstructions.

Download Full Size | PDF

The reconstruction results shown in Fig. 6 confirm the quality enhancement by using the single plane carrier wave and by locating the error suppression band around its spatial frequency. The DOF, however, is not sufficiently shallow. The amount of the blur of the defocused cat object is small and the difference between the focused cat and the defocused cat is not clear as revealed in Figs. 6(f)–6(j). This long DOF is the consequence of the use of the single plane carrier wave.

In order to achieve the high-quality reconstruction and the shallow DOF at the same time, the proposed method uses the time-multiplexing. The spatial frequency of the single plane carrier wave scans over frames within the noise suppression band of the ED weights, achieving the shallow DOF while keeping the high reconstruction quality. In order to reserve large noise suppression band in the SSB area for the carrier wave scanning, the horizontal noise suppression filter shifted in the upper SSB area is designed as shown in Fig. 7. The initial w_initial(r) and the designed ED weight w_shift(r) are w_{initial_horizontal}(r) = 1 and w_shift(r)=−0.0043-j1.0 for r=(r_x,r_y)=(0,-p_y) and 0 otherwise. The spatial frequency along the y direction u_cy is fixed at 3.3 × 10⁴ m⁻¹ for all plane carrier waves in Fig. 7 which is well inside the noise suppression band of w_shift(r).

 

Fig. 7. |H(u)|=|W(u)−1| of the designed ED weight w_shift(r) having shifted horizontal noise suppression band with indicating (a) 3 carrier waves and (b) 11 carrier waves. Blue region means the noise suppression band and the spatial frequency of the carrier waves are indicated by the red dots. (c)-(e) are the |H(u)|, ∠W(u), and |W(u)|, respectively, at the cross-section along the dashed line B in (b).

Download Full Size | PDF

Figure 8 shows the simulated reconstructions with time-multiplexing. The spatial freqeuncies of the plane carrier waves accuumulated over time-muliplexing frames are indicated as the red dots in the left-top inset of each reconstruction. Figure 8(a) shows two reconstructions focused at left or right cat, respectively, when 3 plane carrier waves are used. It is observed that the effect of the defocus is now clear, and the defocused cat is cearly distinguished from the focused one, revealing shallow DOF. It is also observed that the defocused cat is separated into 3 slightly defocused components each of which corresponds to the single plane carrier wave. By increasing the number of the plane carrier waves, for instance 11 as shown in Fig. 8(b), the natural and continuous blur can be achieved. The PSNR values calculated by comparing the focused parts in Fig. 8 with the corresponding parts of the complex field reconstruction in Fig. 6(a) also show the enhanced quality.

 

Fig. 8. Simulated time-multiplexing reconstruction results of the binary holograms with the designed ED weight w_shift(r) having shifted horizontal noise suppression band. (a) 3 carrier waves and (b) 11 carrier waves. The PSNR is calculated for the focused parts which are the left (in the upper row of (a),(b)) or the right half region (in the lower row of (a),(b)) of the reconstructions.

Download Full Size | PDF

4. Experimental results

Figure 9 shows the experimental setup. The DMD used in the experiment is DLPLCR6500EVM from Texas Instrument which has 9523 Hz refresh rate, 7.6 µm pixel pitch, and 1920 × 1080 resolution. The light source is the diode pumped green laser (DPGL-2030 from SUWTECH) whose wavelength is 532 nm. The laser is spatially filtered and collimated by the lens 1 in Fig. 9(a) which has 25 cm focal length. The light diffracted by the DMD is filtered by the 4-f optics with 20 cm focal length lenses, i.e. lens 2 and lens 3 in Fig. 9(a). In the Fourier plane of the 4-f optics, an aperture is located to pass only the noise suppression band of the ED weights. The picture of the implemented optical setup is shown in Fig. 9(b).

 

Fig. 9. (a) The experimental setup scheme, and (b) the implemented optical setup.

Download Full Size | PDF

4.1 ED with a single plane carrier wave

Figure 10 shows the experimental reconstructions comparing three cases, i.e. the random phase carrier wave with the simple binarization, single plane carrier wave with the simple binarization, and the proposed single plane carrier wave and the ED binarization. In the proposed ED binarization case, the ED weight of the vertical filter shown in Fig. 4, (i).e. w_shift(r)=w_{initial_vertical}(r) = 1 for r=(r_x,r_y)=(p_x,0) is used. Note that these three cases are corresponding to the simulations shown in Figs. 6(b), 6(c), and 6(e), respectively. The holograms loaded to the DMD are the same as those used in the corresponding simulations. The experimental results in Fig. 10 show the same results as the simulations in Figs. 6(b), 6(c), and 6(e). The reconstruction of the random phase with the simple thresholding binarization case in Fig. 10(a) suffers from severe speckle noise with the additional binarization noises. The use of the single plane carrier wave reduces the speckle noise but the simple thresholding binarization prohibits exact gray scale reconstruction and introduces the ripple noise as shown in Fig. 10(b). The combination of the single plane carrier wave and the proposed ED binarization gives the best image quality as shown in Fig. 10(c).

 

Fig. 10. The optical reconstructions of (a) random phase carrier wave with simple thresholding binarization, (b) single plane carrier wave with simple thresholding binarziation, and (c) proposed single plane carrier wave with ED biniarzation using the vertical noise supppression band. In each figure, the camera is focused at the left cat in the upper row and at the right cat in the lower row.

Download Full Size | PDF

4.2 ED with plane carrier wave scanning by time-multiplexing

Figure 11 shows the results of the time-multiplexing experiment corresponding to the simulations in Fig. 8. The ED weight designed for the shifted horizontal noise suppression band shown in Fig. 7, (i).e. w_shift(r)=−0.0043-j1.0 for r=(r_x,r_y)=(0,-p_y) and 0 otherwise, is used like the simulation. The physical aperture in the 4-f optics is also implemented to have horizontally long aperture following the noise suppression band of the ED weight. Figure 11 demonstrates that the DOF of the reconstruction is reduced giving clear distinction between the focused and defocused reconstructions while keeping high reconstruction quality. As the number of the time-multiplexing frames increase, the defocus blur becomes continuous, giving more natural results. These experimental results are in good agreement with the corresponding simulations shown in Fig. 8.

 

Fig. 11. The optical reconstructions with time-multiplexing. (a) 3 carrier waves, and (b) 11 carrier waves.

Download Full Size | PDF

Finally, the holographic 3D video is displayed with the proposed method in real-time. Each scene of the video is displayed with 3 time-multiplexing frames with corresponding carrier waves like the case of Fig. 11(a). Two objects are displayed with 3 cm and 1 cm depths from the hologram plane. Each time-multiplexing sub-frame hologram is synthesized with the corresponding plane carrier wave and ED binarized with the shifted horizontal noise suppression band as in Fig. 11(a). All the sub-frames, i.e. 3 sub-frames for each frame, are pre-loaded to the onboard memory of the DMD and played in 90 Hz to result in 30 Hz holographic 3D video. During the recording of the video, the camera focus was changed between two objects to examine the depth reconstruction and the DOF. The snapshots in Fig. 12 and the recorded movie, i.e. Visualization 1, successfully demonstrates the high-quality holographic 3D image reconstruction with shallow DOF by the proposed ED binarization technique.

 

Fig. 12. The reconstructed holographic video with 3 carrier waves using time-multiplexing (see Visualization 1).

Download Full Size | PDF

5. Conclusion

In this paper, we propose a novel technique to enhance the quality of binarized holograms using ED with plane carrier waves for DMD SLMs. The plane carrier wave of the synthesized holograms and the ED filter are designed to suppress the binarization noise around the spatial frequency of the carrier wave where the most signal energy is concentrated. By applying the proposed technique, the speckless enhanced-quality holographic 3D images are reconstructed by using the DMD. The time-multiplexing with scanning plane carrier wave in the binarization error suppression band of the ED filter is also applied to achieve shallow DOF while keeping the enhanced reconstruction quality. The proposed technique is verified both by simulations and optical experiments, demonstrating the 30Hz enhanced-quality holographic 3D video display successfully.