Horizontally scanning holography using a microelectromechanical system spatial light modulator (MEMS-SLM) can provide reconstructed images with an enlarged screen size and an increased viewing zone angle. Herein, we propose techniques to enable color image generation for a screen-scanning display system employing a single MEMS-SLM. Higher-order diffraction components generated by the MEMS-SLM for R, G, and B laser lights were coupled by providing proper illumination angles on the MEMS-SLM for each color. An error diffusion technique to binarize the hologram patterns was developed, in which the error diffusion directions were determined for each color. Color reconstructed images with a screen size of 6.2 in. and a viewing zone angle of 10.2° were generated at a frame rate of 30 Hz.
© 2015 Optical Society of America
Holography can generate three-dimensional (3D) images that satisfy all the physiological factors of the human 3D perception, such as vergence, binocular parallax, motion parallax, and accommodation . Therefore, holographic 3D images are free from the visual fatigue caused by the vergence–accommodation conflict that occurs in most conventional 3D displays . However, electronic implementations of holography are limited with respect to screen size and viewing zone angle. Horizontally scanning holography employing a microelectromechanical system spatial light modulator (MEMS-SLM) [3–5] has been developed to address this issue. In this study, we develop techniques that enable the MEMS-SLM to modulate multiple wavelengths of lights to generate color 3D images using a screen-scanning display system .
Conventional holographic displays using an SLM with a resolution of N × M and a pixel pitch of p can provide 3D images with a viewing zone angle of 2 sin−1(λ/2p) and a screen size of Np × Mp, where λ is the wavelength of light. The pixel pitch needs to be decreased to increase the viewing zone angle, and the resolution needs to be increased to enlarge the screen size. Thus, research on the development of ultrahigh-resolution SLMs has been conducted [6, 7] and techniques using multiple SLMs have been developed [8–11]. Horizontally scanning holography employing a single MEMS-SLM with a high frame rate has been developed to increase both the viewing zone angle and screen size. Three types of scanning systems have been developed. The screen-scanning system [3, 12–14] reduces the pixel pitch using an optical system and enlarges the screen size using a mechanical scanner. Conversely, the viewing zone scanning system  optically enlarges the screen size and mechanically increases the viewing zone. In the 360-degree scanning system , the screen size is optically enlarged and the viewing zone is increased in all directions using a circular scanning system.
Horizontally scanning holography has employed digital micromirror devices (DMDs)  as the MEMS-SLM because of its high frame rate operation. The DMD’s pixels are tilted micromirrors; hence, the DMD’s screen has a structure similar to a reflective blazed grating. Therefore, higher-order diffraction components are used for producing reconstructed images, and the proceeding directions of the diffraction components depend on the wavelength of light. In addition, the DMD can generate only binary patterns, and the error diffusion algorithm was shown to be effective for the binarization of hologram patterns . The directions of the error diffusion, which should be determined considering the orientations of interference fringes, also depend on the wavelength of light. In this study, the above wavelength-dependent characteristics are taken into consideration during the development of techniques that enable the color 3D image generation.
Various holography display systems have been proposed to enlarge the viewing zone angle and screen size; some of these have achieved the color image generation. In the holographic display system developed at MIT , a high-resolution 1D hologram distribution, generated by an acousto-optic modulator (AOM), is two-dimensionally scanned using horizontal and vertical scanners. A three-channel AOM illuminated by R, G, and B laser lights is employed for the color image generation. Recently, the use of anisotropic leaky-mode modulators has been reported . In the active tiling display, developed by QinetiQ , holographic images displayed by a high-speed SLM are demagnified and tiled on an optically addressed SLM. The time-multiplexing technique was used to generate color images; R, G, and B lasers sequentially illuminate the optically addressed SLM. Holographic display systems using multiple SLMs [20, 21] were also developed to increase the viewing zone angle, in which the R, G, and B laser lights were combined using beam splitters to generate color 3D images by applying the time-multiplexing technique. SeeReal is developing a large-screen holographic display that also provides a large viewing zone using an eye-tracking technique . They used two different techniques to generate color holographic images: the time-multiplexing technique and the color filter technique.
In this study, two techniques are developed to enable a single MEMS-SLM to simultaneously modulate R, G, and B laser lights. The first couples the directions of the diffraction components for the three wavelength lights, and the second binarizes hologram patterns using wavelength-dependent error diffusion. These techniques are applied to the screen-scanning type horizontally scanning holographic display.
After a brief explanation about horizontally scanning holography that employs the screen-scanning display system, we will explain the technique to combine R, G, and B higher-order diffraction components as well as the wavelength-dependent error diffusion technique.
2.1 Screen-scanning type horizontally scanning holography
Figure 1 shows a schematic of the screen-scanning display system, which consists of a MEMS-SLM, an anamorphic imaging system, a horizontal scanner, a screen lens, and a vertical diffuser [3, 12–14]. RGB lasers are newly introduced for the color image generation.
The MEMS-SLM generates hologram patterns at a high frame rate. The anamorphic imaging system, which images the display screen of the MEMS-SLM on a vertical diffuser, has a reduced magnification in the horizontal direction and an enlarged magnification in the vertical direction. The vertically stretched hologram pattern, which is an elementary hologram, is scanned horizontally by the horizontal scanner. Since the pixel pitch is reduced in the horizontal direction, the horizontal viewing zone angle increases. A number of elementary holograms are aligned horizontally so that the screen size increases. Because the vertical pixel pitch is increased, this system provides 3D images that have only horizontal parallax. The screen lens redirects light to observers, and a vertical diffuser increases the vertical viewing region. A horizontal slit is placed on the Fourier plane of the anamorphic imaging system to eliminate the conjugate image and zero-order diffraction light .
In our previous studies [3, 12–14], the anamorphic imaging systems consisted of orthogonally aligned cylindrical lenses. In the present study, we modified the anamorphic imaging system by adding one spherical lens in front of the MEMS-SLM, as shown in Fig. 1. The focal plane of the spherical lens is the Fourier plane of this anamorphic imaging system where the horizontal slit is placed. The 2D Fourier transformed images are produced on this Fourier plane where the proper combination of R, G, and B lights is performed. The conjugate image and zero-order diffraction light are spatially separated from the reconstructed image in the vertical direction so that they are eliminated by the horizontal slit, whose height is one half its width.
2.2 Time-multiplexing technique for color 3D image generation
The time-multiplexing technique is used to generate color reconstructed images. As shown in Fig. 2, when the width of the elementary hologram is larger than the horizontal pitch for displaying them, multiple sets of elementary holograms can be displayed. To allow a display of R, G, and B elementary hologram sets, the horizontal displaying pitch should be equal to or smaller than one-third the width. By properly controlling the RGB laser illumination in synchronization with the MEMS-SLM, different elementary hologram sets can be illuminated by different color laser light. This capability of generating multiple elementary hologram sets with different illumination lights was previously used to improve the grayscale representation of reconstructed images .
2.3 Coupling of multiple wavelength lights diffracted by MEMS-SLM
The horizontally scanning holography has employed a DMD as the MEMS-SLM. A DMD screen consists of two-dimensionally aligned micromirrors whose tilt angles can be independently controlled between binary states to modulate light . Therefore, the structure of a DMD screen is similar to that of a reflective blazed grating, as shown in Fig. 3. Thus, as shown in Fig. 3(a), multiple higher-order diffraction components appear around the diffraction peaks. The diffraction peaks become zero-order diffraction light. The directions along which the diffraction peaks are generated follow the grating equation given byFig. 3(b), the light intensity becomes maximal in the direction of the light that is reflected by the micromirror surfaces. The direction of the reflected light is provided by the wave number vector kr, which is given by
The diffraction peaks are eliminated by placing them on the edges of the horizontal slit in the anamorphic imaging system, as shown in Fig. 4(a). Here, we consider the case where the peaks are located on the upper and lower edges of the horizontal slit. The focal length of the spherical lens in the anamorphic imaging system is denoted by fs, and the height of the horizontal slit is denoted by h; thus, this diffraction condition is given by the following equation:
Since horizontally scanning holography provides only horizontal parallax, the directions of maximal light intensity for the R, G, and B lights should coincide in the horizontal direction, as shown in Fig. 4(b). This reflection condition is given by
Six parameters, (kix, kiy), (kdx, kdy), and (krx, kry), should be determined. Equations (3) and (4) provide kdy and krx. As the x- and y-components of Eqs. (1) and (2) give four equations, kix, kiy, kdx, and kry can be solved with regard to the diffraction orders mx and my. To design the display system, we have to determine the direction of the incident light (λ/2π) ki for each color. Hence, the diffraction orders (mx, my) should be chosen appropriately. In this study, we chose the diffraction orders so that the center of the reflection beam passes closest to the center of the horizontal slit to maximize the light usage (Fig. 4(b)), i.e., (λ/2π)kry fs should be as close as possible to h/2.
The advantage of this technique is that no optical component such as a beam combiner is required to combine the three laser lights.
2.4 Wavelength-dependent error diffusion technique for binarizing hologram patterns
A DMD can generate only binary patterns. As the simple binarization of hologram patterns using a fixed threshold level degrades reconstructed images, two techniques have been developed for horizontally scanning holography to improve the grayscale representation of the reconstructed images. One is the time-multiplexing technique in which different elementary holograms sets binarized with different threshold levels are sequentially displayed . The other is the error diffusion technique in which the binarization error is distributed to neighboring pixels in the hologram pattern . In this study, since time-multiplexing is used to generate R, G, and B reconstructed images, the error diffusion technique is used to binarize the hologram patterns.
In a previous study , the error diffusion technique was used to generate monochromatic reconstructed images. At that time, the Floyd–Steinberg error diffusion coefficients were used; these coefficients are widely used for 2D image binarization. In this study, we first used the Floyd–Steinberg coefficients for binarization of the R, G, and B hologram patterns. However, the image quality of the color reconstructed images was not satisfactory; the contrast was not sufficiently high because of noise clouds appearing in the reconstructed images. Hence, the Floyd–Steinberg coefficients are not necessarily appropriate for the binarization of hologram patterns. Hauck et al.  undertook a comprehensive study of the error diffusion technique for the hologram binarization. They focused on the directions of interference fringes in the hologram patterns, i.e., the carrier directions. They showed that the binarization error should be diffused in a direction orthogonal to the carrier direction. Since the phase information is coded in the carrier direction and the amplitude information is coded in the orthogonal direction, the amplitude errors due to binarization should be distributed in the orthogonal direction. In this study, Hack’s technique is used for binarizing the R, G, and B hologram patterns.
Figure 5(a) shows the diffraction peaks and diffraction components on the Fourier plane. The diffraction components for R, G, and B lights appear at different positions because their positions are determined to remove diffraction peaks. To coincide the R, G, and B diffraction patterns on the Fourier plane, the hologram patterns are calculated by adding reference waves proceeding in different directions. The directions of the reference waves projected on the Fourier plane are drawn in Fig. 5(b), which are the carrier directions of the hologram patterns. Figure 5(c) shows the error diffusion directions, which are orthogonal to the carrier directions. The error diffusion directions are different for each color.
The techniques for the color image generation described in the previous section are experimentally verified.
A DiscoveryTM 4100 (Texas Instruments, Inc.) was used as the DMD. The resolution was 1024 × 768, the pixel pitch was d = 13.68 μm, and the frame rate was 22.727 kHz.
The anamorphic imaging system consisted of a spherical lens with the focal length fs of 400 mm, and cylindrical lenses with their lens axes vertically and horizontally aligned with the focal lengths of 400 and 309 mm, respectively. The horizontal and vertical magnifications were 0.183 and 4.74, respectively. The size of the elementary hologram was 2.56 × 49.8 mm2. The horizontal pixel pitch was reduced to p = 2.50 μm.
A galvano mirror was used as the horizontal scanner. The horizontal scan rate was 30.0 scans/s, and the horizontal scan angle was ± 13.6°. The screen size was enlarged to 150 × 49.8 mm2 (6.2 in). The horizontal pitch for displaying the elementary holograms was 0.284 mm, which was smaller than one-third the width of the elementary holograms. Although the DMD can display 757 images during a single scan, in order to avoid nonlinear scan regions, 696 elementary holograms were displayed. Hence, the number of elementary holograms for each color was 232.
The wavelengths of the R, G, and B laser lights were 640, 515, and 445 nm, respectively. The width and height of the horizontal slit are given by (λfs/d) and (λfs/2d), respectively. Since the slit size was determined for the shortest wavelength (B color), the width and height were 13.0 and 6.51 mm, respectively (h = 6.51 mm.) The horizontal viewing zone angles for the R, G, and B colors were 14.7°, 11.8°, and 10.2°, respectively. The horizontal slit limited the viewing zone angle of the display system to 10.2°.
A Fresnel lens with a focal length of 200 mm was used as the screen lens, and a lenticular lens was used as the vertical diffuser. The photograph of the constructed screen-scanning system is shown in Fig. 6.
The wave number vectors ki of the R, G, and B illumination lights were determined. Since each micromirror in the DMD rotates around its diagonal line, the direction normal to the micromirror is given by , where α is the tilt angle of the micromirror. We measured and obtained α = 11.6° and n = (0.142, −0.142, 0.980). Consequently, the illumination directions, given by (λ/2π) ki, were determined as (−0.278, 0.297, −0.914), (−0.279, 0.264, −0.923), and (−0.278, 0.276, −0.920) for R, G, and B lights, respectively. The corresponding diffraction orders (mx, my) were (−6, 6), (−7, 7), and (−9, 8), respectively. Figure 7(a) shows the positions of the three diffraction peaks on the Fourier plane. The horizontal positions of the peaks are given by (λ/2π)kxd fs. Figure 7(b) shows the positions where the centers of the reflected beams intersect the Fourier plane.
We used fiber-coupled laser diodes as the R, G, and B lasers. Three diverging laser beams emitted from the optical fibers were collimated by a common condenser lens to obtain three plane waves that illuminated the DMD from different directions. The positions of the optical fibers were determined from the illumination directions presented in the previous paragraph. An optical fiber array consisting of the three optical fibers was constructed, as shown in Fig. 8. The experimentally obtained light distribution on the Fourier plane is shown in Fig. 9. The three diffraction peaks were aligned almost on the upper and lower edges. The color changed in the vertical direction.
The error diffusion coefficients were determined. Figure 10(a) shows the carrier directions in the hologram patterns obtained from the positions of the diffraction peaks shown in Fig. 7(a). Figure 10(b) shows the error diffusion directions that are perpendicular to the carrier directions. The binarization errors are divided into two pixels, which are the nearest and second-nearest to the desired error diffusion direction. The weights assigned to the two pixels are determined such that their vector sum becomes the diffusion direction, as shown in Fig. 10(c).
The color profile conversion was performed to obtain 3D image data displayed with the three wavelengths. In our previous study using the error diffusion technique for monochromatic image generation , the nonlinearity of the grayscale representation of the reconstructed images was compensated by modifying the intensities of the 3D image data referring to the measured intensities. This technique was also used in this study.
The color image generation was verified. In Fig. 11, the R, G, and B reconstructed images are shown in the top three rows, which were generated by turning one of the three lasers on. The hologram patterns were binarized with the simple binarization technique using a single threshold level, the conventional error diffusion technique using the Floyd–Steinberg coefficients, and the wavelength-dependent error diffusion technique. With the simple binarization technique, the edges of the objects were emphasized so the grayscale representation was insufficient. When the conventional error diffusion technique was used, the grayscale representation was improved. However, noise clouds were observed around the objects in the reconstructed images. When the proposed technique was used, the noise clouds decreased so that the contrast of the images increased. In the bottom row, color reconstructed images are shown; all three lasers were turned on and the RGB images were combined.
Figure 12 shows reconstructed images that contain many colors. When the simple binarization technique was used, some colors in the reconstructed images were different from those in the target images and the grayscale representation was poor. When the conventional error diffusion technique was used, the color and grayscale representations were improved. On the other hand, when the proposed technique was used, the color and grayscale representations were significantly improved and the image contrast was increased.
Figure 13 shows the reconstructed images generated by the proposed technique, which were captured from three different horizontal directions. The images had smooth motion parallax and could easily be observed with both eyes because the viewing zone angle was enlarged.
As seen from the color reconstructed images shown in Fig. 13, the color representation did not change when the observation angle was changed. A dependency of the color representation on the viewing position was not observed; therefore, the proposed color-coupling method functioned properly.
The tilt angle of the micromirrors in the DMD affects the coupling of the three color lights. The typical angle described in the datasheet of the DMD is 12°. First, we designed the fiber array corresponding to` this value. However, the obtained diffraction peak positions and color distribution on the Fourier plane were different from the designed diffraction. Therefore, we measured the tilt angle. The measured tilt angle provided the diffraction on the Fourier plane, as shown in Fig. 9, which was in good agreement with the designed diffraction shown in Fig. 7.
From Fig. 12, colors in the reconstructed images were not exactly the same as those in the target images. This is because the wavelengths of G and B lasers were different from the typical wavelengths of G and B colors. In this study, we prioritized the use of laser diodes. The single-mode laser diode with the typical G-color wavelength was not available when the experiments were conducted. Therefore, we used a laser diode with a wavelength of 515 nm for the G color; the typical wavelength for the G color is approximately 530 nm. For the B color, we first used a laser diode with a wavelength of 445 nm because the typical wavelength for the B color is 430 nm. However, the reconstructed images were not as expected owing to the chromatic aberration of the lenses used in the anamorphic imaging system. The three lenses constituting this system are made of BK7 glass, and the chromatic aberration of these lenses is large at a wavelength of 445 nm. Therefore, we used a laser diode with a wavelength of 488 nm for the B color.
In the reconstructed images, flicker was observed because the frame rate of the experimental system was 30 Hz. In previous screen-scanning systems [3, 12–15], the frame rate was 60 Hz, and no obvious flicker was observed. In this study, we changed the mirror of the galvano scanner. The width of the mirror was increased to 170 mm, as we intend to further increase the viewing zone angle in our subsequent experiments. Unfortunately, this large mirror could not be rotated quickly enough to provide a frame rate of 60 Hz.
From these experimental results, it is evident that the wavelength-dependent error diffusion technique could improve the image quality of the R, G, and B reconstructed images. Since the width of the elementary holograms was 2.56 mm and the pitch for displaying the elementary holograms in each color was 0.852 mm, three elementary hologram sets could be used for each color. Therefore, if different elementary hologram sets were illuminated with different laser powers in each color, the image quality of the reconstructed images would be further improved.
In this study, we proposed techniques for generating color reconstructed images using horizontally scanning holography. An RGB laser illumination system was introduced to a screen-scanning display system employing a single DMD and a time-multiplexing technique was utilized. The higher-order diffraction components for the R, G, and B laser lights were combined by properly designing the R, G, and B illumination directions on the DMD. An error diffusion technique where the error diffusion directions were appropriately determined for the three colors was used to improve the color representation of the reconstructed images. An experimental system was constructed and the color reconstructed image generation was successfully demonstrated; the screen size and viewing zone angle were enlarged to 6.2 in. and 10.2°, respectively. The frame rate of the hologram generation was 30 Hz.
This study was supported by JSPS KAKENHI Grant Number 15H03987.
References and links
1. T. Okoshi, Three-Dimensional Imaging Techniques (Academic, 1976).
6. K. Aoshima, N. Funabashi, K. Machida, Y. Miyamoto, K. Kuga, T. Ishibashi, N. Shimidzu, and F. Sato, “Submicron magneto-optical spatial light modulation device for holographic displays driven by spin-polarized electrons,” J. Disp. Technol. 6(9), 374–380 (2010). [CrossRef]
7. R. Stahl and M. Jayapala, “Holographic displays and smart lenses,” Opt. Photonics 6, 39–42 (2011).
8. K. Maeno, N. Fukaya, O. Nishikawa, K. Sato, and T. Honda, “Electro-holographic display using 15 mega pixels LCD,” Proc. SPIE 2652, 15–23 (1996). [CrossRef]
9. J. Hahn, H. Kim, Y. Lim, G. Park, and B. Lee, “Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators,” Opt. Express 16(16), 12372–12386 (2008). [CrossRef] [PubMed]
11. K. Yamamoto, Y. Ichihashi, T. Senoh, R. Oi, and T. Kurita, “3D objects enlargement technique using an optical system and multiple SLMs for electronic holography,” Opt. Express 20(19), 21137–21144 (2012). [CrossRef] [PubMed]
15. D. Dudley, W. M. Duncan, and J. Slaughter, “Emerging digital micromirror device (DMD) applications,” Proc. SPIE 4985, 14–25 (2003). [CrossRef]
17. P. St-Hilaire, S. A. Benton, M. E. Lucente, and P. M. Hubel, “Color images with the MIT holographic video display,” Proc. SPIE 1667, 73–84 (1992).
19. M. Stanley, M. A. G. Smith, A. P. Smith, P. J. Watson, S. D. Coomber, C. D. Cameron, C. W. Slinger, and A. D. Wood, “3D electronic holography display system using a 100 Mega-pixel spatial light modulator,” Proc. SPIE 5249, 292–308 (2004).
20. K. Sato, A. Sugita, M. Morimoto, and K. Fujii, “Reconstruction of color images of high quality by a holographic display,” Proc. SPIE 6136, 61360V (2006). [CrossRef]
21. T. Senoh, T. Mishina, K. Yamamoto, R. Oi, and T. Kurita, “Viewing-zone-angle-expanded color electronic holography system using ultra-high-definition liquid crystal displays with undesirable light elimination,” J. Disp. Technol. 7(7), 382–390 (2011). [CrossRef]
22. R. Häussler, S. Reichelt, N. Leister, E. Zschau, R. Missbach, and A. Schwerdtner, “Large real-time holographic displays: from prototype to a consumer product,” Proc. SPIE 7237, 72370S (2009). [CrossRef]
24. R. Hauck and O. Bryngdahl, “Computer-generated holograms with pulse-density modulation,” J. Opt. Soc. Am. A 1(1), 5–10 (1984). [CrossRef]