Open Access
Add to BibSonomy
Abstract
A single spatial-light-modulator (SLM) full-color holographic 3-D video display based on image and frequency-shift multiplexing (IFSM) is proposed. In the frequency-shift multiplexing (FSM), three-color holograms are multiplied with their respective phase factors for shifted-separations of their corresponding frequency-spectrums on the Fourier plane. This FSM process, however, causes three-color images to be reconstructed at the center-shifted locations depending on their multiplied phase factors. Center-shifts of those color images due to the FSM can be balanced out just by generation of three-color holograms whose centers are pre-shifted to the opposite directions to those of the image shifts with the novel-look-up-table (NLUT) based on its shift-invariance property, which is called image-shift multiplexing (ISM). These image and frequency-shifted holograms are then multiplexed into a single color-multiplexed hologram and loaded on the SLM, and from which a full-color 3-D image can be reconstructed on the optical 4-f lens system without any color dispersion just by employing a simple pinhole filter mask. Fourier-optical analysis and experiments with 3-D objects in motion confirm the feasibility of the proposed system.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Thus far, the holographic technique has been suffered from several critical problems in most practical applications [1]. One of them is the unavailability of a full-color holographic three-dimensional (3-D) camera for capturing the outdoor scenes under daylight-illumination. Another problem is the unavailability of a large-scale and high-resolution spatial-light-modulator (SLM) to reconstruct the holographic data into 3-D video images. In fact, sizes and pixel pitches of the commercially-available SLMs are ranged from one half to two inches, and 3.74 to 20 micrometers, respectively [2], so that they look too small in size and too low in resolution to display the full-color holographic video images since the resolution of the hologram data is in the order of the light wavelength.
For alleviating this problem, various approaches have been proposed. One of them is the method to build an SLM array just by tiling a number of small-sized SLMs together for providing a large-scale holographic 3-D display [3–5]. Projection-type holographic display was also proposed for displaying large-scale holographic 3-D video images [6,7]. Recently, a natural light full-color holographic camera was also suggested as another means for holographic capturing and reconstruction of the outdoor 3-D scenes [8].
In addition, electro-holographic 3-D displays based on computer-generated holograms (CGHs) have been actively researched since CGHs can precisely record and reconstruct the light waves of the 3-D object [9–13]. But, this electro-holography also suffers from the unavailability of a large-scale high-resolution SLM, as well as the enormous computation-time involved in generation of the CGH patterns [14–17]. Furthermore, optical configuration of the full-color holographic display becomes very complex because of the separated processing of three-color holograms of the input 3-D scene.
Basically, a full-color holographic display requires three SLMs corresponding to each of the red (R), green (G) and blue (B)-color channels because RGB-holograms of the input 3-D scene must be separately generated and reconstructed based on the interference and diffraction optics [18]. Up to now, several methods have been proposed for realizing the full-color holographic 3-D display [19–21]. Moreover, in order to reduce the number of SLMs required in the conventional color-holographic display, time and spatial-multiplexing schemes were also proposed [22,23]. However, the time-multiplexing method requires a SLM with the very high frame-rate to overcome the flickering problem and it also suffers from the complex synchronization operation, while the spatial-multiplexing scheme undergoes other problems such as color dispersion and resolution loss [23].
As the alternative to those time and spatial-multiplexed full-color holographic 3-D displays, a single SLM-based full-color holographic 3-D display based on the color-multiplexed hologram (CMH) was proposed [24–29]. Here, the CMH represents a multiplexed three-color hologram using the color-encoding method. The most challenging issue in the color-encoding method is the color dispersion (CD). There are two kinds of color dispersions due to optical diffractions from the fringe pattern of the CMH and pixelated structure of the SLM, which are called CMH-CD and SLM-CD, respectively. Those two kinds of color dispersions must be removed in the display process, otherwise the reconstructed object image has to be severely distorted in color. Thus far, for solving those color dispersion problems, several color-encoding methods have been proposed, which include depth-division multiplexing (DDM) [24], multiplexing encoding (ME) [25], space-division multiplexing (SDM) [26], frequency-division multiplexing (FDM) [27] and sampling and selective frequency-filtering (SSFF) [28,29] methods.
Thus, in this paper, we propose a simple-structured single SLM full-color holographic 3-D video display based on the image and frequency-shift multiplexing (IFSM) method. In the frequency-shift multiplexing (FSM) process, three-color holograms are multiplied with their respective phase factors to make their corresponding spectrums shifted to the specific locations on the Fourier plane. This FSM process, however, causes three-color images to be reconstructed at the center-shifted locations depending on their multiplied phase factors. But, this problem can be resolved with the image-shift multiplexing (ISM) process employing the NLUT method. That is, center-shifts of those color images due to the FSM can be balanced out just by generation of three-color holograms whose centers are pre-shifted to the opposite directions to those of the image shifts with the NLUT based on its unique property of shift-invariance. These image and frequency-shifted holograms are then multiplexed into the image and frequency-shifted color-multiplexed hologram (IFS-CMH). From this IFS-CMH, a full-color object image can be reconstructed on the optical 4-f lens system just by using a simple pinhole filter mask.
To confirm the feasibility of the proposed method, Fourier-optical analysis and optical experiments with 3-D color objects in motion are carried out and the results are comparatively discussed with those of the conventional methods.
2. Proposed system
Figure 1 shows a flowchart of the proposed single SLM full-color holographic display system based on the image and frequency-shift multiplexing (IFSM) method, which is composed of two systems such as the digital system to generate the IFS-CMH for the input 3-D video, as well as the optical system to reconstruct the IFS-CMH into a full-color 3-D video images on the 4-f lens system with a simple pinhole filter mask.
Fig. 1 Operational flowchart of the proposed single SLM full-color holographic 3-D video display system composed of digital and optical processes.
The digital system also consists of two processes such as the image-shift multiplexing (ISM) and frequency-shift multiplexing (FSM). An input 3-D video is sequentially rearranged into three sets of input image data, which are composed of R, G and B-intensity and depth images. In addition, here in this paper, a new type of the NLUT composed of two sets of center-shifted R and B-PFPs (principal fringe patterns) and a set of original G-PFPs, is employed, which is called a center-shifted NLUT (CS-NLUT). Contrary to the original three-color PFPs whose centers are all located at the origin of the coordinate system, in the proposed CS-NLUT, centers of those R and B-PFPs are shifted from the origin depending on their x and y-directional phase factors multiplied in the FSM process while those of the G-PFPs are remained on the origin.
In the ISM process, R and B-holograms are calculated with their respective center-shifted PFPs, while the G-hologram is calculated with its original PFPs. Thus, only the G-image is reconstructed on the center location, whereas R and B-images are reconstructed on the center-shifted locations. In the FSM process, those R and B-holograms, which are calculated with their center-shifted versions of the PFPs in the ISM process, are multiplied with their specific x and y-directional phase factors whose values are to be matched to the opposite values of the center-shifts of their corresponding PFPs, and then multiplexed into a single IFS-CMH together with the G-hologram calculated with its original versions of the PFPs.
Here, the FSM process enables those R and B-spectrums of the IFS-CMH to be shifted to the different locations, being separated from the G-color spectrum on the Fourier plane, which means that with this FSM process, the CMH-CD issue can be resolved. In addition, center-shifts of those reconstructed R and B-color images due to the FSM can be cancelled out with the ISM process, which enables three-color images to be completely overlapped and display the full-color object image.
In the optical system, which is implemented with a 4-f lens system, the IFS-CMH is loaded on the SLM and transformed into the Fourier domain with the 1st lens of the 4-f lens system by being illuminated with a multi-wavelength light source. Then, only the wanted three-color spectrums of the IFS-CMH can be made separated from the others and overlapped together right on the center spot at the Fourier plane. Thus, by employing a single-pinhole filter mask whose pinhole is just located on the center, only those three wanted spectrums can be made passed. Finally, on the display plane, the filtered IFS-CMH is reconstructed into a full-color 3-D object image without any color dispersion by being inversely Fourier-transformed with the 2nd lens of the 4-f lens system.
2.1 FSM process for the spatial separations of the RGB-frequency spectrums on the Fourier plane
Hologram is a kind of diffraction grating. Basically, both of the real and virtual images reconstructed from the hologram can be used for the holographic display, but here we adopt the real image for our holographic display. Because of the zero-order diffraction and conjugate image effects, an off-axis holography is adopted in this paper, where the reference wave is perpendicularly illuminated on the hologram plane while the object wave is illuminated on the hologram plane with an off-axis angle. When the R-hologram is illuminated with a multi-wavelength light source, three color images can be reconstructed as seen in Fig. 2(a), which is called a SLM-CD. Due to the small diffraction-angle caused by the large pixel-size of the SLM, two color-dispersed images of Gr and Br diffracted from the R-hologram happen to be partly overlapped with the wanted red-color image of Rr diffracted from the R-hologram, which means it looks impossible to completely separate the Rr from the Gr and Br on the display plane as seen in Fig. 2(a). However, frequency-spectrums of those three-color images can be well separated on the frequency plane, so that frequency-spectrums corresponding to those color-dispersed images can be easily removed on the Fourier plane.
Fig. 2 Two kinds of color dispersions in the CMH-based single SLM full-color holographic display: (a) Reconstructed three-color images due to the SLM-CD, (b) Reconstructed three-color images due to the CMH-CD, (c) Reconstructed nine-color images due to both of the SLM-CD and CMH-CD.
Nevertheless, three-color spectrums of the CMH for the R-laser illumination occupy the almost same spatial positions on the Fourier plane due to the small differences in wavelength among those three-color holograms of the CMH. That is, even though three kinds of red-color images of Rr, Rg and Rb reconstructed from the CMH are to be separated each other, their spectrums are almost overlapped on the same spatial positions on the frequency domain as seen in Fig. 2(b), which is called CMH-CD. Thus, when a parallel multi-wavelength light source composed of the R, G and B-lasers, is illuminated on the CMH, there appear three kinds of color spectrum spots on the Fourier plane, and three sets of three-color images are reconstructed on the display plane as seen in Fig. 2(c). Thus, it seems to be impossible to separate only the wanted three-color images, which are reconstructed from the spectrums of the Rr, Gg and Bb, from the other color-dispersed images reconstructed from the spectrums of Rg, Rb, Gr, Gb, Bg and Bg.
Thus, in the proposed method, for the effective separation of those RGB-spectrums on the Fourier plane, three different phase-factors are multiplied to each color hologram in the FSM process, which is given by Eq. (1).
In Eq. (1), hori(x, y) and hfs(x, y) represent the original hologram pattern and its frequency-shifted version by being multiplied with its phase factor, respectively, where fa and fb represent the spatial-frequency along the x and y-directions, respectively. Then, the spectrum of the frequency-shifted hologram due to the FSM process is given by Eq. (2).Where Hori(fx, fy) = F{hori(x, y)}, Hori and Hfs represent the spectrums of the original and frequency-shifted holograms, respectively. Parameters of fx, fy and fa, fb represent the spatial frequencies of the original and frequency-shifted holograms by being multiplied with its phase factors along the spatial-frequency coordinates of X and Y on the Fourier plane, respectively. Thus, spatial positions of the frequency spectrums of the original hologram can be shifted to the X and Y-directions with the pre-determined distances of fa and fb as seen in Fig. 3. Depending on the values of fa and fb, frequency spectrums of the RGB-holograms can be made properly separated on the Fourier plane.Fig. 3 Operational configuration of the FSM process: (a) Shifted spatial position of the spectrum and center-shifted image due to the FSM process, (b) Propagation-direction rotation due to the FSM process (Dori: depth of original reconstruction, Drot: depth of rotated reconstruction).
For the FSM process, R and B-holograms are multiplied with the phase factors of exp[-j2π(fa_R·x-fb_R·y)] and exp[-j2π(-fa_B·x + fb_B·y)] for center-symmetrical shifts of their spectrums to the right-lower and left-upper directions, respectively, while the G-hologram is kept intact. Thus, reconstructed R and B-images from the R and B-holograms on the display plane are to be oppositely displaced from the center with their corresponding amounts of spectrum-shifts on the Fourier plane, which results in a color distortion of the reconstructed object image.
Here, the two-dimensional (2-D) phase factor of exp[-j2π(fa_R·x-fb_R·y)] multiplied to the R-hologram causes the R-hologram to be diffracted to the x and y-directions with the angles of θa and θb, respectively, as seen in Fig. 3, which means that the phase of the R-hologram after being multiplied with the phase factor should be same with the initial phase planes as seen in Fig. 3. Here, the red lines represent the propagation directions of the rays diffracted from the original R-hologram, whereas the blue lines show those of being diffracted from the R-hologram multiplied with the phase factor. The propagation direction is then changed with the tilted angles of θa and θb, thus the phase factor to be multiplied becomes exp[-j2π(x·sinθa/λ-y·sinθb/λ)], where sinθa and sinθb can be calculated from Eq. (3) for the case of dx = dy <<z.
Here, the relationship between the multiplied phase factors of fa and fb, and corresponding locational displacements of the image of dx and dy can be given by Eqs. (4)-(5).
Where px, py and dx, dy represent the x and y-directional phase factors, and locational displacements of the image, respectively. Thus, the spatial position of the image can be controlled to be shifted to any location just by changing the tilted angles of θa and θb. In the actual phase shifting process, the frequency value has to be transformed into the spatial coordinates, which can be derived by the Abbe image formation equations [33] as follows.In Eq. (6), Fx, Fy, L and λ represent the spatial distance values corresponding to the frequency values along the X and Y-axis on the Fourier plane, focal length of the lens and wavelength of the light source, respectively. At the same time, this FSM process also makes depth changes of Dx and Dy due to the rotated image position to the propagation direction when the degree of rotation is large enough as seen in Fig. 3(b). Degrees of depth changes can be calculated with Eq. (7) where Dx, Dy, Δx, Δy, θa, θb and z represent the depth changes due to the image rotations, image-shifted distances, rotational angles along the x and y-axis, and reconstructed distance far away from the hologram, respectively.
As seen in Eq. (7), the total depth changes are proportional to the shifted distances and rotated angles along the x and y-axis, where the rotated angles are inversely proportional to the reconstruction distance of z. Thus, if the reconstruction distance is much farther than the shifted distances of Δx and Δy, the total depth changes of Dx and Dy can be ignored. Otherwise, depth changes should be taken into account. In this paper, these depth changes can be neglected since the reconstruction distance z becomes much larger than the shifted distances of Δx and Δy.
2.2 ISM process for compensating the center-shifts of the reconstructed color images
With the FSM process, three-color spectrums of the CMH can be effectively separated on the Fourier plane, which enables the CMH-CD to be resolved. The FSM process, however, also makes those three-color images to be reconstructed at the center-shifted locations on the display plane depending on their multiplied phase factors, which means that the reconstructed image gets distorted in color due to the spatially-separated reconstructions of those three-color images. Thus, another scheme is needed to reconstruct all those three-color images at the same center locations to display a full-color object image.
For this, an image-shift multiplexing (ISM) process based on the NLUT method is employed. The conventional NLUT is composed of three-color sets of PFPs, where each PFP has a unique property of shift-invariance, which enables the shifted reconstruction of those three-color images on the display plane just by generating three-color holograms with their center-shifted-PFPs in advance [30–32]. Basically, a PFP looks like a concentric circle form of the Fresnel zone plate (FZP). When centers of the PFPs are pre-shifted along the x and y-axis with the controlled distances, the location of the reconstructed image from the hologram calculated with these center-shifted PFPs is also shifted to the x and y-directions with the same amounts of the center-shifted distances of the PFPs.
Thus, for the ISM, three-color sets of PFPs used in the conventional NLUT are transformed into the center-shifted versions, where three-color holograms are calculated with its corresponding color sets of center-shifted PFPs, which is called the center-shifted NLUT (CS-NLUT). This CS-NLUT enables color images to be reconstructed at the center-shifted locations depending on the center-shifts of their corresponding PFPs. For instance, as seen in Fig. 4(a), the hologram pattern for three object points of A(0, 0, zp), B(-x1, -y1, zp) and C(x2, y2, zp) of an input 3-D object on the depth plane of zp can be generated from three shifted and cropped versions of the conventional PFP, where the PFP is located at the center of the hologram plane. From this hologram, the object image is reconstructed right on the center of the display plane as seen in Fig. 4(a).
Fig. 4 Operational configuration of the ISM process for the R-hologram: (a) Conventional NLUT-based hologram generation and its reconstruction processes, (b) CS-NLUT-based hologram generation and its reconstruction processes.
On the other hand, when the conventional sets of PFPs are replaced with those of the center-shifted PFPs with the distances of -dx and + dy along the x and y-directions, positions of the hologram pattern and reconstructed image would be also shifted to the x and y-directions with the same distances of -dx and + dy, respectively, as seen in Fig. 4(b), where three object points are shown to be shifted to A′(-dx, dy, zp), B′(-x1-dx, -y1 + dy, zp) and C′(x2-dx, y2 + dy, zp), respectively, according to the shift-invariance property of the PFP.
In fact, the ISM process enables the x and y-directional shifts of the reconstructed object image on the display plane with the same amounts of the center-shifts of the PFPs of the CS-NLUT, which can be described by Eq. (8).
Where h, ap, and PFP represent the final hologram, intensity values of the pth object point and pre-calculated PFP of the depth plane of zP, respectively. In addition, x, y and dx, dy denote the coordinates and pre-shift-distances of the calculated hologram along the x and y-axes, respectively. In addition, xp and yp represent the shift-distances of the center of the pre-determined cropping area for the pth object point. The center-shifted PFPs can be pre-calculated with Eq. (9), where k0 = 2π/λ and λ mean the wave number and wavelength of the light source. The resolution of the PFP has to be set to be larger enough for their shifting, adding and cropping operations in the hologram generation processes of the NLUT method. That is, the minimum resolution of the PFP can be given by Eqs. (10)-(11), where hx and hy represent the horizontal and vertical resolutions of the hologram pattern, and s, Ox and Oy denote the sampling step and horizontal and vertical image sizes, respectively. Moreover, the effect of the ISM process on the frequency-spectrum of the center-shifted hologram can be given by Eq. (12).
Where fori(fx, fy) = F{hori(x, y)}, and fcs and fori mean the spectrums of the center-shifted and original holograms, respectively. Four parameters of dx, dy and fcs_x, fcs_y represent the pre-shifted distances of the hologram pattern and corresponding phase factors on the frequency along the x and y-axis, respectively, and the subscript c means three R, G and B colors. According to Eq. (12), the spatial position of the spectrum of the center-shifted hologram turns out to be same with that of the spectrum of the original hologram such as fx = fx_c and fy = fy_c, but its propagation direction would be changed due to the phase factor multiplied to the spectrum of the original hologram, which results in the corresponding x and y-directional shifts of the reconstructed object image.2.3 Multiplexing three-color holograms into a single color-multiplexed hologram
In this paper, image and frequency-shifted (IFS) three-color holograms are then multiplexed into a single hologram, which is called an IFS-CMH. Moreover, the IFS-CMH is generated as a form of the simple amplitude-type hologram. The employed SLM can provide an 8-bit data value for each pixel, thus the intensity levels for the amplitude-type hologram may be limited by the range of 0~255 [2]. For multiplexing those three-color holograms into a single IFS-CMH, the range of the intensity must be shared together with three R, G and B-channel holograms. Thus, each pixel-amplitude of those three-color holograms needs to be properly scaled before being multiplexed together. The intensity values of three-color components of the final full-color image can be controlled by three independently-controllable laser light sources instead of controlling the ranges of the intensity values on each channel hologram. Thus, the amplitude scaling factor of the R, G and B-channel can be simply set to be same within the range of 0~85. Three amplitude-scaled R, G and B-channel holograms are finally added up together to generate the IFS-CMH.
2.4 Design of a single-pinhole filter mask for the full-color 3-D display based on the controlled ISM and FSM processes
To reconstruct the full-color object image from the IFS-CMH on the optical 4-f lens system, a pinhole filter mask has to be designed and implemented. When a multi-wavelength light source composed of the R, G and B-lasers is illuminated on the IFS-CMH, three sets of three-color spectrum spots are generated on the Fourier domain due to the FSM process as seen in Fig. 5(b). For comparison, another set of three R, G and B-color spectrum spots, which are generated from the original CMH without the IFSM process, are also shown in Fig. 5(a), where each of the R, G and B-color spectrum spots is composed of three kinds of color spectrum spots such as Rr, Rg, Rb, and Gr, Gg, Gb, and Br, Bg, Bb, respectively. As seen in Fig. 5(b), due to the controlled FSM process, two sets of three-color spectrum spots of Rr, Gr, Br and Rb, Gb, Bb of Fig. 5(a) are shifted to the x and y-directions with their phase factors of (fa_R, -fb_R) and (-fa_B, fb_B), respectively, while three-color spectrum spots of Rg, Gg and Bg are kept to be stayed there.
Fig. 5 Controlled IFSM process for selection of the wanted three-color spectrum spots just by using a single pinhole filter mask on the Fourier plane: (a) Three sets of three-color spectrum spots of the CMH without a ISFM process, (b) Controlled shifts of nine-color spectrum spots of the IFS-CMH, (c) Designed single-pinhole filer mask.
Here it must be noted that only the two-color spectrum spots of Rr and Bb are controlled to be shifted to the center location and mixed up together with Gg, which are the wanted three-color spectrum spots. Thus, with a single-pinhole filter mask being located right on the center as seen in Fig. 5(c), those wanted three-color spectrum spots can be filtered out on the Fourier plane, while all the other color-dispersed spectrum spots are blocked out. Since these three-color beams carrying their respective color information of the input object are passing through the same single pinhole, those light beams propagate on the same directions and completely mixed up together on the center area of the display plane to reconstruct a full-color 3-D object image without any color dispersion.
2.5 Optical reconstruction of the IFS-CMH into a full-color 3-D image with a single pinhole filter mask on the 4-f lens system
Figure 6 shows a simple optical 4-f lens system for reconstruction of the IFS-CMH into a full-color 3-D object image. The IFS-CMH is loaded on the SLM and illuminated with a collimated light source composed of three R, G and B-lasers. The IFS-CMH is then optically transformed into the Fourier domain, where three sets of R, G and B-color spectrums of the IFS-CMH are obtained. That is, as seen in Fig. 6, three sets of three-color spectrum spots of the IFS-CMH are aligned along the same diagonal direction on the Fourier plane according to the controlled x and y-directional shifts of those three-color spectrums. As mentioned above, only the wanted three-color spectrum spots of Rr, Gg and Bb are controlled to be just overlapped right on the center location, whereas all the other color-dispersed versions are diagonally shifted to the different directions as seen in Fig. 6(a).
Fig. 6 Optical configuration for reconstructing the IFS-CMH into a full-color 3-D image with a single-pinhole filter mask on the 4-f lens system.
Thus, as seen in Fig. 6(b), just by employing a simple single-pinhole filter mask, whose pinhole is located right on the center area, all those color-dispersed spectrums can be made blocked, while only the three wanted color spectrums of Rr, Gg and Bb, which are overlapped on the center area, can be made filtered out as seen in Fig. 6(c). Thus, with this simple single pinhole filter mask, the color dispersion problem can be resolved in the proposed system.
3. Experiments and the results
3.1 Overall configuration of the experimental setup
Figure 7 shows an overall experiment setup of the proposed system, which is composed of digital and optical processes. In the experiment, a volumetric 3-D color object of ‘Rubik’s Cube’ whose size is 10mm × 10mm × 10mm, is computationally generated as the test object. The test object is rendered into R, G and B-channel intensity and depth images whose resolution and number of depths are 320 × 240 pixels and 256, respectively. These intensity and depth data are used as the input image data for generating their corresponding CGH patterns with the CS-NLUT method.
Fig. 7 Overall experiment setup: (a) Digital system for generation of the IFS-CMH based on the proposed IFSM method, (b) Optical system for reconstruction of the full-color 3-D object image on the 4-f lens system with a single-pinhole filter mask (BE: Beam expander, BS: Beam splitter, M: Mirror).
In the CS-NLUT method, three sets of PFPs for each color are pre-calculated and stored, where centers of the R-PFPs and B-PFPs are pre-shifted to the left-upper and right-lower directions with (-dx_R, dy_R) and (dx_B, -dy_B), respectively, depending on their phase factors of (fa_R, -fb_R) and (-fa_B, fb_B) to be multiplied in the FSM process, while G-PFPs are remained intact. R, G and B-holograms for the test object are then calculated with this CS-NLUT. As seen in the digital system of Fig. 7(a), the G-hologram is centered, but R and B-holograms are shifted to the x and y-directions with the distances of (-dx_R, dy_R) and (dx_B, -dy_B), respectively. For the FSM process, R and B-holograms are multiplied with their respective phase factors of (fa_R, -fb_R) and (-fa_B, fb_B), where these phase factor values are reversely related with the center-shifts of those holograms as seen in Fig. 7(a). These three-color holograms processed with both of the ISM and FSM schemes are then multiplexed into the IFS-CMH.
3.2 Performance analysis of the ISM & FSM processes on three-color holograms, their frequency-spectrums and reconstructed object images
Figure 8 shows the experimental results on three sets of the original and image-shifted color holograms, their spectrum spots and reconstructed object images. In the experiments, three-color sets of PFPs for each depth plane of the test object are initially calculated for the original NLUT, and then for implementing the proposed CS-NLUT, center locations of the R and B-PFPs are center-symmetrically shifted to the left-upper and right-lower directions, respectively, while those of the G-color PFPs are remained intact.
Fig. 8 Experimental results of the ISM effect on the positions of three-color spectrum spots and reconstructed images for each color hologram on the Fourier and reconstruction planes.
From the Abbe image-formation equations [33], the relationship between the spatial distances and frequencies is given by Eq. (6). In the experiments, the sampling interval of the 3-D object image is set to be three times of the pixel pitch of the SLM, which makes the spatial frequency of the hologram 1/(3 × 6.4µm). Thus, the spatial coordinates of the spectrums for each color wavelength are given by (−8.2mm, 8.2mm), (−6.9mm, 6.9mm) and (−6.2mm, 6.2mm), respectively. Then, the spatial shift-distances of the R and B-spectrum spots along the x and y-directions are set to be (Fx_R, Fy_R) = (1.3mm, −1.3mm) and (Fx_B, Fy_B) = (−0.7mm, 0.7mm), respectively, in the FSM process for the overlapping of those three wanted spectrum spots of Rr, Gg and Bb on the same center. Since the reconstruction distance is set to be 40cm far away from the hologram plane, the actual center-shifted distances of the reconstructed images from the R and B-holograms due to the FSM can be also calculated to be (dx_R, dy_R) = (−2.08mm, 2.08mm) and (dx_B, dy_B) = (1.12mm, −1.12mm), respectively, with Eq. (5).
To cancel out these x and y-directional shifts of the reconstructed images, the pre-shifting distances should be opposite to the actual center-shifted distances due to the FSM process. Because of the position-inversion property of the optical 4-f lens system, the pre-shifted center distances of the R and B-PFPs in the ISM process must be set to be the inverse values of the wanted image-shifted distances in the ISM process, so that they are given by (dx_R, dy_R) = (−2.08mm, 2.08mm) and (dx_B, dy_B) = (1.12mm, −1.12mm), respectively. Here, the x and y-directional image-shifts of 2.08mm and 1.12mm in the ISM process correspond to 325 and 175 pixels of those hologram images, respectively.
As seen in Fig. 8, for the R, G and B-holograms, three sets of three-color spectrum spots are generated due to the SLM-CD on the Fourier plane. All those spectrum spotss are located at the same locations as indicated by the white lines vertically passing through their centers even though each color hologram contains different information of the test object. It shows that the spatial positions of three-color spectrum spots for each color hologram would not be changed due to the ISM process on the Fourier plane. On the other hand, as seen in Fig. 8, positions of the reconstructed R and B-images are shifted to the x and y-directions with the distances of (dx_R, dy_R) = (2.08mm, −2.08mm) and (dx_B, dx_B) = (−1.12mm, 1.12mm), respectively, which are directly related with the center-shifts of the R and B-PFPs due to the ISM process.
Figure 9 shows the experimental results on the three sets of three-color spectrum spots and reconstructed object images of those image and frequency-shifted color holograms. As seen In Fig. 9, each of the image-shift color holograms is multiplied with their respective phase factors.
Fig. 9 Experimental results of the IFSM effect on the positions of three-color spectrums and reconstructed images for each color hologram on the Fourier and reconstruction planes.
As seen in Figs. 8 and 9, for the image-shift color holograms, three sets of three-color spectrum spots due to the SLM-CD are all located at the same positions on the Fourier plane, whereas positions of the reconstructed R and B-images are shifted to their locations with their respective distances of (dx_R, dy_R) = (2.08mm, −2.08mm) and (dx_B, dx_B) = (−1.12mm, 1.12mm), while the G-image is reconstructed on the center location. On the other hand, as seen in Fig. 9, for the image and frequency-shifted color holograms, three sets of three-color spectrum spots would not be located at the same areas, but six-color spectrum spot related to the R and B-holograms are diagonally shifted with their respective spatial frequency-shifts of (Fx_R, Fy_R) = (1.3mm, −1.3mm) and (Fx_B, Fy_B) = (−0.7mm, 0.7mm), respectively, which are directly related to the multiplied phase factors. At the same time, displacements of the positions of the R and B-images due to the FSM process are calculated to be (dx_R, dy_R) = (−2.08mm, 2.08mm) and (dx_B, dx_B) = (1.12mm, −1.12mm), respectively, which are exactly inverse values of those image-shifts in the ISM process.
Thus, by combined use of the ISM and FSM processes, those center-shifts of the reconstructed R and B-images can be balanced out if the center-shift values of (dx_R, -dy_R) = (2.08mm, −2.08mm) and (-dx_B, dx_B) = (−1.12mm, 1.12mm) are set to be exactly matched with the frequency-shift values of (fa_R, fb_R) = (1.3/(250*0.633), −1.3/(250*0.633)) and (fa_B, fb_B) = (−0.7/(250*0.473), 0.7/(250*0.473)), respectively.
For this case, spectrums of the R and B-hologram are generated at their diagonally-shifted spatial locations of (Fx_R, Fy_R) = (1.3mm, −1.3mm) and (Fx_B, Fy_B) = (−0.7mm, 0.7mm), respectively, instead of occupying the same spatial positions on the frequency-domain.
3.3 Optical reconstruction of the IFS-CMH on the 4-f lens system using a simple single-pinhole filter mask
Figure 10 shows an optical reconstruction system based on the 4-f lens system using a simple single-pinhole filter mask. As seen in Fig. 10, the IFS-CMH of Fig. 7(a) is loaded on the SLM, and a multi-wavelength light source composed of the R, G and B-lasers whose wavelengths are 633nm, 532nm and 473nm, respectively, is then illuminated into the SLM. Figures 10(a) and 10(b) show the optical experiment setup and optically-generated nine spectrum spots of the IFS-CMH on the Fourier plane. As seen in Fig. 10(b), three sets of three-color spectrum spots due to the CMH-CD and SLM-CD are well separated by being diagonally rearranged with the proposed IFSM process. Among them, three sets of two spectrum spots of fGr, fBr and fRg, fBg and fRb, fGb, representing the color-dispersed spectrums from the R, G and B-holograms, respectively, have to be removed. On the other hand, only those overlapped three spectrums of fRr, fGg and fBb, representing the original spectrums of the R, G and B-holograms, respectively, must be saved, with which a full-color object image can be finally reconstructed on the reconstruction plane without any color dispersion.
Fig. 10 Optical setup for full-color reconstruction of the CMH on the 4-f lens system with the fabricated optical filter mask.
For filtering out only those three original spectrums of fRr, fGg and fBb of the IFS-CMH, a single-pinhole filter mask is fabricated and inserted on the optical Fourier plane of the 4-f lens system. This optical filter mask, as seen in Fig. 10(c), blocks those six kinds of color-dispersed spectrums terms not to be passed, as well as the zero-order term designated as fzero-order, while only passing those overlapped three original color spectrums as seen in Fig. 10(d). The reconstructed images without and with the optical filter mask are shown in Figs. 10(e) and 10(f), respectively, where we can find that the zero-order and other undesired color-dispersed images have been removed with this optical filter mask and only three original color images of IRr, IGg and IBb are mixed up together to generate a full-color object image on the same center location.
As mentioned above, the pixel pitch of the reconstructed image is set to be 19.2µm in this paper. Then, the frequency of the first-order diffraction image can be given by X = Y = 1/(19.2µm). Thus, for the G-hologram, the spatial coordinate of the spectrum fRg can be given by Eq. (6). Thus, the original spatial coordinates of the fRg is given by (-8.2mm, 8.2mm), as shown in Fig. 10(b). Similarly, the coordinates of the spectrums fGg and fBg due to color dispersion can be calculated to be (−6.9mm, 6.9mm) and (−6.2mm, 6.2mm), respectively.
On the other hand, spectrums of the R and B-holograms are shifted to the right-upper and left-lower directions with (Fx_R, Fy_R) = (1.3mm, −1.3mm) and (- Fx_B, Fy_B) = (−0.7mm, 0.7mm) by being multiplied with their respective phase factors, respectively. In the ISM process, R and B-color images are shifted with the amount of 325 and 175 pixels to the right-upper and left-lower directions with (dx_R, dy_R) = (2.08mm, −2.08mm) and (dx_B, dy_B) = (−1.12mm, 1.12mm), respectively. In the FSM process, the image-shifted R and B-holograms are multiplied with their respective phase factors for returning their shifted positions due to the ISM process back.
Finally, coordinates of those nine spectrums can be calculated with the Abbe image formation theory, which are shown in Table 1. Because the optical filter mask plays a role of removing the unwanted spectrums on the Fourier plane of the 4-f lens system in the proposed method, only the overlapped three original frequency spectrums of fRr, fGg and fBb, must be passed through the filter. Thus, the filter position should be the same with the coordinates of those main spectrums on the Fourier plane of the 4-f lens system, and detailed coordinate parameters of the optical filter mask is (−6.9mm, 6.9mm) as shown in Table 1.
Table 1. Spatial coordinates of the spectrums, center locations and diameter of the pinhole fabricated on the optical filter mask.
Here it is noted that the viewing zone expects to decrease due to the introduction of the pinhole filtering process in the proposed display system. As the diameter of the circular pinhole filter increases, the corresponding viewing zone can be increased, but it also happens to make it harder optical filtering., which means there may exist a trade-off.
3.4 Reconstructed full-color holographic 3-D video images for two kinds of 3-D objects in motion
For the test video scenarios, two kinds of full-color 3-D video images composed of the self-revolving ‘Rubik’s Cube’ and ‘Soccer ball’ rotating around the ‘Rubik’s Cube’, are generated with the 3Ds Max. The first one is composed of 100 frames of video images of a self-revolving 3-D ‘Cube’ from the ‘shuffled’ to the ‘solved’ states with a speed of 24 frames per second as seen in Fig. 11(a). In addition, the second one is also composed of 100 frames of video images of the ‘Soccer ball’ rotating around the self-revolving 3-D ‘Cube’ with a speed of 24 frames per second along the clockwise direction as seen in Fig. 11(b). In particular, four video images of the 8th, 40th, 66th and 93rd frames in the second scenario, show that the ‘Soccer ball’ is supposed to be situated at the right, front, left and back positions, respectively, at those frames.
Fig. 11 Two kinds of test video scenarios composed of 100 frames: (a) Five sample video images of the self-revolving ‘3-D cube’ at the 1st, 25th, 50th, 75th and 100th frames, (b) Five sample images of the ‘Soccer ball’ rotating around the self-revolving 3-D ‘Cube’ along the clockwise direction at the 1st, 8th, 40th, 66th and 93rd frames.
For the optical reconstruction of two sets of CMHs for each of the test scenarios of Fig. 11 are generated and optically reconstructed on the 4-f lens system. Those reconstructed video object images are directly captured with the digital camera (Nikon D7100) focused on the reconstruction plane, which may simulate the human eyes for acquisition. As seen from the experiments of the reconstructed object images in Figs. 12(a) and 12(b), all those full-color 3-D video images have been successfully reconstructed on the proposed system. For their capturing, the depth of field of the camera is set to span the entire ‘Cube’. Thus, the reconstructed object images of the ‘Cube’ in all frames have been clearly reconstructed even though the ‘Cube’ is a 3-D object. But, the object ‘Soccer’ rotating around the ‘Cube’, moves in and out of the depth of field of the camera. Thus, when the ‘Soccer ball’ is rotating around the same depth plane of the ‘Cube’ such as the right and left-hand side planes of the ‘Cube’ as shown in Fig. 11(b), their corresponding ‘Soccer ball’ images have been found to be clearly reconstructed as seen in the 8th and 66th frames of Fig. 12(b). On the other hand, when the ‘Soccer ball’ is rotating out of the depth planes of the ‘Cube’ such as the front and back side planes of the ‘Cube’ of Fig. 11(b), the reconstructed ‘Soccer ball’ images have been found to be somewhat blurred as seen in the 40th and 90rd frames of Fig. 12(b). Since these experimental results have been found to be well consistent with those of the test scenario, these experiments confirm that the proposed system can reconstruct and display the 3-D video images in motion. The total video frames for each of the first and second video scenarios have been compressed into two kinds of video files of ‘Visualization 1[Fig. 12(a)]’, and ‘Visualization 2[Fig. 12(b)]’ and attached.
Fig. 12 Experimental results on the two kinds of test video scenarios of Fig. 10: (a) Five samples of the reconstructed self-revolving 3-D ‘Cube’ images at the 1st, 25th, 50th, 75th and 100th video frames (see Visualization 1), (b) Five samples of the reconstructed ‘Soccer ball’ rotating around the self-revolving 3-D ‘Cube’ along the clockwise direction at the 1st, 8th, 40th, 66th and 93rd (see Visualization 2).
All those successful experimental results show that the proposed system can reconstruct the full-color 3-D holographic video images based on the ISFM method on the 4-f lens system without any color dispersion, which confirms the feasibility of the proposed system in the practical application fields. Detailed analysis on the operational performance of the proposed system depending on the proposed FSM and ISM processes will be discussed in the following paper.
4. Conclusions
In this paper, a new single SLM full-color holographic 3-D display system based on the image and frequency-shift multiplexing (IFSM) scheme has been proposed. By combined use of the ISM and FSM processes on each of the image and Fourier planes, only the three original color spectrums of the IFS-CMH are made to move to the center area, where they are completely overlapped together and filtered out with a simple single-pinhole filter mask. Those filtered three-color spectrums are then reconstructed into the color dispersion-free full-color 3-D object images right on the center of the display plane on the optical 4-f lens system. Fourier-optical analysis and successful experimental results with two kinds of test 3-D objects in motion finally confirm the feasibility of the proposed system.
Funding
MSIT (Ministry of Science and ICT), Korea, under the ITRC support program (IITP-2017-01629) supervised by the IITP, Basic Science Research Program through the NRF of Korea funded by the Ministry of Education (No. 2018R1A6A1A03025242) and Research Grant of Kwangwoon University in 2019.
References
1. P. W. M. Tsang and T. C. Poon, “Review on the state-of-the-art technologies for acquisition and display of digital holograms,” IEEE Trans. Ind’l. Info. 12(3), 886–901 (2016). [CrossRef]
2. Ag, HoloEye Photonics, GAEA 10 Megapixel Phase Only Spatial Light Modulator (Reflective), http://holoeye.com/spatial-light-modulators/gaea-4k-phase-only-spatial-light-modulator/.
3. H. Sasaki, K. Yamamoto, K. Wakunami, Y. Ichihashi, R. Oi, and T. Senoh, “Large size three-dimensional video by electronic holography using multiple spatial light modulators,” Sci. Rep. 4(1), 6177 (2014). [CrossRef] [PubMed]
4. T. Kozacki, M. Kujawińska, G. Finke, B. Hennelly, and N. Pandey, “Extended viewing angle holographic display system with tilted SLMs in a circular configuration,” Appl. Opt. 51(11), 1771–1780 (2012). [CrossRef] [PubMed]
5. J. Hahn, H. Kim, Y. Lim, G. Park, and B. Lee, “Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators,” Opt. Express 16(16), 12372–12386 (2008). [CrossRef] [PubMed]
6. K. Wakunami, P. Y. Hsieh, R. Oi, T. Senoh, H. Sasaki, Y. Ichihashi, M. Okui, Y. P. Huang, and K. Yamamoto, “Projection-type see-through holographic three-dimensional display,” Nat. Commun. 7(1), 12954 (2016). [CrossRef] [PubMed]
7. K. Wakunami, Y. Ichihashi, R. Oi, M. Okui, B. J. Jackin, and K. Yamamoto, “Geometric Deformation Analysis of Ray-Sampling Plane Method for Projection-Type Holographic Display,” IEICE Trans. Electron. E101(11), 863–869 (2018). [CrossRef]
8. M. K. Kim, “Full color natural light holographic camera,” Opt. Express 21(8), 9636–9642 (2013). [CrossRef] [PubMed]
9. P. Su, W. Cao, J. Ma, B. Cheng, X. Liang, L. Cao, and G. Jin, “Fast computer-generated hologram generation method for three-dimensional point cloud model,” J. Disp. Technol. 12(12), 1688–1694 (2016). [CrossRef]
10. T. Zhao, J. Liu, Q. Gao, P. He, Y. Han, and Y. Wang, “Accelerating computation of CGH using symmetric compressed look-up-table in color holographic display,” Opt. Express 26(13), 16063–16073 (2018). [CrossRef] [PubMed]
11. Y. Zhao, L. Cao, H. Zhang, D. Kong, and G. Jin, “Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method,” Opt. Express 23(20), 25440–25449 (2015). [CrossRef] [PubMed]
12. T. Shimobaba and T. Ito, “Fast generation of computer-generated holograms using wavelet shrinkage,” Opt. Express 25(1), 77–87 (2017). [CrossRef] [PubMed]
13. D. Wang, C. Liu, L. Li, X. Zhou, and Q. H. Wang, “Adjustable liquid aperture to eliminate undesirable light in holographic projection,” Opt. Express 24(3), 2098–2105 (2016). [CrossRef] [PubMed]
14. K. Wakunami and M. Yamaguchi, “Calculation for computer generated hologram using ray-sampling plane,” Opt. Express 19(10), 9086–9101 (2011). [CrossRef] [PubMed]
15. M. Lucente, “Interactive computation of holograms using a look-up table,” J. Electron. Imaging 2(1), 28–34 (1993). [CrossRef]
16. Y. Zhao, K. C. Kwon, M. U. Erdenebat, M. S. Islam, S. H. Jeon, and N. Kim, “Quality enhancement and GPU acceleration for a full-color holographic system using a relocated point cloud gridding method,” Appl. Opt. 57(15), 4253–4262 (2018). [CrossRef] [PubMed]
17. T. Shimobaba, N. Masuda, and T. Ito, “Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane,” Opt. Lett. 34(20), 3133–3135 (2009). [CrossRef] [PubMed]
18. J. W. Goodman, Introduction to Fourier Optics (Roberts & Company Publishers, 2005).
19. J. Roh, K. Kim, E. Moon, S. Kim, B. Yang, J. Hahn, and H. Kim, “Full-color holographic projection display system featuring an achromatic Fourier filter,” Opt. Express 25(13), 14774–14782 (2017). [CrossRef] [PubMed]
20. Z. Han, B. Yan, Y. Qi, Y. Wang, and Y. Wang, “Color holographic display using single chip LCOS,” Appl. Opt. 58(1), 69–75 (2019). [CrossRef] [PubMed]
21. Z. Zeng, H. Zheng, Y. Yu, A. K. Asundi, and S. Valyukh, “Full-color holographic display with increased-viewing-angle [Invited],” Appl. Opt. 56(13), F112–F120 (2017). [CrossRef] [PubMed]
22. T. Shimobaba and T. Ito, “A color holographic reconstruction system by time division multiplexing with reference lights of laser,” Opt. Rev. 10(5), 339–341 (2003). [CrossRef]
23. W. Zaperty, T. Kozacki, and M. Kujawińska, “Native frame rate single SLM color holographic 3D display,” Photonics Lett. Pol. 6(3), 93–95 (2014). [CrossRef]
24. M. Makowski, M. Sypek, and A. Kolodziejczyk, “Colorful reconstructions from a thin multi-plane phase hologram,” Opt. Express 16(15), 11618–11623 (2008). [PubMed]
25. G. Xue, J. Liu, X. Li, J. Jia, Z. Zhang, B. Hu, and Y. Wang, “Multiplexing encoding method for full-color dynamic 3D holographic display,” Opt. Express 22(15), 18473–18482 (2014). [CrossRef] [PubMed]
26. T. Shimobaba, T. Takahashi, N. Masuda, and T. Ito, “Numerical study of color holographic projection using space-division method,” Opt. Express 19(11), 10287–10292 (2011). [CrossRef] [PubMed]
27. T. Kozacki and M. Chlipala, “Color holographic display with white light LED source and single phase only SLM,” Opt. Express 24(3), 2189–2199 (2016). [CrossRef] [PubMed]
28. S. F. Lin and E. S. Kim, “Single SLM full-color holographic 3-D display based on sampling and selective frequency-filtering methods,” Opt. Express 25(10), 11389–11404 (2017). [CrossRef] [PubMed]
29. S. F. Lin, Y. S. Hwang, and E. S. Kim, “Full-color holographic 3D display on a single SLM based on sampling and selective frequency-filtering of color holograms,” Proc. SPIE 10558, 22 (2018). [CrossRef]
30. S.-C. Kim and E.-S. Kim, “Effective generation of digital holograms of three-dimensional objects using a novel look-up table method,” Appl. Opt. 47(19), D55–D62 (2008). [CrossRef] [PubMed]
31. S.-C. Kim, J.-M. Kim, and E.-S. Kim, “Effective memory reduction of the novel look-up table with one-dimensional sub-principle fringe patterns in computer-generated holograms,” Opt. Express 20(11), 12021–12034 (2012). [CrossRef] [PubMed]
32. X.-B. Dong, S.-C. Kim, and E.-S. Kim, “MPEG-based novel-look-up-table method for accelerated computation of digital video holograms of three-dimensional objects in motion,” Opt. Express 22(7), 8047–8067 (2014). [CrossRef] [PubMed]
33. B. E. A. Saleh and M. C. Teich, Fundamentals of Photonics (Wiley-Interscience, 1991).
