1. Introduction

Holographic video is considered the holy-grail of 3D auto-stereo displays because it can generate the desired wavefronts which provide the reconstructed images with all the necessary 3D visual cues [1–6]. The information content of a hologram is usually represented by the space bandwidth product (SBP) and the optical extent which is the product of area and fields-of-view (FOV) of the reconstructed light controllable by the hologram [7]. The SBP is calculated by

To produce the information content required of holographic video displays, many 3D holographic display systems have been proposed, each using different scanning mechanisms to solve the technical issues from using SLMs with limited SBPs. Y. Takaki et al. proposed a horizontally scanning holographic display system [8]. An image generated by a high-speed SLM is squeezed in the horizontal direction and enlarged in the vertical direction by an anamorphic imaging system. The anamorphic imaging system, consisting of two orthogonally aligned cylindrical lenses, has different magnifications in the horizontal and vertical directions. Ridwan B. et al. use a two-axis scanning mirror for spatially tile reconstructed sub-holograms to achieve a 50 Mpixel holographic image display with 24 reconstructed objects [9]. MIT has implemented a holographic video display system [10], in which acousto-optic modulators produce travelling high-resolution one-dimensional holographic fringes which are then descanned using a polygonal scanner. Xuewu X. et al. developed a full-color full-parallax digital 3D holographic display system by using 24 physically tiled SLMs with optically scanned tiling and two sets of RGB lasers, which could display 378-Mpixel holograms at 60 Hz, with a displayed image size of 10 inch in diagonal [11].

We previously proposed Coarse Integral Holography (CIH) [12–15] to form a modest size wide field-of-view display with a large optical extent. CIH uses coarse integral optics to angularly tile an array of several low SBP subholograms. Each of the subholograms has full 3D information but is rendered from a slightly different view angle. The subhologram array is arranged behind coarse integral optics consisting of a matching lenslet array and a large common transform lens. CIH displays angularly tile SLM images to produce wider field-of-view and larger dynamic holograms than produced by a single SLM image. However common SLMs can only diffract light into a very small angle, thus possibly needing many devices for a decent field of view display. For illustration, we can calculate the diffraction angle when using a digital micro-mirror device (DMD) as the SLM. To calculate the diffraction angle, we use the 1D diffraction of a hologram presented on the SLM [16–18] expressed as

Based on Eq. (4), the hologram’s horizontal and vertical diffraction view zone can be expressed by

Therefore, our previous dCIH display was only able to use 53% of the SLMs bandwidth due to the limitations of the galvanometric mirror scanner. Due to its limited scan angle and frequency, the scanning system could not appropriately tile (without overlap) the total number of sub-holograms the SLM was capable of producing in one frame period. We have investigated adding an auxiliary resonant vertical dither scanner to create vertical sub-lines for each horizontal scan lines, thereby using the previously unused bandwidth to create more vertical line resolution [21]. Alternatively, we also investigated using two cascaded synchronous scanners in tandem [22] to double the effective horizontal scan angle, permitting us to tile without overlap twice the previous number of sub-holograms, resulting in twice the hologram’s horizontal field of view and utilizing the full bandwidth of the spatial light modulator. However, these methods increase the optical path length and make the whole system larger. Further work is also needed to reduce the optical aberrations due to the latter method’s increased field of view. To fully utilize the SLM bandwidth of the current CIH display, the scanner’s mirror size, horizontal deflection angle, or scanning frequency must be increased. However, using the best commercially available galvanometer for the aluminum mirror’s size, deflection, and frequency is insufficient. We could consider using custom beryllium mirrors, which have less inertia and greater stiffness for the same size as the current aluminum mirrors, thereby allowing us larger size, deflection or higher frequencies; however they are very expensive. Considering these issues, we use a hybrid resonant scanning (HRS) – a 2D scanning system with a vertical galvo scanner and a horizontal resonant scanner – to replace the scanning system [23]. In the HRS-based full bandwidth CIH displays, we replace the horizontal galvanometric mirror scanner with a single large mirror resonant scanner with double the scan angle of the galvo scanner at the desired horizontal scan frequency. This enables us to tile twice the previous number of sub-holograms without overlap, resulting in twice the hologram’s horizontal field of view and utilizing the full bandwidth of the spatial light modulator. Although the single SLM’s bandwidth is fully utilized, the modest size was only able to be provided because only single SLM is used.

In this paper, we propose a scalable CIH display using seamless spatial tiling of the full bandwidth images generated by two high bandwidth SLMs to fully utilize the bandwidth of its (two) SLMs and its scanners and create twice of the final image size and doubled horizontal FOV. In the HRS-based full bandwidth CIH displays, the resonant scanner’s capabilities are under-utilized. The resonant scanner is capable of achieving the same scan angle and frequency for a mirror twice the size. It is desirable to optimize the system so all components – the SLM, scanner, and optics – are fully utilized. We therefore spatially tile images generated by two spatial light modulators and scan the resulting double-wide sub-hologram with a single large mirror resonant scanner. Additional optics are provided to spatially tile the two SLMs without their bezels creating an image gap.

The display as proposed here is different from the dynamic full-bandwidth CIH display published previously [23]. The dynamic full-bandwidth CIH uses an HRS to fully utilize the SLM’s bandwidth and obtain a large FOV, which provides a valuable method to address the small FOV issue due to the very small diffraction angles of the common SLMs currently available. However, how to increase the image size in a dynamic full-bandwidth CIH display using the limited area on an SLM while maintaining the same FOV remains a significant challenge. Here an integrated spatial image titling method is proposed under the full-bandwidth CIH framework to increase the size of the reconstructed holographic images using the same size SLMs and maintain the same full-bandwidth FOV at the same time. The spatially tiling of the full bandwidth images generated by two high bandwidth SLMs using a proper optical configuration allows us to create a scalable CIH display by adding multiple modulators and scanners as required for the information content of the intended holograms. This opens a door to the generation of holographic images with both a large image size and a large FOV in both directions.

2. Scalable CIH approach using integrated spatial image tiling with two high bandwidth DMDs

The full bandwidth CIH display system includes a HRS, two 4f-systems, a Fourier transform lens, a DMD and a laser sub-system. The HRS is composed by a one-dimensional resonant scanner (1D-RS) and a one-dimensional galvanometric scanner (1D-GS). The 1D-RS is located on the focal-plane of the first 4f-system, which is used to change the angle of incidence on the 1D-RS scanner, thus producing the horizontal fields of subholograms. The first 4f-system, consisting of two convex lens, is located the between Fourier transform lens and 1D-RS and enlarge the hologram size from the Fourier transform lens. The 1D-RS and 1D-GS is used to raster the sub-holograms (with virtual holographic lenses) into an array, and equivalently scan the horizontal and vertical fields of view. Another 4f-system is used as the final optical system in scanning system to arrange the subholograms and virtual lenslets and angularly tile them into a larger wider field of view hologram, or equivalently relay the angularly scanned hologram from the scanner into free space. The resonant scanner scans sub-holograms to form the horizontal FOV. The maximum scanning angle represents the maximum FOV of the display system. We can lay out different view sub-holograms over the whole scanning range. The galvanometric scanner scans to form the vertical FOV by distributing horizontal sub-holograms onto the desirable vertical sub-view. The space bandwidth product (SPB) of the full bandwidth dynamic CIH display is expressed by

The resonant scanner is an electromagnetically driven moving mirror device oscillating at a fixed frequency at or near the scanner’s resonance [24–27]. Due to the scanner’s resonant amplification, it can achieve over twice the scan angle as the galvanometric scanner at the same frequency and mirror size. This allows us to double the hologram’s horizontal field of view and to utilize the full bandwidth of the spatial light modulator. The total view is calculated by

The use of the resonant scanner as a replacement or the original galvanometric scanner provides the total horizontal view of FOV'_x= 2×FOV_x, thus, the number of views of each scan line becomes VN' = 2VN. The bandwidth utilization of the SLM becomes SBP'=2SBP, or twice that of the original dynamic CIH system. The capability of the proposed scanner system, C, is calculated by the product of the scanner mirror’s area, deflection angle (field of view), and frequency F. C=2×FOV’_x ×FOV_y×F×W×H=2C₁, or twice the scanning capability, C₁, of the non-resonant galvanometric scanner system of the original dynamic CIH.

Using a resonant scanner as described (or two tandem scanners [22]) allowed us to use the full bandwidth of the DMD SLM in the CIH system. To further increase the resolution, size or field of view of the CIH system, we need to add information content, and thus additional SLMs. The rest of the display system, i.e. the scanner and the optics, must also be able to accept, transform, and distribute the additional information (i.e. resolution, extent, or angle) of light modulated by the additional SLMs. A scalable CIH system must be able to create a display of a desired resolution, size and field of view ability (that is to accommodate arbitrary information content) by adding additional modulators, scanners and optics.

The original all-solid-state CIH system was scalable. The display’s information content could be increased by using additional SLMs with a corresponding lenslet to the sub-hologram and lenslet arrays, which would then be angularly tiled by the final common transform lens. However, if DMD SLMs were to be used, the modulators’ bezels introduce large gaps in the sub-hologram array. However, this only reduces the size of the final hologram, since sub-holograms spatially overlap and are angularly tiled at the final hologram plane. As long as the corresponding lenslet in the lenslet array are centered on their corresponding SLMs and, there will be neither view gaps nor spatial gaps in the final hologram. However, using individual physical DMDs to present the subholograms restricts the subholograms to the DMDs low space bandwidth product and does not take advantage of the DMDs high frame rate and large bandwidth.

In the scanned dynamic CIH system, a single DMD is scanned to form multiple DMD sub-holograms spatially tiled on the sub-hologram plane. By paring one of the transform lenses (either physical or incorporated into the hologram) – an “attached lens” – an equivalent system is created which uses the scanner to angularly tile multiple sub-holograms imaged onto the scanner mirror, and a relay to demagnify the hologram form a real image in space. To further increase the information content using multiple DMDs would require a scanning system to tile the multiple subhologram arrays without gaps, or equivalently angularly tile multiple DMD images on the scanner. Although work is being done on such systems, here we look at spatially tiling multiple DMDs on the scanner (and real-images in space), or equivalently by angularly tiling the subholograms from different DMDs in the subhologram array. However, when spatially tiling a system, care must be taken to ensure spatial gaps are not formed in the final holographic image.

In the full bandwidth CIH, we use a resonant scanner oscillating at the same frequency (70 Hz) but with a larger area mirror (50 mm) as the horizontal scanner, while a high performance galvanometric scanner with a larger area mirror (90 mm) as the vertical scanner. The mirror size and the scan angle of resonant scanners are inversely proportional to the frequency. The maximum mirror size is 50 mm using the current scanning frequency (70 Hz) and FOV of 40°, while the current hologram image size is only 12mm×10 mm without optical demagnification. This means we were only able to use half of the mirror size of the resonant scanner without the decrease of the display resolution. Here, we propose a scalable CIH display with spatial tiling by using the current full bandwidth CIH display configuration but replacing the single DMD with multiple ganged DMDs (at the full bandwidth condition), to maximize the use of the large resonant scanner. We also propose an optical system – delay mirror array structure – to seamlessly spatially tile multiple DMDs.

Two DMDs are tiled horizontally to combine into a large hologram, attached two Fourier lens to get two holographic images, which is scanned by the common large resonant scanner and the common large scanning lens. Based Eq. (4), the holographic image size is determined by

Figure 1 shows the principle of the proposed CIH display. The display system includes three lasers, two SLMs, two Fourier transform lens, two imaging lens, a delay mirror array structure (DMAS), a HRS scanner and a 4f-system. The HRS is composed of a one-dimensional resonant scanner (1D-RS), a one-dimensional galvanometric scanner (1D-GS). The DMAS removes the gap between spatially tiled holographic images from two different DMDs. The combined holographic image is scanned by the HRS, where the 1D-GS is located on the focal-plane of the Fourier transform lens, still keeping the integral image at different scanning views. The 4f-system is used as the final optical system in scanning system to arrange the virtual subholograms and lenslets and angularly tile them into a larger array, or equivalently relay the angularly scanned hologram from the scanner into free space. The scan frequency of the horizontal raster scanner is 70 Hz with the FOV of 40° in order to ensure the same frame frequency as the current CIH video displays. Twice as many frames per second are displayed than previously, thus utilizing the entire bandwidth of the SLM.

 

Fig. 1. Display principle using two DMDs in full bandwidth CIH, R1 and R2 are two reflective mirrors, D1 is a beam splitter, and L1∼L6 are the lens.

Download Full Size | PDF

The scalable CIH display with spatial tiling has a noticeable seam between two parts of holographic images from two SLMs due to the gap between the two SLMs from the bezels and electronics [see Figs. 2(a) and 2(b)]. There is a seam in the actual image, not in the views. Each image is composed of one half from one DMD, and the other half from the other DMD. To address this issue, we propose a delay mirror array structure abutting multiple SLMs with minimal spatial gaps to maximize the use of the large resonant scanner. Figure 2 shows the principle of the DMAS. The DMAS is composed of four mirrors, where two outer mirrors could draw the distance of two holographic images; while inner mirrors could adjust the two holographic images to implement the seamless conjoin [see Fig. 2(c)].

 

Fig. 2. Removing the gap between two DMDs, (a) DMAS structure, (b) actual optical configurations, (c) optical ray tracking of DMAS.

Download Full Size | PDF

The optical parameters of the DMAS determine the tiled holographic image positions on the scanner plane. For a seamless conjoining, these optical parameters can be obtained through two methods: an optical ray trace mathematical model or Zemax simulation. The optical ray trace mathematical model can be constructed through the ray geometric vector relationship of the DMAS. This method can know the accurate position at each plane (i.e. R1-R4), but it is complicated to construct and process. Zemax simulation is relatively straightforward to use in analyzing the DMAS. The simulation results are shown in Fig. 2(c), from which we can obtain the rotation angles and the distance between R1 and R2 (R3 and R4) to control accurately the holographic image positions and design the DMAS for seamless conjoining of the delivered holographic 3D images.

In the proposed system, we use a shared light source, composed of three lasers, to simultaneously illuminate two DMDs. To implement the image spatial tiling, the outgoing light direction vectors from two holograms should be completely parallel in the horizontal and vertical directions. The outgoing light direction vectors of the two holograms are determined by the incoming light direction vectors, while the incoming light direction vectors are determined by the positions of two DMDs and relay optics (i.e. mirrors and lenses) of collimated laser beams. Considering the micro-mirror structure of the two DMDs, the incoming light vector direction has a fixed relationship. A micro-mirror has four energy states, illumination state, on state energy, flat state, and off state energy, when it is at different position (see Fig. 3).

 

Fig. 3. Vector relationship between incoming light and two DMDs.

Download Full Size | PDF

The proposed system keeps the outgoing light vector direction horizontal. So, the on energy state is used as the outgoing direction of holograms, while the illumination state is used as the incoming direction. The vertical projection of the incoming light vector has an angle of 24° to the hologram outgoing direction. The horizontal projection of the incoming light vector has an angle of 2a to the hologram outgoing direction, where a is the rotation angle of the DMD to the to the hologram outgoing direction. Let the normal vector of the DMD be denoted by $\vec{\eta } = (0;0;1)$ when a micro-mirror is located at a flat state. When the DMD rotates a angle of θ, the normal vector of the DMD becomes

3. Experimental results

3.1 Setup

Based on the principle of the proposed system shown in Fig. 1, a proof-of-concept system for a full bandwidth CIH displays with two SLMs is established, which is shown in Fig. 4. In our system, we use DMDs as the MEMS-SLMs, just as in the dynamic CIH system. In this tiled system, the two DMDs are the same device and model number (V-7000 VIS, ViALUX GmbH). The DMD has 1024×768 micro mirrors, with a micro mirror pitch of 13.7µm, which means the active mirror array area is 14×10.5 mm². The DMD can achieve outstanding pattern frequencies of 22,272 global array updates per second, which means the full bandwidth is 17.5Gbit/s. The usable spectral range covers all wavelengths from 350 nm to 2500 nm. In the proposed system, the two DMDs provide 35Gb/s because they can use the full bandwidth.

 

Fig. 4. Experimental setup.

Download Full Size | PDF

The lasers are used to sequentially illuminate the DMD. The lasers’ red spectral band is centered at 450 nm, the green band at 532 nm, and the blue band at 635 nm. The optical configuration is divided into two parts: laser optical subsystem and HRS optical subsystem. The laser optical subsystem adopts a collimating and expanding structure with lenses of focal lengths 15 mm (L1092A) and 150 mm (l1374A). Moreover, we also use a long-pass dichroic mirror (DMLP490L) with a 490 nm cutoff, 550 nm cutoff (DMLP550L), and a reflective mirror to implement the beam combiner. The HRS optical configurations include a 4f-system, which is labeled in Fig. 1. The L1 lens and L2 lens have the focal length of 150 mm and L3 lens and L4 lens have focal lengths of fl=200 mm. In 4f-system, the L5 lens has the focal length of 100 mm and L6 lens has 200 mm. Moreover, the f/# of the L5 lens and L6 lens should be less than 1.123 because of the scanning angle of 48°.

To implement full bandwidth CIH display, we customized a resonant scanner (SC-21) from Electro-Optical Products Corporation with 50 mm mirror. The scanner operates at fixed 70 Hz frequency. The auto gain control AGC driver (AGC-220-BNC), a feedback amplifier driver using the scanner as a frequency sources, is used to sustain the scanner operation at resonant frequency. The AGC driver can provide a high stability (0.01%) and a position output signal (POS). We use the POS to synchronously control the DMD and Y scanner. The maximal scanning angle is 70° and the scanning angle can be set by AGC amplitude adjustment. In the proposed system, the scanning angle is set to 48° and thus double of horizontal fields of view scanned by the resonant scanner. The galvanometric scanner (6260HM44A) is integrated with a 50 mm mirror (6M2650X44B050S4) with beam rotation of +/-22°. The mirror is a beryllium substrate, flatness of λ/2, multilayer hard dielectric over silver, R>98% over 550nm-15µm. The scanner driver is a MicroMax single axis integrated with a high power servo driver amplifier.

To manage two DMDs, the synchronization matching and controlling of all devices is implemented by a Labview software platform (NI LabVIEW 2013) in combination with a multi-function DAQ card (X Series DAQ), where we use a repeatable trigger way to produce the control signals required. Figure 5 shows the control structure for operating our CIH display system. Two DMDs have the same configuration, with each DMD’s serial number used as its identification address. The frame sync unit sets the frame output sync signal to control the three lasers. The slave mode unit sets the display mode of two DMDs, where the slave mode is used because the DMD displays frames and produces synch pulses based on the trigger input. We could use the same trigger to synchronously display two holograms. We also use the infinite loop mode for continuous hologram sequence video display to achieve the full bandwidth display. Two scanners use a boustrophedon scanning pattern, where the galvanometric scanner is controlled by a vector signal generated based on the resonant scanner. The DMD trigger is also generated based the resonant scanner. Therefore, all devices synchronously work together, enabling the DMD and scanner mirrors to tilt simultaneously to achieve a large FOV and the double holographic image size at the full combined bandwidth of the two DMDs.

 

Fig. 5. (a) Controller structure; (b) DMD working mode.

Download Full Size | PDF

3.2 Results

A 2D word and a 2D target, designed in 3ds Max (Autodesk 3ds Max 2012), is used to calibrate the holographic stitching of two DMDs. Fourier holograms of these objects are calculated and presented on the DMD and scanning system. Figure 6 shows the holographic stitching results using the proposed method. Without DMAS, the seams between two holographic images are noticeable due to the gap between the DMDs from the borders and electronics. After the introducing, the seams between two holographic images could be removed. The experimental results show the DMAS structure could conveniently adjust two holographic images to create a seamlessly mosaic. Moreover, the differences of brightness can also be observed between two holographic images from two DMDs.

 

Fig. 6. Holographic stitching using the proposed system, (a) and (c) reconstructed holographic image without DMAS, (b) and (d) reconstructed holographic image with DMAS.

Download Full Size | PDF

Figure 7 shows the doubling of the horizontal holographic image is achieved due to the spatial tiling of two DMDs. The tiling of two DMDs increases the horizontal hologram size and the reconstructed holographic image size to be double that of one DMD, where each DMD provided the half of the whole holographic image. In addition, in the proof of concept system, a mask is used to eliminate the zero-order, high-order image and conjugate image, while only selectively keeping the holographic image of the object.

 

Fig. 7. Reconstructed holographic images with two DMDs, (a) and (e) a holographic image independently displayed by two DMD, (d) and (h) reconstructed holographic image when a DMD displays half part of the whole image, (b),(c), (f) and (g) the display result of a DMD in the whole holographic image.

Download Full Size | PDF

We also use 3ds Max (Autodesk 3ds Max 2012) to design a 3D object model and to produce different image parallax views (subholograms). A variety of computer generated hologram (CGH) rendering algorithms can be used to generate the holographic diffraction patterns [28–32]. Here a layer-based CGH algorithm [23] is employed. The calculated holograms are displayed on the DMD and reconstructed with the scanning system. The DMD, lasers, resonant scanner, and galvanometric scanner synchronously work under the control of the LabView soft controller (NI LabVIEW 2013). Different parallax sub-holograms are delivered to their corresponding array positions by the scanners.

In the prototype, we use the resonant scanner to synchronously control the DMD, lasers, and the galvanometric scanner. Upon system startup, the DMD working parameters are configured (e.g. the DMD display time is set to 44µs) and the scanners are set to produce a boustrophedon scanning pattern with the resonant scanner oscillating at 70 Hz. and the galvanometric scanner operating at 23.33 Hz. The software controller sets the frame output synchronization, which is used to synchronously control the laser. At every rising edge signal generated each scan period by the resonant scanner, the software controller produces trigger signals for the DMD trigger and the driver signal for the galvanometric scanner. At each rising edge of the trigger signals, the DMD updates and displays a hologram pattern. During the half period of the resonant scanner (1/140s), the DMD displays 162 holograms, which includes 54 views with each view composed of three color component images. The proposed system can continuously display hologram sequences because the resonant scanner to synchronously control the DMD, lasers, and the galvanometric scanner.

Figure 8 shows the reconstructed holographic images of a combined 3D teapot at different viewing angles. The figure shows the horizontal size of the holographic image us doubled due to the spatial tiling of two DMDs in the horizontal direction, the display achieves the full-bandwidth horizontal field-of-view because two DMDs are appended to the CIH system behind a common large resonant scanner.

 

Fig. 8. reconstructed holographic images of a 3D teapot, (a) and (b) a holographic image of two DMDs at 0°, (d) Whole holographic image at °0, (c),(e),(f),(g) reconstructed holographic image at +20°, -20°,+1.6°, and °-1.6°.

Download Full Size | PDF

Finally, we analyze the display capability of the proposed method. The display’s scanner capability is calculated by:

When the original dynamic CIH system displayed a hologram with a 10mm×10 mm extent, the scanning capability of the sub-scanning system is C=0.8°×0.8°×6×30×23.33×10×10. = 2.7×10⁵ mm² deg² line/sec Displaying the same size hologram on the proposed system, the display capability achieved is C= (0.8°×0.8°×6×60×23.33×20×10) =10.8×10⁵ mm2 deg² /sec which is the 4-times of our previous system. We doubled not only the size but also the field of view of the original system. Moreover, both DMDs operate at full bandwidth, and the scanners are used to their full capabilities.

4. Conclusion

To utilize the large bandwidth of holographic video display using SLMs with limited space-bandwidth-products but high frame rates, a dynamic Coarse Integral Holographic display rapidly scans a single low SBP SLM image to form a subhologram array for integral optics to angularly tile into a hologram with a large field-of-view and modest size. Previous prototypes were able to fully utilize the SLMs bandwidth but modest size despite of introducing a resonance scanner in addition to the existing galvanometer mirror scanner. Here, we demonstrate a new CIH display structure-full bandwidth scalable CIH displays to fully accommodate the bandwidth of two SLMs spatially tiled seamlessly using a delay mirror array structure. Comparing with our previous dynamic CIH display, the proposed method could double the horizontal size, double the horizontal field-of-view, and use the full bandwidth of both SLMs and capabilities of the scanner. The two SLMs are used to increase the hologram’s horizontal size, while the resonant scanner allows the system to increase the display’s horizontal field of view and fully utilize the two SLMs bandwidth. This method enables spatially tiling of the full bandwidth images generated by two high bandwidth SLMs onto a resonant scanner and utilizing fully the capability of both SLMs. Combined with our previous technique of using multiple scanners in tandem to further increase the effective scanning capabilities, it is now possible to create scalable CIH displays by adding multiple modulators and scanners as required by the desired information content of holograms.