A full-color compact 3D see-through near-eye display system based on complex amplitude modulation

Zhiqi Zhang; Juan Liu; Qiankun Gao; Xinhui Duan; Xueliang Shi

doi:10.1364/OE.27.007023

1. Introduction

Since its appearance, augmented reality (AR) technique has been attracting more and more attention continuously. Among existing AR devices, near-eye display (NED) systems are the most popular research hotspots as they are portable for personal use. NED devices can be used for social communication, navigation, education and entertainment, and are considered to be the personal device of next generation after personal computer and smartphone. Current see-through NED devices provide stereoscopic 3D perception basically by binocular parallax which also brings vergence-accommodation conflict (VAC) and causes visual fatigue to the wearer. Therefore, in most recently, several methods beyond binocular parallax have been proposed and investigated, including light field display [1,2], integral imaging (II) [3–6], multi-focal-plane (MFP) display [7–9], Maxwellian display [10,11], and holographic 3D display [12–16]. For light field display, Lee et al. utilized projectors and HOE layers to realize light field AR display [1]. Each layer diffuses a projection beam from a projector to form additive light field and real world scenes can be seen through the HOEs. But the optical blur and ghost effect of the display results degrade the image quality to some extent. Integral imaging could be regarded as one way to realize light field display. An II based NED system can provide full-parallax 3D images with good occlusion effect. The system structure is compact so it can cooperate with extra elements such as liquid lens [3] or holographic optical elements (HOEs) [4] to improve the performance. Nevertheless, for II-based NED systems, the trade-off among the viewing angle, lateral resolution and the depth range still exists. For MFP display, Hu et al. applied free-form optics on the MFP system to present see-through AR performance [7]. They further analyzed the engineering challenges in the integration of MFP system [8]. The variation of the image planes, however, depends on the use of electro-optical modulator which makes the system structure quite bulky and complex. Maxwellian display is another approach to alleviate the VAC. Instead of reconstructing 3D images with true depth cues, Maxwellian display presents images which are always in focus regardless of the focal length of human eyes. Kim et al. proposed a see-through Maxwellian NED by aid of holographic waveguide [10]. But as Maxwellian display systems guide light into human eyes through pupil directly, the wearers may feel uncomfortable or irritation after wearing a long time. Holographic 3D display is regard as a promising technology since it provides full depth cues in a natural viewing way. In such NED systems computer generated holograms (CGHs) are uploaded on a spatial light modulator (SLM) to reconstruct 3D signals. An optical combiner superimposes virtual signals on real world scene. Li et al. proposed a holographic 3D NED system by employing a multi-functional HOE which performs the optical functions of a mirror and a lens simultaneously [12]. The system is simplified by the proposed HOE and good see-through AR effect is presented. Chen et al. improved the layer-based CGH algorithm and built their holographic NED test system [13]. Despite of the improvements so far, there still exists limitations to be overcome. Phase-only SLMs are commonly used in holographic display because of its high efficiency. So there is an information loss during the CGH encoding as the amplitude distribution is omitted. The other issue is the time consumption. Iteration operation is often performed during CGH calculation to improve the reconstruction quality which makes it difficult to realize dynamic display.

The complex amplitude modulation (CAM) technique is one way to realize holographic 3D display with advantage of reconstructing high quality 3D images without information loss and iterations. In recent years, the CAM based NED systems experience fast development and a lot of relevant studies have been done by our research group already [17–19]. In our previous work, the CAM technique was applied in the design of optical see-through 3D-NED systems for the first time [17]. The proposed system presents high-quality 3D images with sufficient depth cues to eliminate VAC and visual fatigue. We further improved the structure to be more wearable and a compact prototype is implemented [18]. However, the proposed NED systems present only single-color images which makes the viewing effect less vivid and degrades the viewing experience. This is mainly caused by the chromatic dispersion effect of the SLM. Besides, in existing CAM based 3D-NED systems, the grating plays an essential role in complex amplitude reconstruction. As another chromatic dispersive component, the grating makes it more difficult to realize colorful 3D display.

In this paper, a full-color compact 3D see-through NED system by CAM is proposed. The system involves a standard coherent image processing system with a holographic grating at the Fourier plane. Two position-shifted phase holograms are extracted from object wavefront and uploaded on a single phase-only SLM. The holograms can be coupled via the grating and overlapped automatically at the output plane of the system to synthesize a complex hologram. Colorful display is mainly realized by time-multiplexing. CGHs for different wavelengths are calculated separately and carefully calibrated. Color managements are performed to show the color space and chromatic aberration of the system is analyzed. Pre-compensation is operated before optical experiments to improve the image quality. Dynamic colorful display ability is also tested for the proposed 3D NED system.

2. Principle and system

2.1 Principle of complex amplitude modulation method for colorful display

Figure 1 depicts the principle of the proposed CAM method for colorful display. Two Fourier lenses (L₁ and L₂) constitute a 4-f system. The focal length of two lenses are f₁ and f₂ respectively. The grating is located at the back focal plane of L₁ and plays a role as frequency filter. Firstly, the target colorful image is decomposed into three color components and CGHs for different wavelengths are calculated separately. For each wavelength, two position-shifted phase holograms are uploaded on the input plane. If the distance between two holograms $d_{h}$ and the grating period $Δ$ satisfies a specific condition, the holograms will be coupled via the grating and overlapped at the output plane automatically to synthesize a complex hologram. After propagating distance $z_{0}$ , the complex wavefront for single color will be reconstructed [20]. Finally, through time-multiplexing, the target colorful image can be observed with high image quality.

Fig. 1 Schematic of CAM method for colorful display.

Download Full Size | PDF

The process can be further explained mathematically in detail. For single-wavelength component, as the image reconstruction is usually implemented by Fresnel diffraction, we represent a complex Fresnel hologram as follows:

H = A \exp (i θ)

where

A

and

θ

represent the amplitude and phase distributions respectively and

i

denotes

\sqrt{- 1}

. We extract two phase holograms from the complex hologram because phase-only SLM has higher diffraction efficiency and is commonly used for holographic display. The decomposition can be expressed as follows:

\exp (i θ_{1}) + \exp (i θ_{2}) = A \exp (i θ)

where

θ_{1}

and

θ_{2}

refer to two phase holograms. After some mathematically deduction,

θ_{1}

and

θ_{2}

can be solved as:

{\begin{cases} θ_{1} = θ +arc \cos (\frac{A}{2}) \\ θ_{2} = θ - arc \cos (\frac{A}{2}) \end{cases}

The solved two phase holograms are uploaded on the input plane of the 4-f system. They propagate to the back focus of L₁ and are multiplied by the grating. The complex amplitude at the Fourier plane is given by:

\begin{array}{l} u (x_{1}, y_{1}) = \frac{1}{i λ f_{1}} F {t_{1} (x_{0}, y_{0} - \frac{d_{h}}{2}) + t_{2} (x_{0}, y_{0} + \frac{d_{h}}{2})} \cdot G (x_{1}, y_{1}) \\ = \frac{1}{i λ f_{1}} T (\frac{x_{1}}{λ f_{1}}, \frac{y_{1}}{λ f_{1}}) \cdot [\frac{1}{2} + \frac{m}{2} \cos (\frac{2 π y_{1}}{Δ})] \end{array}

where

(x_{1}, y_{1})

and

(x_{0}, y_{0})

denote the coordinate of the Fourier plane and the input plane, respectively.

F {}

stands for Fourier transformation operator and

T

means frequency spectrum.

λ

is the wavelength of the laser.

t_{1}

and

t_{2}

represent two phase holograms with a separated distance

d_{h}

.

G

represents the superposed grating. We assume the grating to be an amplitude cosine grating for brief description while

Δ

is the period of the grating and

m (0 < m \leq 1)

is the modulation depth of the grating. After some mathematical manipulations, the complex amplitude at the output plane can be represented as:

\begin{array}{l} u (x_{2}, y_{2}) = \frac{1}{i λ f_{2}} F {\frac{1}{i λ f_{1}} T (\frac{x_{1}}{λ f_{1}}, \frac{y_{1}}{λ f_{1}}) \cdot [\frac{1}{2} + \frac{m}{2} \cos (\frac{2 π y_{1}}{Δ})]} \\ = \frac{1}{M} {\frac{1}{2} [h_{1} (\frac{x_{2}}{M}, \frac{y_{2}}{M} - \frac{d_{h}}{2}) + h_{2} (\frac{x_{2}}{M}, \frac{y_{2}}{M} + \frac{d_{h}}{2})] \\ + \frac{m}{4} [h_{1} (\frac{x_{2}}{M}, \frac{y_{2}}{M} - \frac{d_{h}}{2} + \frac{λ f_{1}}{Δ}) + h_{2} (\frac{x_{2}}{M}, \frac{y_{2}}{M} + \frac{d_{h}}{2} + \frac{λ f_{1}}{Δ})] \\ + \frac{m}{4} [h_{1} (\frac{x_{2}}{M}, \frac{y_{2}}{M} - \frac{d_{h}}{2} - \frac{λ f_{1}}{Δ}) + h_{2} (\frac{x_{2}}{M}, \frac{y_{2}}{M} + \frac{d_{h}}{2} - \frac{λ f_{1}}{Δ})]} \end{array}

where

(x_{2}, y_{2})

denotes the coordinate of the output plane and

M = - f_{2} / f_{1}

is the magnification of the system. If the separated distance

d_{h}

and the grating period

Δ

satisfy the following condition:

d_{h} = \frac{2 λ f_{1}}{Δ}

Equation (5) can be rewritten as:

\begin{array}{l} u (x_{2}, y_{2}) = \frac{1}{M} {\dots + \frac{m}{4} [h_{1} (x_{2}, y_{2}) + h_{2} (x_{2}, y_{2})] + \dots} \\ = \frac{1}{M} {\dots + \frac{m}{4} [\exp (i θ_{1}) + \exp (i θ_{2})] + \dots} \\ = \frac{1}{M} {\dots + \frac{m}{4} A \exp (i θ) + \dots} \end{array}

where the second term is the magnified complex hologram by recognizing the definition in Eq. (1). The other terms are higher diffraction orders and are omitted for simplicity. So far, the complex hologram is synthesized at the output plane. Then the wavefront propagates a

z_{0}

distance to the reconstruction plane where the desired real image can be observed by Fresnel diffraction:

O (x_{2}, y_{2}) = \frac{1}{i λ z_{0}} \exp (i k z_{0}) \iint u (x_{2}, y_{2}) \cdot \exp {i \frac{k}{2 z_{0}} [{(x - x_{2})}^{2} + {(y - y_{2})}^{2}]} d x_{2} d y_{2}

where

k = 2 π / λ

is the wave number.

In fact, the principle can be easily understood from Eqs. (5) and (7). The grating at the Fourier plane produces three duplicates of each phase hologram at the output plane. The first order duplicate of $h_{1}$ and the minus first order duplicate of $h_{2}$ overlap at the center of the output plane to synthesize a complex hologram when Eq. (6) is satisfied. Consequently, the designed image is obtained with high quality on the reconstruction plane via Fresnel diffraction.

The relationship between the grating and CGHs of different components are further illustrated in Fig. 2. The distances for three wavelengths are written as d_r, d_g and d_b respectively. From Eq. (6), we can see that when the grating period is settled, the separated distances between two holograms are proportional to the wavelengths which means the holograms for red component has the biggest separated distance while the blue component has the smallest. This is because the grating is chromatic dispersive for different wavelengths so the CGHs should be designed and calculated separately. The function of the grating is to synthesize two sub-holograms into a complex hologram at exactly the same position for each wavelength. While displaying, the CGHs are uploaded on the SLM in time sequence and synchronized with three-color lasers. By time-multiplexing, colorful 3D images can be seen through persistence of vision.

Fig. 2 Relationship between the grating and CGHs of different components.

Download Full Size | PDF

2.2 Schematic of 3D full-color near-eye display system

Based on the principle above, a 3D full-color NED system is designed and the schematic is illustrated as Fig. 3(a). Three color lasers are used as light source with wavelengths of $λ_{1}$ , $λ_{2}$ and $λ_{3}$ respectively. After collimating and beam expanding, the combined beam illuminates the SLM through a beam splitter (BS). A 4-f system composed of two doublet lenses (DL) follows the SLM. The doublet lenses are used to correct chromatic aberration. A sinusoidal grating (G) is placed right at the back focus of the first doublet lens to realize complex amplitude modulation. The mirror (M) reflects light to another BS which is used to superimpose the virtual signals on real world scene to realize see-through effect. A compact and lightweight prototype is designed to assemble the optical elements. The proposed prototype has a simple structure thus it is quite suitable for portable and wearable application.

Fig. 3 (a) Schematic of the proposed 3D full-color see-through NED system. BS means beam splitter, DL means doublet lens, M is mirror and G refers to grating, (b) viewing effect of the proposed system.

Download Full Size | PDF

Figure 3(b) shows the colorful viewing effect of the system. The real world scene can be seen through the viewing window of the proposed system. At the same time, colorful 3D virtual images with correct depth cues are presented in front of human eyes and are superposed on the real world scene. The virtual signals can be reconstructed with full color and displayed at continuous depths along with real objects. The system can present colorful images with right depth cues and are free of VAC problem.

2.3 Color management analysis

For colorful display, one issue to be considered is the color management. According to the International Commission on Illumination (CIE) color system, the tristimulus value of three colors are defined as:

{\begin{cases} X = k \sum_{λ} φ (λ) \bar{x} (λ) Δ λ \\ Y = k \sum_{λ} φ (λ) \bar{y} (λ) Δ λ \\ Z = k \sum_{λ} φ (λ) \bar{z} (λ) Δ λ \end{cases}

where

k

is the coefficient,

φ (λ)

is the color stimulus function of the light seen by the observer,

\bar{x}

,

\bar{y}

,

\bar{z}

are the color matching functions of the CIE 1931 standard observer. The wavelengths of the laser source used in the proposed system are 639nm, 532nm and 473nm respectively. Based on it, the color gamut of the system is shown in the CIE x-y color space in Fig. 4. Compared with standard Red Green Blue (sRGB), the color gamut of the proposed NED system is much bigger which means the proposed system can present more plentiful colors.

Fig. 4 Color gamut of the proposed system.

Download Full Size | PDF

Taking optical elements used in the system into consideration, the chromatic aberration of the system needs to be discussed. Two lenses are employed to constitute the 4-f system. As we know, singlet lenses suffer from severe chromatic aberration. For paraxial ray situation, images with different colors are imaged on different positions which is called longitudinal chromatic aberration. So in the proposed system, doublet lenses are employed to correct the chromatic aberration. Besides, all the mirrors and beam splitters used in the system are carefully selected so that they work in a wide spectrum that covers the spectral width of the lasers. Therefore the system is able to reconstruct colorful images with high color fidelity. Still, there exists some cumulative chromatic aberration in the system so it’s necessary to make pre-compensation and fine adjustment of the holograms which will be discussed in section 3.2.

3. Optical experiments and discussions

3.1 Optical setup and specifications

The prototype and optical setup of the proposed 3D full-color see-through near-eye system is illustrated as Fig. 5(a). The shell of the prototype is fabricated through 3D printer technology and the material used is polylactic acid (PLA) which is cheap and hard enough. The size of the structure is 176.4mm × 34.8mm × 35.4mm with a 65.2mm observation accessory. Two doublet lenses constitute the 4-f system to filter the background noise of the SLM panel. The focal lengths of two lenses are $f_{1}$ : 60mm and $f_{2}$ : 75mm respectively. The bigger focal length of the second lens can realize a magnification of the images. The grating is placed right at the back focal plane of the first lens and the grating period is 100μm after proper choice according to Eq. (6). The grating is fabricated by holographic interference for its simple process. The parameters of the fabrication process are as follows: working wavelength $λ = 532 n m$ , angle of two interference beams $β = 0.305 °$ , then the period of the holographic grating is decided by: $Δ = λ / 2 \sin (\frac{β}{2})$ . Figure 5(b) shows a fabricated example of the holographic grating. A band-pass filter is introduced behind the grating to obtain a better displaying effect. For the best viewing experience of human eyes, the eye relief of the system is designed to be 10mm with an eyebox of 21.7mm(H) × 16.9(V)mm. The calculation of eyebox is further illustrated in the section of discussion in details. The weight of the module with inner optics is 220.0g.

Fig. 5 The developed 3D full-color NED prototype, (a) optical setup with inner assembling details, BPF means band-pass filter, (b) a fabricated example of grating.

Download Full Size | PDF

Optical experiments are performed to test the proposed 3D full-color see-through NED system. The illumination light consists of three color lasers with wavelengths of 639nm, 532nm and 473nm respectively. The SLM used is Holoeye Pluto with 8μm pixel pitch, 1920 × 1080 resolution, and the phase modulation range is [0, 2π]. For better displaying effect, the SLM panel is rotated 90° to fit the viewing habit of human eyes. The resolution of the complex field is the same with the resolution of the sub-hologram which is set to be 400 × 1000 here. Three color lasers and the SLM are synchronized by synchronization controllers and the working frequency is set to be 60fps here. Cards with colorful pictures are placed at different distances as real objects (O₁ and O₂). A charge coupled device (CCD) camera (Lumenera Infinity 4) with a camera lens (Nikon AF-S 105mm) is put at the exit pupil to test the performance of the system.

The detailed parameters of the system are listed in Table 1.

Table 1. Parameters of the Proposed Prototype

View Table

3.2 Experimental results

Firstly, pre-compensation is operated for the proposed 3D full-color NED system as is mentioned in section 2.3. According to Eq. (6), the separated distance for CGHs of each wavelengths can be calculated by the given parameters: $d_{r} = 958$ pixels, $d_{g} = 798$ pixels and $d_{b} = 710$ pixels. Though the proposed CAM method is designed to be free of chromatic aberration, there still remains some cumulative chromatic aberration because of the assembling errors in the system which will cause degradation on the colorful effect. From the principle of CAM method illustrated in section 2.3, we conclude that when the grating is placed right at the Fourier plane of the 4-f system and the separated distance satisfies the specific condition of Eq. (6), the complex amplitude of objects will be reconstructed via superposition on image plane. However, for actual experiment, it is hard to position the grating at exactly the right place. Tiny movement of the grating will cause position shift of the reconstructed images. So pre-compensation is necessary for a colorful display system. In the proposed system, the separated distance of two holograms are finely adjusted to make offsets for the chromatic aberration. The actual distance are calibrated to be: $d_{r} = 976$ pixels, $d_{g} = 816$ pixels and $d_{b} = 728$ pixels. Figure 6 is an example of the calculated CGHs for different wavelengths after calibration.

Fig. 6 Example of CGHs after calibration, (a) CGH for red component ( $λ = 639$ nm), (b) CGH for green component ( $λ = 532$ nm), (c) CGH for blue component ( $λ = 473$ nm).

Download Full Size | PDF

Figure 7 depicts the effect of the pre-compensation. The word ‘colour’ with six characters is chosen to be the target colorful image. Each single character is set to be different colors while ‘c’, ‘l’, ‘u’ are in red, green and blue respectively and ‘o’, the second ‘o’, ‘r’ are in cyan, magenta and yellow respectively. The reconstruction distance is set to be 90mm. The card with a school badge is put at the same distance with the reconstructed image. Before pre-compensation, the diffracted images of two sub-holograms fail to overlap in the same position. The reconstructed colorful images are indistinct as shown in Fig. 7(a). After pre-compensation, the SLM are uploaded with CGHs after calibration. The reconstructed characters are clear and in good shape as shown in Fig. 7(b). The colorful CAM is realized and the colors of the characters are all displayed correctly.

Fig. 7 Effect of the pre-compensation, (a) before pre-compensation, (b) after pre-compensation.

Download Full Size | PDF

Then the system is tested with 3D full-color reconstruction. The six characters are divided into ‘col’ and ‘our’ and the reconstruction distances to the SLM plane are 90mm, 120mm respectively. Theoretically, the max reconstruction distance of images is mainly decided by the coherence length of the used laser source. The better monochromaticity the lasers have, the larger modulation depth range can be achieved. Meanwhile, two cards with colorful images (school badge and tree) are placed at the same depths with two groups of characters. The 3D colorful display effect is shown in Fig. 8. From Figs. 8(a)-7(b), we can see the characters are all reconstructed faithfully with the colors we previously designed. Besides, two groups of characters are focused and blurred the same way as the real objects. When the camera lens focus on the school badge, the characters ‘col’ are clear and in good shape. As the focus position moves further, the characters ‘our’ along with the tree becomes clear. The process is recorded in a video which is attached as Visualization 1. Figure 8(c) illustrates the spatial distribution of two groups of characters. The experimental results show that the system can provide good 3D full-color see-through performance and correct depth cues without VAC.

Fig. 8 Reconstructed 3D colorful images at different depths (Visualization 1), (a) focus at ‘col’, $d_{1} = 90$ mm, (b) focus at ‘our’, $d_{2} = 120$ mm, (c) spatial distribution of reconstructed colorful images.

Download Full Size | PDF

As no iterations operations are applied in the CGH calculation process, the system is quite suitable for dynamic display. So a dynamic colorful display is performed. A car navigation system is simulated and used as colorful dynamic signals. The signals contain the real-time speed of the car and the direction instruction arrow. The color of the numbers are designed to be cyan, the speed unit ‘km/h’ set to be yellow and the directional arrow to be magenta respectively. The content of the dynamic signal is a process of a car passing a right corner. While displaying, multiple CGHs are calculated and uploaded in real time. The virtual signal is reconstructed 100mm from the SLM. A smartphone displaying real scene video from a driving cab is placed the same distance with the dynamic images. The virtual instructional signals are correspond with the content of the real scene video such as slowing down, accelerating and turning right. The dynamic colorful display effect is also recorded in a video (Visualization 2). Figures 9(a) and 9(b) are two frames exacted from the video. The visual colors of different parts of the images are correctly reconstructed. The results show good colorful dynamic display effect of the system which proves the potential to interactive NED display.

Fig. 9 Dynamic colorful display (Visualization 2), (a) and (b) are two frames exacted from the video.

Download Full Size | PDF

3.3 Discussions

From the experimental results above, the proposed 3D full-color see-through NED system is proved to be able to display colorful 3D images in good quality. But we can also see from the results that the image resolution is not high enough. This is due to several reasons. Firstly, it is mainly because the pixels of the SLM are not used sufficiently. In the experiment, the resolution of the SLM is 1920 × 1080 pixels, but the resolution of the complex holograms is only 400 × 1000 pixels which means for each frame, only a few parts of the pixels are used for display. This is mainly restricted by the CAM method we use. According to section 2.2, two separated holograms are coupled via the grating on the output plane. When the separated distance is settled, bigger width of the holograms will cause interference between two holograms. Secondly, random phase is introduced in the calculation of the hologram to diffuse the wavefront information to the SLM plane. But meanwhile, it will bring speckle noise and drags the image quality. Besides, the assembly error of the optical elements and the dust on the lens will also cause degradation to the images.

For a see-through NED system, the field of view (FOV) can be described in two ways. One is the FOV for real scene ( $θ_{r}$ ), the other is the FOV for 3D virtual images ( $θ_{v}$ ) as is illustrated in Fig. 10. $θ_{r}$ is decided by the eye relief and the structure of the viewing window. Since the eye relief is 10mm, $θ_{r}$ is calculated by $θ_{r} = 2 \times \arctan (\frac{\frac{1}{2} \times 24 m m}{e y e r e l i e f + B S s i z e}) = {37.5}^{\circ}$ according to simple geometrical relationship in Fig. 10. From the equation, we can see that when the optical structure is settled, $θ_{r}$ is decided by the eye relief of the system. If the eye relief gets bigger, $θ_{r}$ will become smaller. $θ_{v}$ is mainly decided by the structure of the SLM. It is calculated by $θ_{v} = \pm \arcsin (λ / 2 p)$ where p is the pixel size. For the proposed 3D full-color NED system, given $λ_{blue} = 473$ nm and $p = 8$ μm, $θ_{v} = \pm 1.7 °$ . The pupil size is another key parameter for near-eye AR system which means the effective range from where human eye could see the reconstructed images. In the proposed system, it is calculated by $p u p i l s i z e = R \times p + 2 \times L \times \tan \frac{θ_{v}}{2}$ according to Fig. 10 where R denotes to image resolution, p means pixel size, R × p means image size and L represents the distance between image plane and human eye respectively. Given p = 8μm, R is 1000(H) × 400(V) and L is measured to be 230mm, the pupil size is calculated to be 21.7mm(H) × 16.9mm(V). In conclusion, for a full scope, the FOV for real scene $θ_{r}$ and the pupil size of our system is acceptable for a near-eye AR design. But the FOV for virtual image $θ_{v}$ and the image resolution still need to be improved in our future work. For further improvement, if SLM with smaller pixel size and higher resolution is applied, the FOV for virtual images and resolution will be enlarged effectively and the optical performance will be much better.

Fig. 10 Illustration of the system parameters, $θ_{v}$ indicates FOV for virtual image, $θ_{r}$ indicates FOV for real scene, L means distance between image plane and human eye.

Download Full Size | PDF

Last thing to mention is the refreshing frequency. The refreshing rate of the used SLM is 60fps, so the holograms are uploaded at a frequency of 60fps. As time-multiplexing technique is applied, each laser works at 20fps which is a bit lower than the minimum frequency human eyes can sense a continuous video (24Hz). As a result, there is a sense of flickering during colorful dynamic display. So if a SLM with higher refreshing rate is used, the problem will be solved perfectly and there will be no sense of flickering.

4. Summary

A full-color compact 3D see-through near-eye display system by CAM is proposed. A CAM method for colorful display is proposed based on a grating. Colorful display is realized by time-multiplexing and the CGHs for each wavelength are properly managed and finely calibrated. A prototype is fabricated. The structure is compact and lightweight which is quite suitable for portable and wearable applications. Optical experiments are performed and the results show the proposed system can reconstruct high quality 3D full-color images without VAC and eyes fatigue. Dynamic colorful display ability is also tested which shows good potential for interactive near-eye display. It is expected that the proposed method and system could make a promotion for the development of 3D see-through NED systems. Several aspects of the systems such as image resolution and the FOV still remain to be improved. In our future work, other methods such as HOEs, curved reflective optics or freeform optics could be applied to improve the effect. We will further investigate on optimizing the system structure and the modulation algorithm to realize better performance for actual application in the future.

Funding

National Natural Science Founding of China (NSFC) (61575024, 61420106014); National Key R&D Program of China (2017YFB1002900); Newton Fund.

References

1. S. Lee, C. Jang, S. Moon, J. Cho, and B. Lee, “Additive light field displays,” ACM Trans. Graph. 35(4), 1 (2016). [CrossRef]

2. S. Xie, P. Wang, X. Sang, and C. Li, “Augmented reality three-dimensional display with light field fusion,” Opt. Express 24(11), 11483–11494 (2016). [CrossRef] [PubMed]

3. X. Shen and B. Javidi, “Large depth of focus dynamic micro integral imaging for optical see-through augmented reality display using a focus-tunable lens,” Appl. Opt. 57(7), B184–B189 (2018). [CrossRef] [PubMed]

4. H. Zhang, H. Deng, W. Yu, M. He, D. Li, and Q. Wang, “Tabletop augmented reality 3D display system based on integral imaging,” J. Opt. Soc. Am. B 34(5), B16–B21 (2017). [CrossRef]

5. H. Huang and H. Hua, “High-performance integral-imaging-based light field augmented reality display using freeform optics,” Opt. Express 26(13), 17578–17590 (2018). [CrossRef] [PubMed]

6. Y. Yamaguchi and Y. Takaki, “Asymmetric integral imaging system for a see-through three-dimensional display with background imaging function,” Opt. Express 25(17), 20369–20380 (2017). [CrossRef] [PubMed]

7. X. Hu and H. Hua, “High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics,” Opt. Express 22(11), 13896–13903 (2014). [CrossRef] [PubMed]

8. X. Hu and H. Hua, “Design and tolerance of a free-form optical system for an optical see-through multi-focal-plane display,” Appl. Opt. 54(33), 9990–9999 (2015). [CrossRef] [PubMed]

9. S. Liu, Y. Li, P. Zhou, Q. Chen, and Y. Su, “Reverse-mode PSLC multi-plane optical see-through display for AR applications,” Opt. Express 26(3), 3394–3403 (2018). [CrossRef] [PubMed]

10. S. B. Kim and J. H. Park, “Optical see-through Maxwellian near-to-eye display with an enlarged eyebox,” Opt. Lett. 43(4), 767–770 (2018). [CrossRef] [PubMed]

11. N. Fujimoto and Y. Takaki, “Holographic Maxwellian-view Display System,” in Digital Holography and Three-Dimensional Imaging, OSA Technical Digest (online) (Optical Society of America, 2017), paper Th3A.4.

12. G. Li, D. Lee, Y. Jeong, J. Cho, and B. Lee, “Holographic display for see-through augmented reality using mirror-lens holographic optical element,” Opt. Lett. 41(11), 2486–2489 (2016). [CrossRef] [PubMed]

13. J. S. Chen and D. P. Chu, “Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications,” Opt. Express 23(14), 18143–18155 (2015). [CrossRef] [PubMed]

14. P. Zhou, Y. Li, S. Liu, and Y. Su, “Compact design for optical-see-through holographic displays employing holographic optical elements,” Opt. Express 26(18), 22866–22876 (2018). [CrossRef] [PubMed]

15. Y. Su, Z. Cai, Q. Liu, L. Shi, F. Zhou, and J. Wu, “Binocular holographic three-dimensional display using a single spatial light modulator and a grating,” J. Opt. Soc. Am. A 35(8), 1477–1486 (2018). [CrossRef] [PubMed]

16. J. S. Lee, Y. K. Kim, and Y. H. Won, “Time multiplexing technique of holographic view and Maxwellian view using a liquid lens in the optical see-through head mounted display,” Opt. Express 26(2), 2149–2159 (2018). [CrossRef] [PubMed]

17. Q. Gao, J. Liu, J. Han, and X. Li, “Monocular 3D see-through head-mounted display via complex amplitude modulation,” Opt. Express 24(15), 17372–17383 (2016). [CrossRef] [PubMed]

18. Q. Gao, J. Liu, X. Duan, T. Zhao, X. Li, and P. Liu, “Compact see-through 3D head-mounted display based on wavefront modulation with holographic grating filter,” Opt. Express 25(7), 8412–8424 (2017). [CrossRef] [PubMed]

19. X. Li, Y. Wang, J. Liu, J. Jia, Y. Pan, and J. Xie, “Color holographic display using a phase-only spatial light modulator,” in Digital Holography and Three-Dimensional Imaging, OSA Technical Digest (online) (Optical Society of America, 2013), paper DTh2A.3.

20. J. P. Liu, W. Y. Hsieh, T. C. Poon, and P. Tsang, “Complex Fresnel hologram display using a single SLM,” Appl. Opt. 50(34), H128–H135 (2011). [CrossRef] [PubMed]

Name	Description
Visualization 1	Performance of 3D colorful images at different depths
Visualization 2	Dynamic colorful display performance

parameters	values
Wavelengths of light source	639nm, 532nm, 473nm
Size of prototype	176.4mm × 34.8mm × 35.4mm (accessory: 65.2mm)
Weight of prototype	220.0g
SLM resolution	1920 × 1080 pixels
SLM pixel pitch	8μm
Refreshing rate	60fps
Grating period	100μm
Complex field resolution	400 × 1000 pixels
Eye relief	10mm
Eye box	21.7mm(H) × 16.9mm(V)

A full-color compact 3D see-through near-eye display system based on complex amplitude modulation

Abstract

1. Introduction

2. Principle and system

2.1 Principle of complex amplitude modulation method for colorful display

2.2 Schematic of 3D full-color near-eye display system

2.3 Color management analysis

3. Optical experiments and discussions

3.1 Optical setup and specifications

3.2 Experimental results

3.3 Discussions

4. Summary

Funding

References

Supplementary Material (2)

Cited By

Figures (10)

Tables (1)

Equations (9)

Optics Express