Open Access
Add to BibSonomy
Abstract
We propose an integrated holographic waveguide display system. An infrared volume holographic grating (IVHG) and a visible light grating are recorded on the same waveguide to achieve the purpose of a common light path for system miniaturization. Simulated and experimental results verify the feasibility of this method. The coupling efficiencies of the infrared module for eye tracking and the visible light module for augmented reality (AR) display are 40% and 45%. The holographic waveguide has a weight of only 4.3 grams. It is believed that this technique is a good way to achieve a light and thin eye tracking near-eye display.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
The AR near-eye display is that when the observer observes the real environment of the outside world, the virtual image or information superimposed in the real environment can be observed at the same time. As the next generation of the advanced display technology, it can be widely used in industrial, military, medical, consumer, entertainment and other aspects of life. Common near-eye display technology solutions include free-form prisms scheme [1], deformable mirror scheme [2], projection systems scheme [3,4], holographic projection scheme [5,6], and holographic waveguide scheme [7–11].
There are three types of eye tracking technology: one is to track according to the changes of the characteristics of the eyeball, the second is to track according to the change of the iris angle, and the third is to actively project infrared rays to the iris for extracting features. Among them, the way of infrared projection has a great advantage, especially in terms of recognition accuracy [12]. In recent years, the trend of applying eye tracking to AR near-eye display devices has become more apparent, because this method uses eye gaze instead of traditional manual interaction, which can significantly improve head-up performance and provide users with a new interactive experience.
The development of eye tracking and near eye display technology has been previously explored at various levels. Many technology companies, such as Magic Leap, Lumus, Oculus, Google, and Microsoft, have introduced their AR near-eye display devices with eye tracking. However, the eye tracking module of these devices is independent of the imaging module, which will make the device bulky and lack of portability. Duchowski integrated an eye-tracker by ISCAN Corporation with a V8 HMD by Virtual Research Corporation to study software-based fovea-contingent display scheme [13]. Hong Hua proposed an eye-tracking AR near-eye display device based on free-form optical technology, which successfully combined the free-form AR display device with infrared eye tracking technology [14,15]. However, the above solutions still meet the difficulty of fabrication accuracy and complicated equipment. How to effectively integrate infrared eye tracking with AR near-eye display is still a challenge we face today.
The holographic waveguide AR display has the advantages of small volume, light weight, easy processing, high efficiency and the like. Therefore, combining holographic waveguide technology with infrared eye tracking technology is an effective method to solve the above problems.
In this paper, in order to realize a light and small display system, we propose an integrated holographic waveguide display system with common optical path for visible and infrared light, and high diffraction efficiencies are achieved. This design method of sharing the optical path between the infrared eye tracking module and the AR display module significantly reduces the system volume, making the system lighter and easier to process.
2. Basic Idea and principle
The Basic structure diagram of the integrated holographic waveguide display system is shown in Fig. 1. The diffractive optical elements in the figure are all VHG. VHG1 is the in-coupled VHG for AR display. VHG2 is the out-coupled IVHG for eye tracking. MVHG is the multiplexed holographic grating, which is the visible light VHG and the IVHG multiplexed in a same area. The system has two channels of the common light path. The first channel is that the collimated image emitted by the image source is coupled in by the in-coupled volume holographic grating and then transmitted by the waveguide via total internal reflection and then coupled out into the human eye by the the out-coupled volume holographic grating. The second channel is that the near-infrared light reflected by the human eye is coupled in by the in-coupled IVHG and then transmitted by the waveguide via total internal reflection and then coupled out into the detector by the out-coupled IVHG.
We firstly fabricate infrared holographic gratings to achieve high diffraction efficiency. International research on IVHG has been carried out for many years. For instance, Sergio Calixto et al investigate the infrared light-sensitive material - Lexan polycarbonate to obtain infrared grating, while the diffraction efficiency is very low, only 0.6% [16]. Moreover, since the infrared light is not visible, it is more difficult to directly use the infrared light exposure to obtain the infrared volume grating. Subsequently, Qiang Huang and Paul R. Ashley et al. propose a method for recording an infrared volume holographic grating using a 514 nm laser, which has a reproduction efficiency of 23% at 850 nm [17]. Robbins propose a Bragg grating for diffracting infrared light and apply it to human eye tracking, but the Bragg grating has only an angular bandwidth of no more than 5° [18].
Our theoretical analysis is based on the Kogelnik theory [19], which is suitable for VHG. As is well known, if the incident light satisfies Bragg incidence, the incident light vector Kr, the grating vector K, and the diffracted light wave vector Ks form a closed isosceles triangle, as shown in Fig. 2(a). When the incident light wave deviates from Bragg incidence, as shown in Fig. 2(b) and Fig. 2(c), the Bragg mismatch parameter will be introduced.
Fig. 2 Geometric relationship of volume grating reconstruction. (a) satisfied Bragg incidence. (b)and (c) deviated from Bragg incidence. (d) satisfied Bragg incidence when reconstructed with infrared light. The x axis is parallel to the surface of the material and the z axis is perpendicular to the surface of the material.
Bragg mismatch parameter can be calculated by [19]:
where is the grating thickness, is the diffraction angle, is the bulk index of refraction of the material, is the incident angle of reconstruction wave, is the tilt angle of grating vector, and is the wavelength of reconstruction wave.Grating coupling strength can be calculated by [19]:
where is refractive index modulation.Diffraction efficiency can be calculated by [19]:
It can be seen from the above formulas that whether it is an angular offset or a wavelength shift, the Bragg mismatch parameter is generated, resulting in a decrease in diffraction efficiency.
Our proposed method is to compensate the wavelength offset by the angular offset, so as to ensure that the Bragg matching condition is still satisfied when the reconstruction wavelength is inconsistent with the recording wavelength, as shown in Fig. 2(d). Therefore, we can use the visible light laser to record the IVHG suitable for near-infrared wavelength.
3. Parameter design and simulation
We set the recording wavelength of the holographic waveguide to 532 nm, which is practically accessed with high power Genesis Laser. And the near infrared light wavelength λ is 785 nm when reconstruction. Since the waveguide refractive index is 1.51 (Relative to 589.3 nm wavelength), the total reflection angle should be greater than 41.47° to satisfy the total reflection condition. Considering that the total reflection angle of long wavelength reconstruction is smaller than that of short wavelength reconstruction, we set the the object light angle to 85° much larger than 41.47°. The grating thickness of the recording material is 13.5 μm, and the refractive index modulation is 0.02.
Then theoretical calculations are carried out based on the Kogelnik coupled wave theory and the above parameters. when the recording wavelength is 532 nm and the reconstruction wavelength is 785 nm, the wavelength offset is 253 nm. The relationship between the angle offset and the diffraction efficiency is analyzed. we find that when the recording angle deviates from the normal incidence by 23.71°, the diffraction efficiency of the reconstructed light wave is the highest, about 85%, as shown in Fig. 3. For the designed infrared volume holographic grating, the grating period is 304nm, and the slanting angle to the Z-axis is 30.65°.
Fig. 3 The relationship between the angle offset and the diffraction efficiency. The premise is to ensure the vertical reconstruction of 785 nm.
Under the premise that the above grating parameters have been determined, we simulate the relationship between reconstruction wavelength, reconstruction angle and diffraction efficiency, as shown in Fig. 4. It can be seen that the incident angle is 0° when reconstructed with 785 nm. 0° means perpendicular incidence to the surface of the material.
Fig. 4 Relationship between the incident angle of reconstruction wave, the wavelength of reconstruction wave and the diffraction efficiency.
The above parameters are designed for the recording of the IVHG. Other than that, the recording angles of the reference light and the object light of the visible light VHG are 0° and 71°. We calculated that when the visible light VHG and the IVHG are reconstructed vertically at 532 nm and 785 nm, their diffraction angles are 71° and 61.31°, respectively, both of which satisfy the total reflection condition. The hologram recording material used herein has an angular bandwidth of 8°, and the difference between the Bragg angles of the visible light VHG and the IVHG is about 10°, which is larger than the angular bandwidth of the volume holographic grating, thus avoiding mutual crosstalk between different gratings.
Moreover, we set the size and spacing of the individual gratings in the entire system, as shown in Fig. 5. The dimensions of the three gratings are all 7mm*7mm. The distance between VHG1 and VHG2 is 5.3mm. And the distance between VHG2 and MVHG is 15.5 mm. The length, width and thickness of the waveguide is 50mm, 12mm and 2mm. The visible light and the near infrared light are totally reflected six times during reconstruction.
Based on the above parameter design, a system simulation is carried out, as shown in Fig. 6. The three gratings from right to left correspond to VHG1, VHG2, and MVHG, respectively. The green line indicates 532 nm light. The red line indicates 785 nm near-infrared light.
4. Experimental verification
In order to verify the availability of this method, we carry out exposure experiments according to the theoretical parameters. The material used is photopolymer, which is processed by our laboratory. In order to obtain the highest diffraction efficiency, the recording dosage is about 22 mJ/cm2 for green light at 532 nm. The holographic exposure path is similar to the reference [6]. The holographic grating is obtained by two parallel light interference recordings, while the prism is used to satisfy the total reflection condition. VHG1, VHG2, and MVHG are single-layer gratings, of which MVHG is obtained by angle multiplexing. In order to get the best MVHG, we used three exposure methods. The first method is to expose both sets of angles simultaneously. The second method is to first expose the angles corresponding to the visible light display, and then expose the angles corresponding to the infrared tracking. The third method is to first expose the angles corresponding to the infrared tracking, and then expose the angles corresponding to the visible light display. The results show that the MVHG obtained by the third exposure method is better than the first two methods. We conduct the exposure experiments according to the initial parameters, and the results show that the near-infrared light could not be reconstructed vertically. The main cause of this phenomenon is the shrinkage of the material. To compensate influence of material shrinkage on grating vector, we introduce a material shrinkage factor for correction, including a vertical shrinkage factor S1 perpendicular to the surface of the recording material, and a horizontal shrinkage factor S2 parallel to the surface of the recording material.
The relationship between the grating vector after shrinkage of the material and the theoretical grating vector is as follows:
where and are the Z and X components of the theoretical value of the grating vector , respectively. Where and are the Z and X components of the actual value of the grating vector after the material shrinks, respectively.To obtain the value of the shrinkage factor, we conduct a set of experiments. First, we record the VHG with a set of parameters and measure its angle selection curve (diffraction efficiency versus incident angle). Then, we use the parameters of the VHG and the different values of S1 and S2 to obtain a set of angle selection curves by numerical simulation. Finally, it is found that the simulation results are most consistent with the experimental data when S1 = 1.0346 and S2 = 1.1315.
We correct the previous theoretical simulations after introducing the material shrinkage factor. The actual recorded angle after re-correction is 28.11°, which corresponds to an angle of 45° in air, as shown in Fig. 7. At this time, the infrared light can be reconstructed vertically.
Fig. 7 The relationship between the angle offset and the diffraction efficiency after introducing the material shrinkage factor.
Experimentally measured diffraction efficiencies of VHG1, VHG2, MVHG are listed in Table 1. Experimental coupling efficiencies of visible light module and infrared module of the system are listed in Table 2. We especially analyze the characteristics of IVHG. The relationship between the reconstruction wavelength and the diffraction efficiency is as shown in Fig. 8. It can be seen that in the case of normal incidence, the grating coupling efficiency at 785 nm is the highest, up to 40%. 40% is the diffraction efficiency of the 785 nm laser beam in the system. If the system is used for actual human eye tracking, the efficiency will be reduced because the light reflected by the eyeball is not parallel. The diffraction efficiency of this system is sufficient for capturing human eye images for eye tracking. The above diffraction efficiency values are average values under optimal experimental conditions. These values fluctuate by less than 1%.And the fabricated holographic waveguide is shown in Fig. 9.
Table 1. Diffraction efficiency of a single grating
Table 2. Coupling efficiency of visible light module and infrared module of the system
We have set up a prototype for AR display, as shown in Fig. 10. The image source used is an OLED. The relay optical system is a custom-designed lens group. The image collimated by the relay optical system is coupled into the waveguide by VHG1 and transmitted by total reflection. The MVHG couples the image out and guides it to the camera.
Then, we test the display effect of visible light module of holographic waveguide system by using binary image, grayscale image and color image. The observation distance of the images is 20mm. The experimental results show that the system has good imaging ability and the image quality is clear, as shown in the Fig. 11. Since the VHG of the holographic waveguide system works for green, the system in this paper is for monochrome display. The display effect indicates that the holographic structure has a good wavelength selectivity. At the same time, we can see the slight blur visible in the images (d), (e), (f), which is mainly due to the bandwidth of the OLED source. The FWHM of the OLED display is 53.8 nm.
Fig. 11 (a)-(c) are the input original image. (d)-(e) are the photographs of the displayed results. (a) is a binary image. (b) is a grayscale image. (c) is a color image.
The system is a near-eye display device. So we perform experiments to verify the AR display function of this system. The real fruit models and the virtual image can be clearly observed by the human eye at the same time, as shown in Fig. 12. The virtual image is imaged at infinity, and the actual object is at 70 cm.
In brief, the total infrared and visible grating waveguide systems have a weight of 4.3 grams. And the size of the waveguide is 50mm × 12mm × 2mm. The coupling efficiencies of the infrared module for eye tracking and the visible light module for AR display are 40% and 45%. The system proposed in this paper can achieve color display in the future. We can achieve color display by wavelength multiplexing or layering [20]. In order to avoid crosstalk between different gratings during color display, the difference between the Bragg angles of different gratings to be larger than the angle selection bandwidth of the volume holographic grating. Meanwhile, optical tracking without direct contact to the eye can be used for infrared eye tracking. Eye tracking requires hardware and software such as image processing algorithm, which is what we need to do further in the future. One possible method is the pupil center cornea reflection (PCCR) technology [21].
5. Conclusion
We propose an integrated holographic waveguide display system with common optical path for visible and infrared light. We present, analyze and design the entire system theoretically, and realize it experimentally. Theoretical and experimental results are in good agreement. The actually fabricated holographic waveguide system has a size of 50mm × 12mm × 2mm and a weight of 4.3 grams. The system can provide high quality images with high brightness uniformity when performing AR display. The coupling efficiencies of the infrared module for eye tracking and the visible light module for AR display are 40% and 45%. However, the diffraction efficiency of the IVHG can be further improved by optimizing the post-treatment process or using other materials. It is worth mentioning that the method of this paper can also be used to achieve color display or multi-channel display in the future. The proposed method is expected to be widely used in various fields of the eye tracking AR display, such as medical, entertainment, military, and industrial.
Funding
National Natural Science Founding of China (NSFC) (61575024, 61420106014), and the UK Government’s Newton Fund.
References
1. H. Hoshi, N. Taniguchi, H. Morishima, T. Akiyama, S. Yamazaki, and A. Okuyama, “Off-axial hmd optical system consisting of aspherical surfaces without rotational symmetry,” Proc. SPIE 2653, 234–242 (1996). [CrossRef]
2. D. Dunn, C. Tippets, K. Torell, P. Kellnhofer, K. Akşit, P. Didyk, K. Myszkowski, D. Luebke, and H. Fuchs, “Wide field of view varifocal near-eye display using see-through deformable membrane mirrors,” IEEE Trans. Vis. Comput. Graph. 23(4), 1322–1331 (2017). [CrossRef] [PubMed]
3. H. Hua, A. Girardot, C. Gao, and J. P. Rolland, “Engineering of head-mounted projective displays,” Appl. Opt. 39(22), 3814–3824 (2000). [CrossRef] [PubMed]
4. R. Martins, V. Shaoulov, Y. Ha, and J. Rolland, “A mobile head-worn projection display,” Opt. Express 15(22), 14530–14538 (2007). [CrossRef] [PubMed]
5. E. Moon, M. Kim, J. Roh, H. Kim, and J. Hahn, “Holographic head-mounted display with RGB light emitting diode light source,” Opt. Express 22(6), 6526–6534 (2014). [CrossRef] [PubMed]
6. R. Shi, J. Liu, H. Zhao, Z. Wu, Y. Liu, Y. Hu, Y. Chen, J. Xie, and Y. Wang, “Chromatic dispersion correction in planar waveguide using one-layer volume holograms based on three-step exposure,” Appl. Opt. 51(20), 4703–4708 (2012). [CrossRef] [PubMed]
7. A. A. Cameron, “Optical waveguide technology and its application in head-mounted displays,” Proc. SPIE 8383, 83830E (2012). [CrossRef]
8. Z. Wu, J. Liu, and Y. Wang, “A high-efficiency holographic waveguide display system with a prism in-coupler,” J. Soc. Inf. Disp. 21(12), 524–528 (2013). [CrossRef]
9. J. Han, J. Liu, X. Yao, and Y. Wang, “Portable waveguide display system with a large field of view by integrating freeform elements and volume holograms,” Opt. Express 23(3), 3534–3549 (2015). [CrossRef] [PubMed]
10. A. Maimone, A. Georgiou, and J. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 1–16 (2017). [CrossRef]
11. C. Yu, Y. Peng, Q. Zhao, H. Li, and X. Liu, “Highly efficient waveguide display with space-variant volume holographic gratings,” Appl. Opt. 56(34), 9390–9397 (2017). [CrossRef] [PubMed]
12. Wikipedia, “Eye tracking,” https://en.wikipedia.org/wiki/Eye_tracking.
13. A. T. Duchowski and A. Çöltekin, “Foveated gaze-contingent displays for peripheral LOD management, 3D visualization, and stereo imaging,” ACM Trans. Multimed. Comput. Commun. Appl. 3(4), 1–18 (2007). [CrossRef]
14. H. Hua, C. W. Pansing, and J. P. Rolland, “Modeling of an eye-imaging system for optimizing illumination schemes in an eye-tracked head-mounted display,” Appl. Opt. 46(31), 7757–7770 (2007). [CrossRef] [PubMed]
15. H. Hua, X. Hu, and C. Gao, “A high-resolution optical see-through head-mounted display with eyetracking capability,” Opt. Express 21(25), 30993–30998 (2013). [CrossRef] [PubMed]
16. S. Calixto and C. Menchaca, “Infrared recording of transient and permanent interference gratings with commercial plastics,” Appl. Opt. 28(20), 4370–4374 (1989). [CrossRef] [PubMed]
17. Q. Huang and P. R. Ashley, “Holographic Bragg grating input-output couplers for polymer waveguides at an 850-nm wavelength,” Appl. Opt. 36(6), 1198–1203 (1997). [CrossRef] [PubMed]
18. S. Robbins, I. A. Nguyen, and X. Lou, “Waveguide eye tracking employing volume Bragg grating,” U.S. patent US 9377623 (2016).
19. H. Kogelnik, “Coupled wave theory for thick hologram gratings,” Bell Syst. Tech. J. 48(9), 2909–2947 (1969). [CrossRef]
20. J.-A. Piao, G. Li, M.-L. Piao, and N. Kim, “Full color holographic optical element fabrication for waveguide-type head mounted display using photopolymer,” J. Opt. Soc. Korea 17(3), 242–248 (2013). [CrossRef]
21. Z. Zhu and Q. Ji, “Novel eye gaze tracking techniques under natural head movement,” IEEE Trans. Biomed. Eng. 54(12), 2246–2260 (2007). [CrossRef] [PubMed]
