Open Access
Add to BibSonomy
Abstract
Maxwellian view systems can be employed to circumvent the vergence-accommodation conflict in near-eye displays (NEDs), which directly project images onto the retina regardless of the human eye’s depth of focus. However, Maxwellian view optics typically have a limited eyebox, which prevents broader applications of this architecture in NEDs. Here, we demonstrate a thin-film two-dimensional beam deflector composed of multi-twist broad-band Pancharatnam-Berry deflectors to mitigate this limitation via eyebox replication. Based on experimental validation, our proposed design can display always-focused full-color images within a 9 mm × 9 mm eyebox and thus mitigate the limitation of conventional Maxwellian displays while adding negligible weight and volume.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Augmented reality (AR) near-eye displays (NEDs) [1] are proliferating and playing an ever-increasingly important role in education, engineering, healthcare, training, and entertainment, just to name a few. However, most of the current NEDs cannot resolve the mismatch of accommodation and converging distance, called vergence-accommodation conflict (VAC) [2], which causes discomfort and visual fatigue. Several methods have been proposed to alleviate it, including light field display [3,4], holographic display [5–8], multi-focus display [9–11], and Maxwellian display [12–21]. In Maxwellian displays (also known as retina-projection displays), the displayed images are presented at retina by converging the rays to the eye pupil. It thereby can provide the always in-focus images regardless of the focus depth of human eye. Moreover, the Maxwellian displays have an added advantage of relatively simple and compact optical system configuration. Since light rays from the display engine are directly projected to retina, Maxwellian displays also manifest a relatively high optical throughput [13]. Despite of these advantages, nevertheless, it can only provide a narrow eyebox size around the single focus point. Firstly, it is time-consuming and inefficient for the manufactures to customize the exit pupil location for each user. The user may still miss the eyebox from time to time due to the device displacement or eyeball rotation. This inherent drawback significantly hinders the user’s experience and prevents the Maxwellian display from wider applications.
Recently, several approaches have been proposed to overcome this obstacle [13–20]. Based on the pinhole imaging principle, Hedili et al. used an LED (light-emitting diode) array at different positions to steer the eyebox with low motion-to-photon latency [13]. Jang et al. tilted the probe beam incident on the holographic image combiner by taking the advantages of the optomechanical scanning device and angular tolerance range of holographic optical elements [14]. These approaches generate a dynamic eyebox but lead to a bulky and expensive setup with the implementation of an additional eye pupil tracker [13–16,20]. In [15], a holographic method is demonstrated to electrically move the convergent light point in accordance with the eye movement in the horizontal and vertical directions. In [16], the focal spot of viewpoint can be steered in the eye pupil plane by adjusting the angular spectrum range of objects in the computer-generated hologram synthesis. A drawback is that the spatial light modulator increases the cost substantially. Kim et al. used a holographic optical element recorded by multiple concave mirrors [17,18]. Yoo et al. designed a lightguide based viewpoints-switchable Maxwellian display through the combination of polarization grating, multiplexed holographic optical elements, and polarization-dependent eyepiece lens [19]. These proposals effectively enlarge the eyebox without additional mechanical movement of elements. However, the eyebox expansion is achieved only in horizontal direction, and there is a trade-off between waveguide thickness and field-of-view under a specific eye relief. Chang et al. multiplexed different directional plane carrier waves to virtual image target to duplicate the pupils into a two-dimensional (2D) array [20]. This system suffers from a dual image overlap problem because the eye pupil size is larger than the adjacent pupil space, which is limited by the pixel size of the spatial light modulator. Moreover, the above-mentioned methods are difficult to implement in a full-color NED because of the narrow bandwidth of diffractive optical elements.
In this paper, we demonstrate a full-color Maxwellian NED with an enlarged eyebox in 2D space to overcome the small eyebox limitation. This objective is achieved by employing a broadband 2D beam deflector composed of two Pancharatnam-Berry deflectors (PBDs) and a quarter-wave plate (QWP). The first PBD deflects light in horizontal direction, and the QWP converts the light to linear polarization so that the second PBD can deflect each beam vertically to three directions. Ideally, the proposed method can multiplex the single focus spot into a 3-by-3 points array. In the following sections, detailed operating principle and system configuration will be illustrated in the first place. Then, we will demonstrate a proof-of-concept system and discuss its pros and cons based on our experimental results.
2. Principle and system configuration
Our objective is to realize a full-color Maxwellian-view NED with an expanded eyebox. We employ two optimized PBDs to split light due to their special polarization features. A 2D beam deflector is also developed to expand the eyebox in two dimensions.
2.1 Broadband PBD design
Here, we use a polymer based PBD to split the light because of its nearly 100% diffraction efficiency and strong polarization selectivity. PBDs have shown great potential in display applications such as edge detection, resolution enhancement, and foveated displays [22–24]. A multi-twist liquid crystal (LC) structure based on photo-alignment technology and self-aligning property can fulfill this broadband retardation control with a compact form factor [25–29]. The far-field electric field of m order can be calculated as:
In the following, we summarize the optical properties of the optimized PBD in the visible spectral range. First, only the 0 and ${\pm} 1$ orders exist, and the efficiency of each order is ${\mathrm{\eta }_0} = 1/3$, ${\mathrm{\eta }_{ {\pm} 1}} = ({1 \pm S_3^{\prime}} )/3$, where $S_3^{\prime} = {S_3}/{S_0}$ is a normalized Stokes parameter. The diffraction angle of ${\pm} 1$ orders is $\theta ={\pm} \textrm{arcsin}({\lambda /\mathrm{\Lambda }} )$, where $\lambda $ is the wavelength and $\mathrm{\Lambda }$ is the period of PBD. Second, the zero-order preserves the incident polarization ellipticity and the inclination angle of ellipse rotates about $- 53^{\circ}$. The +1 order is right-handed (RCP) and −1 order is left-handed circular polarization (LCP), respectively.
2.2 Two-dimensional beam deflector
The operation principal of the proposed 2D beam deflector is shown in Fig. 2. Here, two thin PBD films with a QWP are chosen as the fundamental elements to spatially separate the collimated beams with small diffraction angles in two orthogonal directions. First, the collimated light source with linear polarization transmits the PBD1 whose period is in x-axis. The light will be diffracted in x-direction and only the 0 and ${\pm} 1$ orders exist. As mentioned above, the polarization state of zero-order is still linear, and +1 order is RCP and −1 order is LCP with equal efficiency. Then, these three orders transmit through a broadband QWP whose optical axes is parallel to the polarization direction of zero-order light. The function of the QWP is to convert the polarization state of ±1 orders from circular to linear, while keeping the zero-order unchanged. Hence, both ±1 orders and zero-order are linearly polarized. The following element is a PBD2 whose period is in y-axis to diffract light in the y-direction. Here, it should be noted that these two PBDs keep the same phase retardation and period constant, but the LC directors linearly rotate in the horizontal and vertical directions, respectively. By doing so, the incident beam can be diffracted into nine beams with equal intensity in the x-y plane. The light beams get the linear Pancharatnam-Berry phase profiles and therefore they are deflected to some specific angles as Fig. 2 illustrates.
Fig. 2. The proposed 2D beam deflector. LP: Linear polarization; LCP/RCP: Left-/right-handed circular polarization; QWP: Quarter-wave plate; PBD: Pancharatnam-Berry deflector.
2.3 Overall system configuration
Figure 3 depicts the system configuration of our proposed Maxwellian displays. Here, we use a laser scanning projector as the image source and a collimation lens (CL) to collimate the laser beams. The linearly polarized light after polarizer is split into nine sub-beams with 3 rows by 3 columns after passing through the 2D beam deflector marked in the dotted rectangle, i.e. PBD1-QWP-PBD2. Such a 2D beam deflector is attached to the eyepiece lens. This way, all the light can enter the eyepiece lens and be focused on different positions (but in the same plane) to enlarge the effective eyebox for the retinal projection. For a see-through augmented reality, a beam splitter is commonly used as the optical combiner shown in Fig. 3.
Fig. 3. Experimental setup of the proposed see-through near-eye Maxwellian display with an enlarged eyebox. CL: Collimation lens; P: Linear polarizer; EL: Eyepiece lens.
This system can provide a maximum field of view $FOV = 2 \cdot \textrm{arctan}({D/2f} )$, where D is the diameter and f is the focal length of the eyepiece lens. In our configuration, the maximum FOV is determined by the f-number of the eyepiece lens. To achieve our proposed design with a more compact form-factor, the overall size of the system can be directly reduced by utilizing lenses with smaller f-numbers or designing the beam deflector in a free-form optical system. The space of adjacent focal spots is $d = f \cdot tan\theta $, which is determined by the diffraction angle of the PBD and the focal length of the eyepiece lens. Here, although the diffraction angle of PBD changes with the illuminated wavelength, the induced chromatic dispersion is negligible. In practice, this space should be a little larger than the human eye’s pupil width to eliminate the double image problem.
3. Experiment
In experiment, we fabricated two PBDs with 33.3% zero-order efficiency. We use the photo-alignment (PA) method to form planar pattern. The PA material (brilliant yellow) was dissolved in dimethyl-formamide (DMF) and then spin-coated on a cleaned glass. The ratio and speed are listed in Table 1.
Table 1. Materials and spin-coating speed for optimized PBD fabrication.
A Mach-Zehnder interferometer with LCP and RCP lights was set to generate the desired alignment on the PA layer. Detailed procedures have been reported in [29]. Afterward, we spin-coated a reactive mesogen mixture on top of the PA layer to replicate the alignment patterns. In experiment, the thickness of each layer can be precisely controlled by the LC polymer concentration ratio and spin-coating speed. The twist handedness and speed are determined by the handedness and ratio of the chiral dopant. The thickness and twist angle of each layer was measured by the method described in [29]. Because the first layer is relatively thick, it was spin-coated twice. UV curing was applied to form the cross-linked polymeric film after coating of each layer. The spin-coating speed and ratio of reactive mesogen RM257, photo-initiator Irgacure 651, surfactant Zonyl 8857A, chiral R811/S811, and solvent toluene of each time are listed in Table 1.
After each PBD was fabricated, we measured the grating period to be $\mathrm{\Lambda } = 13\; \mu m$ and its microscopic image under crossed polarizers is shown in Fig. 4(a). With a linearly polarized input, the simulated and measured efficiency of each order are plotted in Fig. 4(b). The agreement between experiment and simulation is reasonably good.
Fig. 4. (a) The polarizing optical microscopic image and (b) the simulated and measured spectral response of the optimized PBD.
The focal length of the employed eyepiece lens is f=7.5 cm and the diameter D=5 cm, providing a theoretical maximum FOV=36.9°. Here we used a laser beam scanning projector (Sony MP-CL1) with $1920 \times 720$ resolution as the image source. A smartphone camera with f/2.4 was applied to mimicking human eye. After preparing these optical elements, we built a proof-of-concept system on an optical table, as shown in Fig. 5(a). To test the eyebox, the generated focal spots in the focal plane were captured and results are shown in Fig. 5(b). It can be seen clearly that multiple spots are expanded in both horizontal and vertical directions. Here, the color break-up in the focus plane is caused by the PBD diffraction which will cause the chromatic aberration. For the replicated focus spot in the focus plane, the center distance between blue and green color channel is ${\sim} 0.2$ mm, and the distance between red and green color channel is ${\sim} 0.4$ mm. The displacement or geometry shift between each color component occurs because of the small difference in retina projection positions and it will produce undesirable color separation along boarders within virtual images. The chromatic aberration, as a critical issue in diffractive optical elements, can be optically corrected based on the hybrid diffractive and refractive elements [26] or digitally compensated by pre-processing the images according to the chromatic dispersion [31]. We put a ruler below the spots and measured the space of adjacent focal spots in both directions to be ∼3 mm. Therefore, our simulation results agree well with experimental data, and the eyebox is effectively enlarged to approximately 9-mm x 9-mm. Here, we mainly expand the eyebox range in x-y plane. In fact, by inserting an eye-pupil tracker to track the eye pupil in real time, additional electrical or mechanical movement of the eyepiece lens can also bring the eye-relief variation. By doing so, the system can develop broader applications in near-eye displays. Furthermore, the efficiency at each spot from 1 to 9 were measured to be 10.8%, 11.5%, 10.7%, 11%, 11.8%, 8.6%, 10.5%, 11.2%, and 10.3%, respectively. The phase profiles in PBDs and the polarization transformation in QWP affect the efficiency distribution in each spot. The light efficiency at these spots is distributed relatively uniformly and the total efficiency of these 9 points is 96.4%. The remaining 3.6% is diffracted to higher orders, which can be reduced further by a better grating phase pattern and surface quality. In addition, we put a paper scale 15 cm from the camera to test the maximum FOV. As Fig. 5(c) shows, the projection length in each radical direction is about 4.5 cm. The calculated maximum FOV is 33.5°. To demonstrate the always in-focus property for the Maxwellian view NED, the displayed images (when the camera focused at 10 cm and 200 cm) were captured, as shown in Figs. 5(d) and 5(e). The image focused at 200 cm is a little blurred, which indicates the experimental depth-of-focus range is large but not infinite. Although Maxwellian view display can be ideally implemented for scanning the modulated beam through a single point in the eye pupil, other factors in the experimental setup such as tiny eye-relief vibration, oculogyration, and chromatic aberration will affect the laser scanning beam to form a single spot in the eye pupil. Our experiment shows that, although the spot size is a little large, the image quality remains high for replicated eyebox when the camera focused at 10 cm and 200 cm, and thereby the depth of focus range is still large. It is an effective method to alleviate the vergence-accommodation conflict in current NEDs.
Fig. 5. Photographs of experimental results; (a) experiment setup, (b) light intensity distribution at focus plane, (c) field of view test image mixed with the real scale paper, and captured images with (d) 10 cm and (e) 200 cm focal length of the camera.
The generated red, green, and blue images “UCF” at each viewpoint were taken and displayed in Fig. 6. Figure 6(a) shows the captured images in 9 positions when we removed the proposed 2D beam deflector from our system. The virtual image can only be seen in the central viewpoint, and the intensity is high. Figure 6(b) shows the captured images when the proposed 2D beam deflector was inserted to the system. The virtual images can be seen clearly in a large eye movement range. Moreover, as Fig. 6 shows, the displayed image mixed with the real environment was captured by the camera, which exhibits a good optical see-through feature.
Fig. 6. Experimental photographs of our Maxwellian view displays when the 2D beam deflector was (a) removed and (b) inserted in positions (1)-(9).
4. Conclusions
In conclusion, we have demonstrated a Maxwellian-view display with an enlarged eyebox due to our proposed 2D beam deflector system. In addition, our NED system provides full color based on a broadband double-layer liquid crystal structure. In experiment, the eyebox can be enlarged to $9\; mm \times 9\; mm$ with a maximum $33.5^{\circ} $ field of view. Our approach provides a simple way to enlarge eyebox due to the advantages of adopted elements. Its potential applications for near-eye displays is foreseeable.
Funding
Goertek Electronics.
Disclosures
The authors declare no conflicts of interest.
References
1. T. Zhan, K. Yin, J. Xiong, Z. He, and S. T. Wu, “Augmented reality and virtual reality: perspectives and challenges,” iScience 23(8), 101397 (2020). [CrossRef]
2. G. Kramida, “Resolving the vergence-accommodation conflict in head-mounted displays,” IEEE Trans. Vis. Comput. Graph. 22(7), 1912–1931 (2016). [CrossRef]
3. H. Huang and H. Hua, “Systematic characterization and optimization of 3D light field displays,” Opt. Express 25(16), 18508–18525 (2017). [CrossRef]
4. H. Huang and H. Hua, “Effects of ray position sampling on the visual responses of 3D light field displays,” Opt. Express 27(7), 9343–9360 (2019). [CrossRef]
5. A. Mainmone, A. Georgiou, and J. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 1–16 (2017). [CrossRef]
6. K. Wakunami, P.-Y. Hsieh, R. Oi, T. Senoh, H. Sasaki, Y. Ichihashi, M. Okui, Y.-P. Huang, and K. Yamamoto, “Projection-type see-through holographic three-dimensional display,” Nat. Commun. 7(1), 12954 (2016). [CrossRef]
7. G. Li, D. Lee, Y. Jeong, J. Cho, and B. Lee, “Holographic display for see-through augmented reality using mirror-lens holographic optical element,” Opt. Lett. 41(11), 2486–2489 (2016). [CrossRef]
8. E. Moon, M. Kim, J. Roh, H. Kim, and J. Hahn, “Holographic head-mounted display with RGB light emitting diode light source,” Opt. Express 22(6), 6526–6534 (2014). [CrossRef]
9. T. Zhan, J. Xiong, J. Zou, and S. T. Wu, “Multifocal displays: review and prospect,” PhotoniX 1(1), 10 (2020). [CrossRef]
10. G. Tan, T. Zhan, Y. H. Lee, J. Xiong, and S. T. Wu, “Polarization-multiplexed multi-plane display,” Opt. Lett. 43(22), 5651–5654 (2018). [CrossRef]
11. T. Zhan, J. Zou, M. Lu, E. Chen, and S. T. Wu, “Wavelength-multiplexed multi-focal-plane see-through near-eye displays,” Opt. Express 27(20), 27507–27513 (2019). [CrossRef]
12. G. Westheimer, “The Maxwellian view,” Vision Res. 6(11-12), 669–682 (1966). [CrossRef]
13. M. Hedili, B. Soner, E. Ulusoy, and H. Urey, “Light-efficient augmented reality display with steerable eyebox,” Opt. Express 27(9), 12572–12581 (2019). [CrossRef]
14. C. Jang, K. Bang, S. Moon, J. Kim, S. Lee, and B. Lee, “Retinal 3D: augmented reality near-eye display via pupil-tracked light field projection on retina,” ACM Trans. Graph. 36(6), 1–13 (2017). [CrossRef]
15. Y. Takaki and N. Fujimoti, “Flexible retinal image formation by holographic Maxwellian-view display,” Opt. Express 26(18), 22985–22999 (2018). [CrossRef]
16. M. H. Choi, Y. G. Ju, and J. H. Park, “Holographic near-eye display with continuously expanded eyebox using two-dimensional replication and angular spectrum wrapping,” Opt. Express 28(1), 533–547 (2020). [CrossRef]
17. S. B. Kim and J. H. Park, “Optical see-through Maxwellian near-to-eye display with an enlarged eyebox,” Opt. Lett. 43(4), 767–770 (2018). [CrossRef]
18. J. H. Park and S. B. Kim, “Optical see-through holographic near-eye-display with eyebox steering and depth of field control,” Opt. Express 26(21), 27076–27088 (2018). [CrossRef]
19. C. Yoo, M. Chae, S. Moon, and B. Lee, “Retinal projection type lightguide-based near-eye display with switchable viewpoints,” Opt. Express 28(3), 3116–3135 (2020). [CrossRef]
20. C. Chang, W. Cui, J. Park, and L. Gao, “Computational holographic Maxwellian near-eye display with an expanded eyebox,” Sci. Rep. 9(1), 18749 (2019). [CrossRef]
21. J. Xiong, G. Tan, T. Zhan, and S. T. Wu, “Breaking the field-of-view limit in augmented reality with a scanning waveguide display,” OSA Continuum 3(10), 2730–2740 (2020). [CrossRef]
22. J. Zhou, H. Qian, C. F. Chen, J. Zhao, G. Li, Q. Wu, H. Luo, S. Wen, and Z. Liu, “Optical edge detection based on high-efficiency dielectric metasurface,” Proc. Natl. Acad. Sci. U. S. A. 116(23), 11137–11140 (2019). [CrossRef]
23. T. Zhan, J. Xiong, G. Tan, Y. H. Lee, J. Yang, S. Liu, and S. T. Wu, “Improving near-eye display resolution by polarization multiplexing,” Opt. Express 27(11), 15327–15334 (2019). [CrossRef]
24. G. Tan, Y. H. Lee, T. Zhan, J. Yang, S. Liu, D. Zhao, and S. T. Wu, “Foveated imaging for near-eye displays,” Opt. Express 26(19), 25076–25085 (2018). [CrossRef]
25. C. Oh and M. J. Escuti, “Achromatic diffraction from polarization gratings with high efficiency,” Opt. Lett. 33(20), 2287–2289 (2008). [CrossRef]
26. T. Zhan, J. Zou, J. Xiong, X. Liu, H. Chen, J. Yang, S. Liu, Y. Dong, and S. T. Wu, “Practical chromatic aberration correction in virtual reality displays enabled by large-size ultra-broadband liquid crystal polymer lenses,” Adv. Opt. Mater. 8(2), 1901360 (2020). [CrossRef]
27. R. K. Komanduri, K. F. Lawler, and M. J. Escuti, “Multi-twist retarders: broadband retardation control using self-aligning reactive liquid crystal layers,” Opt. Express 21(1), 404–420 (2013). [CrossRef]
28. R. Komanduri, J. Kim, K. Lawler, and M. Escuti, “Multi-twist retarders for broadband polarization transformation,” Proc. SPIE 8279, 82790E (2012). [CrossRef]
29. J. Zou, T. Zhan, J. Xiong, and S. T. Wu, “Broadband wide-view Pancharatnam–Berry phase deflector,” Opt. Express 28(4), 4921–4927 (2020). [CrossRef]
30. C. Oh and M. J. Escuti, “Achromatic polarization gratings as highly efficient thin-film polarizing beamsplitters for broadband light,” Proc. SPIE 6682, 668211 (2007). [CrossRef]
31. S. W. Chung, B. K. Kim, and W. J. Song, “Removing chromatic aberration by digital image processing,” Opt. Eng. 49(6), 067002 (2010). [CrossRef]
