Open Access
Add to BibSonomy
Abstract
Vergence-accommodation conflict (VAC) is a major challenge in optical-see through augmented reality (AR) system. To resolve this conflict, many approaches are proposed, for instance, by means of adjustment of the projected virtual image to coincide with the surroundings, called image registration, which is more often referred to as varifocal function. In this paper, a varifocal AR system is demonstrated by adopting electrically tunable liquid crystal (LC) plane-parallel plates to solve VAC problem. The LC plates provide electrically tunable optical paths when the directors of LC molecules are re-orientated with applied voltages, which leads to a corresponding change of light speed for an extraordinary wave. To provide a sufficient tunable optical path, three pieces of multiple-layered LC structures are used with the total thickness of the active LC layers (∼510 μm). In experiments, the projected virtual image can be adjusted from 1.4 m to 2.1 m away from the AR system, while the thickness of LC plane-parallel plates are only less than 3 mm without any mechanical moving part. When light propagates in the uniaxial LC layers, the wave vector and the Poynting vector are different. The longitudinal displacement of the image plane is determined by Poynting vectors rather than wave vectors. As a result, the analysis of the AR system should be based on Poynting vectors during geometrical optical analysis. Surprisingly, the tunable range of the longitudinal displacement of Poynting vectors is 2-fold larger than the tunable range of the wave vectors. Moreover, the virtual image shifts in opposite directions with respect to the Poynting vectors and wave vectors. The proposed AR system is not only simple but also thin, and it exhibits a large clear aperture. The investigation here paves the way to a simple solution of the VAC problem for augmented reality systems.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Augmented reality (AR) is a set of technology that superimposes computer-generated images on real world for viewers [1–2]. Peoples are assisted by AR with immersive information that transforms the way we work [3–5]. Currently, most AR experiences still suffer from vergence-accommodation conflict (VAC), which is a well-known optical challenge leading to uncomfortably visual fatigues [6]. Vergence is simultaneous rotation of two eyes in opposite direction in order to maintain single binocular vision and the driving mechanism is retinal disparity [6–7]. Accommodation is a response of a crystalline lens of an eye adjusting its focal length for seeing objects clearly by means of changing the curvature of the crystalline lens. The driving mechanism of the accommodation is retinal blur [7]. The vergence and accommodation of the human vision are coupled together through live experiences. The origin of the VAC in AR optical systems is the difference of the locations between the stereoscopic distance set for viewers and a virtual image projected from a 2-dimensional (2D) display; therefore, vergence and accommodation mismatch [6–9]. The current solutions for VAC are either using 3D projection technology or adopting varifocal elements that moves the location of the image plane (the virtual image) [8–9]. Light field displays and integral imaging technologies provide multiple views to form 3D images; however, the image resolution is the trade-off because a display with fixed resolution needs to be segmented for generating multiple views [10]. As for the varifocal solutions, they can be categorized as power-based approach and distance-based approach according to the thin-lens equation for image formation [11]. On the one hand, the power-based approach commonly adopts tunable lenses, such as liquid lenses and liquid crystal (LC) lenses, with tunable focal lengths [12–15]. In this category, the challenges come from the varifocal elements. For example, liquid lenses are relatively thick for AR systems due to the necessity of volume changes as well as a degradation in image quality from the gravity effect [16]. The main challenge of LC lenses is the paradox of tunable range of the focal length and its aperture size when the LC lenses are switched continuously in focal length [17]. Despite of that fact that the challenges of varifocal elements are studied, and the downsides are mitigated, the varifocal elements are still complex elements in terms of fabrications and designs. On the other hand, distance-based approach mainly manipulates the objective distance (i.e., the distance between a display and a projection module) by usage of multiple displays [18], stacked transparent displays [19], tunable scattering screens [9], and polarization dependent optical paths [20–21]. The merits of distance-based approach are simplicity and less limitation on aperture size. However, the current distance-based approach requires extra space for optical elements and the space for changing the optical path. In addition, when the light passes through the multiple transparent displays or the tunable scattering screens, the image quality may decrease in principle [9]. Researchers are still looking for a simple solution with compactness, effortless design, and independency of aperture size. Uniaxial plane-parallel plates with double refractions were discussed to create different optical path lengths, which results in multiple image planes [22–25]. Park et al. switched the polarization states of input light using a thick uniaxial calcite plate (∼30 mm) in order to extend depth-of-focus of an integral imaging system [22]; Lee et al. adopted two thick calcite plates and a half-wave plate instead of single calcite plate to compensate the astigmatism aberration of extraordinary wave and the chromatic aberration [25]. However, the adoption of uniaxial plane-parallel plate in literature for multiple image planes is not practical due to the thick calcite plates. Since the optics axis of the calcite plate is fixed, intrinsically, it is not possible to create enough of image planes for solving VAC problem [26].
In this paper, we propose a varifocal AR optical system using electrically tunable LC plane-parallel plates (we call them LC plates in what follows) to solve VAC problem. The LC plates provide tunable optical paths when the directors of LC molecules are re-orientated with applied voltage which leads to a corresponding change in refractive index for extraordinary wave. In principle, nematic LC material has limitations in birefringence and thickness of a LC layer due to the chemical properties and soft matter properties which indicate the restriction of the variation in the optical path. However, we adopt a multiple-LC-layered approach to enlarge the total active LC layers (∼ 510 micron) to overcome such a limitation. Nevertheless, the Poynting vector and the wave-vector splits up during light propagation in LC plate. Under consideration of Poynting vector propagation in a LC plate, the longitudinal displacement is twice larger than the one under consideration of wave-vector propagation. As a result, when the LC plate is inserted between object and a thin lens of the AR system, the location of the projected virtual image could be adjusted over 100 cm even though the optical path only changes ∼150 micron and the total thickness of the LC plates is <3 mm. Here, we investigate the longitudinal displacement of a tunable uniaxial plane-parallel plate and then the location of the projected virtual image when the optical system adopts a tunable uniaxial plane-parallel plate. To date, no one investigates and demonstrates how an electrically tunable uniaxial plane-parallel plate in the AR system could exhibit varifocal function as well as discuss the related optics. The structure of the LC plate is simple, thin, and have large clear aperture (no limitation in theory). The LC plate could be placed between the projection module and the display, and it does not impact the original design of optical architecture. For further system integration, the cover glass of the microdisplay or physically-planar substrate of any optical elements may be replaced with the LC plate to save even more space and make system more compact.
2. Operating principles
The proposed AR system with LC plane-parallel plates is illustrated in Fig. 1. In Fig. 1(a), a broadband light source and a pinhole are attached onto a negative type of a resolution chart as an object. Such a resolution chart could be replaced by a LCoS (liquid crystal on silicon) micro-display. For a proof-of-concept, we use a resolution chart here. The light source propagates through the pinhole, the resolution chart, a linear polarizer, a LC plate, a beam splitter, and goes to a concave mirror. The light reflected by the concave mirror is then folded by the beam splitter and arrives to a human eye. The concave mirror here folds the optical path and performs image formation at the same time (e.g., Google Glasses use concave mirrors). The projected image of the resolution chart is located in front of the eye and superimposes on surroundings or real scenes. The main purpose of the LC plate in the AR system is to adjust the effective distance between the resolution chart (or a micro-display) and the mirror by means of a tunable optical path. In the beginning (see Fig. 1(a)), the virtual image is set to align well with the real object of the tall building in Fig. 1(a) as the LC plate is at the voltage-off state. When the LC plate is at the voltage-on state, the effective refractive index of the LC plate changes and then it results in a change of the effective optical path between the resolution chart and the concave mirror. As a result, it leads to a position shift of the virtual image (see Fig. 1(b)).
Fig. 1. Operating principle of the AR system with a LC plane-parallel plate. (a) Initially, the LC plate turns off and the refractive index is na. The projected virtual image is set to align with the further real object. (b) When the LC plate turns on, the refractive index of LC becomes nb, and the effective optical path between resolution chart and the mirror varies. As a result, the virtual image shifts to a location closer to the AR system, and then the virtual image is aligned with the near real object. (c) An equivalent optics system for the proposed optical system without plane-parallel plate. (d) An equivalent optical system after adding a plane-parallel plate in (c).
To further prove the propose AR system, we discuss the image formation in Figs. 1(a)–1(b) under a thin lens approximation [10–11]. The proposed AR system with a concave mirror is equivalent to a system with a positive lens, as shown in Fig. 1(c). The concave mirror in Fig. 1(a) could be regarded as a reflective lens in which light double passes through it. Instead of reflective type, we redrew an equivalent optical system of Fig. 1(a) in an expression of transmissive type, as illustrated in Fig. 1(c). We defined the focal length of the lens (or a concave mirror) as f, and the image formation of the optical system can be expressed as:
where p is the objective distance and q is the image distance of the lens. n1 and n2 are the refractive index of surrounding medium on the left and right side of the lens, respectively. From Eq. (1), q is: $q = (p - f)/p \times f$ when the lens is in air (i.e., n1=n2=1). When we add a plane-parallel plate between the object and the lens, as depicted in Fig. 1(d), the effective objective distance ($p^{\prime}$) turns out [27]: where t is the thickness and n is the refractive index of the plane-parallel plate. Here we define $A \equiv \frac{{n - 1}}{n} \cdot t$ and A is positive (i.e., $A \ge 0$) because t ≥ 0 and n ≥ 1. No matter what value of t or n (n>1), $p^{\prime} < p$. That is the effective objective distance in Fig. 1(d) is always shorter than the objective distance in Fig. 1(c). When p shifts to $p^{\prime}$, assume the corresponding shift in q is from q to $q^{\prime} = q + \Delta q$, where $\Delta q$ is the shift of the image distance or so-called longitudinal displacement of the image plane. Then Eq. (1) can be re-written as: From Eq. (1) and Eq. (3), $\Delta q$ can be expressed as: In Eq. (4), when the thickness of the plane-parallel plate is very small (i.e., $t \to 0$) or the refractive index is close to air (i.e., $n \to 1$), $\Delta q \to 0$. Under the assumption of $n \ge 1$ and $t \ge 0$, $A \cdot {(q - f)^2} > 0$. Since we expect to project a virtual image, the object distance should be shorter than the focal length (i.e., $p - f \le 0$) and then the image distance should be negative (i.e., $q\textrm{ < }0$). As a result, $\Delta q$ in Eq. (4) should be positive (i.e., $\Delta q > 0$). As to the shifting direction of the image (virtual image), we restate Eq. (3) with an absolute value of image distance shown Eq. (5). In Eq. (5), ${1 / {|q + \Delta q|}}$ increase with t (t>0) or n (n>1). This means $|q + \Delta q|$ decreases after we add a plane-parallel plate. In addition, this indicates the location of the virtual image should be closer to the lens or AR system after inserting a plane-parallel plate. Once again, $|q + \Delta q|$ decreases as either t or n increases.Now, let’s consider a plane parallel plate made of uniaxial optical medium (i.e., LC). When the materials of the plane-parallel plate is an uniaxial LC (nematic LC) and assume that the effective refractive index of the LC plate for extraordinary wave (e-wave) changes from na to nb (nb>na) as the applied voltage (V) changes from Va to Vb, the image distance then changes from $|{q(n({V_a}) = {n_a})} |$ to $|{q(n({V_b}) = {n_b})} |$ and $|{q(n({V_b}) = {n_b})} |< |{q(n({V_a}) = {n_a})} |$. The shift of the virtual image distance should vary from $\Delta q(n({V_a}) = {n_a})$ to $\Delta q(n({V_b}) = {n_b})$, where $\Delta q(n({V_b}) = {n_b}) > \Delta q(n({V_a}) = {n_a})$. Therefore, the virtual image is electrically adjustable when we use a LC plate in the AR system. When the refractive index increases for the e-wave, as described above, the virtual image should be closer to the lens. Actually, this is against the experimental results. Because the image formation in Eq. (1) is actually based on wave-vector-propagation in optical systems, not Poynting-vector-propagation (i.e. ray-propagation or a propagation based on energy flows). Basically, image formation through a thin lens is on a basis on Snell’s law at the interfaces, which alternatively means phase matching condition at the interfaces under wave-vector-propagation. This is true when the optical medium is isotropic, but not for anisotropic optical medium, such as liquid crystals. When light propagates in isotropic optical medium, the wave vector of plane waves is parallel to the Poynting vector whose direction represents the direction of rays or energy flow. Thereafter, Eqs. (1)–(5) could assist analyzing for the light propagation in isotropic medium. However, nematic LC material is anisotropic optical medium where the wave vector (denoted as $\hat{k}$) and Poynting vector (i.e., direction of energy flux, denoted as $\hat{S}$) are not parallel to each other. Moreover, it often leads to so-called walk-off between ordinary wave and extraordinary wave. When we placed a LC plate in an optical system, the ray we analyze should be Poynting vectors, instead of wave vectors. Especially, when the thickness of LC plates is relative thick (∼510 micron) compared to traditional LC devices (∼4 microns for LCD).
In order to calculate the longitudinal displacement or longitudinal shift ($\delta {l_S}$) based on Poynting vector when light passes through a LC plate, we assume LC molecules are perpendicular to the interface (or optic axis $\hat{c}$ of the uniaxial medium is perpendicular to the interface) as depicted in Fig. 2(a). The dispersion relation of a uniaxial medium or LC for e-wave is [28]:
where ne and no are extraordinary refractive index and ordinary refractive index for nematic LC, respectively. k0 is wave number in vacuum. ktz and ktx are z-component and x-component of the wave vector propagating in the LC plate. ktz and ktx could be expressed in terms of refractive angle ${\theta _e}$ of the wave vector and effective refractive index ${n_{eff}}$.Fig. 2. The longitudinal shift ($\delta {l_S}$) of the Poynting vector and the longitudinal shift ($\delta {l_e}$) of the wave vector for e-wave (a) when LC molecules are perpendicular to the air-LC interface or optic axis $\hat{c}$ is perpendicular to the interface, and (b) when LC molecules are parallel to the air-LC interface or optic axis $\hat{c}$ is parallel to the interface.
Given ${\theta _i}$, we are able to calculate ${\theta _e}$ from Eq. (9). According to Eq. (8), we further obtain ${n_{eff}}$. Next, we calculate propagation direction of corresponding Poynting vector or direction of “ray” in Fig. 2(a). According to dispersion relation in Eq. (6), the unit vector of the Poynting vector can be obtained [28]:
Similarly, when the LC molecules are parallel to the interface (or optic axis $\hat{c}$ of uniaxial medium is parallel to the interface), as depicted in Fig. 2(b), the angle of the Poynting vector ${\theta _S}$, refractive angle ${\theta _e}$ of the wave vector, the longitudinal shift ($\delta {l_S}$) of the Poynting vector, and the longitudinal shift ($\delta {l_e}$) of the wave vector for e-wave are listed in Eqs. (17)–(18):
According to Eqs. (9), (11), (15), and (16), we plotted ${\theta _S}$ and ${\theta _e}$ as a function of angle of incidence, as shown in Fig. 3(a). The parameters we used based on experiments are: no=1.52, ne=1.81 at λ=633 nm (MLC- 2172, Merck), and d=200 microns. In Fig. 3(a), both ${\theta _S}$ and ${\theta _e}$ increase with the angle of incidence no matter LC molecules are perpendicular or parallel to the interface. ${\theta _S}$ for LC molecules been parallel to the interface is larger than the ${\theta _e}$, at the same time, ${\theta _S}$ for LC molecules been perpendicular to the interface is smaller than the ${\theta _e}$. As a result, when LC molecules are switched from parallel to perpendicular arrangement, the difference in ${\theta _S}$ is larger than that the difference in ${\theta _e}$ for a given angle of incidence. When LC molecules are perpendicular to the interface, ${\theta _S} < {\theta _e}$ for a given angle of incidence. However, when LC molecules are parallel to the interface, ${\theta _S} > {\theta _e}$. The Poynting vector or direction of e-ray is refracted more than the wave vector of e-wave when LC molecules are switched. From Eqs. (12), (14), (17), and (18), we also plotted $\delta {l_S}$ and $\delta {l_e}$ as a function of angle of incidence in Fig. 3(b). Both of $\delta {l_S}$ and $\delta {l_e}$ increase with the angle of incidence. When LC molecules are perpendicular to the interface, $\delta {l_S} > \delta {l_e}$ for a given angle of incidence. However, $\delta {l_S} < \delta {l_e}$ for a given angle of incidence when LC molecules are parallel to the interface. This indicates longer longitudinal shift of e-ray (Poynting vector) than that of the e-wave (wave vector) when LC molecules are switched from parallel to perpendicular arrangement. We actually measure the e-ray (Poynting vector), not wave-vector. We now go back to the imaging system with a thin lens. In Fig. 3(c), the image is virtual and erected image when $p < f$. When we put the LC plate in front of a thin lens in Fig. 3(c), as shown in Figs. 3(d)–3(e), the virtual image is further away from the lens as LC is parallel to the interface; while, the virtual image is more closed to the lens as LC is perpendicular to the interface. The virtual image is denoted red and vertical arrows of $\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} \over S}$-vector in Figs. 3(d)–3(e). In other words, the virtual image is more close to the lens when the refractive index decreases for e-wave. This is totally opposite to the conclusion when we analyze the system using Eq. (5) because Eq. (5) is based on analysis of wave-vector propagation. In Figs. 3(d)–3(e), we also draw the blue vertical arrows of $\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} \over k}$-vector to show the location of virtual images based on analysis of wave-vector propagation. Here we would like to emphasize that Poynting vector analysis should be adopted when we consider a LC plate in an optical system. In Eq. (5), q should be the image distance based on Poynting vector analysis, not the analysis of wave-vector propagation. It should be noted that, the analysis above under the assumption that the incident light is in xz-plane. When the angle of incidence is in yz-plane, extra astigmatism needs to be considered [25].
Fig. 3. (a) Calculated ${\theta _S}$ and ${\theta _e}$ as a function of angle of incidence. (b) Calculated $\delta {l_S}$ and $\delta {l_e}$ as a function of angle of incidence. (c) Image formation when the objective distance is smaller than focal length ($p < f$). (d) From (c) Image formation with a LC plate when the optic axis $\hat{c}$ is parallel to the interface. (e) From (d), image formation when the optic axis $\hat{c}$ is perpendicular to interface. Compared with (d), the virtual image is closer to the lens in (e) which is opposite to the result analyzed by wave-vector propagation.
3. Experiments and discussions
For the proof-of-concept, we fabricated three LC plane-parallel plates (LC plates) based on the multiple-layered structure [29], as depicted in Fig. 4(a). The structure of each LC plane-parallel plate in Fig. 4(a) consists of two glass substrates, two indium-tin-oxide (ITO) layers as electrode, two mechanically-rubbed alignment layers (polyimide, model: SE-7492, Nissan Chemical), four LC layers (nematic liquid crystals, MLC-2172, Merck, Δn = 0.29 at λ = 589.3 nm at 20°C) with thickness of 35 μm, and three polymeric layers with thickness of 35 μm function as planar type alignment layers and separators to separate the LC layers. The thickness of the bottom glass substrates (BK7) is 0.5 mm. The thickness is 0.1mm for the top glass substrate (Willow glass, Corning). The reason why we use Willow glass is to minimize the total thickness of each LC plate. As for the polymeric layers, it was made by a mixture of a reactive mesogen (RM257, Merck), a nematic LC (MLC2144, Merck) and a photoinitiator (IRG184) with a ratio of 79:20:1 wt. %. The fabrication of the polymeric layers was reported previously, and here we briefly summarize the process [29–30]. First, the mixture was filled at 90°C between two ITO-glass substrates coated with mechanically buffered PVA (polyvinyl alcohol) and the gap between glass substrates was 35 μm controlled by a mylar film. Next, the cell was exposed to UV light (λ=365 nm) with an intensity of 3 mW/cm2 at 90°C for an hour for photo-polymerization. After photo-polymerization, we obtain a transparent and scattering-free polymeric layer (transmittance∼95%) [31]. The surface of polymeric layer provides strong alignment capability (anchoring energy ∼10−5J/m2) with a direction parallel to the mechanically buffered direction of PVA [31]. Next, we peeled off the substrates and a polymeric layer with thickness of 35 μm was used to assemble the LC plate, depicted in Fig. 4(a). In Fig. 4(a), when the applied voltage is zero (V=0), the LC directors are aligned in x-direction. When the applied voltage (V) much exceeds threshold voltage (Vth), the LC molecules are orientated perpendicularly to the glass substrates. We fabricated three LC plane-parallel plates (denoted as #1, #2, and #3) for the proposed AR system.
Fig. 4. LC plane-parallel plates and the measurement of electro-optical property. (a) Multiple-layered structure consists of four LC layers and three polymeric layers. (b) The voltage dependent refractive index for wave-vector as the function of applied voltage.
After preparing the LC plane-parallel plates, we measured electro-optical (EO) properties and the total thickness of each LC plate. The measurement setup for EO properties consists of a laser light source (λ=633 nm), a pair of crossed polarizers, a photo-detector (Model: 2031, New Focus). Each LC plate was placed between the crossed polarizers, and the rubbing direction of the LC plate was 45 degrees with respect to the transmissive axis of one polarizer. The voltage-dependent transmittance of each LC plate was measured. Later on, we converted the data of the voltage-dependent transmittance to the phase retardation as a function of the applied voltage [28]. After that, the phase retardation is further converted into effective refractive index (neff) of LC as a function of applied voltage, as shown in Fig. 4(b). In Fig. 4(b), neff of three LC plates decreases with the applied voltage and is closed to the ordinary refractive index no (no ∼1.52) as V > 100Vrms. This indicates that the LC directors (i.e., optics axis $\hat{c}$) are reoriented toward z-axis when the applied electric field increases. The angle of optics axis of LC is closed to 0 degree with respect to z-axis (i.e., $\phi (V = 100{\textrm{V}_{rms}})\sim 0$) as V > 100 Vrms. $\phi$ is defined as the angle between optics axis of LC and the z-axis. From Fig. 4(b), we calculated the total thickness of 4 LC layers of each LC plate as listed in Table 1. We also use an outside micrometer (Series 103, Mitutoyo) to measure total thickness of each LC plate as listed in Table 1. When we add three samples together, the total thickness of 12 LC layers is around 510 μm (i.e., 156 + 166 + 188μm=510 μm) which is thick enough for shifting the imaging plane as proposed. By considering the glass substrates, the total thickness of three LC plates is ∼2.67 mm (i.e., 885 + 896 + 894μm=2675 μm).
Table 1. Specifications of fabricated LC plane-parallel plates.
To demonstrate the varifocal AR system shown in Figs. 1(a)–1(b), the AR system was setup. The system consists of a broadband light source (Fiber-Lite, model: DC950), a pin-hole aperture (< 1 mm in diameter) for confining the light as a point source, a resolution chart (USAF 1951; negative type), three LC plane-parallel plates, a beam splitter (side length=12.7 mm), an Al-coated concave mirror with the focal length of 19 mm (i.e., f = 19 mm) and the diameter of 25.4 mm, and a camera (EOS760D, Canon) which is used to replace the human eye as well as to record the projected virtual image. Three LC plane-parallel plates were attached to the resolution chart. The effective objective distance between the resolution chart and the concave mirror is around 18.83 mm at voltage-off states of three LC plates, and the projected image is a virtual image because the effective objective distance is smaller than the focal length of the concave mirror (f = 19 mm). In order to find out the location of the virtual image, we put a real object in front of the AR system as a reference. We moved around the real object and took pictures after we refocused the camera to see object clearly. We then measured the distance (z) between the real object and the beam splitter (BS). When the real object and the virtual image were at the same location, the camera saw the real object and the image projected from the resolution chart. Assume the BS (i.e, the viewing window of the AR system) was set as z = 0 cm. To quantify the location of virtual image, the recorded photos are calculated in terms of contrast ratio defined as $({I_{\textrm{max}}} - {I_{\min}})/({I_{\textrm{max}}} + {I_{\min}})$, where ${I_{\textrm{max}}}$ and ${I_{\min }}$ are the maximum and minimum intensity of the recorded virtual image. As a result, the location of virtual image can be determined when the contrast ratio of the virtual image reaches the peak value. Figure 5(a) plotted the contrast ratio as a function of the distance z when three LC plates are operated. In Fig. 5(a), the peak contrast ratio was ∼0.5 at z = 209 cm when all the LC plane-parallel plates were turned off. When the LC plane-parallel plates turned on, the location of peak contrast ratio shifted to z = 192 cm as the LC plate#1 was subject to 100 Vrms, z = 170 cm as the LC plate#1 and the LC plate#2 were subject to 100 Vrms, and z = 136 cm as three LC plates were subject to 100 Vrms. This indicates the virtual image could be shifted from 209 cm to 136 cm by changing the orientations of LC molecules in the LC plates. According to Eq. (10), we plotted the relationship between the image distance (q) and the changes of the objective distance due to LC plates (i.e., second term of Eq. (10)), as shown in the blue line in Fig. 5(b). Figure 5(b) indicates that the location of the virtual image could be shifted from 209 cm to 136 cm even though the changes of the objective distance by using LC plates is small ∼150 micron. When all LC plates are off, LC plates provide zero difference of optical path (fixed $\delta {l_S}$). After we changes the optical path via LC plates with applied voltages, the projected virtual image shifts to the location closed to the AR system. The experimental results are similar to the calculations as shown in Fig. 5(b). The deviation of image distance between experiments and calculations is because the calculation is under the assumption that that $\phi = 0$ at V=100Vrms>>Vth and $\phi$=90 degrees at V=0. In practical, the initial tilt angle of LC at V=0 is measured at around 2∼3 degrees (i.e., $\phi (V = 0)$≈87 degrees) and the orientations of LC molecules at the surfaces of alignment layers is fixed and hard to be re-oriented due to strong anchoring energy [28,31]. Those are the reasons to cause the deviation between experiments and calculations.
Fig. 5. (a) The contrast ratio of the virtual image as a function of the distance z. z is the distance between the real object and the beam splitter as the camera see clearly for both of the virtual image and the real object. (b) Converted from (a), the measured image distance of the virtual image (q+Δq) as a function of difference of tunable $\delta {l_S}$ that is provided by LC plates. The blue line stands for the calculation results based on Eq. (3) with with f = 19 mm and p ∼ 18.83 mm.
To demonstrate the performance of the varifocal function in AR system we proposed, two real objects, with “NCTU logo” and “Google logo”, were used. The first real object (“NCTU logo”) was located at z = 2.1 m while another real object (“Google logo”) is located at z = 1.4 m. The camera of the AR system was set to focus at z = 2.1 m and no voltage was applied to the LC plates; therefore, the camera sees the real object of “NCTU logo” and the virtual image clearly, as shown in Fig. 6(a). Next, in order to mimic the eye accommodation, we changed the focusing properties of the camera in order to see the real object at z = 1.4 m clearly, as shown in Fig. 6(b). In Fig. 6(b), the virtual image is blurred because the virtual image is still located at z∼2.1 m. To align the virtual image to coincide with the near real object of “Google logo”, we applied 100 Vrms to three LC plates. As a result, the location of the virtual image was then shifted to the location at z∼1.4 m. Thereafter, the camera saw both of the near real object and the virtual image clearly, as shown in Fig. 6(c). In Fig. 6, the image quality is slightly blurred due to ghost image originated from surface reflection of LC plates, including air-glass interface, LC-polymer interface etc. The reflectivity of three LC plates with six air-glass interfaces is around 4%. The image quality could be improved dramatically by anti-reflection coating technique and consideration of index matching in multi-layered structures.
Fig. 6. Image performance of the proposed AR system with three LC plane-parallel plates. (a) The focusing plane of the camera was set at 2.1 m away from AR system. The projected virtual image was also set around 2.1 m away from AR system. (b) When the camera was refocused and set to see the images at the distance of 1.4 m away from AR system, the near object was clear while the virtual image is blurred. (c) After we applied 100 Vrms to three LC plates, the location of the virtual image was shifted to 1.4 m. Both the near object and the virtual image were clearly recorded by the camera. The upper-left figure is the illustration of relative positions of the objects and virtual image in (a), (b), and (c).
Assuming a small angle of incidence and ni=1, let us go back to Eqs. (13), (14), (17), and (18). From Eqs. (13) and (17), the difference in longitudinal displacement between high voltage (V>>Vth) and V=0 for Poynting vectors is:
Since the ratio stated in Eq. (21) is almost fixed (< 3.2 even with a birefringence of ∼0.8 [32]), the optical path difference is still linearly proportional to the birefringence of LC material and the total thickness of LC layers. Currently, tunable depth of image planes in proposed system is from 1.4 m to 2.1 m while the typical varifocal AR systems are expected to have image plane with tunable depth from 0.33 m to infinite (i.e., 3 Diopter tunable range). In order to enlarge the tunable depth, according to the concept multiple-layered structure [30], we could increase the total thickness by adding the numbers of LC layers without increasing the response time. Taking into account the accommodation ability of human eyes, the response time of LC plane-parallel plates (>10 seconds for switching on and for switching off) should be less than 500 ms [28]. To reduce response time, the thickness of each LC layer should also be reduced. However, to sustain a sufficient distance for the purpose of the image shifting, the total thickness of LC layers cannot be reduced. As a result, we need to divide more sub-layers in a multiple-layered structure to maintain the same total thickness [29–30]. In this way, the response time could be reduced while keeping the same the image shifting range. Another solution for sustaining or increasing the distance of image shifting is to adopt LC materials with higher birefringence. However, the current birefringence of LC materials is still limited (less than 0.8) [32]. As for the total thickness of each LC plane-parallel plate, now we only adopt one Willow glass (Corning) with thickness of 0.1mm as the substrate. One could further reduce the thickness of LC plates by adopting two thin glasses and thinner polymeric layers. For reducing the driving voltage, we can add sheet electrodes within the multi-layered structures to control the sub-LC layer individually; For example, we can replace the polymeric layers with thin ITO-coated plastic films. Instead of reorienting the LC directors, another approach could be pairing a polarization controller with a LC plate, where the polarization of incident light could be switched between e-wave or o-wave [25,33]. If we have N pairs of a polarization controller and a LC plate, the number of image planes could be 2N, and the response time is boosted. For example, 8 image planes are achievable for N=3, and response time could be less than 3 ms with LC-based polarization controllers [25,33–35]. For better systematic integration and compactness, the cover glass substrate of the microdisplay may be replaced with this LC plate. Of course, the driving scheme could be specially for practical applications. In Fig. 6, the virtual images were projected with the broadband light source, and no significant color breakup was observed. However, when the total thickness of a LC layer increases, the chromatic aberration needs to be accounted for according to Eqs. (4)–(5) for a better image quality. Regarding to the spherical aberration, well-known in plane-parallel plate, the LC plates act as a plate with angularly dependent refractive index as shown in Fig. 3(b). This indicates that the intrinsic spherical aberration in the plane-parallel plate could be cancelled as we design it tailored to the system. In addition, the demonstration was operated with voltage of 100 Vrms because we need to orientate the LC directors from the parallel alignment to the perpendicular one to the substrate. To reduce the driving voltage, the thickness of polymeric layer should be reduced, or one can use a material with a higher permittivity.
4. Conclusion
We demonstrated a varifocal function for solving VAC of AR system based on electrically tunable LC plane-parallel plates with total thickness around a few millimeters. For providing a sufficient tunable optical path, three multiple-layered structures are used to increase the total thickness of LC layers (∼510 μm). Compared to the system based on tradition calcite-based plane-parallel plates with thickness∼ 30 mm, the proposed system is more compact. Since the split of the wave vector and the Poynting vector for extraordinary wave cannot be ignored, the longitudinal displacement of the image plane is determined by Poynting vectors rather than wave vectors within the geometrical optical analysis. The tunable range of the longitudinal displacement of Poynting vectors is more than twice bigger compared to the one due to by the wave vectors. Researchers always neglect parallel-plane-plate using LC because of limited birefringence of LC material and the thickness of a LC layer. The multiple-LC-layer sample can provide enough optical path. In addition, the Ponyting vector analysis further extend longitudinal displacement due to the origin of splitting between a Ponyting vector and a wave vector in an uniaxial optical medium. Notably, for Poynting vectors and wave vectors, intuitively, the directions of their longitudinal displacements are opposite which leads to an inverse shifting direction of the projected virtual image in the AR system. Among all the solutions for solving vergence-accommodation conflict (VAC), our tunable LC plate method is not only simple, but the system is also thin and exhibits large clear aperture. The current challenges of the proposed solution are relatively high response time (∼10 seconds) and driving voltage. To improve response time and driving voltage, reduction of the thickness of each LC layers by adding more LC layers could be a way to go as well as applying electric fields to each LC layer separately could lower the driving voltage. One could also improve the response time by combining polarization-switching technique. We expect that the investigation here paves a rather simple way to solve the VAC problem for augmented reality systems.
Funding
Google (Google Daydream University Research Award); Ministry of Science and Technology, Taiwan (107-2112-M-009-019-MY3).
Acknowledgments
The authors are indebted to Guo-Lin Hu, Ming-Long Lee, Chen-Han Lin, Huai-An Hsieh, Chia-Ming Chang, and Yu-Cheng Su for technical assistances. We would like to thank Google Daydream University Research Award for partially sponsoring this work.
Disclosures
YJW: Google LLC (R), YHL: Google LLC (R), OC: Google LLC (E).
References
1. O. Cakmakci and J. P. Rolland, “Head-worn displays: a review,” J. Disp. Technol. 2(3), 199–216 (2006). [CrossRef]
2. B. Kress and T. Starner, “A review of head-mounted displays (HMD) technologies and applications for consumer electronics,” Proc. SPIE 8720, 87200A (2013). [CrossRef]
3. X. Li, W. Yi, H. L. Chi, X. Wang, and A. P. C. Chan, “A critical review of virtual and augmented reality (VR/AR) applications in construction safety,” Autom. Constr. 86, 150–162 (2018). [CrossRef]
4. M. B. Ibáñez, Á. D. Serio, D. Villarán, and C. D. Kloos, “Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness,” Comput. Educ. 71, 1–13 (2014). [CrossRef]
5. M. N. van Oosterom, H. G. van der Poel, N. Navab, C. J. H. van de Velde, and F. W. B. van Leeuwen, “Computer-assisted surgery: virtual-and augmented-reality displays for navigation during urological interventions,” Curr. Opin. Neurol. 28(2), 205–213 (2018). [CrossRef]
6. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence–accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008). [CrossRef]
7. G. K. Von Noorden and E. C. Campos, Binocular Vision and Ocular Motility, 6th ed (Mosby, 2002), Chap. 2.
8. G. Kramida, “Resolving the vergence-accommodation conflict in head-mounted displays,” IEEE Trans. Visual. Comput. Graphics 22(7), 1912–1931 (2016). [CrossRef]
9. S. Liu, Y. Li, and Y. Su, “Multiplane displays based on liquid crystals for AR application,” J. Soc. Inf. Disp. 28(3), 224–240 (2020). [CrossRef]
10. H. Huang and H. Hua, “Effects of ray position sampling on the visual responses of 3D light field displays,” Opt. Express 27(7), 9343–9360 (2019). [CrossRef]
11. E. Hecht, Optics, 4th ed (Pearson Education, 1974).
12. S. Liu and H. Hua, “Time-multiplexed dual-focal plane head-mounted display with a liquid lens,” Opt. Lett. 34(11), 1642–1644 (2009). [CrossRef]
13. P. Chakravarthula, D. Dunn, K. Aksit, and H. Fuchs, “Focusar: auto-focus augmented reality eyeglasses for both real world and virtual imagery,” IEEE Trans. Visual. Comput. Graphics 24(11), 2906–2916 (2018). [CrossRef]
14. H. S. Chen, Y. J. Wang, P. J. Chen, and Y. H. Lin, “Electrically adjustable location of a projected image in augmented reality via a liquid crystal lens,” Opt. Express 23(22), 28154–28162 (2015). [CrossRef]
15. Y. J. Wang, P. J. Chen, X. Liang, and Y. H. Lin, “Augmented reality with image registration, vision, correction and sunlight readability via liquid crystal devices,” Sci. Rep. 7(1), 433 (2017). [CrossRef]
16. H. Ren and S. T. Wu, Introduction to Adaptive Lenses (Wiley, 2012).
17. Y. H. Lin, Y. J. Wang, and V. Reshetnyak, “Liquid crystal lenses with tunable focal length,” Liq. Cryst. Rev. 5(2), 111–143 (2017). [CrossRef]
18. U. Y. Basak, S. Kazempourradi, C. Yilmaz, E. Ulusoy, and H. Urey, “Dual focal plane augmented reality interactive display with gaze-tracker,” OSA Continuum 2(5), 1734–1745 (2019). [CrossRef]
19. J. P. Rolland, M. W. Krueger, and A. Goon, “Multifocal planes head-mounted displays,” Appl. Opt. 39(19), 3209–3215 (2000). [CrossRef]
20. Y. H. Lee, G. Tan, K. Yin, T. Zhan, and S. T. Wu, “Compact see-through near-eye display with depth adaption,” J. Soc. Inf. Disp. 26(2), 64–70 (2018). [CrossRef]
21. Q. Chen, Z. Peng, Y. Li, S. Liu, P. Zhou, J. Gu, J. Lu, L. Yao, M. Wang, and Y. Su, “Multi-plane augmented reality display based on cholesteric liquid crystal reflective films,” Opt. Express 27(9), 12039–12047 (2019). [CrossRef]
22. J. H. Park, S. Jung, H. Choi, and B. Lee, “Integral imaging with multiple image planes using a uniaxial crystal plate,” Opt. Express 11(16), 1862–1873 (2003). [CrossRef]
23. M. Avendaño-Alejo, “Analysis of the refraction of the extraordinary ray in a plane-parallel uniaxial plate with an arbitrary orientation of the optical axis,” Opt. Express 13(7), 2549–2555 (2005). [CrossRef]
24. M. Avendaño-Alejo and M. Rosete-Aguilar, “Optical path difference in a plane-parallel uniaxial plate,” J. Opt. Soc. Am. A 23(4), 926–932 (2006). [CrossRef]
25. C. K. Lee, S. Moon, S. Lee, D. Yoo, J. Y. Hong, and B. Lee, “Compact three-dimensional head-mounted display system with Savart plate,” Opt. Express 24(17), 19531–19544 (2016). [CrossRef]
26. S. Liu and H. Hua, “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010). [CrossRef]
27. M. Katz, Introduction to Geometrical Optics (World Scientific, 2002).
28. D. K. Yang and S. T. Wu, Fundamentals of Liquid Crystal Devices (Wiley, 2006).
29. Y. H. Lin, H. Ren, Y. H. Wu, Y. Zhao, J. Fang, Z. Ge, and S. T. Wu, “Polarization-independent liquid crystal phase modulator using a thin polymer-separated double-layered structure,” Opt. Express 13(22), 8746–8752 (2005). [CrossRef]
30. H. S. Chen, Y. J. Wang, C. M. Chang, and Y. H. Lin, “A polarizer-free liquid crystal lens exploiting an embedded-multilayered structure,” IEEE Photonics Technol. Lett. 27(8), 899–902 (2015). [CrossRef]
31. Y. H. Lin, H. S. Chen, Y. J. Wang, and C. M. Chang, “A liquid crystal and polymer composite film for liquid crystal lenses,” Proc. SPIE 9384, 938410 (2015). [CrossRef]
32. Y. Arakawa, S. Kang, H. Tsuji, J. Watanabe, and G. I. Konishi, “The design of liquid crystalline bisolane-based materials with extremely high birefringence,” RSC Adv. 6(95), 92845–92851 (2016). [CrossRef]
33. S. Pagidi, R. Manda, S. S. Bhattacharyya, S. G. Lee, S. M. Song, Y. J. Lim, J. H. Lee, and S. H. Lee, “Fast Switchable micro-lenticular lens arrays using highly transparent nano-polymer dispersed liquid crystals,” Adv. Mater. Interfaces 6(18), 1900841 (2019). [CrossRef]
34. G. D. Love, D. M. Hoffman, P. J. W. Hands, J. Gao, A. K. Kirby, and M. S. Banks, “High-speed switchable lens enables the development of a volumetric stereoscopic display,” Opt. Express 17(18), 15716–15725 (2009). [CrossRef]
35. Y. H. Lee, F. Peng, and S. T. Wu, “Fast-response switchable lens for 3D and wearable displays,” Opt. Express 24(2), 1668–1675 (2016). [CrossRef]
