Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

An optical system for augmented reality with electrically tunable optical zoom function and image registration exploiting liquid crystal lenses

Open Access Open Access

Abstract

An optical-see-through augmented reality (AR) system assists our daily work by augmenting our sense with computer-generated information. Two of optical challenges of AR are image registration and vision correction due to fixed optical properties of the optical elements of AR systems. In this paper, we demonstrated an AR system with optical zoom function as well as a function of image registration via two LC lenses in order to help people see better by magnifying the virtual image and adjusting the location of virtual image. The operating principles are introduced, and experiments are performed. The concept demonstrated in this paper could be further extended to other electro-optical devices as long as the devices exhibit the capability of phase modulations.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Ageing society is a global phenomenon, and United Nations (UN) suggested that elderly work force should be permitted to work as long as the intention and productivity remain [12]. However, human enter senescence with time, and the descending problems in physiological and mental state appear naturally. To fight with ageing related problems, people equipped with electronic, so-call “bionic people”, are augmented in hearing, vision, memory, and other function [2]. Augmented reality (AR) is a set of technologies that provides computer-generated information to eyes for augmenting the real-world environment [3-7]. The optical-see-through type of AR system allows people see the real world directly instead of seeing through a monitor displaying the camera-captured objects and information of interest. In optical viewpoint, optical-see-through AR system consists of light source, information source (e.g., display panel), light guiding element, and projection module (e.g., lens and mirror) [3-7]. The main challenges of the optical-see-through AR systems are vergence-accommodation conflicts (VAC) in 3D experiences, image registration which means virtual images can not coincide with real objects, and a lack of vision correction for different human eyes [8]. By adding extra and tunable degree of freedom in optical properties to the optical-see-through AR systems, liquid crystal (LC) lenses with electrically tunable focal lengths are the great candidates for solving the challenges of vision correction and image registration [9-13]. In 2017, we had demonstrated an AR system with capability of image registration and vision correction using two LC lenses [14-16]; however, it is still a challenge to realize the polarization-independent LC lens with not only large aperture size (> 20 mm) and continuously tunable optical lens power > 3 Diopter (i.e. an inverse of a focal length) due to limitation of chemical and physical natures of nematic liquid crystals [1618]. In this paper, instead of pursuing polarization-independent LC lens with large aperture and large tunable range, we propose an approach to provide optical zoom function as well as a function of image registration in in AR system via two LC lenses in order to help people see better by magnifying the virtual image and adjusting the location of virtual image. The operating principles of the proposed AR system are introduced. The proof-of-concept is demonstrated theoretically and experimentally. The concept proposed in this paper could be further extended to other electro-optical devices as long as the devices exhibit the capability of phase modulations.

2. Operating principles

The proposed AR system with LC lenses is illustrated in Fig. 1(a)–1(c). The first LC lens, named as “LC lens 1” is placed between a polarizing beam splitter (PBS) and a beam splitter (BS); the second LC lens, named as “LC lens 2”, is placed between the BS and a concaved mirror. The light from LED passes through PBS onto a reflective display (liquid crystal on silicon panel or LCOS panel), and the image on the display panel is then reflected to the PBS, a polarizer, “LC lens 1”, the BS, “LC lens 2”, and the concave mirror. The image reflected by the concave mirror arrives to a retinal of an eye through “LC lens 2” and BS, a cornea and a crystalline lens of eye. Finally, the eye sees a projected image (virtual image) superimposed with the real world (real object) via the BS. In Fig. 1(a), the projected virtual image is spatially registered with the distant real object (i.e. the location of the projected virtual image coincides with the distant real object.) as the two LC lenses are turned off. The eye sees both of the real object and the virtual image clearly. Assume an eye suffers from myopia and presbyopia and the eye cannot see object clearly when the real object is located at different locations. When we turn on “LC lens 2” with a negative lens power, the image and the near real object coincide because the effective optical lens power of “LC lens 2” and mirror changes [1416], as depicted in Fig. 1(b). However, the virtual image is blurred due to presbyopia. To magnify the virtual image while keep the location of virtual fixed (i.e., registration function), we need to operate another LC lens together. When we turn on “LC lens 1” and “LC lens 2” simultaneously (see Fig. 1(c)), the virtual image could be projected at the fixed location and also coincides with near real object. Similarly, for eyes with myopia, the magnification of the virtual image is tunable while its spatial location is registered with a certain real object.

 figure: Fig. 1.

Fig. 1. Operating principle of the AR system with optical zoom function and image registration. (a) Initially, eye sees the distant real object and the virtual image “zoom function” when “LC lens 1” and “LC lens 2” are off. (b) When “LC lens 2” is on and the virtual image is registered with the near real object, for eye, both the virtual image and the real object are blurred due to presbyopia. (c) The project virtual is magnified when “LC lens 1” and “LC lens 2” are operated properly together. Therefore, the magnified virtual image in the retinal becomes clearer.

Download Full Size | PDF

To prove the proposed optical zoom function, we discuss the image formation with a configuration based on two tunable lenses under thin lens approximation [1922]. Assume the “p1” and “q1” are the objective distance and the image distance of the first tunable lens (i.e., “LC lens 1”), and “p2” and “q2” are the objective distance and the image distance of the second tunable lens (i.e., “LC lens 2” integrated with a concave mirror). In our AR system, q2 is the sum of two distances: the distance between the concave mirror and the BS, and the distance between the BS and the projected image. The proposed image system should satisfy the following equations:

$$\frac{1}{{{p_1}}} + \frac{1}{{{q_1}}} = \frac{1}{{{f_1}(V)}} = {\bar{P}_1}(V)$$
$$\frac{1}{{{p_2}}} + \frac{1}{{{q_2}}} = \frac{1}{{{f_2}(V)}} = {\bar{P}_2}(V)$$
where ${p_2} = d - {q_1}$ and “d” is the effective optical path between two tunable lenses; ${f_1}$ and ${f_2}$ are the focal length of the first and the second tunable lens, respectively. The optical lens powers of two tunable lenses are defined as the reciprocal of its focal length in a unit of m-1 (Diopter, D), and they are related to “LC lens 1”, “LC lens 2”, and the concave mirror (i.e., ${\bar{P}_1} = {\bar{P}_{LC1}}(V)$, where $V$ is applied voltage; ${\bar{P}_2} = 2 \cdot {\bar{P}_{LC2}}(V) + {\bar{P}_{mirror}}$). For a given q2, the relationship between ${\bar{P}_1}$ and ${\bar{P}_2}$ is
$${\bar{P}_2} = \frac{{{{\bar{P}}_1} \cdot {p_1} \cdot (d + {q_2}) - ({p_1} + d + {q_2})}}{{{q_2} \cdot [{{{\bar{P}}_1} \cdot {p_1} \cdot d - ({p_1} + d)} ]}}$$
The magnification (M) of projected virtual image is defined as the product of the ratios of image distance to objective distance:
$$M = \frac{{{q_1}}}{{{p_1}}} \cdot \frac{{{q_2}}}{{{p_2}}}$$
To investigate the magnification capability, we further rewrite Eq. (4) based on image formation stated in Eq. (1) and Eq. (2). Then, we obtain the M as the function of ${\bar{P}_1}$ and ${\bar{P}_2}$ for a fixed projection distance (q2), respectively:
$$M({\bar{P}_1}) = \frac{{{q_2}}}{{{{\bar{P}}_1} \cdot {p_1} \cdot d - d - {p_1}}}$$
$$M({\bar{P}_2}) = \frac{{{{\bar{P}}_2} \cdot d \cdot {q_2} - d - {q_2}}}{{{p_1}}}$$
According to Eq. (5) and Eq. (6), the tunable range of M could be confined by ${\bar{P}_1}$ and ${\bar{P}_2}$ which are related to the optical lens power of the LC lenses. Next, to further obtain the optical zoom ratio (ZR) which is defined as the maximum magnification over minimum magnification of the AR system, we need to consider several practical conditions [4,10]: (1) the projected image is a magnified image (absolute value of M is larger than 1). (2) The projected image is a virtual image (q2<0). (3) the AR system is compact with small value of p1 and d (∼cm) compared to the image distance (q2) (4) The tunable lens power of an LC lens ranges from −10D to +10D unless we specially design it or integrate it with a solid lens. Due to those constrains, the proper design conditions of the lens power of two tunable lenses are obtained and stated in Eq. (7) and Eq. (8).
$$\frac{{{q_2} + d + {p_1}}}{{d \cdot {p_1}}} < {\bar{P}_1} < \frac{{d + {p_1}}}{{d \cdot {p_1}}}.$$
$${\bar{P}_2} < \frac{{{p_1} + d + {q_2}}}{{d \cdot {q_2}}}.$$
When the lens powers of LC lenses are in the interval stated in Eq. (7) and Eq. (8), $M({\bar{P}_1})$ is a monotonic increasing function while $M({\bar{P}_2})$ is a monotonic decreasing function. Thus, the maximum magnification (Mmax) and minimum magnification (Mmin) could be found at the upper and lower limits of ${\bar{P}_1}$ and ${\bar{P}_2}$ (i.e., ${\bar{P}_{1,\min}}$,${\bar{P}_{1,\max }}$,${\bar{P}_{2,\min }}$, and ${\bar{P}_{2,\max }}$). Assume the range of magnification in Eq. (5) is ${M_A} \le M({\bar{P}_1}) \le {M_B}$, where ${M_A} = M({\bar{P}_1} = {\bar{P}_{1,\min }})$ and ${M_B} = M({\bar{P}_1} = {\bar{P}_{1,\max }})$; the range of magnification in Eq. (6) is ${M_C} \le M({\bar{P}_2}) \le {M_D}$, where ${M_C} = M({\bar{P}_2} = {\bar{P}_{2.\max }})$ and ${M_D} = M({\bar{P}_2} = {\bar{P}_{2,\min }})$. When the tunable range of lens power of “LC lens 1” is ${\bar{P}_{LC1,\min }} \le {\bar{P}_{LC1}} \le {\bar{P}_{LC1,\max }}$, ${\bar{P}_{1,\min}} = {\bar{P}_{LC1,\min }}$ and ${\bar{P}_{1,\max}} = {\bar{P}_{LC1,\max }}$. The second tunable lens consists of an LC lens (“LC lens 2”) and a mirror. When the tunable range of lens power of “LC lens 2” is ${\bar{P}_{LC2,\min }} \le {\bar{P}_{LC2}} \le {\bar{P}_{LC2,\max }}$, $2 \cdot {\bar{P}_{LC2,\min }} + {\bar{P}_{mirror}} \le {\bar{P}_2} \le 2 \cdot {\bar{P}_{LC2,\max }} + {\bar{P}_{mirror}}$. Therefore, the limitation of magnification can be rewrite as:
$${M_A} = \frac{{{q_2}}}{{{{\bar{P}}_{LC1,\min }} \cdot {p_1} \cdot d - d - {p_1}}}$$
$${M_B} = \frac{{{q_2}}}{{{{\bar{P}}_{LC1,\max }} \cdot {p_1} \cdot d - d - {p_1}}}$$
$${M_C} = \frac{{(2 \cdot {{\bar{P}}_{LC2,\max }} + {{\bar{P}}_{mirror}}) \cdot d \cdot {q_2} - d - {q_2}}}{{{p_1}}}$$
$${M_D} = \frac{{(2 \cdot {{\bar{P}}_{LC2,\min}} + {{\bar{P}}_{mirror}}) \cdot d \cdot {q_2} - d - {q_2}}}{{{p_1}}}$$
where ${\bar{P}_{mirror}} = {{({p_1} + d + {q_2})} \mathord{\left/ {\vphantom {{({p_1} + d + {q_2})} {{q_2} \cdot (d + {p_1}}}} \right.} {{q_2} \cdot (d + {p_1}}})$. In general, the maximum lens power of an LC lens is positive lens, and it equals to the absolute value of its negative lens power; ${\bar{P}_{LC1,\max }} = - {\bar{P}_{LC1,\min }}$ and ${\bar{P}_{LC2,\max }} = - {\bar{P}_{LC2,\min }}$. Assume the “LC lens 1” and the “LC lens 2” are identical, we can simplify the LC lenses: ${\bar{P}_{LC1,\max }} = {\bar{P}_{LC2,\max }} = \Delta \bar{P}$ and ${\bar{P}_{LC1,\min }} = {\bar{P}_{LC2,\min}} = - \Delta \bar{P}$. Thus, the limitation of magnification can be further rewrite as:
$${M_A} = \frac{{{q_2}}}{{ - \Delta \bar{P} \cdot {p_1} \cdot d - d - {p_1}}}$$
$${M_B} = \frac{{{q_2}}}{{\Delta \bar{P} \cdot {p_1} \cdot d - d - {p_1}}}$$
$${M_C} = \frac{{(2 \cdot \Delta \bar{P} + {{\bar{P}}_{mirror}}) \cdot d \cdot {q_2} - d - {q_2}}}{{{p_1}}}$$
$${M_D} = \frac{{( - 2 \cdot \Delta \bar{P} + {{\bar{P}}_{mirror}}) \cdot d \cdot {q_2} - d - {q_2}}}{{{p_1}}}$$
According to the ranges of the lens powers of LC lenses in Eq. (7) and Eq. (8) and the practical constrains (i.e., M > 1, q2 < 0, |q2| >> p1, |q2| >> d), we obtain MA > MC and MB < MD. The possible minimum magnification is limited by MA and MC. Since MA is larger than MC, the minimum magnification of the AR system is limited by the minimum lens power of “LC lens 1” (i.e., ${\bar{P}_{LC1,\min }}$). As for the maximum magnification of the AR system, it is limited by MB which is related to the maximum lens power of “LC lens 1” (i.e., ${\bar{P}_{LC1,\max}}$). As a result, the range of M of the AR system is limited by ${\bar{P}_{LC1}}$ ranging from ${\bar{P}_{LC1,\min }}$ to ${\bar{P}_{LC1,\max}}$:
$${M_{\max }} = \frac{{{q_2}}}{{{P_{1,\max}} \cdot {p_1} \cdot d - d - {p_1}}}$$
$${M_{\min }} = \frac{{{q_2}}}{{{P_{1,\min }} \cdot {p_1} \cdot d - d - {p_1}}}$$
Therefore, the optical zoom ratio (ZR) confined by “LC lens 1”:
$$ZR = \frac{{{M_{\max }}}}{{{M_{\min }}}} = \frac{{{P_{LC1,\min }} \cdot {p_1} \cdot d - d - {p_1}}}{{{P_{LC1,\max}} \cdot {p_1} \cdot d - d - {p_1}}}$$
To enlarge the ZR of the proposed AR system, the tunable range of “LC lens 1” is critical. The lens power of a gradient-index (GRIN) type LC lens is:
$${\bar{P}_{LC}}(V) = \frac{{2 \cdot \delta n(V) \cdot t}}{{{r^2}}}$$
where $\delta n$ is the refractive index difference that light experiences between the center of aperture and the edge of the aperture, “t” is the thickness of LC layer, “r” is the radius of the lens aperture. $\delta n$ is limited by the birefringence of LC material (i.e., $\delta n \le {n_e} - {n_o}$, where ne and no are the extraordinary refractive index and ordinary refractive index, respectively). According to Eq. (20), the large tunable range of the lens power of an LC lens could be designed by enlarging thickness of the LC layer and reducing the lens aperture. However, a thick LC layer leads to slow response time of the LC lens and small aperture size limits the field-of-view. Thus, the design parameters should be selected properly. Based on the discussion, we theoretically prove an electrically tunable optical zoom function for the AR system with a given image distance (i.e., registration function).

3. Results and discussion

To demonstrate the concept of the proposed AR system, we prepared two polarized LC lenses. The structures of “LC lens 1” and “LC lens 2” are illustrated in Figs. 2(a) and 2(b), respectively. The structure of “LC lens 1” consists of a LC layer (MLC-2172, Merck, Δn = 0.294 at λ = 589.3 nm at 20°C) with thickness of 50 µm, two mechanically buffered alignment layers (materials: polyvinyl alcohol, PVA), three glass substrates with thickness of 0.7 mm (one of glass substrate is labeled as buffering layer), a high-resistive layer (a mixture of PVA and the conductive polymer (S300, Agfa)), an insulating layer (NOA81, Norland Optical Adhesive) with thickness of 35 μm, and three indium-tin-oxide (ITO) layers as electrodes. One of the ITO layer is hole-patterned with an aperture size of 2.5 mm; and the rest of ITO layers are sheet electrodes without any patterns. Similarly, the structure of “LC lens 2” has a similar configuration, but a 10-mm hole-patterned ITO layer and two LC layers (LN3, Δn = 0.369 for λ = 589.3 nm at 20°C) which is separated by a polymer layer. The polymeric layer was made by a mixture of a reactive mesogen (RM257, Merck), a nematic LC (MLC2144, Merck) and a photoinitiator (IRG184) with a ratio of 79:20:1 wt.%. The mixture was then filled at 90°C between two ITO-glass substrates coated with mechanically buffered PVA and the gap between glass substrates was 35 µm controlled by a mylar film. We then applied voltage of 300 Vrms to the cell and then the cell was exposed to unpolarized UV light (λ=365 nm) with an intensity of 3 mW/cm2 for an hour for photo-polymerization. After photo-polymerization, we peeled off the substrates and a polymeric layer with thickness of 35 μm was used to assemble two samples, depicted in Figs. 2(a) and 2(b). When the applied voltage pair V1=V2=0, two LC lenses has no focusing effect (or zero lens power) since the LC directors are aligned parallel to the glass substrates. When V1>V2, the electric field in the central aperture is smaller than the peripheral region of the aperture. As a result, the LC directors in the peripheral region are more perpendicular to central region. The two LC lenses function as positive lenses (i.e. lens powers are positive) when V1>V2. On the contrary, at V1<V2, electric field in the central aperture is strong than the one in the peripheral region; therefore, two LC lenses function as negative lenses (i.e. lens powers are negative).

 figure: Fig. 2.

Fig. 2. (a) Structure of “LC lens 1”. (b) Structure of “LC lens 2”. (c) The measured lens power as the function of an applied voltage pair of (V1, V2). The “LC lens 1” functions as a positive lens at (60Vrms, V) at frequency (Freq) of 6 kHz (red circles) and a negative lens at (V, 50Vrms) at Freq = 0.2 kHz (red triangles). The “LC lens 2” functions as a positive lens at (70Vrms, V) at Freq = 7kHz (blue squares) and a negative lens at (V, 75Vrms) at Freq = 2 kHz (blue diamonds). The effective aperture size of “LC lens 2” was 4 mm.

Download Full Size | PDF

After preparing two LC lenses, lens powers of two LC lenses were measured by Shack-Hartmann wavefront sensor (WFS150-7AR, Thorlabs). The detail measurement was reported previously [1516]. The light source in the measurement consists of a laser (diode pumped solid state laser, λ = 532 nm), a single mode fiber to form a point source, a solid lens with a focal length of 75.6 cm for converting the point source into a plane wave. The generated plane wave passes through the LC samples, and the output wavefront is relayed onto the wavefront sensor by a relay optics (a paired of solid lens as relay optics with focal length: 12.5 cm and 8.3 cm). In the measurement, the wavefront sensor provides the fitting data in the form of 5th order Zernike coefficients (i = 0-20). The lens power can be obtained using equation ${\bar{P}_{LC}} = - {{4\sqrt {3 \cdot } {c_4}} \mathord{\left/ {\vphantom {{4\sqrt {3 \cdot } {c_4}} {{r^2}}}} \right.} {{r^2}}}$ , where c4 is the Zernike coefficient for i = 4. The measured lens power as the function of a voltage pair of (V1, V2) is shown in Fig. 2(c). The measured lens power of “LC lens 1” (${\bar{P}_{LC1}}$) with aperture of 2.5 mm ranges from −14.1D to +17.5D in Fig. 2(c) and the measured lens power of “LC lens 2” (${\bar{P}_{LC2}}$) ranges from −3.0D to +4.8D. Here, we only measured the lens power of the “LC lens 2” with the effective diameter of 4 mm because the size of the liquid-crystal-on-silicon (LCoS) display is 4 mm in diagonal. Generally speaking, the tunable range of the lens power is large when the aperture is small according to the power law of the LC lens [18].

To prove the proposed concept of AR system, we setup the AR system (Fig. 1) and a camera (EOS500D, Canon) was used to replace the human eye in order to record the projected virtual image. The resolution of the LCoS display was 320 × 240 pixels (HX7033, Himax Display Inc.) and the focal length of the concave mirror was 30.94 mm (i.e., ${\bar{P}_{mirror}}$ = 32.32D). In the experiments, p1 and d were measured as 0.98 cm and 2.3 cm, respectively. Assume BS (i.e., viewing window of the AR system) was located at z = 0 cm, we set the camera captured image clearly when the real object was at a distance of z = 300 cm. Then we turned on and input signals to the LCoS display, the LCoS display exhibited the image of bars of 4 pixel per line pair and 2 pixels per line pair. Only when the location of projected image (virtual image) is also at 300 cm, the camera can capture clear image; otherwise, the image is blurred. By adjusting lens powers of two LC lenses, the location of the virtual image is adjustable. For example, the recorded image is clear when lens powers of “LC lens 1”and “LC lens 2” are +12.9 D and −1.9 D, respectively (Fig. 3(a)). This means the virtual image is located at z = 300 cm. From Fig. 3(a), we changed the lens powers of “LC lens 2” into zero diopter and then the recorded image is blurred, as shown in Fig. 3(b). When we adjusted the focal lengths of the camera, we changed the object distance for the real object and set this object distance to find out the location of the virtual image. The recorded photos are then calculated in terms of contrast ratio (CR) defending as $({I_{\max}} - {I_{\min}})/({I_{\max}} + {I_{\min}})$, where Imax and Imin are the maximum and minimum intensity of the recorded image by the camera. CR as the function of applied voltage V1 of “LC lens 2” at different voltage pair (V1, V2) of “LC lens 1” is shown in Fig. 3(c). In Fig. 3(c), the peak of each curve (i.e. maximum contrast ratio) represents the clear image captured by the camera as well as the virtual image is located at z = 300 cm. From Fig. 3(c) and Fig. 2(c), we recorded lens powers of both LC lenses for the maximum and then plotted the lens power of “LC lens 2” (${\bar{P}_{LC2}}$) as the function of lens power of “LC lens 1” (${\bar{P}_{LC1}}$) in green circles in Fig. 3(d). Similarly, we plotted ${\bar{P}_{LC2}}$ v.s. ${\bar{P}_{LC1}}$ in Fig. 3(d) for z = 50 cm, and 100 cm. The lens power of the “LC lens 2” decreases when the lens power of the “LC lens 1” increases. To obtain clear virtual image, both of LC lenses need to have proper lens powers. In addition, when the range of ${\bar{P}_{LC1}}$ changes from −14.10D to +12.86D, the required total tunable range of ${\bar{P}_{LC2}}$ is around ∼1.30D for three different image locations. According to Eq. (3), we also plotted the relationship between ${\bar{P}_{LC1}}$ and ${\bar{P}_{LC2}}$ (dotted lines in Fig. 3(d)). The experimental results agree well with calculations. This also means the need of a larger ${\bar{P}_{LC1}}$ for AR system with an optical zoom function. The magnification (M) is calculated by comparing the sizes of the projected images with the displayed images on LCoS panel. Then, we plotted M as the function of ${\bar{P}_{LC1}}$, as shown in Fig. 3(e). In Fig. 3(e), the magnification of the projected virtual image increases from 85.7 to 104.2 when ${\bar{P}_{LC1}}$ increases from −14.1 D to +12.9 D for z = 300 cm. Larger virtual image distance z, larger M. According to Eq. (5), we also plotted calculated M v.s. ${\bar{P}_{LC1}}$ in Fig. 3(e). The experimental results and calculations exhibit similar trends. When the virtual image is projected from 50 cm to 300 cm, the magnifications are: 14.0∼16.6 (ZR∼1.2) for z = 50 cm, 28.7∼34.4 (ZR∼1.2) for z = 100 cm, 85.7∼104.2 (ZR∼1.2) for z = 300 cm, where theoretical ZR is ∼1.2 according to Eq. (19). ZR could be enlarged when tunable range of ${\bar{P}_{LC1}}$ is enlarged. To achieve ZR = 2 with the same p1 and d, the ${\bar{P}_{LC1}}$ should range from −50 D to +50 D (or from 0 D to + 80 D). According to Eq. (20), the lens power of ± 50D can be achieved when thickness of LC layer (Δn = 0.294) is up to 140 µm for an aperture size of 2.5 mm. To align the LC molecules well, we can separate the LC layer into several sub-layers as a multilayered structure [18]. Alternatively, we can also reduce the aperture size of an LC lens into 1.5 mm when thickness of LC layer is 50 µm. The response time of the proposed LC lenses are around a few seconds. To reduce the response time, ones can adopt the overdrive method and two-mode-switching method for reducing response time (∼500 ms) [21]. The reduction of the thickness of LC layer and the development of fast-response-time LC lenses [10, 2223], such as ferroelectric LC lenses with response time of 0.171 ms, could also improve the response time [24].

 figure: Fig. 3.

Fig. 3. (a) The virtual image recorded by the camera when the “LC lens 1” was applied voltage pair of (60Vrms, 10Vrms) at Freq = 6 kHz and the “LC lens 2” was applied voltage pair of (20Vrms, 75Vrms) at Freq = 2 kHz. (b) The virtual image recorded by the camera when the “LC lens 1” was applied voltage pair of (60Vrms, 10Vrms) at Freq = 6 kHz and the “LC lens 2” was applied voltage pair of (75Vrms, 75Vrms) at Freq = 2 kHz. In (a) and (b), q2=300 cm. (c) Contrast ratio of the virtual image as the function of applied voltage V1 of “LC lens 2” at different voltage pairs of “LC lens 1” when z = 300 cm. (d) ${\bar{P}_{LC2}}$ as the function of ${\bar{P}_{LC1}}$ at z = 50 cm, 100 cm, and 300 cm, respectively. (e) The magnification of the virtual images as the function of the lens power of “LC lens 1” at q2 = 50 cm, 100 cm, and 300 cm, respectively. “sim” and “exp” in (d) and (e) stand for theoretical calculation and experimental results, respectively.

Download Full Size | PDF

Based on the experimental results, we demonstrated the system with three real objects, the tall building, the medium tall building and short building, located at 300 cm, 100 cm, 50 cm away from the AR system. In the beginning, the camera was setting to see the real object (the tall building) at z = 300 cm clearly. Thereafter, the LCoS display turned on and we input signal with a text of “Taiwan 101 300 cm”. When the “LC lens 1” was off and ${\bar{P}_{LC2}}$ was set as −1.12 D with voltage pair of (31Vrms, 75Vrms) at Freq = 2kHz, both of the virtual image (text of “Taiwan 101 300 cm”) and the real object(tall building) were clearly displayed on camera, as shown in Fig. 4(a). This means both of the virtual image and the real object were located at 300 cm. This is so-called registration. In Fig. 4(a), only the image of the object located at 300 cm is clear; otherwise, objects, such as the medium tall building and short building, are blurred. Next, we added lens power of −14.10 D of “LC lens 1” by applying the voltage pair (0Vrms, 50Vrms) at Freq = 0.2 kHz while the applied voltage of “LC lens 2” was (41Vrms, 75Vrms) at Freq = 2 kHz for ${\bar{P}_{LC2}}$ = -0.56D. The virtual image in Fig. 4(b) is smaller compared to the virtual image in Fig. 4(a). On the contrary, the virtual image (see Fig. 4(c)) is magnified compared to the virtual image in Figs. 4(a) and 4(b) when ${\bar{P}_{LC1}}$= +12.86D with a voltage pair of (60Vrms, 10Vrms) at Freq = 6kHz and the corresponding ${\bar{P}_{LC2}}$= -1.89D with (20Vrms, 75Vrms) at Freq = 2kHz. From Figs. 4(b) and (c), the ZR is around 1.2. By manipulating lens powers of two LC lenses, the proposed AR system indeed exhibits the optical zoom function. Similarly, we adjust the camera in order to see the real object located at 100 cm and then test the optical zoom function of the virtual image by manipulating lens powers of two lenses, as shown in Figs. 4(d)–4(f). In Fig. 4, the image is slightly blurred. The image blur results from low resolution of LCoS. The aberrations from the design of optical system and the LC lenses also affect image quality. In our optical system, a spherical concave mirror is adopted, and the spherical aberration arises because of the deviation from Abbe sine condition [25]. In addition, the LC lenses have spherical aberration at different operating conditions. Aside from high resolution LCoS displays, we could adjust the wavefronts of the liquid crystal lenses electrically with less aberration or add a certain aberration to compensate the aberration of the optical system.

 figure: Fig. 4.

Fig. 4. Images of proposed AR system with optical zoom function. (a) The camera is set to see the object at 300 cm (tall building). The projected virtual image is at 300 cm when ${\bar{P}_{LC1}}$ = 0D and ${\bar{P}_{LC2}}$= −1.12D. (b) The image as ${\bar{P}_{LC1}}$ = −14.10D and ${\bar{P}_{LC2}}$ = −0.56D. (c) The image as ${\bar{P}_{LC1}}$ = +12.86D and ${\bar{P}_{LC2}}$ = −1.89D. (d) The camera is set to see the object at 100 cm (medium tall building). The projected virtual image is at 100 cm when ${\bar{P}_{LC1}}$ = 0.08D and ${\bar{P}_{LC2}}$ = −1.46D. (e) From (d), the image as ${\bar{P}_{LC1}}$ = −14.10D and ${\bar{P}_{LC2}}$ = −0.88D. (f) From (e), the image as ${\bar{P}_{LC1}}$ = +12.86D and ${\bar{P}_{LC2}}$ = −2.11D.

Download Full Size | PDF

4. Conclusion

An optical see-through AR system with the function of tunable optical zoom and image registration is demonstrated exploiting two electrically tunable liquid crystal (LC) lenses. The zoom ratio in this paper is around 1.2 for proof-of-concept, and it can be enlarged by increasing the range of tunable lens power of LC lenses. In practical, the polarized LC lenses are enough for the current AR system since the display is either LCoS display or OLED display. If the display is replaced by polarization-independent display (e.g., micro-LED panel) in the future, we can adopt the polarizer-free LC lenses [10, 18]. Because of limitation of LC materials (e.g., birefringence < 0.8) [17], LC lens still have challenges on the range of tunable lens power, aperture size, and response time [10]. The LC lenses for the AR system need to be tailored to the needs depending on the optical designs of AR systems. The proposed AR system here enables the magnification of the projected images and then easy to read the text of the virtual image. The concept in this paper can be further extended to use different tunable lens, such as liquid lenses. The impact of this study is to provide a solution of AR systems with both of optical zoom function and image registration to help myopic people easily see the projected virtual images coincides with environment objects.

Funding

Ministry of Science and Technology, Taiwan (MOST) (107-2112-M-009-019-MY3); Google (Google Daydream University Research Award).

Acknowledgments

The authors are indebted to Mr. Huai-An Hsieh and Dr. Kuan-Hsu Fan-Chiang (Himax Display Inc.) for technical assistance, Himax Display Inc. for the LCoS panel, as well as Prof. Liang Xiao in Tsinghua University China) for providing LC materials.

References

1. D. Phillips and O. L. Siu, Global Aging and Aging Workers in The Oxford Handbook of Work and Aging (Oxford University, 2012).

2. M. Miodownik, “Materials for the 21st century: What will we dream up next?” MRS Bull. 40(12), 1188–1197 (2015). [CrossRef]  

3. J. P. Rolland and H. Hua, “Head-mounted displays,” in Encyclopedia of Optical Engineering, R.B. Johnson and R.G. Driggers, eds., Taylor and Francis (2005).

4. O. Cakmakci and J. P. Rolland, “Head-worn displays: a review,” J. Disp. Technol. 2(3), 199–216 (2006). [CrossRef]  

5. J. P. Rolland and O. Cakmakci, “Head-worn displays: the future through new eyes,” Opt. Photonics News 20(4), 20–27 (2009). [CrossRef]  

6. B. Kress and T. Starner, “A review of head-mounted displays (HMD) technologies and applications for consumer electronics,” Proc. SPIE 8720, 87200A (2013). [CrossRef]  

7. R. Palmarini, J. A. Erkoyuncu, R. Roy, and H. Torabmostaedi, “A systematic review of augmented reality applications in maintenance,” Robot. Comput.-Integr. Manuf. 49, 215–228 (2018). [CrossRef]  

8. S. K. Feiner, “Augmented Reality: A New Way of Seeing,” Sci. Am. 286(4), 48–55 (2002). [CrossRef]  

9. H. C. Lin, H. S. Chen, and Y. H. Lin, “A review of electrically tunable focusing liquid crystal lenses,” Trans. Electr. Electron. Mater. 12(6), 234–240 (2011). [CrossRef]  

10. Y. H. Lin, Y. J. Wang, and V. Reshetnyak, “Liquid crystal lenses with tunable focal length,” Liq. Cryst. Rev. 5(2), 111–143 (2017). [CrossRef]  

11. S. Sato, A. Sugiyama, and R. Sato, “Variable-Focus Liquid-Crystal Fresnel Lens,” Jpn. J. Appl. Phys. 24(Part 2), L626–L628 (1985). [CrossRef]  

12. G. Q. Li, P. Valley, M. S. Giridhar, et al., “Large-aperture switchable thin diffractive lens with interleaved electrode patterns,” Appl. Phys. Lett. 89(14), 141120 (2006). [CrossRef]  

13. L. Li, D. Bryant, and P. J. Bos, “Liquid crystal lens with concentric electrodes and inter-electrode resistors,” Liq. Cryst. Rev. 2(2), 130–154 (2014). [CrossRef]  

14. H. S. Chen, Y. J. Wang, P. J. Chen, and Y. H. Lin, “Electrically adjustable location of a projected image in augmented reality via a liquid crystal lens,” Opt. Express 23(22), 28154–28162 (2015). [CrossRef]  

15. Y. J. Wang, P. J. Chen, X. Liang, and Y. H. Lin, “Augmented reality with image registration, vision, correction and sunlight readability via liquid crystal devices,” Sci. Rep. 7(1), 433 (2017). [CrossRef]  

16. Y. H. Lin and Y. J. Wang, “Liquid crystal lenses in augmented reality,” SID Symp,” Dig. Tech. Pap. - Soc. Inf. Disp. Int. Symp. 48(1), 230–233 (2017). [CrossRef]  

17. Y. Arakawa, S. Kang, H. Tsuji, et al., “The design of liquid crystalline bisolane-based materials with extremely high birefringence,” RSC Adv. 6(95), 92845–92851 (2016). [CrossRef]  

18. H. S. Chen, Y. J. Wang, C. M. Chang, and Y. H. Lin, “A polarizer-free liquid crystal lens exploiting an embedded-multilayered structure,” IEEE Photonics Technol. Lett. 27(8), 899–902 (2015). [CrossRef]  

19. Y. H. Lin, M. S. Chen, and H. C. Lin, “An electrically tunable optical zoom system using two composite liquid crystal lenses with a large zoom ratio,” Opt. Express 19(5), 4714–4721 (2011). [CrossRef]  

20. H. C. Lin, N. Collings, M. S. Chen, et al., “A holographic projection system with an electrically tuning and continuously adjustable optical zoom,” Opt. Express 20(25), 27222–27229 (2012). [CrossRef]  

21. H. C. Lin and Y. H. Lin, “A fast response and large electrically tunable-focusing imaging system based on switching of two modes of a liquid crystal lens,” Appl. Phys. Lett. 97(6), 063505 (2010). [CrossRef]  

22. W. Choi, D. W. Kim, and S. D. Lee, “Liquid Crystal Lens Array with High Fill-Factor Fabricated by an Imprinting Technique,” Mol. Cryst. Liq. Cryst. 504(1), 35–43 (2009). [CrossRef]  

23. S. Stankovic, D. Dias, M. Hain, et al., “Fast Switching Liquid Crystal Lenses for a Dual Focus Digital Versatile Disc Pickup,” Appl. Opt. 40(5), 614–621 (2001). [CrossRef]  

24. H. S. Chen, Y. H. Lin, A. K. Srivastava, et al., “A large bistable negative lens by integrating a polarization switch with a passively anisotropic focusing element,” Opt. Express 22(11), 13138–13145 (2014). [CrossRef]  

25. M. Katz and M. Kata, Introduction to Geometrical Optics (World Scientific, 2002).

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1.
Fig. 1. Operating principle of the AR system with optical zoom function and image registration. (a) Initially, eye sees the distant real object and the virtual image “zoom function” when “LC lens 1” and “LC lens 2” are off. (b) When “LC lens 2” is on and the virtual image is registered with the near real object, for eye, both the virtual image and the real object are blurred due to presbyopia. (c) The project virtual is magnified when “LC lens 1” and “LC lens 2” are operated properly together. Therefore, the magnified virtual image in the retinal becomes clearer.
Fig. 2.
Fig. 2. (a) Structure of “LC lens 1”. (b) Structure of “LC lens 2”. (c) The measured lens power as the function of an applied voltage pair of (V1, V2). The “LC lens 1” functions as a positive lens at (60Vrms, V) at frequency (Freq) of 6 kHz (red circles) and a negative lens at (V, 50Vrms) at Freq = 0.2 kHz (red triangles). The “LC lens 2” functions as a positive lens at (70Vrms, V) at Freq = 7kHz (blue squares) and a negative lens at (V, 75Vrms) at Freq = 2 kHz (blue diamonds). The effective aperture size of “LC lens 2” was 4 mm.
Fig. 3.
Fig. 3. (a) The virtual image recorded by the camera when the “LC lens 1” was applied voltage pair of (60Vrms, 10Vrms) at Freq = 6 kHz and the “LC lens 2” was applied voltage pair of (20Vrms, 75Vrms) at Freq = 2 kHz. (b) The virtual image recorded by the camera when the “LC lens 1” was applied voltage pair of (60Vrms, 10Vrms) at Freq = 6 kHz and the “LC lens 2” was applied voltage pair of (75Vrms, 75Vrms) at Freq = 2 kHz. In (a) and (b), q2=300 cm. (c) Contrast ratio of the virtual image as the function of applied voltage V1 of “LC lens 2” at different voltage pairs of “LC lens 1” when z = 300 cm. (d) ${\bar{P}_{LC2}}$ as the function of ${\bar{P}_{LC1}}$ at z = 50 cm, 100 cm, and 300 cm, respectively. (e) The magnification of the virtual images as the function of the lens power of “LC lens 1” at q2 = 50 cm, 100 cm, and 300 cm, respectively. “sim” and “exp” in (d) and (e) stand for theoretical calculation and experimental results, respectively.
Fig. 4.
Fig. 4. Images of proposed AR system with optical zoom function. (a) The camera is set to see the object at 300 cm (tall building). The projected virtual image is at 300 cm when ${\bar{P}_{LC1}}$ = 0D and ${\bar{P}_{LC2}}$= −1.12D. (b) The image as ${\bar{P}_{LC1}}$ = −14.10D and ${\bar{P}_{LC2}}$ = −0.56D. (c) The image as ${\bar{P}_{LC1}}$ = +12.86D and ${\bar{P}_{LC2}}$ = −1.89D. (d) The camera is set to see the object at 100 cm (medium tall building). The projected virtual image is at 100 cm when ${\bar{P}_{LC1}}$ = 0.08D and ${\bar{P}_{LC2}}$ = −1.46D. (e) From (d), the image as ${\bar{P}_{LC1}}$ = −14.10D and ${\bar{P}_{LC2}}$ = −0.88D. (f) From (e), the image as ${\bar{P}_{LC1}}$ = +12.86D and ${\bar{P}_{LC2}}$ = −2.11D.

Equations (20)

Equations on this page are rendered with MathJax. Learn more.

1 p 1 + 1 q 1 = 1 f 1 ( V ) = P ¯ 1 ( V )
1 p 2 + 1 q 2 = 1 f 2 ( V ) = P ¯ 2 ( V )
P ¯ 2 = P ¯ 1 p 1 ( d + q 2 ) ( p 1 + d + q 2 ) q 2 [ P ¯ 1 p 1 d ( p 1 + d ) ]
M = q 1 p 1 q 2 p 2
M ( P ¯ 1 ) = q 2 P ¯ 1 p 1 d d p 1
M ( P ¯ 2 ) = P ¯ 2 d q 2 d q 2 p 1
q 2 + d + p 1 d p 1 < P ¯ 1 < d + p 1 d p 1 .
P ¯ 2 < p 1 + d + q 2 d q 2 .
M A = q 2 P ¯ L C 1 , min p 1 d d p 1
M B = q 2 P ¯ L C 1 , max p 1 d d p 1
M C = ( 2 P ¯ L C 2 , max + P ¯ m i r r o r ) d q 2 d q 2 p 1
M D = ( 2 P ¯ L C 2 , min + P ¯ m i r r o r ) d q 2 d q 2 p 1
M A = q 2 Δ P ¯ p 1 d d p 1
M B = q 2 Δ P ¯ p 1 d d p 1
M C = ( 2 Δ P ¯ + P ¯ m i r r o r ) d q 2 d q 2 p 1
M D = ( 2 Δ P ¯ + P ¯ m i r r o r ) d q 2 d q 2 p 1
M max = q 2 P 1 , max p 1 d d p 1
M min = q 2 P 1 , min p 1 d d p 1
Z R = M max M min = P L C 1 , min p 1 d d p 1 P L C 1 , max p 1 d d p 1
P ¯ L C ( V ) = 2 δ n ( V ) t r 2
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.