Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Design of foveated contact lens display for augmented reality

Open Access Open Access

Abstract

We present a design of a contact lens display, which features an array of collimated light-emitting diodes and a contact lens, for the augmented reality. By building the infrastructure directly on top of the eye, eye is allowed to move or rotate freely without the need of exit pupil expansion nor eye tracking. The resolution of light-emitting diodes is foveated to match with the density of cones on the retina. In this manner, the total number of pixels as well as the latency of image processing can be significantly reduced. Based on the simulation, the device performance is quantitatively analyzed. For the real image, modulation transfer functions is 0.669757 at 30 cycle/degree, contrast ratio is 5, and distortion is 10%. For the virtual image, the field of view is 82°, best angular resolution is 0.38′, modulation transfer function is above 0.999999 at 30 cycle/degree, contrast ratio is 4988, and distortion is 6%.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Why is making a good near-eye display (NED) for augmented reality (AR) so hard? A blog recently posted on LinkedIn by Daniel Wagner [1], chief technology officer of DAQRI, truly strikes a chord among the AR community. To digest this blog, there are a lot to take in. Yet we can follow a Liebig’s barrel method [2] to sift out the most decisive factors that hold back the current NEDs from being good. According to the blog, there are two types of NEDs, i.e. the free-space-combiner-based and waveguide-based. For free-space-combiner-based NEDs [312], the trade-off between the field of view (FOV) and exit pupil is identified as its shortest stave. To have a large FOV, the exit pupil will become unacceptably small. To have a big exit pupil, either the FOV needs to be decreased or the size of the device will be increased to be even uncomfortable to wear. For waveguide-based NEDs [1322], the good news is FOV is no longer coupled with exit pupil as the latter can be expanded via a number of techniques. The bad news is FOV will be limited by the condition of total internal reflection. For most consumers, a small FOV is definitely a deal breaker. Although not mentioned in the blog, it is necessary to be aware of another type of NEDs [2329], which exploit the direct retinal projection without the need of a combiner or waveguide to form an intermediate virtual image. The minimal design aside, such displays are often associated with a fairly large FOV of more than 100°. That being said, they are not very successful in making real products. For example, virtual retinal display [23]―also known as retinal scanning display―requires that all beams be converged to the center of lens of eye so that the image would be unaffected no matter how the eye accommodates its diopter. Unfortunately, this condition can be only met if the eye does not move or rotate. Pinlight display [26]―a mimic of the pinhole camera―allows for certain eye motion. But its image is subject to diopter of eye, which is not an invariant. From the above, it can be seen that each type of NEDs has its short staves. To patch up the stave, or to build a new barrel is a question.

Customarily, NEDs are mounted on a helmet or eyewear. In this work, we attempt to shift this paradigm by building the infrastructure directly on top of the eye. Since the jargon NED is not applicable or at least inaccurate for our case, we shall re-name this type of display as contact lens display (CLD). Hereinafter, the principle as well as the potential of CLD is to be unfolded in full.

2. Principles

2.1 Proposed structure

The proposed CLD consists of two components, i.e. a contact lens and a collimated light-emitting diode (LED) array, as shown in Fig. 1. For the sake of symmetry, the contact lens, LED array and eye are center-aligned. Adjacent to the cornea is a thin layer of contact lens for fixing the refractive errors. On top of the contact lens is an array of LEDs, each pixel of which is able to emit a collimated beam of light towards the center of lens of eye. In this regard, our CLD is analogous to the said virtual retinal display or retinal scanning display. The major difference is that in our CLD, eye can rotate freely without losing the image, thanks to the surface tension [30] that tightly adheres the whole device to the eye.

 figure: Fig. 1.

Fig. 1. Cross-section view of the proposed contact lens display. Adjacent to the cornea is a thin layer of contact lens for fixing the refractive errors. On top of the contact lens is fabricated an array of LEDs, each pixel of which is able to emit a collimated beam of light towards the center of lens of eye. For the sake of symmetry, the contact lens, LED array and eye are center-aligned.

Download Full Size | PDF

2.2 Photoreceptor cells

Photoreceptor cells are neuroepithelial cells found in the retina that are capable of converting the photons into biological signals [31]. There are three known types of photoreceptor cells, i.e. rods, cones, and photosensitive ganglions. Rods are extremely sensitive to the brightness, and can be triggered by even a single photon. Cones are less brightness sensitive, but can discern the colors. Photosensitive ganglions are responsible for the circadian rhythm and pupil control. The human retina contains about 120 million rods, 6 million cones, 24 to 60 thousand photosensitive ganglions. Number-wise, we are particularly interested in the number of cones as it is closely linked to the visual acuity, the ability to resolve the spatial details [31]. By retrieving the data from Curcio et al.’s study on the photoreceptor cells [32], the density of cones is plotted with respect to the eccentricity―a rotation angle ɛ about the axis connecting the center of fovea and the center of eye (see Fig. 2)―as shown in Fig. 3. The positive sign of eccentricity signifies the temporal side, while the negative sign the nasal side. It can be seen that cones are vastly concentrated at the fovea, whose eccentricity is up to 3.3° [33]. Between −13.6° to −21.6° at the nasal side is a photoreceptor-free region called blind spot [34].

 figure: Fig. 2.

Fig. 2. Illustration of eccentricity. The eccentricity is a rotation angle ɛ about the axis connecting the center of fovea and the center of eye.

Download Full Size | PDF

 figure: Fig. 3.

Fig. 3. Density of cones versus the eccentricity [32]. The positive sign of eccentricity signifies the temporal side, while the negative sign the nasal side. It can be seen that cones are vastly concentrated at the fovea, whose eccentricity is up to 3.3°. Between −13.6° to −21.6° at the nasal side is a photoreceptor-free region called blind spot.

Download Full Size | PDF

2.3 Visual acuity

Visual acuity (VA) refers to the clarity of vision, which can be quantitatively described as the reciprocal of angular resolution [35], i.e.

$$VA = \frac{1}{{angular\; resolution}}$$
Suppose two LEDs A and B on the cornea are just resolvable and coaxial with two cones on the retina, as shown in Fig. 4. Relative to the center of lens, the angle subtended by A and B is equivalent to the angle α subtended by A′ and B′. With the assumptions that the eye is perfectly healthy having no refractive errors, retinal detachment, macular degeneration etc. and that the beam size of LEDs is small enough compared to the pupil that no diffractions would occur, then angular resolution α will be solely dependent on the density of cones ρ as
$$\alpha = 2{\sin ^{ - 1}}\left[ {\sin \left( {\frac{{\sqrt {1/\rho } }}{{2\sqrt {{{[\textrm{r} \cdot \sin (\varepsilon )]}^2} + {{[\textrm{r} \cdot \cos (\varepsilon )+ {d_{el}}]}^2}} }}} \right)} \right]$$
where r is the radius of eyeball, ɛ the eccentricity, and del the distance from the center of eye to the center of lens. With the above equations and the densities of cones given in Fig. 3, VA can be calculated as a function of eccentricity, as shown in Fig. 5. For a quick look-up, we shall divide the retina into four different regions, including fovea (0° to 3.3°), parafovea (3.3° to 5.5°), macula or perifovea (5.5° to 12.1°), and periphery (12.1° and beyond), and itemize the maximum density of cones, maximum angular resolution, and the best VA for each region, respectively, as in Table 1. Within the fovea, the best VA can be as high as 2.6. For the majority, whose vision is more or less compromised by optical and/or neural factors, the normal VA is 1.0―another version as a fraction is 20/20 vision [35]. Only a small portion, about 1% of the population, can have a VA above 2.0 [36].

 figure: Fig. 4.

Fig. 4. Suppose two LEDs A and B on the cornea are just resolvable and coaxial with two cones A′ and B′ on the retina. Relative to the center of lens, the angle subtended by A and B is equivalent to the angle α subtended by A′ and B′.

Download Full Size | PDF

 figure: Fig. 5.

Fig. 5. Visual acuity (decimal) versus the eccentricity. The visual acuity drops dramatically as the eccentricity rises. Within the fovea, the best visual acuity can be as high as 2.6.

Download Full Size | PDF

Tables Icon

Table 1. Maximum angular resolution of four different regions on the retina

2.4 Contact lens

The design of contact lens should conform to the prescription acquired from an optometrist or ophthalmologist. It is important to be aware that a prescription for contact lens is not the same as a prescription for eyeglasses. This is because the working distance of contact lens is obviously much shorter than that of eyeglasses. Say a user is nearsighted and his/her eyeglasses have a diopter Pg of −3.00 m−1 for both eyes. Then, his/her contact lens shall have a diopter Pc [37], i.e.

$${P_c} = \frac{{{P_g}}}{{1 - {d_g} \cdot {P_g}}}$$
where dg is the distance between the eyeglasses and eye. When dg = 12 mm, Pc= −2.896 m−1. Treating the contact lens as a thin lens [38], we have
$${P_c} = ({n - 1} )\left( {\frac{1}{{{R_1}}} - \frac{1}{{{R_2}}}} \right)$$
where n is refractive index of contact lens, R1 the radius of curvature of front surface, and R2 the radius of curvature of back surface or anterior cornea, as shown in Fig. 6. In order to be compatible as an encapsulation layer for the LED array [39], contact lens is supposed to be air impermeable. To meet this purpose, polymethyl methacrylate (PMMA)―the raw material for the old-fashioned hard contact lens [40]―is selected as the lens material. Besides PMMA, other plastic polymers, e.g. polysiloxane (silicone) and polyethylene terephthalate (PET), might work as well [41]. Per the above rules, a contact lens can be tentatively designed with the parameters listed in Table 2, which also includes a couple of physical dimensions, such as the thickness tc, overall diameter d, optical zone diameter do, and radius of curvature of edge Re.

 figure: Fig. 6.

Fig. 6. Geometry of contact lens. R1 is the radius of curvature of front surface, R2 the radius of curvature of back surface, Re the radius of curvature of edge, tc the thickness, d the overall diameter, and do the optical zone diameter.

Download Full Size | PDF

Tables Icon

Table 2. Parameters of contact lens

2.5 Collimated light-emitting diode

Figure 7 is a schematic of a collimated LED―an LED in tandem with a collimator―being sandwiched between the substrate and contact lens. For the reason that the position of LED is beyond the near point of eye, which is normally 25 cm [37], it is required that the etendue of LED be adequately small so that a clear image would be formed on the retina. To satisfy this requirement, a collimator is inserted at the end of LED to narrow down the etendue. Rather, this collimator is an optical fiber with a high-refractive-index core in the middle surrounded by a low-refractive-index cladding. In the most extreme case―namely the so-called single-mode fiber―the light will be collimated into a straight line. This occurs when

$${D_c}\;<\;\frac{{2.4\lambda }}{{\pi \sqrt {n_1^2 - n_2^2} }}$$
where Dc is the diameter of core, λ the wavelength, n1 the refractive index of core, and n2 the refractive index of cladding [38]. When λ = 532 nm, n1 = 1.468, and n2 = 1.460, Dc < 2.655 µm. Considering that the diameter of cone is approximately 2.7 µm, very close to the above value, a one-to-one correspondence between the LEDs and cones could be fulfilled. In fact, other than the single-mode, multimode collimator would work as well, as long as the etendue is not too big.

 figure: Fig. 7.

Fig. 7. Schematic of a collimated LED―an LED in tandem with a collimator―being sandwiched between the substrate and contact lens. The collimator is basically an optical fiber with a high-refractive-index core in the middle surrounded by a low-refractive-index cladding.

Download Full Size | PDF

2.6 Field angle versus eccentricity

Picture a ray (yellow line) is incident to the contact lens at a field angle θ, then refracted towards the center of lens at an angle β, and finally hits the retina at an eccentricity ɛ, as shown in Fig. 8. We could derive

$$\theta = \beta + {\sin ^{ - 1}}\frac{{n \cdot {d_{cl}}\sin \beta }}{{{R_1}}} - {\sin ^{ - 1}}\frac{{{d_{cl}}\sin \beta }}{{{R_1}}}$$
where
$$\beta ={-} {\sin ^{ - 1}}\frac{{{{(\textrm{r} \cdot \sin \varepsilon )}^2}}}{{\sqrt {{{(\textrm{r} \cdot \sin \varepsilon )}^2} + {{(\textrm{r} \cdot \cos \varepsilon + {d_{el}})}^2}} }}$$
in which dcl is the distance from the vertex of contact lens to the center of lens. For the human eye, its eccentricity ɛ varies from −96° (nasal) to + 80° (temporal) [32]. The corresponding range of field angle is therefore −66° (temporal) to + 81° (nasal). To avoid confusion about the plus/minus sign, it shall be noted that, due to the mirror imaging of lens, the orientation of field angle θ is opposite to that of eccentricity ɛ.

 figure: Fig. 8.

Fig. 8. Picture a ray (yellow line) is incident to the contact lens at a field angle θ, then refracted towards the center of lens at an angle β, and finally hits the retina at an eccentricity ɛ. del is the distance from the center of lens to center of eye, dcl the distance from contact lens to center of lens, r the radius of eye, and R1 the radius of curvature of front surface of lens.

Download Full Size | PDF

2.7 Pupil size

Pupil, a black hole in the center of iris, acts as an aperture stop to regulate the amount of light reaching the retina [42]. As is known, the size or diameter of pupil―more exactly and by default, the size of entrance pupil, which is the image of pupil formed by the cornea―is tunable in response to the ambient brightness. Plus, it will be also affected by the age. To take into account both the brightness and age, we shall modify a formula [43] to determine the pupil size D, given by

$$D = \frac{{ - 0.11Y + 14.64}}{{{{\left( {\frac{{L\pi FO{V^2}}}{{4 \times 846}}} \right)}^{0.41}} + 2}} + 0.0022Y + 1.937$$
where Y is the age, and L the luminance. Say the user is 24 years old and FOV = 82°. Then, we could calculate the pupil size against the luminance, as shown in Fig. 9, from which it can be seen that the pupil size ranges from 2 to 8 mm. When L = 500 cd/m2, D = 2.40 mm. Incidentally, different levels of luminance will trigger different modes of vision. When L > 5 cd/m2, D < 3.96 mm and photopic vision takes effect, in which cones dominate [44]. When 5 cd/m2 > L > 0.005 cd/m2, 3.96 mm < D < 7.35 mm and mesopic vision takes effect, in which both cones and rods are active. When L < 0.005 cd/m2, D > 7.35 mm and scotopic vision takes effect, in which only rods are active.

 figure: Fig. 9.

Fig. 9. Pupil size versus the luminance. Pupil diameter ranges from 2 to 8 mm. When L = 500 cd/m2, D = 2.40 mm. Incidentally, different levels of luminance will trigger different modes of vision. When L > 5 cd/m2, D < 3.96 mm and photopic vision takes effect, in which cones dominate. When 5 cd/m2 > L > 0.005 cd/m2, 3.96 mm < D < 7.35 mm and mesopic vision takes effect, in which both cones and rods are active. When L < 0.005 cd/m2, D > 7.35 mm and scotopic vision takes effect, in which only rods are active.

Download Full Size | PDF

2.8 Foveated LED array

The concept of foveated imaging originates from the computer graphics, which is intended to speed up the rendering of high-resolution images [45]. However, the physical resolution of the image remains intact, and an additional device to track the eye is needed [46,47]. Alternatively, this concept can be realized by re-arranging the pixels to match with the distribution of cones. Under the presumption that, upon the arrival at the retina, the collimated beam of each LED is received by one single cone only, it is legitimate to equate the number of LEDs to that of cones. Let the area of LEDs be decomposed into M rings and the width of each ring be equal to that of LED. For the ith ring, the number of LEDs Ni it contains can be roughly estimated by

$${N_i} = \frac{{\pi \textrm{sin}\beta }}{{\textrm{sin}\left( {\frac{{{\alpha_i}}}{2}} \right)}}$$
where αi is the angular resolution on the ith ring. Hence, the total number of LEDs N is
$$N = \mathop \sum \limits_{i = 1}^M {N_i}$$
Substituting Eq. (6) into Eq. (9), the number of LEDs on each ring can be calculated as a function of field angle, as shown in Fig. 10. Interestingly, the blind spot will leave a blank area not covered by LEDs on the contact lens. It not just increases the transparency of the device, but saves quite a few pixels. Size-wise, blind spot is 1.76 mm (horizontal) by 1.92 mm (vertical)―or, in terms of field angle, 6° (horizontal) and 7° (vertical). According to Eq. (10), the number of pixels saved by blind spot is 20,000, give or take. To better appreciate the benefit of foveated pixel arrangement, the minimal number of pixels―at an aspect ratio of 16:9―required to yield an angular resolution of 1′ is computed with respect to the diagonal FOVs, as shown in Fig. 11. Take a FOV of 100° as an instance. The minimal numbers of pixels for the non-foveated and foveated displays are 15.38 and 3.20 million, respectively. The latter is merely about 1/5 of the former.

 figure: Fig. 10.

Fig. 10. Number of LEDs versus the field angle. Interestingly, the blind spot will leave a blank area not covered by LEDs on the contact lens.

Download Full Size | PDF

 figure: Fig. 11.

Fig. 11. Number of pixels―at an aspect ratio of 16:9―required to yield an angular resolution of 1′ is computed with respect to the diagonal FOVs. Take a FOV of 100° as an instance. The minimal numbers of pixels for the non-foveated and foveated displays are 15.38 and 3.20 million, respectively. The latter is merely about 1/5 of the former.

Download Full Size | PDF

3. Results and discussion

3.1 Simulation setting

The whole idea of CLD is validated through the numerical simulation on Code V (Synopsys). The design wavelength is 532 nm. Figure 12 outlines the optical surfaces defined in Code V, which are in turn (1) contact lens, (2) anterior cornea, (3) posterior cornea, (4) pupil, (5) anterior lens, (6) virtual plane, (7) posterior lens, and (8) retina. The initial parameters of eye are adapted from our previous eye model [48], in which the lens has gradient refractive indices and it is split into anterior and posterior lenses with a virtual plane being inserted in between. The total length of eye is 24 mm. For the real image, the object is positioned at 3 m ahead of the eye. For the virtual image, the object coincides with the contact lens. During the optimization, the radii of anterior and posterior lenses are set as the variables. Table 3 summarizes the as-optimized parameters for each surface. For more details, parameters for aspherical surfaces and gradient refractive indices of lens are disclosed in Table 4 and Table 5, respectively.

 figure: Fig. 12.

Fig. 12. Optical surfaces defined in Code V, which are in turn (1) contact lens, (2) anterior cornea, (3) posterior cornea, (4) pupil, (5) anterior lens, (6) virtual plane, (7) posterior lens, and (8) retina. For the real image, the object is positioned at 3 m ahead of the eye. For the virtual image, the object coincides with the contact lens.

Download Full Size | PDF

Tables Icon

Table 3. Parameters of optical surfaces used in Code V

Tables Icon

Table 4. Parameters for aspherical surfaces

Tables Icon

Table 5. Parameters for gradient refractive indices of anterior and posterior lenses

3.2 Field of view

Referring to Fig. 13, the following geometric relationship among FOV and other parameters can be obtained

$$FOV = 2\left( {{{\tan }^{ - 1}}\frac{D}{{2{d_{pl}}}} + {{\sin }^{ - 1}}\frac{{n \cdot D \cdot {d_{cl}}}}{{{R_1}\sqrt {{D^2} + 4{d_{pl}}^2} }} - {{\sin }^{ - 1}}\frac{{D \cdot {d_{cl}}}}{{{R_1}\sqrt {{D^2} + 4{d_{pl}}^2} }}} \right)$$
where dpl is the distance from the center of the lens to the pupil. Assigning the above parameters with the values provided in Table 6, FOV is calculated as 82°. A careful examination of Eq. (11) implies that FOV is proportional to the pupil size. As can be seen in Fig. 14, when the pupil dilates to 8 mm in diameter, FOV reaches up to 142°. Unlike the rectangular NED, our CLD is round in shape. If both are of the same FOV, the round one will indisputably have a bigger image size.

 figure: Fig. 13.

Fig. 13. Geometric relationship among FOV and other parameters. D is the pupil size, dpl the distance from the center of the lens to the pupil, dcl the distance from contact lens to center of lens, and R1 the radius of curvature of front surface of lens.

Download Full Size | PDF

 figure: Fig. 14.

Fig. 14. FOV versus the pupil size. When the pupil is 2.4 mm in diameter, FOV is 82°. When the pupil dilates to 8 mm in diameter, FOV reaches up to 142°.

Download Full Size | PDF

Tables Icon

Table 6. Parameters for calculating FOV

3.3 Angular resolution

For the resolution of our CLD is foveated, so is its angular resolution. If measured in arcminute (′), the angular resolution for the ith ring of the field angle θ can be calculated with

$$Angular\; resolution = \frac{{21600\sin \theta }}{{{N_i}}}$$
As shown in Fig. 15, the best angular resolution is 0.38′ at 0°, whereas the worst is 3.11′ at −41°.

 figure: Fig. 15.

Fig. 15. Angular resolution versus the field angle. The best angular resolution is 0.38′ at 0°, whereas the worst is 3.11′ at −41°.

Download Full Size | PDF

3.4 Modulation transfer function

Modulation transfer function (MTF) is calculated for both real and virtual images, as shown in Fig. 16. For the real image, MTF of the field of 0° is 0.669757 at 30 cycle/degree. For the virtual image, MTFs of all fields are above 0.999999 at 30 cycle/degree.

 figure: Fig. 16.

Fig. 16. MTFs calculated as a function of spatial frequency in cycle/degree. For the real image, MTF of the field of 0° is 0.669757 at 30 cycle/degree. For the virtual image, MTFs of all fields are above 0.999999 at 30 cycle/degree.

Download Full Size | PDF

3.5 Contrast ratio

Contrast ratio (CR) of image is defined as

$$CR = \frac{{C{R_0} + 1 + MTF \cdot ({C{R_0} - 1} )}}{{C{R_0} + 1 - MTF \cdot ({C{R_0} - 1} )}}$$
where CR0 is the CR of object [49]. For the real image, CR of object can be infinitely large. For the virtual image, CR of object or LED, is set as 5000. For the field of 0° at a spatial frequency of 30 cycle/degree, CRs of real and virtual images are calculated as 5 (MTF = 0.669757) and 4988 (MTF = 0.999999), respectively.

3.6 Distortion

Distortion is defined as

$$\textrm{Distortion} = \frac{{{h_a} - {h_p}}}{{{h_a}}}$$
where ha is the height of actual image, and hp the height of paraxial image calculated with the first-order approximation [49]. As shown in Fig. 17, distortions of real and virtual images are 10% and 6%, respectively.

 figure: Fig. 17.

Fig. 17. Distortion versus the field angle. Distortions of real and virtual images are 10% and 6%, respectively.

Download Full Size | PDF

3.7 Simulated imaging

Figure 18 shows the original image (Snellen chart [50]) alongside the real and virtual images formed on the retina. As expected, the virtual image is sharper and less distorted than the real image especially at the marginal fields, which agrees with the foregoing MTF and distortion. However, the foveated effect of the virtual image is not visible. The reason is that, in Code V and other simulation tools, the influence of photoreceptor cells on the image has yet to be factored into.

 figure: Fig. 18.

Fig. 18. (a) Original image (Snell chart), (b) real image, and (c) virtual image. The foveated effect of the virtual image is not visible. The reason is that, in Code V and other simulation tools, the influence of photoreceptor cells on the image has yet to be factored into.

Download Full Size | PDF

4. Conclusions

As an alternative to the established NEDs, a foveated CLD has been proposed. To justify this concept, the working principles for each component have been explained. To evaluate its performance, numerical simulations have been carried out. For the real image, MTF is 0.669757 at 30 cycle/degree, CR is 5, and distortion is 10%. For the virtual image, FOV is 82°, best angular resolution is 0.38′, MTF is above 0.999999 at 30 cycle/degree, CR is 4988, and distortion is 6%. Compared to the retinal-projection-based NEDs, our CLD has its own pros and cons. First (pro), eye is allowed to move or rotate freely without the help of exit pupil expansion nor eye tracking. This is a big advantage over its counterparts, especially those with small exit pupils [23,26,48,51]. Second (pro), the physical resolution is foveated to match with the distribution of cones. This will significantly reduce the total number of pixels as well as the latency incurred by the image processing. Third (pro), no burden or weight on the head and no worries about head-related human factors, such as the shape of head, interpupillary distance etc. Fourth (con), the fabrication and popularization of CLD will face a bunch of challenges. Among others, the safety of CLD is arguably the number one issue. In addition, the power supply of CLD necessitates either the standalone built-in battery―e.g. glucose biofuel cell―or the wireless charging by means of electromagnetic induction [52]. Sadly, those issues will bring us back to the question raised at the beginning. Why is it so hard to make a good AR display? In Wagner’s closing remarks, he expressed a kind of pessimistic mood towards the possibility of breakthroughs in the short term. Still, we believe if we keep thinking outside the box, a dream solution might be just around the corner.

Funding

National Natural Science Foundation of China (61831015); Science and Technology Commission of Shanghai Municipality (1701H169200, 1801H163000, 19ZR1427200); Shanghai Shadow Creator Inc. (18H100000559).

Disclosures

The authors declare no conflicts of interest.

References

1. D. Wagner, A. Stannard, and L. Noui, “Why is making good AR displays so hard?” https://www.linkedin.com/pulse/why-making-good-ar-displays-so-hard-daniel-wagner/?trackingId=ak3AnPVYQCyTEzZq2F0bwg%3D%3D.

2. Wikipedia, “Liebig’s law of the minimum,” https://en.wikipedia.org/wiki/Liebig%27s_law_of_the_minimum.

3. J. P. Rolland, “Wide-angle, off-axis, see-through head-mounted display,” Opt. Eng. 39(7), 1760–1767 (2000). [CrossRef]  

4. S. Liu, H. Hua, and D. Cheng, “A novel prototype for an optical see-through head-mounted display with addressable focus cues,” IEEE Trans. Visual. Comput. Graphics 16(3), 381–393 (2010). [CrossRef]  

5. Z. Zheng, X. Liu, H. Li, and L. Xu, “Design and fabrication of an off-axis see-through head-mounted display with an x-y polynomial surface,” Appl. Opt. 49(19), 3661–3668 (2010). [CrossRef]  

6. H.-S. Chen, Y.-J. Wang, P.-J. Chen, and Y.-H. Lin, “Electrically adjustable location of a projected image in augmented reality via a liquid-crystal lens,” Opt. Express 23(22), 28154–28162 (2015). [CrossRef]  

7. R. Zhu, G. Tan, J. Yuan, and S.-T. Wu, “Functional reflective polarizer for augmented reality and color vision deficiency,” Opt. Express 24(5), 5431–5441 (2016). [CrossRef]  

8. Q. Gao, J. Liu, X. Duan, T. Zhao, X. Li, and P. Liu, “Compact see-through 3D head-mounted display based on wavefront modulation with holographic grating filter,” Opt. Express 25(7), 8412–8424 (2017). [CrossRef]  

9. L. Zhou, C. P. Chen, Y. Wu, Z. Zhang, K. Wang, B. Yu, and Y. Li, “See-through near-eye displays enabling vision correction,” Opt. Express 25(3), 2130–2142 (2017). [CrossRef]  

10. A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 1–16 (2017). [CrossRef]  

11. D. Dunn, C. Tippets, K. Torell, P. Kellnhofer, K. Aksit, P. Didyk, K. Myszkowski, D. Luebke, and H. Fuchs, “Wide field of view varifocal near-eye display using see-through deformable membrane mirrors,” IEEE Trans. Visual. Comput. Graphics 23(4), 1322–1331 (2017). [CrossRef]  

12. G.-Y. Lee, J.-Y. Hong, S. Hwang, S. Moon, H. Kang, S. Jeon, H. Kim, J.-H. Jeong, and B. Lee, “Metasurface eyepiece for augmented reality,” Nat. Commun. 9(1), 4562 (2018). [CrossRef]  

13. Y. Amitai, “Extremely compact high-performance HMDs based on substrate-guided optical element,” Dig. Tech. Pap. - Soc. Inf. Disp. Int. Symp. 35(1), 310–313 (2004). [CrossRef]  

14. T. Levola, “Diffractive optics for virtual reality displays,” J. Soc. Inf. Disp. 14(5), 467–475 (2006). [CrossRef]  

15. H. Mukawa, K. Akutsu, I. Matsumura, S. Nakano, T. Yoshida, M. Kuwahara, and K. Aiki, “A full-color eyewear display using planar waveguides with reflection volume holograms,” J. Soc. Inf. Disp. 17(3), 185–193 (2009). [CrossRef]  

16. D. Cheng, Y. Wang, H. Hua, and M. M. Talha, “Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism,” Appl. Opt. 48(14), 2655–2668 (2009). [CrossRef]  

17. Y. Weng, D. Xu, Y. Zhang, X. Li, and S.-T. Wu, “Polarization volume grating with high efficiency and large diffraction angle,” Opt. Express 24(16), 17746–17759 (2016). [CrossRef]  

18. J. Yang, P. Twardowski, P. Gerard, and J. Fontaine, “Design of a large field-of-view see-through near to eye display with two geometrical waveguides,” Opt. Lett. 41(23), 5426–5429 (2016). [CrossRef]  

19. Y. Wu, C. P. Chen, L. Zhou, Y. Li, B. Yu, and H. Jin, “Design of see-through near-eye display for presbyopia,” Opt. Express 25(8), 8937–8949 (2017). [CrossRef]  

20. Y. Wu, C. P. Chen, L. Zhou, Y. Li, B. Yu, and H. Jin, “Near-eye display for vision correction with large FOV,” in SID Display Week (2017), pp. 767–770.

21. Z. Shi, W. T. Chen, and F. Capasso, “Wide field-of-view waveguide displays enabled by polarization-dependent metagratings,” Proc. SPIE 10676, 1067615 (2018). [CrossRef]  

22. J. Xiao, J. Liu, J. Han, and Y. Wang, “Design of achromatic surface microstructure for near-eye display with diffractive waveguide,” Opt. Commun. 452, 411–416 (2019). [CrossRef]  

23. T. A. Furness III and J. S. Kollin, “Virtual retinal display,” US Patent 5,467,104 (1992).

24. S. C. McQuaide, E. J. Seibel, J. P. Kelly, B. T. Schowengerdt, and T. A. Furness III, “A retinal scanning display system that produces multiple focal planes with a deformable membrane mirror,” Displays 24(2), 65–72 (2003). [CrossRef]  

25. R. Sprague, A. Zhang, L. Hendricks, T. O’Brien, J. Ford, E. Tremblay, and T. Rutherford, “Novel HMD concepts from the DARPA SCENICC program,” Proc. SPIE 8383, 838302 (2012). [CrossRef]  

26. A. Maimone, D. Lanman, K. Rathinavel, K. Keller, D. Luebke, and H. Fuchs, “Pinlight displays: wide field of view augmented reality eyeglasses using defocused point light sources,” ACM Trans. Graph. 33(4), 1–11 (2014). [CrossRef]  

27. T. North, M. Wagner, S. Bourquin, and L. Kilcher, “Compact and high-brightness helmet-mounted head-up display system by retinal laser projection,” J. Disp. Technol. 12(9), 982–985 (2016). [CrossRef]  

28. C. P. Chen, L. Zhou, J. Ge, Y. Wu, L. Mi, Y. Wu, B. Yu, and Y. Li, “Design of retinal projection displays enabling vision correction,” Opt. Express 25(23), 28223–28235 (2017). [CrossRef]  

29. Y. Wu, C. P. Chen, L. Mi, W. Zhang, J. Zhao, Y. Lu, W. Guo, B. Yu, Y. Li, and N. Maitlo, “Design of retinal-projection-based near-eye display with contact lens,” Opt. Express 26(9), 11553–11567 (2018). [CrossRef]  

30. Wikipedia, “Surface tension,” https://en.wikipedia.org/wiki/Surface_tension.

31. Wikipedia, “Photoreceptor cell,” https://en.wikipedia.org/wiki/Photoreceptor_cell.

32. C. A. Curcio, K. R. Sloan, R. E. Kalina, and A. E. Hendrickson, “Human photoreceptor topography,” J. Comp. Neurol. 292(4), 497–523 (1990). [CrossRef]  

33. Wikipedia, “Macula of retina,” https://en.wikipedia.org/wiki/Macula_of_retina.

34. Wikipedia, “Optic disc,” https://en.wikipedia.org/wiki/Optic_disc.

35. Wikipedia, “Visual acuity,” https://en.wikipedia.org/wiki/Visual_acuity.

36. LaserFit, “20/10 vision explained and achievable,” https://laserfitlens.com/laserfit-scleral-lenses-2010-vision-who-can-benefit.

37. M. Bass, C. DeCusatis, J. Enoch, V. Lakshminarayanan, G. Li, C. MacDonald, V. Mahajan, and E. V. Stryland, Handbook of Optics 3rd Edition Volume III: Vision and Vision Optics (McGraw-Hill Education, 2009).

38. F. L. Pedrotti, L. M. Pedrotti, and L. S. Pedrotti, Introduction to Optics 3rd Edition (Pearson, 2006).

39. G. P. Crawford, Flexible Flat Panel Displays (Wiley, 2007).

40. C. S. A. Musgrave and F. Fang, “Contact lens materials: a materials science perspective,” Materials 12(2), 261 (2019). [CrossRef]  

41. J. De Smet, A. Avci, P. Joshi, D. Schaubroeck, D. Cuypers, and H. De Smet, “Progress toward a liquid crystal contact lens display,” J. Soc. Inf. Disp. 21(9), 399–406 (2013). [CrossRef]  

42. Wikipedia, “Pupil,” https://en.wikipedia.org/wiki/Pupil.

43. A. B. Watson and J. I. Yellott, “A unified formula for light-adapted pupil size,” J. Vision 12(10), 12 (2012). [CrossRef]  

44. W. van Bommel, Road Lighting: Fundamentals, Technology and Application (Springer, 2014), Chap. 6.

45. P. Kortum and W. Geisler, “Implementation of a foveated image coding system for image bandwidth reduction,” Proc. SPIE 2657, 350–360 (1996). [CrossRef]  

46. G. Tan, Y.-H. Lee, T. Zhan, J. Yang, S. Liu, D. Zhao, and S.-T. Wu, “Foveated imaging for near-eye displays,” Opt. Express 26(19), 25076–25085 (2018). [CrossRef]  

47. J. Kim, Y. Jeong, M. Stengel, K. Aksit, R. Albert, B. Boudaoud, T. Greer, J. Kim, W. Lopes, Z. Majercik, P. Shirley, J. Spjut, M. McGuire, and D. Luebke, “Foveated AR: dynamically-foveated augmented reality display,” ACM Trans. Graph. 38(4), 99 (2019). [CrossRef]  

48. L. Mi, C. P. Chen, Y. Lu, W. Zhang, J. Chen, and N. Maitlo, “Design of lensless retinal scanning display with diffractive optical element,” Opt. Express 27(15), 20493–20507 (2019). [CrossRef]  

49. R. E. Fischer, B. Tadic-Galeb, and P. R. Yoder, Optical System Design 2nd Edition (McGraw-Hill Education, 2008).

50. Wikipedia, “Snellen chart,” https://en.wikipedia.org/wiki/Snellen_chart.

51. E. Tremblay, M. Guillaumee, and C. Moser, “Method and apparatus for head worn display with multiple exit pupils,” US Patent 9,846,307 B2 (2017).

52. T. Takamatsu, Y. Chen, T. Yoshimasu, M. Nishizawa, and T. Miyake, “Highly efficient, flexible wireless-powered circuit printed on a moist, soft contact lens,” Adv. Mater. Technol. 4(5), 1800671 (2019). [CrossRef]  

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (18)

Fig. 1.
Fig. 1. Cross-section view of the proposed contact lens display. Adjacent to the cornea is a thin layer of contact lens for fixing the refractive errors. On top of the contact lens is fabricated an array of LEDs, each pixel of which is able to emit a collimated beam of light towards the center of lens of eye. For the sake of symmetry, the contact lens, LED array and eye are center-aligned.
Fig. 2.
Fig. 2. Illustration of eccentricity. The eccentricity is a rotation angle ɛ about the axis connecting the center of fovea and the center of eye.
Fig. 3.
Fig. 3. Density of cones versus the eccentricity [32]. The positive sign of eccentricity signifies the temporal side, while the negative sign the nasal side. It can be seen that cones are vastly concentrated at the fovea, whose eccentricity is up to 3.3°. Between −13.6° to −21.6° at the nasal side is a photoreceptor-free region called blind spot.
Fig. 4.
Fig. 4. Suppose two LEDs A and B on the cornea are just resolvable and coaxial with two cones A′ and B′ on the retina. Relative to the center of lens, the angle subtended by A and B is equivalent to the angle α subtended by A′ and B′.
Fig. 5.
Fig. 5. Visual acuity (decimal) versus the eccentricity. The visual acuity drops dramatically as the eccentricity rises. Within the fovea, the best visual acuity can be as high as 2.6.
Fig. 6.
Fig. 6. Geometry of contact lens. R1 is the radius of curvature of front surface, R2 the radius of curvature of back surface, Re the radius of curvature of edge, tc the thickness, d the overall diameter, and do the optical zone diameter.
Fig. 7.
Fig. 7. Schematic of a collimated LED―an LED in tandem with a collimator―being sandwiched between the substrate and contact lens. The collimator is basically an optical fiber with a high-refractive-index core in the middle surrounded by a low-refractive-index cladding.
Fig. 8.
Fig. 8. Picture a ray (yellow line) is incident to the contact lens at a field angle θ, then refracted towards the center of lens at an angle β, and finally hits the retina at an eccentricity ɛ. del is the distance from the center of lens to center of eye, dcl the distance from contact lens to center of lens, r the radius of eye, and R1 the radius of curvature of front surface of lens.
Fig. 9.
Fig. 9. Pupil size versus the luminance. Pupil diameter ranges from 2 to 8 mm. When L = 500 cd/m2, D = 2.40 mm. Incidentally, different levels of luminance will trigger different modes of vision. When L > 5 cd/m2, D < 3.96 mm and photopic vision takes effect, in which cones dominate. When 5 cd/m2 > L > 0.005 cd/m2, 3.96 mm < D < 7.35 mm and mesopic vision takes effect, in which both cones and rods are active. When L < 0.005 cd/m2, D > 7.35 mm and scotopic vision takes effect, in which only rods are active.
Fig. 10.
Fig. 10. Number of LEDs versus the field angle. Interestingly, the blind spot will leave a blank area not covered by LEDs on the contact lens.
Fig. 11.
Fig. 11. Number of pixels―at an aspect ratio of 16:9―required to yield an angular resolution of 1′ is computed with respect to the diagonal FOVs. Take a FOV of 100° as an instance. The minimal numbers of pixels for the non-foveated and foveated displays are 15.38 and 3.20 million, respectively. The latter is merely about 1/5 of the former.
Fig. 12.
Fig. 12. Optical surfaces defined in Code V, which are in turn (1) contact lens, (2) anterior cornea, (3) posterior cornea, (4) pupil, (5) anterior lens, (6) virtual plane, (7) posterior lens, and (8) retina. For the real image, the object is positioned at 3 m ahead of the eye. For the virtual image, the object coincides with the contact lens.
Fig. 13.
Fig. 13. Geometric relationship among FOV and other parameters. D is the pupil size, dpl the distance from the center of the lens to the pupil, dcl the distance from contact lens to center of lens, and R1 the radius of curvature of front surface of lens.
Fig. 14.
Fig. 14. FOV versus the pupil size. When the pupil is 2.4 mm in diameter, FOV is 82°. When the pupil dilates to 8 mm in diameter, FOV reaches up to 142°.
Fig. 15.
Fig. 15. Angular resolution versus the field angle. The best angular resolution is 0.38′ at 0°, whereas the worst is 3.11′ at −41°.
Fig. 16.
Fig. 16. MTFs calculated as a function of spatial frequency in cycle/degree. For the real image, MTF of the field of 0° is 0.669757 at 30 cycle/degree. For the virtual image, MTFs of all fields are above 0.999999 at 30 cycle/degree.
Fig. 17.
Fig. 17. Distortion versus the field angle. Distortions of real and virtual images are 10% and 6%, respectively.
Fig. 18.
Fig. 18. (a) Original image (Snell chart), (b) real image, and (c) virtual image. The foveated effect of the virtual image is not visible. The reason is that, in Code V and other simulation tools, the influence of photoreceptor cells on the image has yet to be factored into.

Tables (6)

Tables Icon

Table 1. Maximum angular resolution of four different regions on the retina

Tables Icon

Table 2. Parameters of contact lens

Tables Icon

Table 3. Parameters of optical surfaces used in Code V

Tables Icon

Table 4. Parameters for aspherical surfaces

Tables Icon

Table 5. Parameters for gradient refractive indices of anterior and posterior lenses

Tables Icon

Table 6. Parameters for calculating FOV

Equations (14)

Equations on this page are rendered with MathJax. Learn more.

V A = 1 a n g u l a r r e s o l u t i o n
α = 2 sin 1 [ sin ( 1 / ρ 2 [ r sin ( ε ) ] 2 + [ r cos ( ε ) + d e l ] 2 ) ]
P c = P g 1 d g P g
P c = ( n 1 ) ( 1 R 1 1 R 2 )
D c < 2.4 λ π n 1 2 n 2 2
θ = β + sin 1 n d c l sin β R 1 sin 1 d c l sin β R 1
β = sin 1 ( r sin ε ) 2 ( r sin ε ) 2 + ( r cos ε + d e l ) 2
D = 0.11 Y + 14.64 ( L π F O V 2 4 × 846 ) 0.41 + 2 + 0.0022 Y + 1.937
N i = π sin β sin ( α i 2 )
N = i = 1 M N i
F O V = 2 ( tan 1 D 2 d p l + sin 1 n D d c l R 1 D 2 + 4 d p l 2 sin 1 D d c l R 1 D 2 + 4 d p l 2 )
A n g u l a r r e s o l u t i o n = 21600 sin θ N i
C R = C R 0 + 1 + M T F ( C R 0 1 ) C R 0 + 1 M T F ( C R 0 1 )
Distortion = h a h p h a
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.