1. Introduction

Augmented reality (AR) is a technology that displays computer-generated graphics over objects in the real world [1–7]. The first implementation of AR can be traced back to 1968 when Ivan Sutherland built the first AR display, The Sword of Damocles [1]. The emergence of powerful computer hardware, digital content, and technologies such as smartphone display panels, pico-projector micro-display panels, inertial measurement units, and camera and depth map sensors have been reshaping AR technology developments [3]. Various approaches have been proposed for AR displays, such as retinal scanning [8], freeform mirrors [9–11], birdbath [12], Maxwellian view displays [13], and waveguide combiners [14,15]. Optical waveguide combiners are currently the most common in commercial AR devices because they can provide a large eyebox while remaining lightweight and compact.

Waveguide combiners trap light in a thin glass substrate via total internal reflection (TIR) [15–18]. One distinctive feature of a waveguide combiner is the exit pupil expansion process, which increases the eyebox without greatly increasing the form factor. As demonstrated in Fig. 1(a), rays coming from the display optics are guided into the waveguide by the in-coupler. The rays propagate inside the waveguide via TIR until they reach the out-coupler, where they are extracted from the waveguide. When first encountering the out-coupler, some light is extracted from the waveguide toward the user. The rest continues propagating in the waveguide until it interacts with the out-coupler again. This process of partial extraction and continued propagation allows a single ray to be replicated many times and increases the eyebox of the combiner while preserving the field-of-view (FOV), thus increasing the system etendue.

 

Fig. 1. The device application: (a) Schematic diagram illustrating the method of pupil expansion; the chosen in-coupler is the metagrating shown in (b) instead of the commonly used SRG shown in (c). (b) Schematic diagram showing the metagrating in-coupler. (c) Schematic diagram showing the surface relief grating (SRG) in-coupler.

Download Full Size | PDF

Waveguide displays face challenges, including limited FOV, eyebox uniformity, image plane brightness and uniformity, and imaging quality. It is well known that the refractive index of the waveguide restricts the maximum FOV of a waveguide combiner because of the critical angle of TIR [3,4]. Additionally, inherent trade-offs exist between system parameters, like maximum image brightness and light uniformity in the image and pupil plane. While waveguide architectures are conventionally designed by constraining the system and optimizing the waveguide and the in- and out-coupler together, we think it is also beneficial to break the system down and understand the limitations and performance of each component individually. To the best of our knowledge, these component and system-level fundamental trade-offs and limitations have not been well documented in the existing literature.

A good place to begin investigating these limitations is at the in-coupler since any light lost here bottlenecks the brightness and imaging quality of the whole waveguide. This paper will focus on quantifying a limit for the in-coupling efficiency, which translates to a limit on the system's brightness. One of the primary issues that limit the efficiency of the in-coupler is that incident light may interact with it multiple times. An in-coupler bends light into the waveguide such that it TIRs inside the glass until it reaches the out-coupling regions. In the ideal case, each incident ray will interact with the grating exactly once. However, the glass is typically thin enough that some of the rays intersect the in-coupler grating multiple times after being diffracted, as shown by the red ray in Fig. 2. Any time a ray interacts more than once with the grating, there will be losses, thus decreasing the efficiency of the in-coupler. This problem of multiple interactions will be the basis for deriving the in-coupling efficiency limit.

 

Fig. 2. Demonstration of multiple interactions with the in-coupler. An in-coupler where two parallel rays are incident on the in-coupler grating. After diffracting and undergoing TIR, the red ray interacts for the second time in the same location as the blue ray when it interacts for the first time with the in-coupler. The red and blue rays are drawn offset vertically for clarity.

Download Full Size | PDF

The derived limit is general for any diffractive waveguide in-coupler (even gratings that vary spatially in geometry). This component-level analysis (in contrast to the full-system optimization) is useful to determine the possible solution space and avoid attempting to design a system with parameters that are not physically possible (e.g., a system with image plane brightness higher than the upper bound determined by the system geometry). We demonstrate this design approach by designing both an SRG and a metasurface-based in-coupler for a specific example waveguide geometry.

Metasurfaces are made of nanostructures at a scale comparable to or smaller than the wavelength of light [19–24]. They can tune the phase, amplitude, and polarization of light through dipolar resonances [25,26], coupled modes [27,28], or geometric phase [19,20,29]. Recent years have witnessed the great potential of metasurfaces in multiple technology applications, including AR [30–38]. For example, metasurfaces can be used as the eyepiece for near-eye displays (NEDs) [30] or used with or in place of freeform optics as a free-space AR combiner [31,32]. Metasurfaces can also be used for eye-tracking systems [39], as the couplers for waveguide combiners [34–37], or applied to see-through displays for contact lens-based AR NEDs [38]. The current work explores how metasurfaces can help with the issue of low output brightness by leveraging the ability of metagratings to be designed to work over a broad angular range [40].

In Section 2, we first show that, given a waveguide geometry (i.e., the specific choice of waveguide material, thickness, in-coupler size, and grating period), an efficiency limit originates from the multiple interactions of rays with the in-coupler. We then derive the efficiency limit of the in-coupler and confirm the calculation with a ray tracing simulation of an example waveguide geometry. With the understanding of the multiple interaction problem and the limit it places on in-coupler efficiency, we design a metagrating such as the one shown in Fig. 1 (b) that meets the fundamental efficiency limit benchmark when used as an in-coupler for a pupil expansion architecture. Note that this grating is not designed for the highest possible efficiency at each point in the field but rather to achieve the system-level efficiency limit we derived based on the waveguide geometry. We further demonstrate that this metagrating shows better efficiency than the commonly used SRG in-couplers [see Fig. 1(c)].

2. Efficiency limit based on waveguide geometry

The geometry of the waveguide limits the in-coupling efficiency of the in-coupler for a given FOV. We derive this efficiency limit, calculate it for each point in the FOV, and use a ray-tracing simulation to confirm the calculation.

2.1 Problem of multiple interactions

As explained in Section 1, some rays interact more than once with the in-coupler. The first time a ray interacts with the in-coupler to couple into the waveguide, it diffracts into the 1^st -order with some diffraction efficiency, ${\eta _{ + 1}}(\theta )$ where $\theta $ is the angle of incidence (AOI) as shown by the blue ray in Fig. 2. After being diffracted, some rays interact with the in-coupler again. To successfully in-couple, they should continue into the 0^th-order with a diffraction efficiency ${\eta _0}({{\theta_{ + 1}}} )> 0$ where ${\theta _{ + 1}}$ is the angle of the light after being diffracted, as demonstrated by the red ray in Fig. 2. As shown in Appendix A, ${\eta _{ + 1}}(\theta )+ {\eta _0}({{\theta_{ + 1}}} )\le 1$, which implies there is an upper bound on the in-coupling efficiency.

To understand this upper bound better, consider two parallel rays interacting with an in-coupler, as shown in Fig. 2. The in-coupler efficiency may vary spatially. The red ray interacts once, TIRs, and then interacts again. The parallel blue ray interacts for the first time at the location where the red ray has its second interaction. At this location, we can assume that ${\eta _0}({{\theta_{ + 1}}} )+ {\eta _{ + 1}}(\theta )= 1$ to find an upper bound on efficiency. In any area of the in-coupler where there is more than one interaction, any gains for the rays interacting the first time [increased ${\eta _{ + 1}}(\theta )$] will be offset by losses for rays that have already been diffracted [decreased ${\eta _0}({{\theta_{ + 1}}} )$]. The inverse is also true, i.e., that if ${\eta _0}({{\theta_{ + 1}}} )$ increases, ${\eta _{ + 1}}(\theta )$ will decrease an equal amount. Therefore, the upper bound on efficiency for each point in the field can be determined by assuming ${\eta _{ + 1}}(\theta )= 1$.

Decreasing and even spatially varying the grating efficiency in some areas of the in-coupler will affect which specific rays from the pupil make it through the waveguide and thus affect imaging quality. However, this variation in grating efficiency will not increase the in-coupling efficiency at each field point when compared to the same field point for an ideal 100% efficient grating. Therefore, the light uniformity in the entrance pupil plane (and therefore the imaging quality) typically comes at the cost of decreasing efficiency. While this relationship is implied in the existing literature and can be observed indirectly by brute force full-system optimization, to the best of our knowledge, it has not been explicitly explored nor quantified [35,37,41–43]. Therefore, to begin quantifying the problem, we will consider the ideal grating case to derive the upper bound benchmark.

For an ideal grating ${\eta _{ + 1}}(\theta )= 1$ and therefore ${\eta _0}({{\theta_{ + 1}}} )= 0.\; $ Thus, only rays that interact with the in-coupler once contribute to the in-coupler efficiency because rays that interact a second time will not make it past the in-coupler, as demonstrated by the red ray in Fig. 2 and Eq. (9) in Appendix A. The area where incident rays will only interact once with the in-coupler is referred to as the effective aperture and is the blue region of the in-coupler in Fig. 3. It is called an aperture since it is typically coplanar with the exit pupil of the lens that collimates the light from the display (see the example in Section 2.2). As shown in Fig. 4, the effective aperture size changes based on the FOV. The maximum possible efficiency for a given field is calculated by finding the ratio of the effective aperture area to the area of the entire in-coupler. This ratio can be expressed as

 

Fig. 3. Demonstration of effective aperture at the in-coupler: (a) Top inset: waveguide geometry. Bottom inset: zoomed-in view of the in-coupler showing two parallel incident rays (red and blue solid lines); the red ray interacts twice with the in-coupler and will experience loss after its first interaction. The blue ray only interacts once with the in-coupler and thus will experience no losses after its first interaction. The thickness of the waveguide, t, and the interaction separations, ${s_x}\; $ and ${s_y}$, are shown; (b) A top-down view of the in-coupler showing the incident and diffracted blue ray projected onto the x-z plane, enabling ${s_x}$ computation. The incident angle, diffracted angle, and interaction separation in x are labeled as ${\theta _x}$, ${\theta _{x,D}}$, and ${s_x}$ respectively; (c) A side view of the in-coupler showing the incident and diffracted blue ray projected onto the y-z plane, enabling ${s_y}$ computation. The incident angle, diffracted angle, and interaction separation in y are labeled as ${\theta _y}$, ${\theta _{y,D}}$, and ${s_y}$ respectively; (d) The in-coupler is shown in two colors. The red region is where incident rays interact multiple times with the in-coupler, and the blue is where incident rays only interact once (i.e., effective aperture). The blue region is subdivided into two regions, ${A_1}$ and ${A_2}$. The height and width of the in-coupler are labeled as H and W respectively.

Download Full Size | PDF

 

Fig. 4. Calculation of the in-coupler efficiency map over the full FOV based on waveguide geometry. The six in-coupler insets show how rays at different incident angles will each have a different effective aperture area (i.e., one interaction) shown in blue. The effective aperture area dictates the in-coupling efficiency limit for each part of the FOV.

Download Full Size | PDF

To find the effective aperture area, the separation in the x and y dimensions between bounces for a single ray needs to be calculated. These distances are indicated by ${s_x}$ and ${s_y}$ in Fig. 3. These can be calculated using

Knowing ${s_x}$ and ${s_y}$, the areas of each of the subsections in Fig. 3(d) (${A_1}$ and ${A_2}$) can be expressed as

Equation (6) can be used to calculate ${\eta _{max}}$ for each point in a chosen FOV, assuming a waveguide with a known refractive index, thickness, in-coupler size, and grating period. The derivation of Eq. (6) assumes a rectangular in-coupler, but the concepts behind the derivation could be used to derive a similar equation for any in-coupler. A derivation for a circular in-coupler is provided as another example in Appendix B. As seen in Fig. 4, ${\eta _{max}}$ is not uniform over the FOV, which is undesirable for AR/VR displays. In a practical application, software can decrease the brightness of the most efficient (i.e., brightest) points in the FOV. In contrast, the darker points cannot be made brighter as the optical architecture limits them. Therefore, the uniform image plane brightness of the complete display architecture is limited by the point in the FOV with the minimum in-coupling efficiency, as the rest of the field points can be dimmed to match that efficiency. Thus, the overall brightness of the complete display architecture is still limited by the point in the FOV that has the minimum in-coupling efficiency. This minimum field efficiency (MFE) for in-coupling is given by

In this paper, the waveguide is made of N-BK7 (n = 1.52 @ 532 nm) glass, a common glass for low-cost optical components. The N-BK7 thickness is 0.5 mm, the common thickness of the N-BK7 wafer. The in-coupler is 3 mm by 3 mm, close to the human vision pupil size under photopic illumination. In practice, a thicker waveguide or a smaller grating can mitigate some of the multiple interactions but at the cost of increasing weight and decreasing pupil replication density, both of which are undesirable for AR/VR displays. Thus, the geometry example we are exploring is impactful as it represents a design space that is desirable from an application perspective but challenging due to the multiple interaction problem. The light source is green light with a wavelength of 532 nm. The full FOV is 20° by 20°, which is near the FOV limit dictated by the waveguide, as demonstrated in Appendix C. The period of the coupler gratings is 453 nm, calculated such that the minimum diffracted angle within the waveguide over the FOV is just greater than the critical angle for TIR. Given this waveguide thickness, grating period, wavelength, and FOV, the maximum interaction separation expected is about 2 mm. Having the maximum interaction separation be smaller than the in-coupler avoids gaps between replicated pupils when the beam is extracted at the out-coupler. As seen in Fig. 4, the MFE occurs at the X-FOV of -10° because it has the smallest effective aperture over this FOV. The MFE value of 29% (i.e., the minimum efficiency value over the efficiency map in Fig. 4) will benchmark the performance of two in-couplers in Section 3.

2.2 Verifying the theoretical limit with simulation

To verify the theoretical calculation of the in-coupler efficiency limit, we implemented a ray-tracing simulation of this geometry using LightTools from Synopsys, as demonstrated in Fig. 5(b).

The simulation first measures how much light is incident on the in-coupler for each FOV point. The input lens collimates light from the display, and the exit pupil of the input lens is set to be coplanar with the in-coupler. The exitance from the display is filtered to exclude any emitted rays that do not intersect the in-coupler. The in-coupler is a theoretical grating with 100% efficiency into the 1^st-order. There are losses at the in-coupler due to the effective aperture for each field. The light that successfully passes the in-coupler (i.e., light from the effective aperture for each field) then TIRs until it intersects the out-coupler.

The simulation measures the amount of light that is out-coupled and focused to the image plane. The out-coupler is another theoretical grating with 100% efficiency into the 1^st-order. Because there is no need to expand the eyebox in this case, the large out-coupler region from Fig. 1(a) has been replaced by a smaller out-coupler region shown in Fig. 5(a). The out-coupler experiences no losses because the output lens focuses every extracted ray onto the image plane. The image plane irradiance is divided by the filtered display plane irradiance to get the efficiency of incident light from the in-coupler that reaches the image plane for each point in the FOV, shown by the efficiency map in Fig. 5(d). Figure 5(c) shows the efficiency map predicted by Eq. (6) above and shows excellent agreement with the results from the LightTools simulation in Fig. 5(d). The MFE in both cases is 29%. In Section 3, this model will be used to evaluate the in-coupling efficiency over the FOV for an optimized SRG and an optimized metagrating.

 

Fig. 5. Simulation architecture to verify the theoretical prediction: (a) The waveguide display with the expanding out-coupler region replaced by a 100% efficient out-coupler; (b) LightTools rendering of the simulated geometry. The input lens collimates light from the display before it gets to the in-coupler. The in-coupler diffracts light into the waveguide, and the out-coupler extracts it back to free space. The collimating and the focusing lenses are identical to provide 1-to-1 imaging. The ray bundles correspond to -10, 0, and 10 degrees in the x-z plane colored in red, green, and blue, respectively; (c) The theoretical in-coupling efficiency map over the full FOV as calculated by Eq. (6); (d) The simulated in-coupling efficiency map over the full FOV from LightTools.

Download Full Size | PDF

3. Metagrating simulations

Now that the limit for the efficiency over the field is known for a given waveguide geometry, we can design gratings and evaluate their performance in the system using the simulation setup from Section 2.2. The initial grating design is done using a photonic simulation package. These results are then converted to a bidirectional scattering distribution function (BSDF) for implementation in the ray-tracing simulation.

The photonic simulations are done using the rigorous coupled-wave analysis method in RSoft from Synopsys. The metagrating is made of titanium dioxide, which is desirable for optical waveguide combiners because of its large refractive index and low loss of visible light. The metagrating consists of periodic nano-beams and nano-pillars, as shown in Fig. 6(a). As mentioned at the end of Section 2.1, the period of the metagrating is 453 nm. The optimizing variables are the width of the nano-beam, the diameter of the nano-pillar, their thickness, the distance between the nano-beam and nano-pillar, and the unit cell size along the y direction. The output of each simulation was set to be the 1^st-order diffraction efficiency. The MFE benchmark of 29% from Section 2.1 is used for optimizing the metagrating.

 

Fig. 6. Optimized in-coupler gratings: (a) Optimized metagrating of nano-beam and nano-pillar geometry. The center-to-center distance between nano-beams and nano-pillars in the unit cell is 270 nm; (b) Optimized SRG that is continuous along the y-direction; (c) & (d) Simulated diffraction efficiency as a function of the angle of incidence in the 1^st-order for the optimized metagrating and SRG, respectively; (e) & (f) Simulated efficiency map over the full FOV for the optimized metagrating and SRG, respectively.

Download Full Size | PDF

The optimization was established using the multi-variable optimization and scanning tool in RSoft. The light source is TE-polarized green light with a wavelength of 532 nm. From Fig. 5(c) and (d), it is apparent that there is an insignificant change in efficiency when varying the FOV angle along the y-axis. Thus, to accelerate optimization, each simulation only scans incident angles from -10° to 10° along the x direction. A constraint is used to maximize the efficiency of the -10° X-FOV in the merit function to compensate for the small effective aperture at that FOV point. Having the grating be as efficient as possible for this field will help offset losses due to the small effective aperture. A second constraint is added to keep the minimum efficiency over the field as high as possible to avoid losing efficiency at other angles of incidence. This second constraint is weighted the same as the first constraint.

The metagrating optimized using the described merit function is shown in Fig. 6(a). The 1^st-order diffraction efficiency plot of this optimized metagrating is shown in Fig. 6(c). The diffraction efficiency at -10° X-FOV of this metagrating is as high as 92%, but the efficiency for the 10° X-FOV is only 51%.

The metagrating is now integrated into the LightTools simulation from Section 2.2 to evaluate its viability as an in-coupler using an efficiency map. The metagrating BSDF replaces the theoretical in-coupler grating from Section 2.2, enabling RSoft results to interface with the LightTools simulation. The MFE for the metagrating in-coupler is 28% and occurs at the X-FOV of -10°. The efficiency loss for the positive angles of incidence [Fig. 6(c)] of this metagrating does not decrease the MFE, instead resulting in a more uniform efficiency over the full FOV [Fig. 6(e)].

To show the advantage of this metagrating as the in-coupler for a waveguide combiner, we optimized an SRG for comparison. The top ridge width, base width, slant angle, and groove depth of the SRG are optimized using the same merit function as for the metagrating. The optimized SRG is shown in Fig. 6(b), and its diffraction efficiency plot is shown in Fig. 6(d). From the diffraction efficiency plot, the SRG achieves an efficiency of 85% for -10° X-FOV. However, the efficiency drops faster than the metagrating, and it only has 30% efficiency for 10° X-FOV. Figure 6(f) shows the efficiency map using this optimized SRG as the in-coupler. Compared with Fig. 6(e), the efficiency drops on the right half of the FOV. The MFE for the SRG is only 20% and occurs for the X-FOV of +10° rather than the X-FOV of -10°. The lower MFE occurs because the positive incident angles’ efficiencies drop so fast that even the larger effective aperture area cannot compensate for this loss, leading to the minimum efficiency over the field occurring on the right half of the FOV. Thus, despite emphasizing efficiencies at both sides of the X-FOV, the metagrating can obtain high diffraction efficiency over the entire FOV, while the SRG cannot.

4. Discussion

Section 2 showed that the effective aperture area (i.e., the area of the in-coupler where rays only interact once with the in-coupler) limits waveguide in-coupler efficiency. The effective aperture area for a given FOV is determined by the refractive index and thickness of the waveguide, the size of the in-coupler, and the grating period. As discussed in Section 3, this limit can be used to calculate the MFE as a benchmark for designing in-coupler gratings. Knowing the MFE to aim for can save time that may be fruitlessly spent trying to achieve higher efficiencies than the limit set by the waveguide geometry.

As shown in Table 1, the metagrating in-coupler achieves a higher MFE than the SRG. The MFE jump from 20% to 28% shows a relative improvement of 40% for the metagrating over the SRG. The metagrating achieves over 96% of the theoretical MFE benchmark, while the SRG achieves almost 69%. Additionally, the higher efficiency over the entire FOV seen in Fig. 6(e,f) results from the metagrating being less sensitive to the AOI than the SRG. In other words, using the same merit function, we can achieve better control over the angular efficiency response of the metagrating compared to the SRG. An expanded view of the efficiency vs. AOI plots can be found in Appendix E. The additional plots show that the metagrating maintains a higher efficiency than the SRG over an even larger AOI range than the designed FOV. In terms of in-coupling efficiency, the metagrating is the superior candidate.

Table 1. Minimum field efficiency for different in-couplers

View Table

A question remains about how the effective aperture area affects the system's imaging performance. A larger aperture means better resolution assuming diffraction-limited imaging from the lenses. In the case of a waveguide in-coupler where the exit pupil of the collimating lens (i.e., the aperture) is coplanar with the in-coupler, the effective aperture area is independent of the collimating lens. In the theoretical case where the in-coupler is 100% efficient, only rays from within the effective aperture for each FOV would contribute to the image. In the case of the metagrating, some light from outside the effective aperture could still make it to the image plane. Thus, more of the rays from the original pupil would be able to make it to the image plane and possibly increase the resolution, assuming a diffraction-limited system. Future work could study the effect of contributing rays on imaging performance.

5. Conclusion

In this paper, we showed that the multiple interactions of light with the in-coupler effective aperture set a fundamental efficiency limit specific to a waveguide's geometry. We further showed how to calculate the associated MFE of an AR waveguide in-coupler. The values provided in this paper are specific to the example waveguide geometry, but Eq. (6) is general to any waveguide in-coupler using a rectangular diffractive in-coupler. The method for deriving Eq. (6) could be used to derive a similar relation for any shaped in-coupler. We then designed a metagrating in-coupler whose MFE approached the derived geometry-based limit. The metagrating showed a relative MFE improvement of 40% over the SRG designed using the same optimization method. Thus, the geometry-based efficiency limit derived in the paper can be a valuable tool when designing AR waveguide architectures. The advanced angular efficiency response of metagratings makes them an ideal candidate for meeting this limit, opening a promising path towards more efficient in-couplers and brighter AR displays.

Appendix A

This section shows that for a diffraction grating (i.e., in-coupler), the sum of the diffraction efficiencies for two rays from different incident angles but diffracting into the same angle is less than or equal to one.

As discussed in Section 2.1, many rays interact with the in-coupler more than once. Figure 7(a) shows an incident ray diffracting into the 1^st-order, undergoing TIR, and then reflecting into the 0^th-order. In Fig. 7, $\theta $ is the incident angle before any diffraction, ${\theta _{ + 1}}$ is the incident angle after being diffracted into the 1^st-order, and $- {\theta _{ + 1}}$ is the angle for the ray traveling in reverse to the 1^st-order diffracted ray. The diffraction efficiency for a given incident angle into a given order is denoted by ${\eta _m}(\theta )$ where $\eta $ is the diffraction efficiency, m is the diffracted order of interest, and $\theta $ is the incident angle.

 

Fig. 7. Illustration of the multiple interaction problem. Only diffracted orders relevant to the discussion are shown, but more exist. (a) A ray diffracts into the 1^st-order with efficiency ${\eta _{ + 1}}(\theta )$, totally internally reflects (blue) and interacts with the grating a second time (red). Light reflects into the 0^th-order with efficiency ${\eta _0}({{\theta_{ + 1}}} )$. The top left inset shows the sign convention used in this work. (b) Both types of interactions from (a) are overlayed. (c) A ray traveling backward along the 1^st-order diffraction path ($- {\theta _{ + 1}}$) interacts with the grating and diffracts into the 0^th- and 1^st-order with efficiencies ${\eta _0}({ - {\theta_{ + 1}}} )$ and ${\eta _{ + 1}}({ - {\theta_{ + 1}}} )$ respectively.

Download Full Size | PDF

Considering the case shown in Fig. 7(c), by conservation of energy, it is clear that

(8)$${\eta _0}({ - {\theta_{ + 1}}} )+ {\eta _{ + 1}}({ - {\theta_{ + 1}}} )\le 1.$$

In other words, the sum of light going into the two diffracted orders will not be greater than the amount of light incident on the grating at that location.

By the principle of reversibility, ${\eta _0}({ - {\theta_{ + 1}}} )= {\eta _0}({{\theta_{ + 1}}} )$ and ${\eta _{ + 1}}({ - {\theta_{ + 1}}} )= {\eta _{ + 1}}(\theta )$. Using these identities, Eq. (8) can be rewritten as

(9)$$\boxed{{{\eta _0}({{\theta_{ + 1}}} )\, + {\eta _{ + 1}}(\theta )\le 1}}$$

Appendix B

This section calculates the in-coupling efficiency limit assuming a circular in-coupler. We rely on the same assumptions as in Section 2.1. Again, the efficiency limit for each FOV angle is determined by the ratio of the effective aperture area [i.e., the blue area of the in-coupler in Fig. 8(a)] to the total in-coupler area.

 

Fig. 8. (a) Effective aperture area for a circular in-coupler. $R\; $ is the radius of the in-coupler, ${s_x}$ and ${s_y}$ are the interaction separations in x and y respectively, ${s_t}$ is the total interaction separation, and $\phi $ is the half-angle subtended by the region where incident rays interact twice. Note that the red region is equivalent to two circle segments added together. (b) In-coupling efficiency map assuming a circular in-coupler with a radius of 3 mm. The point indicated in the top right corner of the efficiency map roughly corresponds to the effective aperture area in (a) (i.e., most of the in-coupler is in the effective aperture, so the efficiency limit is high).

Download Full Size | PDF

To determine the effective aperture area for a circular pupil, we need to know the total interaction separation, ${s_t}$. This is the distance between the first interaction of a ray with the in-coupler and its second interaction after being diffracted. This can be calculated by

(10)$${{s_t} = \sqrt {s_x^2 + s_y^2} } $$

where ${s_x}$ and ${s_y}$ are calculated using Eqs. (2) and (3), respectively. By inspection of Fig. 8(a), we can see that for a ray that diffracts in the positive x-direction with interaction separation ${s_t}$, rays in the red region will interact more than once. The angular subtense of this region is equal to 2$\phi $ with

(11)$${\phi = {{\cos }^{ - 1}}\left( {\frac{{{\raise0.7ex\hbox{${{s_t}}$} \!\mathord{\left/ {\vphantom {{{s_t}} 2}} \right.}\!\lower0.7ex\hbox{$2$}}}}{R}} \right)} $$

where R is the radius of the in-coupler. The area of the red region, ${A_{red}}$, is twice the area of a circle segment given by the following equation

(12)$${{A_{red}} = 2{A_{segment}} = 2\left( {\phi - \frac{{\sin 2\phi }}{2}} \right){R^2}.} $$

The maximum efficiency ratio can be expressed as

(13)$${{\eta _{max,circ}} = \frac{{{A_T} - {A_{red}}}}{{{A_T}}} = 1 - \frac{{{A_{red}}}}{{{A_T}}}.} $$

In this case, ${A_T} = \pi {R^2}$ where ${A_T}$ is the total area of the in-coupler. Combining Eq. (12) with Eq. (13) yields an equation for in-coupling efficiency

(14)$$\boxed{{{{\eta _{max,circ}} = 1 - \frac{2}{\pi }\left( {\phi - \frac{{\sin 2\phi }}{2}} \right).\; } }}$$

Using this new equation for ${\eta _{max,circ}}$, we can calculate an in-coupling efficiency map over the full FOV, as shown in Fig. 8(b). The circular in-coupler has a diameter of 3 mm, and all other parameters match the rectangular in-coupler used in the rest of the paper.

Appendix C

This section shows how to calculate a 2D FOV limit given a refractive index for the waveguide. This limit is proven using a k-space diagram like the one shown in Fig. 9.

 

Fig. 9. A k-space diagram for a 2D-FOV coupling into a waveguide. The radius of the inner circle is equal to 1, indicating the critical angle, and the radius of the outer circle is n, the refractive index of the waveguide. The FOV is drawn in gray on the plot with initial coordinates $({{x_i},q\mathrm{\ast }{x_i}} )$ and final coordinates $({{x_i} + {k_g}\,,q\mathrm{\ast }{x_i}} )$ where q is the aspect ratio of the FOV and ${k_g}$ is the grating vector of the in-coupler grating.

Download Full Size | PDF

As shown in the diagram, the radius of the inner circle is equal to 1, and the radius of the outer circle is n, the refractive index of the waveguide. The FOV is drawn in gray on the plot. Here we assume an aspect ratio of q where ${y_i} = q\mathrm{\ast }{x_i}$ and ${x_i}$ and ${y_i}$ are the sines of the x and y components of the incident FOV (${\theta _{x,i}},\,{\theta _{y,i}}$), respectively. Expressed mathematically, ${x_i} = \sin {\theta _{x,i}}$ and ${y_i} = \sin {\theta _{y,i}}$. The in-coupler shifts the FOV to the right in k-space by some amount ${k_g}$. The magnitude of ${k_g}$ is chosen so that the left side of the FOV is at the critical angle after being diffracted. Expressed mathematically, ${k_g} = 1 + {x_i}$. Thus, for some initial field point $({{x_i},{y_i}} )$ in the FOV, its k-space coordinate after being diffracted is

(15)$${({{x_f},{y_f}} )= ({{x_i} + {k_g},{y_i}} )= ({{x_i} + ({1 + {x_i}} ),q\mathrm{\ast }{x_i}} )} $$

where ${x_f},{y_f}$ are the k-space coordinates of the diffracted ray. The magnitude of (${x_f},{y_f}$) can be calculated via

(16)$${{R^2} = x_f^2 + y_f^2.} $$

To find the FOV limit, we want $R = n$, (i.e., the diffracted beam to be right at the edge of being evanescent). Substituting $R = n$, ${x_f} = 2{x_i} + 1$, and ${y_f} = q\mathrm{\ast }{x_i}$, we get the equation

(17)$${{n^2} = {{({2{x_i} + 1} )}^2} + {q^2}x_i^2.} $$

This can be rearranged to yield

(18)$${({4 + {q^2}} )x_i^2 + 4{x_i} + ({1 - {n^2}} )= 0.} $$

We now assume $q = 1$ for a square FOV and solve the quadratic equation

(19)$${{x_i} = \frac{{ - 2 \pm \sqrt {5{n^2} - 1} }}{5} \approx{-} 1.0497,0.2497} $$

This assumes that the index of the waveguide is $n$=1.52. We do not need to consider the $|{{x_i}} |> 1$ value as it is non-physical. To return to the input FOV, we take the inverse sine of ${x_i}$

(20)$${\boxed{{{\theta _x} = \textrm{si}{\textrm{n}^{ - 1}}({{x_i}} )\approx 12.2^\circ \,}}} $$

Thus, the half-FOV along one dimension is about 12 degrees, so the full-FOV in 2D is limited to about 24$^\circ $ x 24$^\circ $. However, at this limit, the diffracted rays at the corners of the FOV reach the evanescent limit (i.e., nearly parallel to the surface). Thus, for this paper, we chose a +/- 10-degree FOV, so the interaction separation is about 2 mm.

Appendix D

This section shows how the nano-pillar and nano-beam design was achieved. Originally, the metagrating used two TiO₂ pillars to form the gradient phase due to the very short grating period. During the optimization process, one TiO₂ pillar tended to expand out of the period boundary. Thus, the first version of the metagrating consisted of one large pillar and one smaller pillar in each grating period. The metagrating drawing, diffraction plot, and in-coupler efficiency are shown in Fig. 10.

 

Fig. 10. Drawing and data for the original two-pillar metagrating design. (a) Drawing of the two-pillar metagrating design. Note how three of the four sides of the larger pillar are clipped by the edges of the unit cell. (b) Simulated diffraction efficiency as a function of the angle of incidence in the 1^st-order for the optimized two-pillar metagrating. (c) Simulated efficiency map over the full FOV for the optimized two-pillar metagrating.

Download Full Size | PDF

Although the pillar height is only 340 nm - shorter than the typical 500 nm of other TiO₂ metagratings in the literature - the aspect ratio of the smaller nano-pillar is still around 3, which is typical for TiO₂ metasurfaces [45,46]. However, as can be seen in Fig. 10(a), the edges of the larger pillar are clipped by the unit cell. The shape of the larger chopped pillars connecting along the y-direction is similar to a continuous nano beam extending along the y-direction. For this reason, we later shifted to the new geometry presented in the main text consisting of a nano-pillar and a nano-beam. The MFE for both designs is still 28%, although the original chopped pillar design did exhibit slightly higher in-coupling efficiency on the right side of the FOV.

Appendix E

This section shows RSoft simulations of 1^st-order diffraction efficiency vs. AOI for the optimized metagrating and SRG. This plot doubles the AOI range from +/-10 degrees used for the in-coupler design to +/-20 degrees. The extended curves shown in Fig. 11 demonstrate that the metagrating continues to have higher diffraction efficiency outside the AOI range for which it was optimized. Inside the optimization range, the SRG has slightly higher efficiency between -8 and -4 degrees, but its efficiency drops off much faster than for the metagrating. These curves further show that the metagrating allows for superior control over the angular efficiency response compared to the SRG.

 

Fig. 11. First-order diffraction efficiency vs. AOI for the metagrating and SRG out to +/- 20 degrees.

Download Full Size | PDF

Funding

National Science Foundation (DGE-1922591).

Acknowledgments

We thank Synopsys Optical Solutions for the use of RSoft Photonic Device Tools and LightTools in this paper.

Disclosures

The authors declare no conflicts of interest.

Data availability

All data needed to evaluate the conclusions are present in the paper and/or the Supplementary Materials. Additional data related to this paper may be requested from the authors.

References

1. I. E. Sutherland, “A head-mounted three dimensional display,” in Proceedings of the December 9-11, 1968, fall joint computer conference, part I, (Association for Computing Machinery, 1968), pp. 757–764.

2. J. P. Rolland, K. P. Thompson, H. Urey, and M. Thomas, “See-Through Head Worn Display (HWD) Architectures,” in Handbook of Visual Display Technology, J. Chen, W. Cranton, and M. Fihn, eds. (Springer, 2012).

3. B. C. Kress and C. Peroz, “Optical architectures for displays and sensing in augmented, virtual, and mixed reality (AR, VR, MR),” in Proc. SPIE, (2020), 1131001.

4. B. C. Kress and I. Chatterjee, “Waveguide combiners for mixed reality headsets: a nanophotonics design perspective,” Nanophotonics 10(1), 41–74 (2020). [CrossRef]  

5. K. Yin, E.-L. Hsiang, J. Zou, Y. Li, Z. Yang, Q. Yang, P.-C. Lai, C.-L. Lin, and S.-T. Wu, “Advanced liquid crystal devices for augmented reality and virtual reality displays: principles and applications,” Light: Sci. Appl. 11(1), 1–22 (2022). [CrossRef]  

6. R. T. Azuma, “A survey of augmented reality,” Presence: Teleoperators & Virtual Environments 6(6), 355–385 (1997). [CrossRef]  

7. M. Billinghurst, A. Clark, and G. Lee, “A survey of augmented reality,” FNT in Human–Computer Interaction 8(2-3), 73–272 (2015). [CrossRef]  

8. M. Bayer, “Retinal scanning display: a novel HMD approach for Army aviation,”, in Helmet- and Head- Mounted Display VII 4711, pp. 202–213 (2002).

9. O. Cakmakci and J. Rolland, “Design and fabrication of a dual-element off-axis near-eye optical magnifier,” Opt. Lett. 32(11), 1363–1365 (2007). [CrossRef]  

10. A. Bauer and J. P. Rolland, “Visual space assessment of two all-reflective, freeform, optical see-through head-worn displays,” Opt. Express 22(11), 13155–13163 (2014). [CrossRef]  

11. J. P. Rolland, M. A. Davies, T. J. Suleski, C. Evans, A. Bauer, J. C. Lambropoulos, and K. Falaggis, “Freeform optics for imaging,” Optica 8(2), 161–176 (2021). [CrossRef]  

12. Y. Itoh, T. Kaminokado, and K. Akşit, “Beaming Displays,” IEEE Trans. Visual. Comput. Graphics 27(5), 2659–2668 (2021). [CrossRef]  

13. S.-B. Kim and J.-H. Park, “Optical see-through Maxwellian near-to-eye display with an enlarged eyebox,” Opt. Lett. 43(4), 767–770 (2018). [CrossRef]  

14. J. Han, J. Liu, X. Yao, and Y. Wang, “Portable waveguide display system with a large field of view by integrating freeform elements and volume holograms,” Opt. Express 23(3), 3534–3549 (2015). [CrossRef]  

15. D. Cheng, Y. Wang, C. Xu, W. Song, and G. Jin, “Design of an ultra-thin near-eye display with geometrical waveguide and freeform optics,” Opt. Express 22(17), 20705–20719 (2014). [CrossRef]  

16. A. Frommer, “Lumus: maximus: large FoV near to eye display for consumer AR glasses,” in SPIE AVR21 Industry Talks II, 1176403 (2021). [CrossRef]  

17. G. Lin, T. Panigrahi, J. Womack, D. J. Ponda, P. Kotipalli, and T. Starner, “Comparing order picking guidance with Microsoft hololens, magic leap, google glass xe and paper,” in Proceedings of the 22nd international workshop on mobile computing systems and applications, (2021), 133–139.

18. T. Yoshida, K. Tokuyama, Y. Takai, D. Tsukuda, T. Kaneko, N. Suzuki, T. Anzai, A. Yoshikaie, K. Akutsu, and A. Machida, “A plastic holographic waveguide combiner for lightweight and highly-transparent augmented reality glasses,” J. Soc. Inf. Disp. 26(5), 280–286 (2018). [CrossRef]  

19. G. Zheng, H. Mühlenbernd, M. Kenney, G. Li, T. Zentgraf, and S. Zhang, “Metasurface holograms reaching 80% efficiency,” Nat. Nanotechnol. 10(4), 308–312 (2015). [CrossRef]  

20. D. Lin, P. Fan, E. Hasman, and M. L. Brongersma, “Dielectric gradient metasurface optical elements,” Science 345(6194), 298–302 (2014). [CrossRef]  

21. X. Ni, A. V. Kildishev, and V. M. Shalaev, “Metasurface holograms for visible light,” Nat. Commun. 4(1), 2807 (2013). [CrossRef]  

22. H.-T. Chen, A. J. Taylor, and N. Yu, “A review of metasurfaces: physics and applications,” Rep. Prog. Phys. 79(7), 076401 (2016). [CrossRef]  

23. N. Yu and F. Capasso, “Flat optics with designer metasurfaces,” Nat. Mater. 13(2), 139–150 (2014). [CrossRef]  

24. A. V. Kildishev, A. Boltasseva, and V. M. Shalaev, “Planar photonics with metasurfaces,” Science 339(6125), 1232009 (2013). [CrossRef]  

25. J. Sautter, I. Staude, M. Decker, E. Rusak, D. N. Neshev, I. Brener, and Y. S. Kivshar, “Active tuning of all-dielectric metasurfaces,” ACS Nano 9(4), 4308–4315 (2015). [CrossRef]  

26. V. E. Babicheva and A. B. Evlyukhin, “Resonant lattice Kerker effect in metasurfaces with electric and magnetic optical responses,” Laser Photonics Rev. 11(6), 1700132 (2017). [CrossRef]  

27. Z. Li, M.-H. Kim, C. Wang, Z. Han, S. Shrestha, A. C. Overvig, M. Lu, A. Stein, A. M. Agarwal, and M. Lončar, “Controlling propagation and coupling of waveguide modes using phase-gradient metasurfaces,” Nat. Nanotechnol. 12(7), 675–683 (2017). [CrossRef]  

28. J. Park, J.-H. Kang, S. J. Kim, X. Liu, and M. L. Brongersma, “Dynamic reflection phase and polarization control in metasurfaces,” Nano Lett. 17(1), 407–413 (2017). [CrossRef]  

29. W. Luo, S. Sun, H.-X. Xu, Q. He, and L. Zhou, “Transmissive ultrathin Pancharatnam-Berry metasurfaces with nearly 100% efficiency,” Phys. Rev. Appl. 7(4), 044033 (2017). [CrossRef]  

30. G.-Y. Lee, J.-Y. Hong, S. Hwang, S. Moon, H. Kang, S. Jeon, H. Kim, J.-H. Jeong, and B. Lee, “Metasurface eyepiece for augmented reality,” Nat. Commun. 9(1), 4562 (2018). [CrossRef]  

31. D. K. Nikolov, A. Bauer, F. Cheng, H. Kato, A. N. Vamivakas, and J. P. Rolland, “Metaform optics: Bridging nanophotonics and freeform optics,” Sci. Adv. 7(18), eabe5112 (2021). [CrossRef]  

32. C. Hong, S. Colburn, and A. Majumdar, “Flat metaform near-eye visor,” Appl. Opt. 56(31), 8822–8827 (2017). [CrossRef]  

33. J.-H. Song, J. van de Groep, S. J. Kim, and M. L. Brongersma, “Non-local metasurfaces for spectrally decoupled wavefront manipulation and eye tracking,” Nat. Nanotechnol. 16(11), 1224–1230 (2021). [CrossRef]  

34. D. Gu, C. Liang, L. Sun, H. Chen, Y. Chen, and L. Yang, “Optical metasurfaces for waveguide couplers with uniform efficiencies at RGB wavelengths,” Opt. Express 29(18), 29149–29164 (2021). [CrossRef]  

35. Z. Liu, C. Zhang, W. Zhu, Z. Huang, H. J. Lezec, A. Agrawal, and L. J. Guo, “Compact Stereo Waveguide Display Based on a Unidirectional Polarization-Multiplexed Metagrating In-Coupler,” ACS Photonics 8(4), 1112–1119 (2021). [CrossRef]  

36. W.-Q. Chen, D.-S. Zhang, S.-Y. Long, Z.-Z. Liu, and J.-J. Xiao, “Nearly dispersionless multicolor metasurface beam deflector for near eye display designed by a physics-driven deep neural network,” Appl. Opt. 60(13), 3947–3953 (2021). [CrossRef]  

37. L. Shangyu, L. Nianyi, L. Zhenzhen, C. Wenqing, Z. Dasen, and X. Junjun, “Color near-eye display with high exit-pupil uniformity based on optimized meta-grating,” Opt. Eng. 61(06), 065101 (2022). [CrossRef]  

38. S. Lan, X. Zhang, M. Taghinejad, S. Rodrigues, K.-T. Lee, Z. Liu, and W. Cai, “Metasurfaces for near-eye augmented reality,” ACS Photonics 6(4), 864–870 (2019). [CrossRef]  

39. J.-H. Song, J. van de Groep, S. J. Kim, and M. L. Brongersma, “Nonlocal metasurfaces for spectrally decoupled wavefront manipulation and eye tracking,” arXiv, arXiv:2102.05790 (2021).

40. Z.-L. Deng, J. Deng, X. Zhuang, S. Wang, T. Shi, G. P. Wang, Y. Wang, J. Xu, Y. Cao, and X. Wang, “Facile metagrating holograms with broadband and extreme angle tolerance,” Light: Sci. Appl. 7(1), 1–8 (2018). [CrossRef]  

41. Z. Liu, Y. Pang, C. Pan, and Z. Huang, “Design of a uniform-illumination binocular waveguide display with diffraction gratings and freeform optics,” Opt. Express 25(24), 30720–30731 (2017). [CrossRef]  

42. C. P. Chen, L. Mi, W. Zhang, J. Ye, and G. Li, “Waveguide-based near-eye display with dual-channel exit pupil expander,” Displays 67, 101998 (2021). [CrossRef]  

43. C. Jang, K. Bang, M. Chae, B. Lee, and D. Lanman, “Waveguide Holography: Towards True 3D Holographic Glasses,” arXiv, arXiv:2211.02784 (2022).

44. J. E. Harvey and C. L. Vernold, “Description of diffraction grating behavior in direction cosine space,” Appl. Opt. 37(34), 8158–8159 (1998). [CrossRef]  

45. S. Sun, Z. Zhou, C. Zhang, Y. Gao, Z. Duan, S. Xiao, and Q. Song, “All-dielectric full-color printing with TiO2 metasurfaces,” ACS Nano 11(5), 4445–4452 (2017). [CrossRef]  

46. R. C. Devlin, M. Khorasaninejad, W. T. Chen, J. Oh, and F. Capasso, “Broadband high-efficiency dielectric metasurfaces for the visible spectrum,” Proc. Natl. Acad. Sci. U. S. A. 113(38), 10473–10478 (2016). [CrossRef]