Open Access
Add to BibSonomy
Abstract
It has been a challenge to design an optical see-through head-mounted display that is compact, lightweight, and stray-light-suppressed by using freeform optics. A new type of design based on freeform prisms is presented. The system consists of three optical elements and a micro-display. Two prisms serve as near-eye viewing optics that magnify the image displayed by the micro-display, and the other freeform lens is an auxiliary element attached to the main wedge-shaped prism to provide an undistorted see-through view of a real-world scene. The overall thickness of the optical system does not exceed 9.5 mm, and the weight is less than 9.8 g per eye. The final system is based on a 0.49-inch micro-display and features a diagonal field of view of 38°, an F/number of 1.8, with a 10 mm × 7 mm exit pupil diameter, and a 19 mm eye relief. The causes of stray light in the optical-mechanical system are investigated, and effective solutions or theoretical suppression of stray light are given. The freeform optical elements are successfully fabricated, and the system performance is carefully investigated. The results show that the performance of the optical see-through head-mounted display is adequate for practical applications.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Recently, mobile near-eye display technologies, especially augmented reality (AR) displays, have received increasing attention. The AR devices are widely used in various applications [1–4]. In the AR ecosystem, an optical see-through head-mounted display (OST-HMD), that can visually overlay computer-generated virtual scenes with views of a real-world scene, is one of the core components. Conveniently, the volume, weight, and image quality of the OST-HMD directly affect the user experience. These characteristics should be given special consideration during development.
However, it is difficult to design an OST-HMD that is compact, lightweight, good optical performance, and highly efficient at the same time. Various solutions have been tried to achieve these goals [5,6]. In previous development works, aspherical elements [7,8], waveguides [9,10], diffractive elements [11,12], holographic elements [13,14], or metasurface [15] have been used to achieve compact and lightweight systems. However, the stray light of the geometrical waveguide, the low light efficiency of the diffractive or holographic elements, and the limited processing methods of the metasurface, are urgent problems to be solved.
The development of freeform surface description and design methods has brought about a revolutionary leap for the advancement of OST-HMD technology. OST-HMD based on freeform prisms is compact in size and light in weight, leading to potentially wearable displays. Another fundamental difference from waveguides is that light efficiency of freeform prisms is no longer limited by the diffraction efficiency, or reflectance of partially reflective mirrors array (PRMA). Therefore, the light efficiency is higher than waveguides and general catadioptric systems. Ultraprecision diamond turning machining is an enabling technology for freeform optics, making the mass production of ultra-precise freeform optics possible [16,17]. Therefore, freeform prism can be an effective solution to achieve OST-HMD with high performance.
Due to the advantages of freeform technology, HMD with freeform optics can achieve a large field of view (FOV) and a small F-number [18]. Some previous research can avoid see-through distortion [19], achieve wearable off-axis HMD [20], or enable a vision correction capability [21]. However, these systems tend to be thick or complex, with the thickness of previous designs usually larger than 12 mm. Stray light analysis is also few to be mentioned. It is an urgent problem to design a compact and stray-light-suppressed OST-HMD with freeform optics.
In this paper, the design of an OST-HMD system consisting of three freeform optics is presented. The total thickness of the final system is only 9.5 mm. Using a 0.49-inch micro-display, the final system provides a diagonal FOV of 38°, an F/number (F/#) of 1.8, an eye relief of 19 mm, and an exit pupil diameter of 10 mm × 7 mm. The distortion of the virtual image light path and see-through light path is also well controlled, being only 0.60% and 0.46% respectively. The rest of the paper is organized as follows: In Section 2, previous research on the use of freeform prisms in HMD designs is reviewed and compared. In Section 3, the design process of the optical system, including prism structural constraint design, the optical performance evaluation, and tolerance analysis is described. In Section 4, the stray light sources of the system are analyzed in detail, and appropriate improvements are made to suppress them. Section 5 shows the test results of the prototype.
2. Previous research
The design, fabrication, and evaluation methods of the OST-HMD, which is based on a wedge-shaped prism, can date back to the 1990s [22]. Considering many factors, it is a good choice to design the prism with optical resin. The granular raw material can be injection molded and made into a prism with the desired shape. An unlimited number of copies can be produced from the mold. Mass production is therefore guaranteed. Unlike optical glass, the resin material is lightweight and impact resistant. Due to its high stability and excellent manufacturability, OST-HMDs have been designed and manufactured with wedge-shaped prisms. A number of commercial companies have launched HMDs or optical modules based on freeform prisms [23–26].
We have conducted many studies on OST-HMDs based on freeform prisms. In our previous research paper [18], a design based on two freeform optics was presented. The wedge-shaped freeform prism magnifies the image displayed on the micro-display, and the freeform lens is used to provide an undistorted see-through view. However, the shortcomings of the design in [18] still affect the wearing experience. The thickness of the lens is 12 mm in the design result. And the distortion of the virtual image light path is up to 12%, so digital distortion correction is required. Another problem is that the design of the auxiliary lens is complicated and difficult. Although the see-through distortion is only 1.4%, the image quality of the real scene decreases.
In the latter research paper [21], another OST-HMD which uses a combination of three freeform elements and two lenses was designed. A wedge-shaped auxiliary lens and two rotationally symmetric lenses were added for the virtual image light path. The auxiliary wedge-shaped lens was also used for the see-through light path. The additional optics improved the performance of the system. Compared to the system presented in [18], the distortion of the virtual and see-through light paths decreased to 0.6% and 0.4%, respectively, and the aspect ratio of the see-through view is not affected by the prisms. By changing the radius of surface close to the eyes, the optical system has the function of diopter adjustment. Users with myopia or hyperopia did not have to wear two pairs of glasses simultaneously. However, the thickness of the final system is 14 mm, which is not comfortable enough to wear. In addition, the display quality of OST-HMD is also affected by stray light, which has been little discussed in previous studies [18,21].
The design presented in this study consists of two freeform prisms and one freeform lens. The optical path development process is listed in Table 1. The previous designs [18,21] and the design presented here are compared in all aspects. To solve the problem of the thickness of the freeform prism OST-HMD, a secondary prism was added between the main wedge-shaped prism and the micro-display. This secondary prism further folds the light path, making the design thinner.
Table 1. Specifications of different freeform prism systems
The currently presented solution entails that the thickness is less than 9.5 mm, while the distortion of the virtual image light path is less than 0.60% without digital distortion correction. With this new design an exit pupil diameter of 10 mm × 7 mm is achieved. Although the FOV of the present design is slightly smaller than that of our previous designs, it has more potential to meet the usage requirements of mobile AR applications because of its compactness [5]. In addition, the sources of stray light are investigated and classified, and effective improvements are made to suppress the disorderly light.
3. Design process of the freeform prism HMD
As mentioned in Section 2, the thickness of the freeform-prism-based OST-HMD is usually unsatisfactory. The design parameters of the OST-HMD optical system, such as field of view, exit pupil size, eye relief, distortion, etc., are mutually restrictive. It is difficult to consider all parameters simultaneously. For example, in previous design I, a micro-display with a size of 0.61-inch was selected to achieve a larger field of view, but the distortion of the virtual and see-through light path was not well controlled. The design presented in this paper prioritizes the compactness of the OST-HMD optical system. A smaller thickness allows the appearance of entire optical module to be close to an ordinary eyeglass. It is an urgent problem to reduce the thickness of the design, while ensuring high optical performance for both the virtual image light path and the see-through light path. To solve this problem, a novel optical layout is presented in this paper as shown in Fig. 1.
Fig. 1. Layout of the final OST-HMD optical system. S1-S3 denote the optical surfaces of the main wedge-shaped prism E1, S4-S6 denote the optical surfaces of the secondary prism E2, S2’ and S7 denote optical surfaces of the auxiliary lens E3.
The overall specifications of the system are listed in Table 2. Based on various considerations of volume, resolution, availability, and cost, a 0.49-inch organic LED (OLED) display with a subpixel rendering (SPR) RGB resolution of 1920 × 1080 pixels and a 5.62 µm pixel size was selected. In the development of vision devices, especially binocular HMDs, a large exit pupil diameter (EPD) is essential to avoid vignetting or image loss due to the eye rotation. A large EPD is very adaptable to the interpupillary distances of different users and avoids the mechanical adjustment of the binocular optics interval. However, a large EPD affects the compactness, weight, and FOV of the optical system, greatly increasing the challenge of designing low F/# systems.
Table 2. Specifications of the HMD optical system
Taking these factors into account, the exit pupil diameter is set to be 10 mm × 7 mm. During the optimization process, other parameters, such as focal length, were constantly adjusted. After much thought and consideration, an 18.1 mm focal length is finally determined, resulting in an F/# of 1.81. According to the paraxial imaging formula Eq. (1), the FOV of the system is proportional to the diagonal size of the active display area of the micro-display. Using a micro-display with a size of 0.49-inch, a 38° diagonal full FOV is finally achieved.
where y is the diagonal size of the micro-display, and f is the effective focal length of the OST-HMD system.The system presented in this paper does not have the function of diopter adjustment. It is only suitable for users whose vision has been corrected with prescription glasses. Therefore, the system should have a large eye relief of 19 mm, allowing the user to wear glasses. According to our previous experience [21], the image distortion should be controlled within 0.60% to avoid digital distortion correction of the original image source. Modulation transfer function (MTF) is selected to evaluate the overall image sharpness [27]. The MTF value should be better than 30% at a spatial frequency of 30 lp/mm throughout the FOV.
As shown in Fig. 1. The final system includes three optical elements: the main wedge-shaped mirror E1, the secondary prism E2, and the auxiliary lens E3. The virtual image is projected into the human eye through the two prisms E1 and E2, and the auxiliary lens E3 is used to achieve optical see-through without distortion. The main wedge-shaped prism E1, which provides the main optical power of the system, enables the magnification of the image on the micro-display. The secondary prism E2 is placed between the micro-display and the main wedge-shaped prism to further fold the optical path, effectively reducing the thickness of the system, and achieving a compact design. On the other side of the main wedge-shaped prism, the auxiliary lens E3 is used to correct the distortion of see-through optical path.
In this system, the main wedge-shaped prism E1 and the secondary prism E2 each consist of three optical surfaces. The auxiliary lens, E3, has only two optical surfaces. Reverse ray tracing is used when modelling the virtual optical light path in the CODE V optical software. Starting from the pupil position where the human eye is located, the light passes through S1-S3 of the main prism E1, and S4-S6 of the secondary prism E2 and finally reaches the micro-display. The see-through light path is modeled with forward ray tracing. The light from the real scenes successively passes through S7, S2’ of the auxiliary lens E3, S2 and S1 of main prism E1, and finally enters the human eyes.
In the previous design [18], the optical surface near the human eye had a freeform shape, which greatly improved the imaging quality of the virtual optical path. The S1 surface is not only a transmission surface, but also provides total reflection for the system, and the reuse of this optical surface effectively folds the light path. As this surface takes on more functions, it must be described by a freeform surface with more degrees of freedom. The problem, however, is that this is not conducive to optimizing the see-through light path, especially controlling see-through distortion. It is easy to create a large difference in focal length between the X and Y directions, resulting in an anamorphic distortion ratio for the see-through distortion, making the image of the see-through light path look wider than the original image. In [21], the problem is solved by replacing the expression of the optical surface near the human eye with a spherical surface. The distortion of the see-through light path is well corrected, and the anamorphic ratio is eliminated. Therefore, S1 of the main prism E1 in this design is also set as a spherical surface to achieve the good see-through effect.
The design process of this system is divided into three parts including establishing structural specifications, performing optical performance analysis, and performing tolerance analysis. Each of these parts are presented in more detail.
3.1 Structural constraints
In the virtual image light path, the physical structure affects the optical performance and manufacturability of the final design. It is important to control the structure of the main freeform prism E1 and the secondary prism E2, to ensure that rays from all fields pass through the optical system correctly and reach the user’s eyes without errors. It should be noticed that no vignetting factors are added to the system. As for the manufacturing requirements, the three surfaces of E1 and E2 should each form a shape with manufacturability. And the value of the center and edge thickness of these prisms should be appropriate. According to our previous experience, the minimal edge thickness should be greater than 0.7 mm. For the sake of the compactness of the system, the center thickness should be as small as possible. Since the system has the plane symmetry around the YOZ plane, it is only necessary to focus on the correctness of the structure in the YOZ plane. The methods for optimizing the physical structure of the wedge-shaped freeform prism E1, and the secondary prism E2, are as follows.
When modeling the system in optical design software, the position of each surface is described in the same global coordinates, with the origin of the global coordinate system located at the center of the pupil, as shown in Fig. 2. In general, the marginal ray determines the physical shape of the prism. The marginal rays and the intersection points of the marginal rays with the optics are defined to construct the essential boundary conditions for a positive optical path and appropriate lens thickness. The rays and points selected in this paper are shown in Fig. 2, where R1’ is the lower marginal ray of the minimum Y-direction field, R2 is the upper marginal ray of the maximum Y-direction field, and R3 is the central ray of the central Y-direction field. R3 coincides with the optical axis of the system. Before reaching the micro-display, the ray R1’ intersects with the main prism E1 at points Pa1, Pa2, Pa3, and intersects with the secondary prism E2 at points Pa4, Pa5, respectively. Ray R2 intersects with the main prism E1 at points Pb1 and Pb2, and intersects with the secondary prism E2 at points Pb3, Pb4, respectively. Ray R3 intersects with E1 and E2 at points Pc1 and Pc2, respectively. L1-L4 represents different lines coinciding with the different segments of the marginal rays. By constraining the positional relationship between the marginal rays and each point, the rationality of the physical structure of the virtual image light path can be fully guaranteed.
Fig. 2. Optical paths of the rays of different object fields for the virtual image. R1’, R2, R3 denote characteristic rays. Pa1−Pa5, Pb1−Pb4, and Pc1−Pc2 are the characteristic points. L1-L4 are straight lines coinciding with different segments of marginal rays.
For E1, the coordinate value of Pa1 along the Z-axis direction must be less than Pa2. Then, the position of the point Pb1 must be below the line L1, and the position of the point Pb2 and Pa3 must be above the line L2. For E2, Pa4, Pb3 should be located below the line L3, and Pa5, Pb4 should be located above the line L4. To ensure that there is no overlap between E1 and E2, you must also ensure that the coordinate value of Pa3 in the Y direction is less than Pa4, the coordinate value of Pb2 in the Y direction is less than Pb3, and the coordinate value of Pc1 along the Y-axis is less than Pc2.
In the commercial optical design software CODE V, constraints on the coordinate values of points can be easily set using real ray tracing data. The built-in JMRCC function can be used to implement the above constraints in terms of point and line [21]. According to the above discussion, the constraints for maintaining the physical structure of the main prism E1 and the secondary prism E2 can be given as follows:
Similarly, for the see-through light path, the physical structure of the auxiliary lens E3 is constrained by controlling the coordinate values of the critical points. As shown in Fig. 3, four points are labeled. The position of Pa2’ corresponds to Pa2 mentioned in Fig. 2, and the position of Pb1’ corresponds to that of Pb1. To ensure that the edge thickness of E3 meets the manufacturing requirements and the overall thickness of the system is minimized, the difference between the coordinate values of points Pb5 and Pb1’ in the Z direction can be equal to 0.7 mm.
Fig. 3. Optical paths of the rays of different object fields for the see-through image. R1’ and R2 are the same as noted in Fig. 2. Pa2’ and Pa6 are the intersection points of ray R1’ with the surfaces of E3. Pb1’ and Pb5 are the intersection points of ray R2 with the surfaces of E3.
In the process of optimization, the thickness of the system is one of the important points to consider, while the distortion and the lateral color aberration are also taken into account to obtain the final system. All prisms are made of optical resin materials, which can be mass produced by injection molding. The materials of E1 - E3 are E48R from ZEONEX compony. Chromatic aberration cannot be completely corrected because all materials of the optical elements are the same, electronic methods can be considered if required [28]. To achieve the display function, the lens surfaces should be coated appropriately. As shown in Fig. 1, S2 must be coated with half-mirror coating, and S5 with a reflective coating. When the light from the micro-display first passes through S1, the total internal reflection condition is satisfied, so S1 has no reflective coating. Other surfaces can be coated with anti-reflective film to reduce the stray light. Although the OST-HMD system based on freeform prisms employs a half-mirror coating, the light only passes through this surface once. When the beam splitting ratio is 1, the light efficiency of the whole system can theoretically reach 50%, which is much higher than geometric waveguide.
After all optimization steps are completed, the final system is obtained, the structure of which is shown in Fig. 4. The total thickness of the final system is reduced to 9.5 mm after complete optimization, and weighs less than 9.8 g. Compared with the previous freeform HMD design, the thickness has been significantly reduced.
To ensure the imaging performance of the virtual light path, S2, S3, S5 are described using freeform expressions, and S4, S6 are even asphere. To ensure that there is no deformation distortion for the see-through light path, even asphere had been used to describe S1 and S7. But the shape at the edge of the clear aperture is difficult to control. Besides, as the system was optimized, its asphericity in the clear aperture remains low. Finally, the spherical surface was used for S1 and S7. S2 and S2’ are similar, so that E1 and E3 can be bonded with glue. The equation for the freeform surface is shown in Eq. (3).
3.2 Image quality analysis
For the virtual image light path, Fig. 5(a) shows the polychromatic modulation transfer function (MTF) curves of selected fields. It is evaluated up to the spatial frequency of 30 lp/mm with an exit pupil of 4 mm. The MTF curves of 9 fields, including the center and marginal fields, are provided. In the optimization process, the weights of different field points are adjusted to make the image quality in the full field of view as balanced as possible. All the MTF values are higher than 0.3 at 30 lp/mm.
Fig. 5. Optical performance analysis of the final system for the virtual image light path. (a) The values of MTF are higher than 0.30 for all fields at 30 lp/mm. (b) RMS spot diameter for full pupil. (c) The distortion grids.
Fig. 6. MTF plot and distortion for the see-through light path. (a) The MTF values are higher than 0.78 for all fields at 50lp/mm. (b) The maximum distortion value is 0.46%.
Figure 5(b) and (c) show the other evaluation results of the imaging quality of the virtual image light path for the final system. The average RMS spot diameter of the full field of view is only 24 µm when the pupil diameter is 10 mm × 7 mm. At the same time, the distortion of the system was well controlled, and the maximum distortion rate of the whole field of view is only 0.60%.
MTF plot and distortion grid for the see-through light path is shown in Fig. 6, the MTF values for all fields are higher than 0.78 at 50 lp/mm. Since the optical surface near the human eye is a spherical surface, the distortion of see-through light path is well controlled. The maximum optical distortion is only 0.46%.
3.3 Tolerance analysis
Tolerance analysis is an essential part of optical design because it is closely related to optical fabrication, alignment, and cost. By carefully evaluating the interaction between parameter-variant and image-effect and paying particular attention to the negative and positive aspects of these interactions, overdesign can be avoided and tight tolerances only where necessary are assigned. Such an analysis helps determine the acceptable limits for various fabrication and alignment errors.
Unlike the rotationally symmetric system, the tolerance analysis of the freeform system is more complex, and each surface has many optical parameters that affect the result. In previous work [29], the process of tolerance analysis of a system with freeform was presented in detail. For the design presented in this paper, an overall tolerance analysis was performed for the virtual image light path using the tolerance items and values listed in Table 3. The position and the tilt of the micro-display are defined as the compensators.
Table 3. Tolerance Items
Among the various tolerance errors, the thicknesses of the surfaces are insensitive to the system performance. A 20 µm deviation is acceptable for surface displacements along each axis. The delta sag at a clear aperture (DLS) of S2 and S5 is more sensitive than the others, and it should be controlled within 4 µm.
Figure 7 plots the overall analysis results of the probable changes in MTF values with different cumulative probabilities. For the largest probable MTF change in with a cumulative probability of 90%, the MTF values of sampled fields at 30 lp/mm is higher than 0.2 for both radial and tangential azimuths.
Fig. 7. Probable change of MTF value with different cumulative probabilities for overall tolerance analysis using the tolerance values listed in Table 4. F1-F9 denote the sampled fields as shown in Fig. 5. (a) Result for radial azimuth. (b) Result for tangential azimuth.
4. Stray light analysis of the freeform prism HMD
Although OST-HMDs based on freeform prisms have certain advantages over other technologies, there are fewer studies involving its stray light analysis. Analyzing and suppressing the stray light caused by undesirable reflection/refraction can further improve the user's experience. Since it is a visual imaging system, not only the stray light entering inside the effective FOV, but also outside the effective FOV will be observed. The stray light inside the effective FOV degrades the image contrast and the display quality, while the stray light outside the effective FOV distracts the user’s attention. Therefore, it is necessary to simulate and suppress these two types of stray light.
In this study, the stray light in the virtual image light path was investigated in detail. To make the simulation results more realistic, an optical-mechanical system is modeled in the commercial optical software LightTools. Besides the freeform optics and micro-display, a holder is integrated into the system as shown in Fig. 8(a). The holder fixes the micro-display, the main wedge-shaped prism E1, and the secondary prism E2 together. Different materials and optical properties are assigned to the individual components depending on the situation.
Fig. 8. (a) The OST-HMD system is modeled in LightTools. The whole system includes freeform optics, micro-display, and holder. (b) For the virtual image, the ideal ray path is Microdisplay-S6-S5-S4-S3-S1-S2-S1-Pupil (Perfect lens)-Receiver. For simplicity, the holder is hidden. (c) The non-optical surface with diffuser property is denoted by N1.
A complete system consists of at least a light source, an optical lens and support structure, and a receiver in LightTools. For the OST-HMD system presented here, the light-emitting surface of the micro-display is set as the surface light source, and the wavelengths of the light source are set to 457 nm, 525 nm, and 633 nm. The viewing angle of the micro-display is about 70°, and the true illumination distribution along the X and Y axis is shown in Fig. 9(b). The illumination distribution of the light source in the simulation model is set according to the real luminous properties of the micro-display.
Fig. 9. (a) Top view of the micro-display. (b) Illumination distribution of the selected micro-display along the X and Y axis.
Three freeform optics and a lens holder make up the optical-mechanical part. As a visual system, the image projected by the OST-HMD should be received by the human eye. To simulate the crystal lens of the human eye, an ideal lens is placed on the pupil surface, and a receiver is placed in the focal plane of the ideal lens to simulate the retina of the human eye. When the stray light within the effective FOV is analyzed. According to Eq. (1), the ideal dimension of the receiver should be equal to the product of the ideal lens focal length and the tangent of the half FOV of the OST-HMD system. Under this condition, the image displayed in the active area of the micro-display can accurately fill the receiver. When simulating the stray light outside the effective FOV, the size of the receiver should be larger than the ideal dimension.
Setting appropriate optical properties for the elements is the most important step before starting the simulation. Under ideal conditions, S5 is set with total reflection properties. The ray tracing mode of S2 is split rays where 50% are for transmittance and 50% for reflectance. The remaining optical surfaces are set with the transmission property, the transmittance is 95% and the reflectance is 5%. The non-optical surfaces of the optical elements are set with a simple scattering property. Without any surface treatment process, the reflectance of the scattering is set to 70%, and transmittance is set to 30%. The holder’s surfaces are set with a simple scattering property, and the ray tracing mode is set to be reflected where the reflectance is 90% and the absorption is 10%. The ray tracing power threshold for the Monte Carlo ray trace is set to 0.1%. If the power of any Monte Carlo ray, relative to its starting power, falls below this threshold, its ray tracing is terminated.
For the presented OST-HMD system, the propagation of the ideal ray path is shown in Fig. 8(b). The light emitted from the micro-display passes through the optical surfaces of E1 and E2 in sequence and exits the optical system from the S1. Then the light is converged by the ideal lens and imaged on the receiver. The ideal ray path can be expressed as follows: Microdisplay-S6-S5-S4-S3-S1-S2-S1-Pupil (Ideal lens)-Receiver. The upper non-optical surface of E2 is labeled as N1.
4.1 Stray light within the effective FOV
Inside the effective FOV, the light takes a different path from the ideal ray path is shown in Fig. 8(b) is referred to as stray light. To analyze the stray light inside the effective FOV, a white dot array is displayed in the micro-display. The total number of rays tracked is set to 100 million. Run the Monte Carlo ray tracing and obtain the preliminary simulation results. All ray paths are shown in Fig. 10(a).
Fig. 10. (a) All ray paths that can be received by the receiver are shown. (b) The illumination distribution on the receiver when a white dot array image displayed in the micro-display.
A ray path is the path followed by one or more rays when they intersect a unique sequence of surface zones in LightTools. After a simulation, ray paths can be displayed in different colors in the design view so that the paths can be distinguished. The power of each path can also be examined. Ray paths can be a valuable diagnostic tool when trying to analyze stray light or unexplained illumination patterns. To detect the specific ray path from the multiple paths in the system, ray path filtering capabilities can be helpful.
Using ray path filters in a proper way, stray light paths can be selected. The number, strength, size, and sources of stray light can then be determined for the proposed OST-HMD system. Then, the system is evaluated to find the optical surfaces that contribute to stray light so that baffle building or other effective improvements can be made. In addition, the ray paths are evaluated to determine which non-optical surfaces need to be treated to reduce the scattered light from becoming incident on the receiver.
It is difficult to directly determine the stray light paths from the result shown in Fig. 10(a). The ray path filtering functions are used to mask out the ideal ray paths and highlight the stray light. When the relative ray power threshold is 0.1%, no stray light path is found. Based on the illumination distribution on the receiver as shown in Fig. 10(b), no obvious ghost image can be seen. Therefore, the effect of stray light on the effective FOV can be ignored.
4.2 Stray light outside the effective FOV
The unexplained illumination patterns are attributed to stray light outside the effective FOV. A completely white image is displayed on the micro-display. Also, the total number of rays tracked is set to 100 million. The illumination distribution on the receiver is shown in Fig. 11(a) after the Monte Carlo ray tracing is completed. The yellow dashed box denotes the ideal illumination pattern of the effective FOV, which has a similar shape to the active area of the micro-display. The maximum irradiance value of the ideal illumination pattern in the receiver is normalized. It is also assumed that the rest of the visible illumination pattern is thought to be generated by disordered stray light.
Fig. 11. The illumination distribution on the receiver, and the propagation paths of the stray light. (a) The yellow dot box indicates the illumination distribution within the effective FOV. Others are considered as the stray light. According to the ray path, the stray light is divided into five types. (b) The green colored ray shows the path of stray light type I. Unexpected total internal reflection in S4 has occurred. The red colored ray shows the path for stray light type II. The scattering happened in the non-optical surface N1. The blue colored ray shows the path for stray light type III. Unexpected total internal reflection in S6 has occurred.
To make clear the causes of stray light in different positions, it is divided into five types according to the ray path, as shown in Fig. 11(a). The green box denotes the stray light that is below the ideal illumination pattern. This is also the most severe type of stray light, with a comparable brightness to the ideal illumination pattern. The relative irradiance value of stray light type I is about 0.56, and it has the greatest impact on the working process of the system. The red box indicates the stray light at the top. The relative irradiance value of the type II stray light is about 0.07. The blue boxes indicate the stray light located on both sides of the type II and are symmetrically distributed. The relative irradiance value of the stray light type III is also about 0.07. The shape of stray light marked with the orange box is regular and corresponds to the non-optical surfaces of the holder. Its relative irradiance value is about 0.05. The purple box indicates the weakest stray light and is located on either side of the ideal illumination pattern. The relative irradiance value of the stray light type V is about 0.04.
By intuitive observation, it can be seen that the stray light type IV is caused by scattering from the surfaces of the holder. This type of stray light can be suppressed or even eliminated by improving the holder's surface treatment process and reducing the reflectance of the diffuse reflection. For the type V, the relative illumination is less than 4%, and is not easily noticeable. It is mainly generated by the scattering of the non-optical surfaces of E2 and the holder. Therefore, adjusting the optical properties of the non-optical surfaces can suppress the stray light type V.
To embody the stray light path succinctly and intuitively, the schematic diagrams of the other three types of stray light paths are shown in Fig. 11(b). The ray path colored in green is the first type of stray light. When the light exits the micro-display with a larger angle and crosses the optical surface S6, it does not reach S5 directly, but an unexpected total internal reflection occurs at the optical surface S4. This results in a ghost image that is below the ideal image. Since the viewing angle of micro-display cannot be controlled, a baffle could be used to block the path of this stray light.
In Fig. 11(b), the ray path colored in red shows the stray light type II, that occurs when light is transmitted in the E2. The cause of this type of stray light is that the light first undergoes a scattering on the non-optical surface N1, and then directly reaches the S4 without passing through the S5. This type of stray light can be reduced by using a suitable surface treatment process for the non-optical surface N1.
The ray path colored in blue shows the stray light type III. An additional total internal reflection takes place in the optical surface S6 after the ray has been reflected by the optical surface S5. A baffle could be used to block the path of this type of stray light, while the baffle can also block the stray light type II.
4.3 Improvements and result
Based on the above analysis results, two baffles are inserted into the gap between the optical prisms E1 and E2 to block the stray light path as shown in Fig. 12. This effectively eliminates the stray light type I, II and III. The non-optical surface N1 of E2 is treated by spraying matt paint to reduce stray light caused by scattering. In this way, the stray light type II can be further suppressed. The surfaces of the holder are sprayed with matting paint to reduce the stray light type IV and V caused by scattering. In addition, optical surfaces with transmission properties are coated with anti-reflective coatings. Based on the level that can be reached, the coating requirements were developed, and the optical properties were set to surfaces in the simulation model.
Fig. 12. For stray light type I, II and III, two baffles are placed in the gap between E1 and E2 to block the stray light path.
After making these improvements, the optical properties of the model are adjusted in LightTools. The reflectance of the simple scattering is set to 30%, the transmittance is set to 0%, and the absorption is set to 70% for the non-optical surfaces of the optics. For the holder’s surfaces, the reflectance is set to 20% and the absorption to 80%. The transmittance is 99% and the reflectance is 1% for optical surfaces with transmission properties.
The improved system is shown in Fig. 13(a). Two baffles have been added, and the holder is hidden. To verify the effectiveness of the above improvements, the modified system was simulated again. The final illumination distribution is shown in Fig. 13(b), and it can be seen that the five types of stray light are almost eliminated. The relative irradiance value of the stray light is less than 2%, which is satisfactory.
Fig. 13. (a) Improved system with baffles, the holder is hidden. (b) Simulation result for the improved system. Five types of main stray light were greatly reduced.
Even though the improved system has greatly suppressed the stray light, there is a problem to be noted. As shown in Fig. 13(a), the baffles block part of the ideal ray path, which means that vignetting occurs for the marginal fields. The maximum vignetting factors are about 0.25 for the lower fields, and 0.1 for the upper fields. The vignetting reduces the size of eye box. This is a major problem that needs to be solved in the next generation of the design.
5. Prototype and experimental results
The system was fabricated successfully. An exploded view of the module is shown in Fig. 14(a). The overall appearance is shown in Fig. 14(b). The main prism and the auxiliary lens were glued with ultraviolet glue, and the holder fixed to the micro-display, the main wedge-shaped prism, and the secondary prism together. The total weight of the single display module is 9.8 g, which meets the lightweight requirements of the OST-HMD.
Fig. 14. Components and prototype of the optical system. (a) Exploded view showing all elements of the system. (b) Overall appearance of the prototype.
To demonstrate the quality of the virtual and see-through light path image of the OST-HMD, simple tests are performed. A commercial camera is used to simulate a human eye and record the projected image transmitted through the OST-HMD system. The camera was placed in front of the optical module. The stop aperture and optical axis of the camera were matched with the pupil and viewing axis of the optical module.
An image resolution test chart shown in Fig. 15(a) was displayed on the micro-display. Figure 15(b) shows a clear and undistorted image, expressing the good virtual imaging performance of the optical module. Figure 15(c) shows the result of the fusion of a virtual clock with a real clock, demonstrating the potential applications of this module in AR.
Fig. 15. Test results of the optical prototype. (a) The input image displayed on the micro-display when testing the performance of the system. (b) The output image captured by the camera at the exit pupil of the OST-HMD system. (c) The result of the fusion of the virtual clock and the real electronic clock.
It is concluded that the stray light outside the field of view was tolerable based on the experimental results shown in Fig. 15 and feedback from the tester's experience. Therefore, baffles are not included in the prototype for the time being. The stray light problem should be further addressed before the system is put to commercial use.
6. Conclusions
In this paper, a compact, lightweight, and stray-light-suppressed OST-HMD optical system is proposed. The total thickness of the finished system is only 9.5 mm. The overall optical system achieves a diagonal FOV of 38° and an F/# of 1.8, with a 10 mm × 7 mm exit pupil diameter and a 19 mm eye relief. Folding the light path with freeform prisms reduces the overall thickness of the final system to 9.5 mm. All optical elements in this system are made of plastic materials, and the total weight including micro-display is only 9.8 g per eye. Thus, the optical system is very compact and lightweight. The methods and constraints for the design and optimization of the optical system were presented in detail. The performance of the optical system was analyzed. Tolerance and stray light analyses were performed to obtain an outstanding experimental prototype. Effective improvements were made to reduce the relative irradiance of the stray light to less than 2%. This compact optical see-through HMD could be a potential opportunity for the widespread adoption of near-eye display technology.
The baffles that are added to suppress the stray light block part of the ideal ray path, generating vignetting for the marginal fields. Hence, the size of eye box of the OST-HMD is small than desired. In the next generation of design, it should be carefully considered how to eliminate the stray light path without affecting the ideal ray path. For example, the air gap between S3 and S4 can be eliminated by combining E1 and E2 into a single optical element. As a result, the stray light type I caused by total internal reflection at S4 will be avoided.
Funding
National Key Research and Development Program of China (2021YFB2802100); Young Elite Scientist Sponsorship Program by CAST (2019QNRC001).
Acknowledgments
We would like to thank Synopsys for providing the educational license of CODE V and LightTools.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. H. Hua, L. D. Brown, and C. Gao, “Scape: supporting stereoscopic collaboration in augmented and projective environments,” IEEE Comput. Grap. Appl. 24(1), 66–75 (2004). [CrossRef]
2. A. Sisodia, A. Riser, M. Bayer, and J. McGuire, “Advanced Helmet Mounted Display (AHMD) for simulator applications,” Proc. SPIE 6224, 62240O (2006). [CrossRef]
3. L. Jensen and F. Konradsen, “A review of the use of virtual reality head-mounted displays in education and training,” Educ. Inf. Technol. 23(4), 1515–1529 (2018). [CrossRef]
4. H. Li, X. Zhang, G. Shi, H. Qu, Y. Wu, and J. Zhang, “Review and analysis of avionic helmet-mounted displays,” Opt. Eng. 52(11), 110901 (2013). [CrossRef]
5. O. Cakmakci and J. P. Rolland, “Head-Worn Displays: A Review,” J. Disp. Technol. 2(3), 199–216 (2006). [CrossRef]
6. D. Cheng, Q. Wang, Y. Liu, H. Chen, D. Ni, X. Wang, C. Yao, Q. Hou, W. Hou, G. Luo, and Y. Wang, “Design and manufacture AR head-mounted displays: A review and outlook,” Light Adv. Manuf. 2(3), 336–355 (2021). [CrossRef]
7. Z. Zheng, X. Liu, H. Li, and L. Xu, “Design and fabrication of an off-axis see-through head-mounted display with an x–y polynomial surface,” Appl. Opt. 49(19), 3661–3668 (2010). [CrossRef]
8. A. Wilson and H. Hua, “Design and demonstration of a vari-focal optical see-through head-mounted display using freeform Alvarez lenses,” Opt. Express 27(11), 15627–15637 (2019). [CrossRef]
9. Q. Wang, D. Cheng, Q. Hou, L. Gu, and Y. Wang, “Design of an ultra-thin, wide-angle, stray-light-free near-eye display with a dual-layer geometrical waveguide,” Opt. Express 28(23), 35376–35394 (2020). [CrossRef]
10. L. Gu, D. Cheng, Q. Wang, Q. Hou, and Y. Wang, “Design of a two-dimensional stray-light-free geometrical waveguide head-up display,” Appl. Opt. 57(31), 9246–9256 (2018). [CrossRef]
11. C. Pan, Z. Liu, Y. Pang, X. Zheng, H. Cai, Y. Zhang, and Z. Huang, “Design of a high-performance in-coupling grating using differential evolution algorithm for waveguide display,” Opt. Express 26(20), 26646–26662 (2018). [CrossRef]
12. T. Levola, “Novel Diffractive Optical Components for Near to Eye Displays,” Dig. Tech. Pap. - Soc. Inf. Disp. Int. Symp. 37(1), 64–67 (2006). [CrossRef]
13. I. Kasai, Y. Tanijiri, T. Endo, and H. Ueda, “A Practical See-Through Head Mounted Display Using a Holographic Optical Element,” Opt. Rev. 8(4), 241–244 (2001). [CrossRef]
14. J. Han, J. Liu, X. Yao, and Y. Wang, “Portable waveguide display system with a large field of view by integrating freeform elements and volume holograms,” Opt. Express 23(3), 3534–3549 (2015). [CrossRef]
15. G.-Y. Lee, J.-Y. Hong, S. Hwang, S. Moon, H. Kang, S. Jeon, H. Kim, J.-H. Jeong, and B. Lee, “Metasurface eyepiece for augmented reality,” Nat. Commun. 9(1), 4562 (2018). [CrossRef]
16. F. Fang, X. Zhang, A. Weckenmann, G. Zhang, and C. Evans, “Manufacturing and measurement of freeform optics,” CIRP Ann. 62(2), 823–846 (2013). [CrossRef]
17. J. P. Rolland, M. A. Davies, T. J. Suleski, C. Evans, A. Bauer, J. C. Lambropoulos, and K. Falaggis, “Freeform optics for imaging,” Optica 8(2), 161–176 (2021). [CrossRef]
18. D. Cheng, Y. Wang, H. Hua, and M. M. Talha, “Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism,” Appl. Opt. 48(14), 2655–2668 (2009). [CrossRef]
19. S. J. Robbins, “Three piece prism eye-piece,” U.S. patent 8, 937,771 B2 (January 20, 2015).
20. L. Wei, Y. Li, J. Jing, L. Feng, and J. Zhou, “Design and fabrication of a compact off-axis see-through head-mounted display using a freeform surface,” Opt. Express 26(7), 8550–8565 (2018). [CrossRef]
21. D. Cheng, J. Duan, H. Chen, H. Wang, D. Li, Q. Wang, Q. Hou, T. Yang, W. Hou, D. Wang, X. Chi, B. Jiang, and Y. Wang, “Freeform OST-HMD system with large exit pupil diameter and vision correction capability,” Photonics Res. 10(1), 21–32 (2022). [CrossRef]
22. H. Hoshi, N. Taniguchi, H. Morishima, T. Akiyama, S. Yamazaki, and A. Okuyama, “Off-axial HMD optical system consisting of aspherical surfaces without rotational symmetry,” Proc. SPIE 2653, 234–242 (1996). [CrossRef]
23. https://www.olympus-global.com/en/news/2000b/nr000831fmd250e.html.
24. https://www.rockwellcollins.com.
25. https://www.emagin.com/products/application-note.
26. http://www.nedplusar.com/en/index.
27. J.-Y. Wu and J. Kim, “Prescription AR: a fully-customized prescription-embedded augmented reality display,” Opt. Express 28(5), 6225–6241 (2020). [CrossRef]
28. https://xinreality.com/wiki/Chromatic_aberration#cite_note-2.
29. Q. Wang, D. Cheng, Y. Wang, H. Hua, and G. Jin, “Design, tolerance, and fabrication of an optical see-through head-mounted display with free-form surface elements,” Appl. Opt. 52(7), C88–C99 (2013). [CrossRef]
