Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Automated design of freeform imaging systems for automotive heads-up display applications

Open Access Open Access

Abstract

The freeform imaging system is playing a significant role in developing an optical system for the automotive heads-up display (HUD), which is a typical application of augmented reality (AR) technology. There exists a strong necessity to develop automated design algorithms for automotive HUDs due to its high complexity of multi-configuration caused by movable eyeballs as well as various drivers’ heights, correcting additional aberrations introduced by the windshield, variable structure constraints originated from automobile types, which, however, is lacking in current research community. In this paper, we propose an automated design method for the automotive AR-HUD optical systems with two freeform surfaces as well as an arbitrary type of windshield. With optical specifications of sagittal and tangential focal lengths, and required structure constraints, our given design method can generate initial structures with different optical structures with high image quality automatically for adjusting the mechanical constructions of different types of cars. And then the final system can be realized by our proposed iterative optimization algorithms with superior performances due to the extraordinary starting point. We first present the design of a common two-mirror HUD system with longitudinal and lateral structures with high optical performances. Moreover, several typical double mirror off-axis layouts for HUDs were analyzed from the aspects of imaging performances and volumes. The most suitable layout scheme for a future two-mirror HUD is selected. The optical performance of all the proposed AR-HUD designs for an eye-box of 130 mm × 50 mm and a field of view of 13° × 5° is superior, demonstrating the feasibility and superiority of the proposed design framework. The flexibility of the proposed work for generating different optical configurations can largely reduce the efforts for the HUD design of different automotive types.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Augmented reality (AR) is a technology that enhances people’s perception of the physical world with virtual digital information [13]. With the development of AR technology, the head-up display (HUD) system, which was used in aviation aircraft in the last century, has also developed rapidly and is gaining its popularity in the automotive field. The automotive HUD system is typically designed by an off-axis three-mirror system consisting of an irregular windshield and two freeform surfaces compensating aberrations caused by the windshield. The asymmetry of the windshield and the multiple configurations of pupil location make it burdensome to balance the optical aberrations of the AR-HUD system. Traditional trial and error approach to design an HUD optical system for any specific windshield requires a large amount of human effort, and the design performance is quite dependent on designer’s experience. Due to its high complexity, the design process may probably fall into a locally optimized state, which in most cases will fail to satisfy the design requirements. Considering the large amount of windshield types, it is strongly necessitated to propose an automated design method for the AR-HUD optical system with any specific windshield.

With the fast development of aberration theory [4,5] and optical manufacturing technologies [6,7], freeform surfaces have been applied in many different areas, such as LED illumination [8,9], head-mounted displays [10,11], hyperspectral systems [12], remote sensing systems [13], and so on. Many scholars have explored various automated design methods for freeform surfaces, such as the Simultaneous Multiple Surface (SMS) method [14,15] and partial differential equations (PDEs) method [16,17]. However, all these methods are only feasible for designing systems with a single or small field of view (FOV), as well as a limited number of freeform surfaces. The First-time right design method presented by Duerr [18,19] can design full FOV freeform surface system directly, which needs to solve complex power series equations. In recent years, Yang et al. proposed an iterative design method for free-form surfaces, i.e., construction-iteration (CI) method [2022]. The method directly carries out point cloud calculation on sampled points, where the calculation of each surface is relatively independent, and thus the number of freeform surfaces is relatively relaxed. As a result, the design of most optical systems with multiple freeform surfaces can be realized by CI method. However, for a system with wide FOV and multiple pupil coordinates, the footprint corresponding to each pupil on optical surface is sparse. Current direct design methods for freeform surfaces cannot tackle with the HUD system with such complex multi-configurations well, where problematic designs or discontinuous surfaces can probably be generated.

Lots of research have been conducted to develop the AR-HUD system with wide FOV as well as multiple configurations with various pupils. Peng presented a HUD system based on an optical waveguide [23]. Due to the limited installation space, an off-axis optical system configuration is widely adopted to design HUDs with compact structures. For off-axis designed optical system, additional astigmatism and coma will be introduced due to the off-axis components and aberration distributions over pupils will become much more complex. Moreover, it is difficult to correct asymmetric aberrations induced by windshields using traditional rotationally symmetric surfaces. Freeform surfaces are defined as non-rotational symmetric surfaces, which can effectively balance asymmetric aberrations. Wei presented a small-volume HUD based on the lateral distribution of a single freeform surface [24]. Gu presented an aircraft HUD system based on an off-axis reflective configuration with four freeform surfaces [25]. An initial configuration with good performance is very important for system design, which can greatly reduce the design difficulty for engineers. For traditional off-axis reflective systems, the optimal initial configuration is retrieved by calculating a spherical coaxial system with no primary aberrations under paraxial conditions [26,27]. However, the AR-HUD optical system consists of a fixed freeform windshield, the design process of paraxial method is rather tedious and is not suitable for HUD. To the best of our knowledge, there is little research on developing direct design methods for automotive AR-HUD systems up to now.

In this paper, we propose an automated design method of freeform imaging systems for the automotive AR-HUD optical systems with wide FOV as well as multiple pupil coordinates, which can be also feasible for any other optical system with wide-field or multiple pupils. In the design process, we firstly apply XY polynomials or Zernike polynomials to fit the irregularly shaped windshield of the AR-HUD system. We trace all the characteristic rays on the full FOV for each sampled pupil inside the whole eye-box, and then apply CI method to construct the initial structures of freeform surfaces with only low-order polynomial terms. With the initial structure, the final design with extraordinary performance for all pupils on the full FOV can be iteratively optimized by tracing all rays over pupils. To demonstrate the feasibility the algorithm of this paper, we first present the design of a common two-mirror HUD system with longitudinal and lateral structures are presented with high optical performances. We then analyze the imaging performance and volume of several typical HUD off-axis layouts. Finally, we select the layout scheme that is most suitable for the future dual-mirror HUD. The most suitable layout scheme for future two-mirror HUD is selected.

2. Design principles and methods

AR-HUD system is a typical multi-pupil design that regulates rays from different pupils on the whole eye-box for its full FOV. In this section, we propose an automated design method to generate a feasible initial structure for the wide-field and multi-pupil AR-HUD system. The method consists of two mains procedures. Firstly, construct an initial structure for the AR-HUD by tracing characteristic rays for each sampled pupil inside the eye-box over the full FOV. Secondly, iteratively redesign the AR-HUD system by tracing all sampled light rays over all sampled pupils inside the eye-box.

2.1 Construction method of initial structures

To design an AR-HUD optical system with superior image quality over the whole eye-box, we should firstly sample all pupil positions inside the whole eye-box. Figure 1 show that the eye-box with width P and height Q is divided into S sampled pupil positions, where each eye-pupil is regarded as an 8 mm circular aperture. For each pupil position sampling radial sampling, the angle step of sampling is θ, and the length step under each sampling angle is d. For each sampled pupil, N characteristic rays are traced for each sampled single field by polar coordinate sampling over the pupil as shown in Fig. 1. The whole FOV of the AR-HUD system is sampled into K equivalently spaced fields. As a result, the total number of characteristic rays that requires to be traced is R = K × N × S.

 figure: Fig. 1.

Fig. 1. Sampling of characteristic rays in the entrance pupil plane.

Download Full Size | PDF

Since the first surface in the AR-HUD optical system is the windshield, which is provided by automotive vendors, the way to construct the freeform surfaces in AR-HUD systems will be different from that in traditional off-axis three-mirror systems as shown in Fig. 2. For traditional off-axis three-mirror system, the order of optical surface construction starts from the surface nearby the image plane and is reverse to the ray tracing sequence. While for the AR-HUD system, we should firstly construct the primary mirror next to the windshield surface and then construct the secondary mirror nearby the image plane, where the stop surface is set as the eye pupil. The details of the main construction steps can be described as follows.

 figure: Fig. 2.

Fig. 2. Construction order of freeform surfaces: (a) traditional off-axis three-mirror anastigmatic system; (b) automotive AR-HUD system.

Download Full Size | PDF

Step One. The freeform surface can be constructed mathematically if we know the incident ray direction as well as the output ray direction corresponding to each surface point, whose surface type is initially assumed to be plane mirror surface for simplicity as shown in Fig. 3. Since both surfaces are initially set as flat surface, then the target points for the output rays of the primary mirror should be exactly the conjugate points of reference image points related to the secondary mirror, both of which are symmetric with respect to the secondary mirror. With the information of target points, the surface normal for the primary mirror can be determined by Snell’s law. Thus, the point cloud of the primary surface can be calculated by a surface construction method [2022]. While for the secondary mirror surface, the final target points of its output rays are the reference image points. To escape overfitting caused by too many variables, we adopt the 2nd-order XY polynomial terms to conduct the surface fitting in the initial construction in this paper. The formula for XY polynomial surface can be expressed as:

$$z(x,y) = \frac{{c({x^2} + {y^2})}}{{1 + \sqrt {1 - (1 + k){c^2}({x^2} + {y^2})} }} + \sum\limits_{i = 0}^2 {\sum\limits_{j = 0}^2 {{A_{ij}}{x^i}{y^j}} } \textrm{ }(i + j \ge 1),$$
where c, k, Aij are the curvature, conic coefficient, and coefficients of polynomial terms respectively.

 figure: Fig. 3.

Fig. 3. Method of deciding target points for the firstly constructed unknown freeform surface.

Download Full Size | PDF

Step Two. After the first iteration of calculation for the two surfaces, both surfaces have been replaced by a freeform surface with up to second-order polynomial terms. In this step, both surfaces are reconstructed iteratively to realize higher performance for the AR-HUD optical system. For the construction of the primary mirror, the target points of output rays cannot be the symmetrical points of the reference imaging points with respect to the secondary mirror, because the secondary mirror is not a plane surface again. Therefore, the target points of output rays for the primary mirror can be recalculated by the following equations based on the Fermat’s principle:

$$\left\{ \begin{array}{l} z = \frac{{c({x^2} + {y^2})}}{{1 + \sqrt {1 - (1 + k){c^2}({x^2} + {y^2})} }} + \sum\limits_{i = 0}^n {\sum\limits_{j = 0}^n {{A_i}_j{x^i}{y^j}\textrm{ }(i + j \ge 1)} } ,\\ OP = F(x,y,z),\\ \frac{{\partial OP}}{{\partial x}} = 0,{\kern 1pt} \\ \frac{{\partial OP}}{{\partial y}} = 0,{\kern 1pt} \end{array} \right.$$
where z is the freeform surface sag expressed by a 5-order XY polynomial terms, and OP is the optical path length. While for the secondary mirror, the target points for output rays of the mirror surface are still the reference image points. To flexibly adjust the optical power distribution between the two freeform surfaces, we further constrain the coordinates of the target points using the following equation:
$$\left[ {\begin{array}{c} X\\ Y\\ Z \end{array}} \right] = \left[ {\begin{array}{c} {{X_{\textrm{current}}} + \rho ({X_{\textrm{target}}} - {X_{\textrm{current}}})}\\ {{Y_{\textrm{current}}} + \rho ({Y_{\textrm{target}}} - {Y_{\textrm{current}}})}\\ {{Z_{\textrm{current}}} + \rho ({Z_{\textrm{target}}} - {Z_{\textrm{current}}})} \end{array}} \right],$$
where [Xcurrent; Ycurrent; Zcurrent] are the intersections between sampled rays and the current surface, [Xtarget; Ytarget; Ztarget] are coordinates of calculated points by Eq. (2), and ρ is the surface factor, whose sign decides whether the derived surface is convex or concave. The group of nonlinear functions can be solved by quasi-Newton’s method. Then we can recalculate the surface normal and gradually increase the polynomial order of XY polynomial surface in the surface fitting step until the 5th-order.

2.2 Iterative optimization of the AR-HUD optical system

The initial structure of the freeform surfaces in the AR-HUD system can be determined through the design procedure in Section 2.1. However, as shown in Fig. 4, the footprints of sampled rays at each eye pupil of the whole field are very widespread distributed on optical surfaces. The initially generated design by the direct design method proposed previously cannot constrain all rays from all eye pupils for the whole field into a relatively small area, which, however, is strongly necessitated for generating a compact AR-HUD optical system. Moreover, freeform surfaces generated by the CI method are still far from optimal design due to the calculation error, which will cause non-negligible deviation between actual image points and reference image points. In addition, the position of the image surface varies with the pupil position. Therefore, if the image plane position needs to be fixed, the focal length of the AR-HUD optical system needs to vary with the pupil position, it will be problematic to use the CI method to directly construct the AR-HUD optical system by simply controlling its focal length.

 figure: Fig. 4.

Fig. 4. Footprint diagram on the freeform surface over its full eye-box and full FOV.

Download Full Size | PDF

In this section, we are aimed to propose an automatic optimization algorithm to iteratively optimize the AR-HUD optical system design to improve the image quality as well as its mechanic performance such as compactness. The key point to develop the iterative optimization algorithm is to establish the merit function to evaluate the final performance of the AR-HUD optical system. Considering the complexity of the AR-HUD optical system, the merit function required to iteratively optimize its performance should include the following three parts: imaging quality, absence of obscuration, and image shift for different eye-box heights.

Imaging performance. With the initially constructed structure for the AR-HUD optical system, actual image points on the image surface obtained by ray tracing are different from the reference ones due to surface construction errors. Since the focal lengths are not identical for different fields and pupils, we trace the chief ray of the central eye pupil at the central field to acquire the paraxial image point as the first reference point, and all other reference points corresponding to other fields can be acquired based on the given magnification ratios in X and Y directions compared to the first reference point. The merit function of imaging performance can be defined as the root-mean-square (RMS) deviation of all sampled rays’ intersections on the image surface from ith sampled pupil inside the eye-box at jth sampled field compared to the corresponding reference image point during the iterative optimization process:

$$\textrm{M}{\textrm{F}_1} = {\omega _1}\sum\nolimits_i {\sqrt {\frac{{\sum\limits_j {({{({x_{i,j}} - {x_{ref,j}})}^2} + {{({y_{i,j}} - {y_{\textrm{ref},j}})}^2} + {{({z_{i,j}} - {z_{ref,j}})}^2})} }}{N}} } ,$$
where [xi,j yi,j zi,j] represents the intersection point between the image surface and the traced light ray from the ith sampled pupil inside the eye-box at the jth sampled field, [xref,j yref,j zref,j] represents the reference point for the jth sampled field on the image surface, ω1 is the weight of this term in the final merit function, and N represents the total number of sampled rays. It is worth to be noted that the reference points will shift in each cycle of optimization.

Absence of obscuration. The ray intersecting with optical surfaces always happens in compact off-axis reflective systems such as AR-HUDs. Therefore, it is necessary to set constraints that make the AR-HUD optical system unobscured. Lots of research have been conducted to develop obscuration elimination algorithms [28]. However, since the multi-pupil characteristic of AR-HUD optical system, fixing the critical light is not suitable for this system. We propose an obscuration elimination for HUD and other multi-pupil systems. As shown in Fig. 5, the obstruction effect of the AR-HUD system usually occurs at the freeform secondary mirror. The method to evaluate the obscuration performance of the AR-HUD optical system can be described as follows. Firstly, we trace all sampled characteristic rays through the AR-HUD optical system, and evaluate the distances L from the center point O of the secondary freeform surface S2 to all these traced rays in the optical space between the windshield and the primary mirror, and then specify the light ray I corresponding to the shortest L. We define ray I as critical ray. Secondly, evaluate the distances D from all intersection points between all traced rays and the freeform secondary mirror surface S2 to light ray I in the optical space between the primary mirror and the secondary mirror, and then specify the point P which has closest distance D to the ray I. Finally, judge whether point P is to the left or right of ray I. Here, the relative position of point P can be estimated by evaluating the area of the triangle Δ$\textrm{AQP}$ as follows:

$${S_{\textrm{AQP}}} = \frac{{\overrightarrow {\textrm{AQ}} \times \overrightarrow {\textrm{AP}} }}{2},$$
where × represents cross product operation, point A is the intersection between ray I and the windshield as shown in Fig. 5. If point P is located at the left of ray I, the system is unobscured, and thus the value of SAQP is greater than 0; if point P is located at the right of ray I, the system is obscured, and thus the value of SAQP is less than 0. Therefore, the second part of the merit function to evaluate the obscuration performance of the AR-HUD optical system can be expressed as:
$$\textrm{M}{\textrm{F}_2} = \begin{cases} 0,&{\textrm{if}\textrm{ }{S_{\textrm{AQP}}} \ge 0}\\ {\omega_2}\|{{S_{\textrm{AQP}}}} \|,&\textrm{i}\textrm{f}\textrm{ }{S_{\textrm{AQP}} < 0} \end{cases}$$
where ω2 is the weight of the second term about the system’s obscuration performance in the final merit function, and $\|{{\kern 1pt} {\kern 1pt} {\kern 1pt} \cdot {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} } \|$ represents the modulo operation.

 figure: Fig. 5.

Fig. 5. Obscuration effect in the AR-HUD optical system.

Download Full Size | PDF

Image shift for different eye-box heights. To accommodate drivers with different heights, the AR-HUD system needs to be designed with satisfactory performance for different eye-box heights, which is realized by rotating the primary mirror. However, the rotation of the primary mirror can make image shift for different eye-box heights, which is unacceptable due to stationary image source for the AR-HUD. The actual image positions for different eye-box heights can be obtained by tracing the chief rays for different configurations. As a result, the term for constraining the image shift caused by different eye-box heights in the final merit function can be expressed as:

$$\textrm{M}{\textrm{F}_3} = {\omega _3}\frac{{\sum\limits_1^S {{{\|{{{\mathbf P}_{\textrm{low}}} - {{\mathbf P}_{\textrm{high}}}} \|}_2}} }}{S},$$
where Plow represents the image position for the lowest eye-box height, and Phigh represents the image position for the highest eye-box height, S represents the number of sampled eye pupils inside the eye-box, ω3 denotes the weight of this term in the final merit function, and ${\|{{\kern 1pt} {\kern 1pt} {\kern 1pt} \cdot {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} } \|_2}$ represents second vector norm operation.

The parameters of both freeform surfaces can be determined through local optimization algorithms such as the damped-least-squares (DLSQ) method or the Simplex algorithm based on the merit function formulated above. For example, if we choose an XY polynomial surface, the optimization parameters will be curvature c, conic coefficient k, and polynomial coefficients [A1, A2, …, An]. The flow diagram of the calculation process for automated design of the AR-HUD optical system is shown in Fig. 6.

 figure: Fig. 6.

Fig. 6. Flowchart of the design process.

Download Full Size | PDF

3. AR-HUD design with single eye-box

An AR-HUD system with two freeform surfaces is proposed to demonstrate the feasibility of our proposed automatic design method. The first example is an AR-HUD optical system design with single eye-box. The proposed system has a 10 m virtual image distance (VID) and 13°×5° FOV, and the detailed design specifications are listed in Table 1.

Tables Icon

Table 1. Specifications of the AR-HUD system

An initial system with three tilted and decentered flat surfaces without obscuration was set up firstly. The windshield of actual automobile enterprises is fitted by the XY polynomial surface. An aperture stop is set up as eye-box 10 m away from the virtual image surface. The initial planar system is shown in Fig. 7. The system uses a biased FOV of 5° in the vertical direction (from -3.5°to -8.5°) and FOV of 13° in the horizontal direction. Here, nine sampled biased fields in our design are (0°, -3.5°), (0°, -6°), (0°, -8.5°), (6.5°, -3.5°), (6.5°, -6°), (6.5°, -8.5°), (-6.5°, -3.5°), (-6.5°, -6°), and (-6.5°, -8.5°) respectively.

 figure: Fig. 7.

Fig. 7. The AR-HUD planar structure with single eye-box.

Download Full Size | PDF

Through the automatic design method proposed in Section 2, an AR-HUD design with two 5th-order XY polynomial freeform surfaces is obtained. The optical layout is shown in Fig. 8(c). The 5th-order XY polynomial system has much better image quality compared to the plane mirror system and 2nd-order XY polynomial system as shown in Fig. 8(a) and Fig. 8(b). The RMS spot radius at each sampling position of the eye-box is shown in Fig. 9 and the grid distortion diagram is shown in Fig. 10. Obviously, the image performance of the AR-HUD design with 5th-order XY polynomial freeform surfaces is much better than that of the plane mirror system and 2nd-order XY polynomial system.

 figure: Fig. 8.

Fig. 8. Design layouts as well as their image performance: (a) plane mirror system; (b) 2nd-zorder XY polynomial system; (c) 5th-order XY polynomial system.

Download Full Size | PDF

 figure: Fig. 9.

Fig. 9. The spot size diagram of single eye-box AR-HUD generated by the proposed method.

Download Full Size | PDF

 figure: Fig. 10.

Fig. 10. The distortion performance of the AR-HUD optical system with single eye-box: (a) eye pupil at the center of the eye-box; (b) eye pupil at the edge of the eye-box.

Download Full Size | PDF

Moreover, the lateral layout AR-HUD commonly used in commercial cars proposed is tested. We extended the single freeform surface design in the previous work [24] to a two freeform surface design with a larger eye-box of 130mm × 50 mm. As shown in Fig. 11, the method proposed in this paper can still generate a structure with good imaging performance and is still applicable for lateral layout AR-HUDs with strict volume requirements.

 figure: Fig. 11.

Fig. 11. Lateral layout of AR-HUD and the output system by the algorithm: (a) algorithm output system; (b) the spot size diagram of lateral layout AR-HUD; (c) the distortion performance of the lateral layout AR-HUD.

Download Full Size | PDF

4. AR-HUD design with eye-box at different heights

To further verify the feasibility of the automatic design method, we employ the design algorithms to automatically design another AR-HUD optical system with two different eye-box heights. In the design example, the AR-HUD system's eye-box heights are set to 0 mm and 30 mm. Based on the design algorithms given in Section 2, the initial structure of the AR-HUD can be obtained as shown in Fig. 12 with initial input of a planar system, where θ is the rotation angle of the primary mirror. The distributions of RMS spot sizes for all sampled pupils, sampled fields of both eye-boxes are shown in Fig. 13, and the generated structure for the AR-HUD can have satisfactory image performance as well as no obscuration. There is no doubt that our directly constructed design can be a very good starting point for further iterative design by local optimization algorithms.

 figure: Fig. 12.

Fig. 12. The layout of the AR-HUD system with various eye-box heights: (a) the plane mirror system; (b) the generated structure by the proposed method.

Download Full Size | PDF

 figure: Fig. 13.

Fig. 13. The diagram of the image spot size distributions: (a) 0 mm eye-box height position; (b) 30 mm eye-box height position.

Download Full Size | PDF

The subsequent optimization of the starting point was conducted in Zemax [29]. The surface type of both freeform surfaces is XY polynomial, which is a kind of simple and effective non-rotationally symmetric freeform surface. To reduce the computation burden, the highly nonlinear conic surface term is omitted. Thus, the expressions for both freeform surfaces can be as follows:

$$\begin{aligned} z(x,y) &= {A_1}x + {A_2}y + {A_3}{x^2} + {A_4}xy + {A_5}{y^2} + {A_6}{x^3} + {A_7}{x^2}y + {A_8}x{y^2} + {A_9}{y^3}\\ &\quad +{A_{10}}{x^4} + {A_{11}}{x^3}y + {A_{12}}{x^2}{y^2} + {A_{13}}x{y^3} + {A_{14}}{y^4} + {A_{15}}{x^5} + {A_{16}}{x^4}y\\ &\quad +{A_{17}}{x^3}{y^2} + {A_{18}}{x^2}{y^3} + {A_{19}}x{y^4} + {A_{20}}{y^5}, \end{aligned}$$
where Ai is the coefficient of the XY polynomial term. The rotation of the system is controlled by the first order term coefficients.

The layout of the final design is shown in Fig. 14. Although the curvature of the secondary mirror changed in the final system, the primary mirror still offers most of the optical power which is the same as the initial structure. The MTF plot of the final design is given in Fig. 15, which shows the MTF meets the specifications of the system (>0.2 at 12lps/mm) at the visible wavelength for all sampled pupils and fields at both two eye-box heights. Figure 16 shows the grid distortion performance of our proposed design for different pupils as well as different eye-box heights, which further demonstrates the feasibility of our method.

 figure: Fig. 14.

Fig. 14. Optical system of the final system after optimization.

Download Full Size | PDF

 figure: Fig. 15.

Fig. 15. The MTF diagram of the final HUD system: (a) 0 mm eye-box height position; (b) 30 mm eye-box height position.

Download Full Size | PDF

 figure: Fig. 16.

Fig. 16. The distortion grid diagram of the final HUD system: (a) maximum grid distortion in the center of the eye-box at 0 mm height; (b) maximum grid distortion in the edge of the eye-box at 0 mm height; (c) maximum grid distortion in the center of the eye-box at 30 mm height; (d) maximum grid distortion in the edge of the eye-box at 30 mm height.

Download Full Size | PDF

It is well known that it is very difficult to design an off-axis optical system from a planar system by optical software directly. Moreover, for the AR-HUD optical system, the freeform windshield, multi-pupil and multi-eye-box configuration will make the design more difficult. In this paper, the influence of the multi-eye-box and the freeform surface windshield has been considered during the design process of the initial system and the light rays are well-focused. Meanwhile, the method in this paper can effectively avoid system’s obscuration during the design process. In this example, the final design can be automatically obtained with iterative optimization based on the optimal initial structure, which further validates the feasibility of our presented automated design algorithms for the complex AR-HUD optical system design.

5. Typical layout experiment of AR-HUDs

To further verify the practicality of the automatic design method, we design and analyze the typical configurations of freeform reflective systems for HUD. Since the windshield is fixed, we change the layouts of the two mirrors based on [30] forming three typical structures including α, U, Z configurations. As shown in Fig. 17(a), the α configuration has the most compact optical volume. However, the spot size of the α configuration is the largest among the three configurations. As shown in Fig. 17(b), the spot size performance of the U configuration is better than that of the α configuration, but its optical volume is the largest. On the other hand, as shown in Fig. 17(c), the Z configuration results have the best imaging performance and relatively compact optical volume. HUD optical systems not only have requirements for imaging quality, but also have relatively strict requirements for system volume. Therefore, based on the imaging performance and volume of the three configurations, this paper concludes that Z configuration is more suitable for HUD optical systems, providing a reference for the design of HUD optical systems in the future.

 figure: Fig. 17.

Fig. 17. Typical optical system layouts of two-mirror off-axis and the output system by the algorithm:(a) α configuration; (b) U configuration; (c) Z configuration.

Download Full Size | PDF

6. Conclusion

In this paper, we successfully propose an automated design method of freeform imaging systems for the automotive AR-HUD applications. With our given algorithms, designer can automatically retrieve an initial structure of the AR-HUD optical systems with better imaging performance, and then decide whether to repeat the iterative process described in the paper or to manually optimize it by software. The algorithm can generate different optical structures in limited volumes for adjusting the mechanical construction of different cars. The feasibility and practicality of the design algorithm is verified by single eye-box height in the vertical and lateral direction and various eye-box heights ranging from 0 mm to 30 mm, as well as several typical AR-HUD designs. The design instances demonstrate good optical performance on the entire eye-box of 130 mm × 50 mm and the full field of view of 13° × 5°, and demonstrated no obscuration effect. Our design method of AR-HUD optical system based on real windshield proposed in this paper basically covers the details that need to be taken into consideration when designing AR-HUDs, and can automatically design the layout of most current AR-HUD optical systems, which will be beneficial to industry, saving a lot of manpower. Furthermore, this paper analyses three typical structures of two-mirror reflective systems for HUD and presents the optimal layout of the typical double mirror AR-HUD, which provides a reference for the future development of the AR-HUD field. Moreover, the proposed automated design method for the AR-HUD optical system can also be extended to any other similar multi-pupil systems.

Funding

National Natural Science Foundation of China (12274156); Science, Technology and Innovation Commission of Shenzhen Municipality (JCYJ20190809100811375, JCYJ20210324115812035); Key Research and Development Program of Hubei Province (2020BAB121); Innovation Fund of WNLO.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. S. Xie, P. Wang, X. Sang, and C. Li, “Augmented reality three-dimensional display with light field fusion,” Opt. Express 24(11), 11483–11494 (2016). [CrossRef]  

2. R. T. Azuma, “A survey of augmented reality,” Presence: Teleoperators & Virtual Environments 6(4), 355–385 (1997). [CrossRef]  

3. R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, “Recent advances in augmented reality,” IEEE Comput. Grap. Appl. 21(6), 34–47 (2001). [CrossRef]  

4. A. Bauer, E. M. Schiesser, and J. P. Rolland, “Starting geometry creation and design method for freeform optics,” Nat. Commun. 9(1), 1756 (2018). [CrossRef]  

5. K. Thompson, “Description of the third-order optical aberrations of near-circular pupil optical systems without symmetry: errata,” J. Opt. Soc. Am. A 26(3), 699 (2009). [CrossRef]  

6. M. Elgarisi, V. Frumkin, O. Luria, and M. Bercovici, “Fabrication of freeform optical components by fluidic shaping,” Optica 8(11), 1501–1506 (2021). [CrossRef]  

7. S. Kumar, Z. Tong, and X. Jiang, “Advances in the design and manufacturing of novel freeform optics,” Int. J. Extrem. Manuf. 4(3), 032004 (2022). [CrossRef]  

8. R. Wu, C. Y. Huang, X. Zhu, H. Cheng, and R. Liang, “Direct three-dimensional design of compact and ultra-efficient freeform lenses for extended light sources,” Optica 3(8), 840–843 (2016). [CrossRef]  

9. S. Sorgato, J. Chaves, H. Thienpont, and F. Duerr, “Design of illumination optics with extended sources based on wavefront tailoring,” Optica 6(8), 966–971 (2019). [CrossRef]  

10. D. Cheng, H. Chen, C. Yao, Q. Hou, W. Hou, L. Wei, T. Yang, and Y. Wang, “Design, stray light analysis, and fabrication of a compact head-mounted display using freeform prisms,” Opt. Express 30(20), 36931–36948 (2022). [CrossRef]  

11. D. Cheng, J. Duan, H. Chen, H. Wang, D. Li, Q. Wang, Q. Hou, T. Yang, W. Hou, D. Wang, X. Chi, B. Jiang, and Y. Wang, “Freeform OST-HMD system with large exit pupil diameter and vision correction capability,” Photonics Res. 10(1), 21–32 (2022). [CrossRef]  

12. Y. Liu, A. Bauer, T. Viard, and J. P. Rolland, “Freeform hyperspectral imager design in a CubeSat format,” Opt. Express 29(22), 35915–35928 (2021). [CrossRef]  

13. Y. Xie, X. Mao, J. Li, F. Wang, P. Wang, R. Gao, X. Li, S. Ren, Z. Xu, and R. Dong, “Optical design and fabrication of an all-aluminum unobscured two-mirror freeform imaging telescope,” Appl. Opt. 59(3), 833–840 (2020). [CrossRef]  

14. D. Wang, P. Xu, Z. Wu, X. Fu, R. Wu, M. Kong, J. Liang, B. Zhang, and R. Liang, “Simultaneous multisurface measurement of freeform refractive optics based on computer-aided deflectometry,” Optica 7(9), 1056–1064 (2020). [CrossRef]  

15. J. Mendes-Lopes, P. Benítez, J. C. Miñano, and A. Santamaría, “Simultaneous multiple surface design method for diffractive surfaces,” Opt. Express 24(5), 5584–5590 (2016). [CrossRef]  

16. R. A. Hicks, “Controlling a ray bundle with a free-form reflector,” Opt. Lett. 33(15), 1672–1674 (2008). [CrossRef]  

17. T. Glimm and V. Oliker, “Optical design of two-reflector systems, the Monge-Kantorovich mass transfer problem and Fermat's principle,” Indiana Univ. Math. J. 53(5), 1255–1278 (2004). [CrossRef]  

18. F. Duerr and H. Thienpont, “Freeform imaging systems: Fermat's principle unlocks “first time right” design,” Light: Sci. Appl. 10(1), 95 (2021). [CrossRef]  

19. Y. Nie, D. Shafer, H. Ottevaere, and F. Duerr, “Automated freeform imaging system design with generalized ray tracing and simultaneous multi-surface analytic calculation,” Opt. Express 29(11), 17227–17245 (2021). [CrossRef]  

20. T. Yang, G. Jin, and J. Zhu, “Automated design of freeform imaging systems,” Light: Sci. Appl. 6(10), e17081 (2017). [CrossRef]  

21. T. Yang, J. Zhu, W. Hou, and G. Jin, “Design method of freeform off-axis reflective imaging systems with a direct construction process,” Opt. Express 22(8), 9193–9205 (2014). [CrossRef]  

22. T. Yang, J. Zhu, X. Wu, and G. Jin, “Direct design of freeform surfaces and freeform imaging systems with a point-by-point three-dimensional construction-iteration method,” Opt. Express 23(8), 10233–10246 (2015). [CrossRef]  

23. H. Peng, D. Cheng, J. Han, C. Xu, W. Song, L. Ha, J. Yang, Q. Hu, and Y. Wang, “Design and fabrication of a holographic head-up display with asymmetric field of view,” Appl. Opt. 53(29), H177–H185 (2014). [CrossRef]  

24. S. Wei, Z. Fan, Z. Zhu, and D. Ma, “Design of a head-up display based on freeform reflective systems for automotive applications,” Appl. Opt. 58(7), 1675–1681 (2019). [CrossRef]  

25. L. Gu, D. Cheng, Y. Liu, J. Ni, T. Yang, and Y. Wang, “Design and fabrication of an off-axis four-mirror system for head-up displays,” Appl. Opt. 59(16), 4893–4900 (2020). [CrossRef]  

26. H. Ji, Z. Zhu, H. Tan, Y. Shan, W. Tan, and D. Ma, “Design of a high-throughput telescope based on scanning an off-axis three-mirror anastigmat system,” Appl. Opt. 60(10), 2817–2823 (2021). [CrossRef]  

27. Z. Fan, L. Zhao, S. Cao, J. Peng, H. Ji, Z. Zhu, S. Wei, Y. Mo, H. Chen, and D. Ma, “High performance telescope system design for the TianQin project,” Class. Quantum Gravity 39(19), 195017 (2022). [CrossRef]  

28. X. Chen, D. Cheng, and Y. Wang, “Automatic obscuration elimination for off-axis mirror systems,” Appl. Opt. 56(32), 9014–9022 (2017). [CrossRef]  

29. Zemax optical design software, https://www.zemax.com/.

30. J. B. Volatier, L. Duveau, and G. Druart, “An exploration of the freeform two-mirror off-axis solution space,” J. Phys. Photonics 2(1), 014004 (2020). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (17)

Fig. 1.
Fig. 1. Sampling of characteristic rays in the entrance pupil plane.
Fig. 2.
Fig. 2. Construction order of freeform surfaces: (a) traditional off-axis three-mirror anastigmatic system; (b) automotive AR-HUD system.
Fig. 3.
Fig. 3. Method of deciding target points for the firstly constructed unknown freeform surface.
Fig. 4.
Fig. 4. Footprint diagram on the freeform surface over its full eye-box and full FOV.
Fig. 5.
Fig. 5. Obscuration effect in the AR-HUD optical system.
Fig. 6.
Fig. 6. Flowchart of the design process.
Fig. 7.
Fig. 7. The AR-HUD planar structure with single eye-box.
Fig. 8.
Fig. 8. Design layouts as well as their image performance: (a) plane mirror system; (b) 2nd-zorder XY polynomial system; (c) 5th-order XY polynomial system.
Fig. 9.
Fig. 9. The spot size diagram of single eye-box AR-HUD generated by the proposed method.
Fig. 10.
Fig. 10. The distortion performance of the AR-HUD optical system with single eye-box: (a) eye pupil at the center of the eye-box; (b) eye pupil at the edge of the eye-box.
Fig. 11.
Fig. 11. Lateral layout of AR-HUD and the output system by the algorithm: (a) algorithm output system; (b) the spot size diagram of lateral layout AR-HUD; (c) the distortion performance of the lateral layout AR-HUD.
Fig. 12.
Fig. 12. The layout of the AR-HUD system with various eye-box heights: (a) the plane mirror system; (b) the generated structure by the proposed method.
Fig. 13.
Fig. 13. The diagram of the image spot size distributions: (a) 0 mm eye-box height position; (b) 30 mm eye-box height position.
Fig. 14.
Fig. 14. Optical system of the final system after optimization.
Fig. 15.
Fig. 15. The MTF diagram of the final HUD system: (a) 0 mm eye-box height position; (b) 30 mm eye-box height position.
Fig. 16.
Fig. 16. The distortion grid diagram of the final HUD system: (a) maximum grid distortion in the center of the eye-box at 0 mm height; (b) maximum grid distortion in the edge of the eye-box at 0 mm height; (c) maximum grid distortion in the center of the eye-box at 30 mm height; (d) maximum grid distortion in the edge of the eye-box at 30 mm height.
Fig. 17.
Fig. 17. Typical optical system layouts of two-mirror off-axis and the output system by the algorithm:(a) α configuration; (b) U configuration; (c) Z configuration.

Tables (1)

Tables Icon

Table 1. Specifications of the AR-HUD system

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

z ( x , y ) = c ( x 2 + y 2 ) 1 + 1 ( 1 + k ) c 2 ( x 2 + y 2 ) + i = 0 2 j = 0 2 A i j x i y j   ( i + j 1 ) ,
{ z = c ( x 2 + y 2 ) 1 + 1 ( 1 + k ) c 2 ( x 2 + y 2 ) + i = 0 n j = 0 n A i j x i y j   ( i + j 1 ) , O P = F ( x , y , z ) , O P x = 0 , O P y = 0 ,
[ X Y Z ] = [ X current + ρ ( X target X current ) Y current + ρ ( Y target Y current ) Z current + ρ ( Z target Z current ) ] ,
M F 1 = ω 1 i j ( ( x i , j x r e f , j ) 2 + ( y i , j y ref , j ) 2 + ( z i , j z r e f , j ) 2 ) N ,
S AQP = AQ × AP 2 ,
M F 2 = { 0 , if   S AQP 0 ω 2 S AQP , i f   S AQP < 0
M F 3 = ω 3 1 S P low P high 2 S ,
z ( x , y ) = A 1 x + A 2 y + A 3 x 2 + A 4 x y + A 5 y 2 + A 6 x 3 + A 7 x 2 y + A 8 x y 2 + A 9 y 3 + A 10 x 4 + A 11 x 3 y + A 12 x 2 y 2 + A 13 x y 3 + A 14 y 4 + A 15 x 5 + A 16 x 4 y + A 17 x 3 y 2 + A 18 x 2 y 3 + A 19 x y 4 + A 20 y 5 ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.