Design of off-axis reflective imaging systems based on freeform holographic elements

Tong Yang; Yongdong Wang; Dongwei Ni; Dewen Cheng; Yongtian Wang

doi:10.1364/OE.460351

1. Introduction

As one of the applications of holography technique, holographic optical element (HOE) is essentially a diffraction grating, but its microstructure, fabrication method and wavefront modulation are very different from the surface relief diffraction gratings. Its basic working principle can be concluded as the interference recording and the diffraction reconstruction. Two coherent recording wavefronts or beams (signal wave and reference wave) interference inside the photosensitive medium, and the interference pattern containing the information of recording wavefronts is recorded inside it, which will enable spatial variations generated by the refractive index of photosensitive medium [1]. When one of the recording wavefronts illuminates the HOE, the other recording wavefront can be reconstructed completely.

According to the difference of recording wavefronts, HOEs can act as different optical imaging functions, such as lens [2,3], mirror [4,5], lens array [6,7], diffuser [8–10], etc. Therefore, HOEs can replace the functions of conventional optical elements to some extent, and possess obvious advantages in many aspects. HOE reacts only on the incident light with special angle and wavelength for its angular and wavelength selectivity. In the augmented-reality (AR) near-eye display and other applications, it can be used as the imaging combiner to achieve better optical see-through viewing. Meanwhile, HOE may significantly reduce the weight and volume of overall system to enable the thin and lightweight form factor [11]. HOE can be repeatedly fabricated through the same experimental exposing setup. It is easy to achieve low-cost and rapid mass production.

At present, holography technique has been successfully applied in optical information storage [12], interferometry [13], optical filtering [14], optical interconnection devices [15], etc. For the imaging optical system, HOEs can be used in holographic waveguide [16–19], retinal-projection display [20–22], light field displays [23], head-up display (HUD) [24–25] and other fields.

A phase function can be used to characterize the wavefront modulation of a HOE, that is, the design of imaging optical system based on HOEs needs to calculate the desired phase functions [26]. At present, HOEs are mostly fabricated by simple spherical waves or plane waves to achieve quasi-ideal imaging of an object-image points pair (the points can be located at a finite distance or infinity). That means only for a single field point, good imaging performance can be achieved. Other field points may have severe aberrations. If HOEs are applied to an imaging system with large field of view (FOV) and aperture, it is necessary to select the special freeform wavefronts to record the corresponding HOEs with desired phase functions, so as to achieve good imaging performance over the full FOV and aperture. The corresponding HOEs can be referred to the freeform HOEs.

Some methods of designing diffractive elements can be used to design the phase functions of HOEs, such as Gerchberg-Saxton (G-S) algorithm [27], Yang-Gu (Y-G) algorithm [28], etc. However, the imaging system design needs to control the light rays from multiple fields and pupil coordinates simultaneously. The above phase retrieval or beam-shaping design methods are less applicable. When an imaging system merely consists of off-axis tilted HOEs, feasible design starting points with approximate system specifications and configurations cannot be found in most cases, which will make the optimization design process very difficult or even failed. Some researchers proposed direct or point-by-point optical design method of elements with phase functions. J. Mendes-Lopes et al proposed an improved simultaneous multiple surface (SMS) design method for co-axis diffractive elements according to the system requirements [29]. But this method has certain restriction on the number of fields and the system structure considered in the design process. Yang et al proposed a point-by-point design method of off-axis nonsymmetric imaging systems consisting of flat phase elements [30]. Light rays from multiple fields and aperture positions are considered in the design process. Although the imaging systems with phase elements can be generated using the above methods, they do not involve the practical fabrication of HOEs. C. Jang et al designed and fabricated freeform HOEs and then realized the prototypes of several holographic imaging systems [31]. The holographic imaging system and the recording systems of HOEs are designed individually, meanwhile, the performance and the diffraction efficiency of holographic imaging system were also optimized separately, the final optimization results might not be the optimal solutions. To achieve an overall good imaging performance and high diffraction efficiency in general, the holographic imaging and recording systems should be generated simultaneously in a joint design and optimization process. In addition, the initial system generation, the stray lights elimination, the actual recording system setup and the effect of the cover/substrate glasses of the actual HOEs should also be considered during the design process to get the good final results of the actual systems. Related effective, practical and detailed design methods as well as system fabrication method remain to be explored for the successful development of holographic imaging systems.

In this paper, we propose a design method of off-axis nonsymmetric reflective imaging systems based on freeform HOEs. The starting point of the imaging system based on freeform HOEs is generated using a point-by-point design method and then goes through the preliminary optimization. Based on this the recording system of each HOE in the imaging system are then designed preliminarily. To get the final results, we optimize the holographic imaging system and the corresponding recording systems jointly, simultaneously considering the imaging performance of all sub-systems, the diffraction efficiency, the system constraints and fabrication during the joint optimization design process. To validate the feasibility and effectiveness of the proposed method, an off-axis reflective HUD system with good performance based on freeform HOEs is designed and the corresponding prototype is fabricated. Detailed design and development process of the system are demonstrated.

2. Design method

2.1 Basic theory of holographic imaging system

In this paper, we focus on the volume HOE with high diffraction efficiency and strong selectivity. The ray-tracing properties of HOE defined at the holographic substrate surface can be analyzed numerically, assuming that the holographic grating itself is infinitely thin. At a point on the substrate surface S, the signal wavefront and the reference wavefront are represented as the wave vector r_S and r_R, respectively, as shown in Fig. 1 (considering there is only one pair of recording wavefronts).

(1)$$\begin{aligned} {\boldsymbol{r}_\textrm{S}}(x,y,z) &= \frac{{2\mathrm{\pi }}}{{{\lambda _\textrm{c}}}}[{n_\textrm{c}}({\mu _\textrm{S}}\hat{x} + {\nu _\textrm{S}}\hat{y} + {\xi _\textrm{S}}\hat{z})]\\ {\boldsymbol{r}_\textrm{R}}(x,y,z) &= \frac{{2\mathrm{\pi }}}{{{\lambda _\textrm{c}}}}[{n_\textrm{c}}({\mu _\textrm{R}}\hat{x} + {\nu _\textrm{R}}\hat{y} + {\xi _\textrm{R}}\hat{z})] \end{aligned}, $$

where the vector $({\mu _\textrm{S}}\hat{x} + {\nu _\textrm{S}}\hat{y} + {\xi _\textrm{S}}\hat{z})$ indicates the unit direction vector along the wave vector r_S, and μ_S, ν_S, ξ_S are the direction cosine along this direction relative to the x, y, z axis, respectively. The vector $({\mu _\textrm{R}}\hat{x} + {\nu _\textrm{R}}\hat{y} + {\xi _\textrm{R}}\hat{z})$ indicates the unit direction vector along the wave vector r_R, and μ_R, ν_R, ξ_R are the direction cosine along this direction relative to the x, y, z axis, respectively. $\hat{x},\hat{y},\hat{z}$ are the unit direction vector along the x, y, z axis, respectively. λ_c is the wavelength of recording wavefronts in air, and n_c is the refractive index of recording wavefronts inside the photosensitive medium. At this point, we define a three-dimensional vector Ψ_G lying on the substrate surface to characterize the holographic volume grating, and each component of vector Ψ_G can characterize the spatial frequency (the reciprocal of period) of holographic grating in corresponding direction, Srespectively [32]:

(2)$${\boldsymbol{\varPsi} _\textrm{G}} = ({\varPsi _{\textrm{G,}x}},{\varPsi _{\textrm{G,}y}},{\varPsi _{\textrm{G,}z}}) = {\boldsymbol{r}_\textrm{S}} - {\boldsymbol{r}_\textrm{R}} = \frac{{2\mathrm{\pi }}}{{{\lambda _\textrm{c}}}}[{n_\textrm{c}}({\mu _\textrm{S}} - {\mu _\textrm{R}}),{n_\textrm{c}}({\nu _\textrm{S}} - {\nu _\textrm{R}}),{n_\textrm{c}}({\xi _\textrm{S}} - {\xi _\textrm{R}})]. $$

Fig. 1. The schematic diagram of grating vector of HOE on the substrate surface S with its phase function ϕ(x,y) plotted in a virtual three-dimensional space.

Download Full Size | PDF

During reconstruction (or imaging), the incident wavefront r_I at this point on the holographic substrate surface can be expressed as

(3)$${\boldsymbol{r}_\textrm{I}}(x,y,z) = \frac{{2\mathrm{\pi }}}{{{\lambda _\textrm{p}}}}[{n_\textrm{p}}({\mu _\textrm{I}}\hat{x} + {\nu _\textrm{I}}\hat{y} + {\xi _\textrm{I}}\hat{z})], $$

where λ_p is the wavelength of reconstruction wavefronts in air, and n_p is the refractive index of reconstruction (or imaging) wavefronts inside the photosensitive medium, respectively. Therefore, the first-order diffracted wavefront r_D at this point can be represented as follows:

(4)$${r_{\textrm{D,}x}} = {\varPsi _{\textrm{G,}x}} + {r_{\textrm{I,}x}}, \qquad {r_{\textrm{D,}y}} = {\varPsi _{\textrm{G,}y}} + {r_{\textrm{I,}y}}. $$

The corresponding z-component r_D,z of the diffracted wavefront can be calculated numerically according to the norm of vector r_D, the plus or minus sign chosen depends only on whether the grating is reflection or transmission type. As described above, the grating vector components (Ψ_G,x, Ψ_G,y) completely determine the relationship between the incident light and the diffracted light at each point on the holographic substrate surface.

If the function of a HOE is described by a phase function ϕ(x,y) illustrated by Fig. 1, for the incident wavefront r_I at a point on the HOE, and the corresponding first-order outgoing wavefront r_O at this point can be written as follows [26]:

(5)$${r_{\textrm{O},x}} = \frac{{\partial \phi }}{{\partial x}} + {r_{\textrm{I},x}}, \qquad {r_{\textrm{O},y}} = \frac{{\partial \phi }}{{\partial y}} + {r_{\textrm{I},y}}. $$

The corresponding z-component r_O,z of the outgoing wavefront can also be calculated numerically according to the norm of vector r_O, the plus or minus sign chosen depends only on whether the surface is reflective or transmissive. It should be noted that the above-mentioned three-dimensional vectors r_S, r_R, r_I and r_O are all wave vector. From the above Eqs. (2)–(5), we can get the following equation,

(6)$$\frac{{\partial \phi }}{{\partial x}} = \frac{{2\mathrm{\pi }}}{{{\lambda _\textrm{c}}}}{n_\textrm{c}}({\mu _\textrm{S}} - {\mu _\textrm{R}}), \qquad \frac{{\partial \phi }}{{\partial y}} = \frac{{2\mathrm{\pi }}}{{{\lambda _\textrm{c}}}}{n_\textrm{c}}({\nu _\textrm{S}} - {\nu _\textrm{R}}). $$

Equation (6) indicates that the phase function ϕ(x,y) can fully represent the grating vector Ψ_G of HOE. Therefore, the design of holographic imaging system can be considered as to designing the locations and phase functions of HOEs, while the same phase functions are used during the recording systems design. The phase function should be expressed and computed in the unit of radian, it can have no axis of symmetry and may be described by complicated expressions, such as XY polynomials, Zernike polynomials, NURBS, etc. In this way, the HOE can be considered as a freeform surface to some extent while many degrees of design freedom can be offered and the aberrations induced by nonsymmetric configuration can be corrected. When the incident wavefront r_I is different in wavelength or incident angle compared to the original reference wavefront r_R, the diffracted wavefront r_D will deviate from the signal wavefront r_S, resulting in a decrease in diffraction efficiency, which is called the Bragg mismatch. An evaluation parameter of diffraction efficiency ΔΨ_z is used

(7)$$\Delta {\varPsi _z} = ({r_{\textrm{D},z}} - {\varPsi _{\textrm{G},z}} - {r_{\textrm{I},z}}). $$

When ΔΨ_z = 0, the Bragg condition is satisfied, and the diffraction efficiency η is the highest. η directly decreases with the increase of ΔΨ_z. For the actual holographic imaging system, Bragg condition cannot be fully satisfied in general, which may induce decrease of diffraction efficiency and non-uniformity of light across the FOV and pupil. Therefore, we can use ΔΨ_z during the system optimization to control the magnitude and uniformity of diffraction efficiency. Details can be found in Section 2.4.

The whole design process of the holographic imaging system consisting of three main steps. Firstly, the holographic imaging system is designed preliminarily, in which the starting point can be generated using a point-by-point design process. Next, the recording system of each HOE in the imaging system is designed preliminarily. Then, the imaging system and the recording systems are jointly optimized in order to achieve good imaging performance in each and control the imaging diffraction efficiency. The final design results can be used to fabricate the elements in the recording systems and fabricate the HOEs.

2.2 Preliminary design of the holographic imaging system

The goal of the holographic imaging system design is to obtain the locations and phase functions of HOEs in the system and achieve good imaging performance. The system can be highly nonsymmetric and the phase functions may have no axis of symmetry, which is similar to freeform imaging system design to some extent. Traditional design strategy for this kind of system is to first find a proper starting point and then apply optimization. However, for the nonsymmetric holographic system design, feasible starting points cannot be found in most cases, which may induce great difficulty for the whole design process. Here, a point-by-point method is used here to generate the starting point for further optimization [30]. An initial system consisting of geometric planes or planar phase elements offering no optical power is firstly established for the following design process. The initial elements should be approximately located at the places determined based on the requirements on system configuration and the folding geometry of light path. x² and y² terms of the phase function can be added to realize expected optical power distribution before construction, which can control the system structure and eliminate light obscuration. Light rays from different fields and pupil coordinates are sampled. As shown in Fig. 2(a), the values of phase function as well as the normals of phase function graph in the virtual three-dimensional space corresponding to the intersection points of these rays with HOE can be calculated based on the ray mapping, generalized refractive/reflective equation and Fermat’s principle. Closed-form phase function can be obtained through fitting considering the phase value and the normal, as shown in Fig. 2(b), and the phase functions of multiple HOEs can be obtained one-by-one. Iterative process is further conducted to reduce the ray aberrations, and thus obtaining good starting point for further optimization. Details of this point-by-point design method can be found in Ref. [30].

Fig. 2. (a) The calculation of phase function values as well as the normals of phase function graph corresponding to the intersection points of light rays with HOE. (b) The closed-form phase function after fitting.

Download Full Size | PDF

Further optimization is conducted on this starting point. The optimization algorithm of the holographic imaging system may use the same one for traditional imaging system using geometric elements, such as damped least-square algorithm. Multiple imaging fields sampled across the full FOV (not a small FOV or only the central field) are simultaneously sampled and optimized to achieve good imaging performance across FOV. This process is a tradeoff between different field points and satisfies the general requirements of designing actual imaging systems. The coefficients or parameters in the phase functions of HOEs as well as their locations (including the location of image plane) can be taken as the design variables. Light obscurations can be eliminated by controlling the constraints established using the real ray trace data. Focal length can be calculated by ABCD matrix method and controlled. Distortion as well as system volume can be controlled by constraining the real ray trace data. Finally, we can get the preliminary design result of the imaging system, which will be used to guide the preliminary design of the recording system of each HOE, and be used for the joint optimization.

2.3 Preliminary design of the recording systems

After the holographic imaging system is obtained, the next step is to design the recording system of each HOE. The goal of the recording or exposure process for each HOE is to realize the corresponding ϕ(x,y) obtained during the imaging system design. The recording of a HOE is realized by the interference of the signal and reference waves, which are coming from the same laser source. However, if traditional spherical waves are used as the signal and reference waves, the HOE can realize good imaging performance for only one field point. The required ϕ(x,y) cannot be obtained and other field points will have large aberrations. As a result, at least one extra wavefront correction element (WCE) is needed to regulate the recording waves, in order to form the desired grating vector Ψ_G, and then the required ϕ(x,y) can be generated according to Eq. (6). For the recording of a reflection HOE, the signal and reference waves should be on the different sides of the HOE. Figure 3(a) illustrates the actual recording process of a reflection HOE, the recording wavefronts modulated by WCEs record the HOE in order to realize the phase function ϕ(x,y) obtained by the imaging system design. The recording system design can be seen as designing a stigmatic imaging system considering one field point corresponding to the two recording wavefronts, as shown in Fig. 3(b). The object wave from a point source is modulated by a WCE, modulated by the HOE according to ϕ(x,y), modulated by another WCE, and finally images at a point source (or infinity). The WCEs play the role of correcting the aberrations caused by HOE (or can be considered as to generating the freeform wavefronts) in the recording system. The point source of plane wave can be considered to be located at infinity. As the phase function of HOE as well as the configuration of the recording system is generally nonsymmetric, we can use freeform lenses or mirrors as the WCEs, which will make the system compact and have strong ability to correct aberrations. The obtained phase function ϕ(x,y) can well control the wavefronts of different fields in the actual holographic imaging system.

Fig. 3. (a) The schematic diagram of the actual recording process for a reflection HOE. (b) The corresponding stigmatic imaging system design.

Download Full Size | PDF

During the recording system design, the signal and reference wavefronts should match the directions and features of the wavefronts in the original holographic imaging system as much as possible in order to keep the diffraction efficiency. The locations of laser sources, the locations and surface coefficients of WCEs can be taken as variables while the coefficients of phase function ϕ(x,y) are fixed. Light obscurations can be eliminated by controlling the constraints established using the real ray trace data. The WCE can be either a refractive or a reflective element, which may be determined based on the requirements of system structure and stray lights elimination. It should ensure that only the signal and reference waves interfere inside the photosensitive medium, otherwise other holographic gratings will be generated and unexpected aberrations will be induced in the final imaging system. The aperture size of HOE surface should ensure that the recording wavefronts can completely cover the interference area of two beams, and the interference area should be larger than the actual used area of HOE in the final imaging system.

2.4 Joint optimization of the imaging system and the recording system

Although the holographic imaging system as well as the recording system of each HOE can be obtained using the above methods, however, they are designed individually. In fact, the phase functions of HOEs in the imaging system affect the design of each recording system, while the imaging performance of the recording system also affect the actual performance of the imaging system using the fabricated HOEs. Therefore, all the systems should have good imaging performance while high and uniform diffraction efficiency should be maintained. To solve this problem, all the systems should be simultaneously optimized to obtain good design results, including the imaging system using multiple imaging field points and the corresponding recording systems using one field point. The whole design task can be considered as to designing a system with multi-configuration, and each sub-system can be considered as a configuration during this joint optimization process. The design constraints in each configuration are the same with the preliminary designs.

The whole holographic imaging system can be taken as a configuration. For some special applications, the imaging system contains multiple configurations. For example, when designing some augmented-reality imaging systems, it is optional that the different eye positions in the eyebox can be set as different sub-imaging systems, corresponding to different configurations.

If the imaging system has I configurations in total, the total error function of the imaging system can be represented as follows:

(8)$${\Omega_{\textrm{imaging}}} = \sum\limits_i^I {{w_{\textrm{imaging},i}}}{{\Omega_{\textrm{imaging,i}}}},$$

where Ω_imaging,i is the error function related to imaging performance of the ith (1 ≤ i ≤ I) configuration, which may be represented using transverse ray aberrations, spot size, wavefront error, etc. w_imaging,i is the weight of the ith configuration.

If a total of R HOEs need to be recorded, R different configurations will be designed for these recording systems. The error function related to imaging performance of the rth (1 ≤ r ≤ R) configuration will be written as Ω_recording,_r, which may also be represented using transverse ray aberrations, spot size, or wavefront error, and the corresponding weight is w_recording,r. The total error function of the recording systems will be represented as follows:

(9)$${\Omega _{\textrm{recording}}} = \sum\limits_{r = 1}^R {{w_{\textrm{recording},r}}}{{\Omega_{\textrm{recording,r}}}}.$$

The diffraction efficiency should be also considered during the joint optimization. As shown in Fig. 4(a), consider a imaging ray of field F_k (1 ≤ k ≤ K, K is the total number of sampled imaging fields) and pupil coordinates (x_p,y_p) in the imaging system intersects at point H with local x and y coordinates (x_h,y_h) on the holographic substrate surface S. At this point, ξ_I and ξ_D are the z-component of direction cosine of the incident ray r_I and the diffracted ray r_D, respectively. For the recording system as shown in Fig. 4(b), at the same point H, ξ_R and ξ_S are the z-component of direction cosine of the reference ray r_R and the signal ray r_S, respectively. The above-mentioned data can be directly acquired by the real ray trace data. Referring to Eq. (7), the evaluation parameter of diffraction efficiency for the imaging ray in the imaging system can be calculated as

(10)$$\Delta _p^k = \frac{{\textrm{2}\pi }}{{{\lambda _p}}}[{n_\textrm{p}}{\xi _\textrm{D}} - \frac{{{\lambda _\textrm{p}}}}{{{\lambda _\textrm{c}}}}{n_\textrm{c}}({\xi _\textrm{S}} - {\xi _\textrm{R}}) - {n_\textrm{p}}{\xi _\textrm{I}}], $$

we employ the root-mean-square (RMS) value of evaluation parameters of all the sampled imaging rays across the pupil of field F_k to control the diffraction efficiency of this field.

(11)$${\delta _{\textrm{RMS},k}} = \sqrt {\frac{{\sum\limits_g^G {\Delta {{_g^k}^2}} }}{G}} , $$

where G is the total number of sampled imaging rays across the pupil. Equation (11) is used to construct error function Ω_RMS,k

(12)$${\Omega _{\textrm{RMS},k}} = {w_k}{({\delta _{\textrm{RMS},k}} - {\delta _{\textrm{TAR},k}})^2}, $$

where w_k is the weight, and δ_TAR,k is the optimization target value, which is generally zero. In addition, the uniformity of diffraction efficiency of the imaging rays across FOV should be controlled in order to realize uniform imaging or display performance. We employ the standard deviation of δ_RMS,k of all the sampled imaging fields.

(13)$${\eta _{\textrm{STD}}} = \sqrt {\frac{{\sum\limits_k^K {{{({\delta _{\textrm{RMS},k}} - \frac{{\sum\limits_j^K {{\delta _{\textrm{RMS},j}}} }}{K})^2}}} }}{K}} , $$

Equation (13) is used to construct error function Ω_STD

(14)$${\Omega _{\textrm{STD}}} = {w_{\textrm{STD}}}{({\eta _{\textrm{STD}}} - {\eta _{\textrm{TAR}}})^2}, $$

where w_STD is the weight, and η_TAR is the optimization target value, which is generally zero.

Fig. 4. The schematic diagram of ray tracing for the joint optimization. (a) The imaging system for the field F_k. (b) The recording system of the HOE.

Download Full Size | PDF

The error function related to diffraction efficiency for the rth (1 ≤ r ≤ R) HOE can be written as Ω_efficiency,r, and the total error function related to diffraction efficiency will be represented as follows:

(15)$${\Omega _{\textrm{efficiency}}} = \sum\limits_{r = 1}^R {{w_{\textrm{efficiency},r}}}{{\Omega _{\textrm{efficiency,r}}}},$$

where w_efficiency,r is the weight of the rth HOE, and Ω_efficiency,r can be further decomposed into

(16)$${\Omega _{\textrm{efficiency},r}} = {\Omega _{\textrm{STD}}}\textrm{ + }\sum\limits_k^K {{\Omega _{\textrm{RMS},k}}} . $$

The total error function Ω(ω) of the joint optimization can be represented as follows:

(17)$$\Omega (\boldsymbol{\omega }) = {w_{\textrm{imaging}}}{\Omega _{\textrm{imaging}}} + {w_{\textrm{recording}}}{\Omega _{\textrm{recording}}} + {w_{\textrm{efficiency}}}{\Omega _{\textrm{efficiency}}}, $$

where ω represents the design variables, including the coefficients of phase functions, the surface coefficients of WCEs and the position coordinates of each element in the imaging and recording systems. Therefore, the error function is a function of design variables. The joint optimization of the holographic imaging and recording systems should address the following problems with constraints.

(18)$$\begin{array}{l} \min \; \Omega (\boldsymbol{\omega })\\ \textrm{s.t.}\;\; {f_u}(\boldsymbol{\omega }) \le 0\;\;\textrm{or}\;\;{f_u}(\boldsymbol{\omega }) < 0\;\;\textrm{or}\;\;{f_u}(\boldsymbol{\omega }) = 0, \;1 \le u \le L \end{array}, $$

where f_u(ω) represents the uth design constraint and is a function of design variables, L is the total number of design constraints. The flowchart of the whole design process is shown in Fig. 5.

Fig. 5. The flowchart of the whole design process.

Download Full Size | PDF

3. Design examples

We designed and fabricated of a reflective holographic HUD system with monochromatic green display as an example to verify the above-mentioned design method. The system specifications of the holographic HUD are listed in Table 1. The nonsymmetric system has two reflective freeform HOEs, as shown in Fig. 6(a). HOE1 is taken as the combiner. The light coming from the outside scene can reach the eyebox without aberrations, as the planar glass plate do not induce aberrations and the light is not modulated by the holographic grating due to the selectivity.

Fig. 6. (a) The schematic diagram of off-axis reflective holographic HUD. (b) The initial system using planar phase elements with no power. (c) The system when some power terms are added to realize the power distribution. (d) The imaging system generated by the point-by-point design process. (e) The distortion grid after point-by-point design. (f) The imaging system with the cover glass after preliminary optimization.

Download Full Size | PDF

Table 1. Specifications of the holographic HUD

View Table

In this design example, we select the photopolymer which has high diffraction efficiency, high signal-to-noise ratio and simple post-treatment process as the photosensitive medium [33]. In practical application, if the holographic grating is directly exposed to the air, the grating microstructure may be damaged by water vapor, dust, scratches, etc. Therefore, in order to protect the HOE, after the recording and post-treatment process, the photopolymer attached to the substrate glass needs to be cemented by a cover glass plate. The holographic grating is located between the cover glass and the substrate glass. The thickness of the cover glass and the substrate glass in this design are both 3mm as an example, but in fact the thickness can be further reduced to ultrathin (for example, 1mm) without changing the basic design theory and holographic display. The minimum thickness depends on the capability of glass plate fabrication and cementation. The glass material is K9. Meanwhile, we need to consider ray deflections and aberrations induced by the cover glass and the substrate glass during the design process of the imaging and recording systems. For the recording system of a reflection HOE, the signal wave or the reference wave will be modulated by the substrate glass of HOE. For the actual imaging system, the light coming from the display panel will be modulated by the cover glass.

The next step is to design the imaging system preliminarily. The phase function of each HOE is described by XY polynomials up to the 6th order.

(19)$$\begin{aligned} \phi (x,y) &= \frac{{2\mathrm{\pi }}}{{{\lambda _\textrm{c}}}}({A_2}y + {A_3}{x^2} + {A_5}{y^2} + {A_7}{x^2}y + {A_9}{y^3} + {A_{10}}{x^4} + {A_{12}}{x^2}{y^2} + {A_{14}}{y^4}\\ &+ {A_{16}}{x^4}y + {A_{18}}{x^2}{y^3} + {A_{20}}{y^5} + {A_{21}}{x^6} + {A_{23}}{x^4}{y^2} + {A_{24}}{x^2}{y^4} + {A_{27}}{y^6} ) \end{aligned}. $$

As the system is symmetric about the YOZ plane, the odd-order terms of x are not used. Point-by-point design method demonstrated in Section 2.2 is used to generate the starting point for the imaging system design. An initial system using planar phase elements with no power is firstly established where the two elements located approximately at the expected locations of the two HOEs and realize the expected light folding geometry, as shown in Fig. 6(b). However, in order to avoid light obscuration, optical power should be distributed to the two HOEs without much difference. In the design example, some power terms (x² and y²) are added to HOE1 manually before point-by-point design process to realize the power distribution, as shown in Fig. 6(c). The coefficient of the added power term (x² and y²) for HOE1 is 0.001 (A₃ and A₅ in Eq. (19)). In fact, this value is selected after several trials to get a satisfactory result, and it is not restricted to this specific value. Half-full FOV is sufficient to be considered during the design as the system is plane-symmetric. Discrete light rays from multiple fields and pupil coordinates are sampled. These rays are expected to be redirected by the two HOEs to their ideal image points calculated using the object-image relationship. After the point-by-point design process, the phase functions can be obtained. For the starting point design, the phase functions generated at this stage are up to the 4th order. The corresponding system is shown in Fig. 6(d) and its distortion grid is shown in Fig. 6(e). This system can be taken as a good starting point. During subsequent optimization, the coefficients of phase functions as well as the locations of HOEs and image plane are taken as the design variables. The phase functions are gradually upgraded to 6th order. The light obscurations should be avoided and the system focal length and distortion are controlled. After preliminary design the two phase functions ϕ₁(x,y) and ϕ₂(x,y) corresponding to the HOE1 and HOE2 respectively can be obtained. The imaging system with the cover glass after preliminary optimization is shown in Fig. 6(f).

After we have obtained the imaging system, the next step is to design the recording system of each HOE based on the design result of the imaging system. For each recording system design, the selection of WCE needs to consider stray lights elimination. For the cases when freeform lens is used as the WCE, the reflective lights from front or rear surfaces of lens may introduce stray lights during the exposure process, which will disturb the recording of holographic grating. Therefore, in this design, we select the freeform mirror as the WCE. The lights once reflected by the mirror cannot introduce stray lights during the exposure process. The freeform mirror should be positioned off-axis relative to the substrate glass to ensure that the other recording wavefront transmitted through the substrate glass cannot intersect on this mirror, thus avoiding stray lights reflected by this mirror back to the photopolymer again.

The design of each recording system can be taken as a stigmatic imaging system design task considering only one field point. The light footprint of the recording waves should cover the effective used area of HOE in the imaging system. In this design, one WCE is used in each recording system. We select a freeform mirror to modulate one of the recording wavefronts. In order to keep the diffraction efficiency, the recording wavefronts should be similar in directions and features with the imaging wavefronts. In this design example, the imaging system has a relatively large EPD, and its FOV is not large. For the actual HUD application, the diffracted lights of HOE1 should be the plane wavefronts with relatively large aperture and different imaging fields (within the FOV that is not very large). Therefore, for the recording system design of HOE1, we select a plane wavefront corresponding to the 0° field point instead of a spherical wavefront as the signal wavefront, which is then modulated by HOE1 with ϕ₁(x,y), modulated by freeform mirror M1, and finally images at the point source, as shown in Fig. 7(a). The diffracted lights of HOE2 can be approximately regarded as diverging spherical wavefronts, we select a diverging spherical wavefront as the signal wavefront, which is then modulated by HOE2 with ϕ₂(x,y), modulated by freeform mirror M2, and finally images at the point source, as shown in Fig. 7(b). The freeform type of the freeform mirrors is XY polynomials surface up to the 10th order with a base conic.

(20)$$z(x,y) = \frac{{c({x^2} + {y^2})}}{{1 + \sqrt {1 - (1 + k){c^2}({x^2} + {y^2})} }} + \sum\limits_{i = 0}^{10} {\sum\limits_{j = 0}^{10} {{C_{i,j}}{x^i}{y^j}} } ,\;\;\;1 \le i + j \le 10. $$

The odd-order terms of x are also not used. During the design of each recording system, the freeform surface coefficients and the locations of freeform mirror and laser source are taken as the design variables. The imaging spot size should be optimized to be very small in order to generate the expected phase function using point-like source. The phase function of HOE is the same with the imaging system obtained earlier. The conic constant should be controlled to be not large (within ±10 in this example). The location of freeform mirror relative to the aperture stop should be controlled in order to avoid surface interference and eliminate stray lights. The preliminary design results of two recording systems are shown in Fig. 7.

Fig. 7. The preliminary design results of the recording systems. The direction of the actual reference wavefront is opposite to the direction of the corresponding wavefront during these stigmatic imaging systems design. (a) HOE1. (b) HOE2.

Download Full Size | PDF

After we have obtained the preliminary design results of the imaging system and recording systems, the next step is the joint optimization. As described in Section 2.4, we construct the joint optimization error function related to the imaging performance of three different configurations (the imaging system and two recording systems) as well as the diffraction efficiency for HOE1 and HOE2, according to the real ray trace data of HOEs in the imaging system and corresponding recording systems. These configurations are optimized simultaneously, as shown in Fig. 8, and the phase functions of HOEs are same in different configurations. The design variables used during the joint optimization are the same with the individual design of the systems.

Fig. 8. The joint optimization process. Three configurations are plotted simultaneously.

Download Full Size | PDF

The final design results as shown in Fig. 8, three configurations are plotted simultaneously. The layout of the imaging system is shown in Fig. 9(a), and the distortion grid is shown in Fig. 9(b). The MTF plot for 4mm pupil located at the center of the eyebox is shown in Fig. 9(c). The MTF curves of the system at different eye positions located at (0mm,15mm), (0mm,−15mm), (15mm,0mm) and (−15mm,0mm) across the eyebox are all close to the diffraction limit for 4mm pupil.

Fig. 9. (a) The layout of the imaging system. (b) The distortion grid. (c) The MTF plot of HUD imaging system with 4 mm pupil located at the center of the eyebox.

Download Full Size | PDF

We further flip the whole imaging system and evaluate the performance in visual space. The visual space MTF plot for 4mm pupil located at the center of the eyebox is shown in Fig. 10(a). In order to compare the performance between systems which are fabricated using or not using WCEs, we also designed another HUD system. The system configuration and specifications are exactly the same with the design example. However, the signal and reference waves in the recording systems of HOE1 and HOE2 are both traditional plane wave or spherical wave. The visual space MTF plot for 4mm pupil located at the center of the eyebox of this comparison system is shown in Fig. 10(b). It can be seen that the imaging performance of the system using HOEs fabricated by WCEs is much better than the system using traditional HOEs. In addition, when observing the image at the eyebox of the system using traditional HOEs, the virtual image may move and rotate as the eye moves across the eyebox due to large aberrations of the system.

Fig. 10. (a) The visual space MTF plot of the system using freeform HOEs. (b) The visual space MTF plot of the system using traditional HOEs.

Download Full Size | PDF

Fig. 11. The experimental setups of the recording systems. (a) HOE1. (b) HOE2.

Download Full Size | PDF

The experimental setups of the recording systems are shown in Fig. 11, which are built based on the final design results of the recording systems. The 532nm laser is used to expose holographic grating. The microscope objective focuses the laser beam into a small point to generate the spherical wavefront, and a pinhole locates at the focal spot position to filter higher-order mode and stray lights to improve quality of the recording beam. The plane wavefront is generated using a collimating lens. We use a large aperture aspherical lens as the collimating lens to enhance the collimation effect of plane wavefront. The freeform mirrors are fabricated by single point diamond turning (SPDT), and aluminum is used as the material of the substrate, as shown in Fig. 12(a).

The aperture stop with wedge frame structure is used to locate the holographic substrate glass by the position pins, and to define the effective recording area of holographic grating, as shown in Fig. 12(b). If the aperture stop does not adopt wedge frame structure, the outer frame will produce the shadows on the substrate glass under oblique illumination of the recording beams, which will affect the recording process of holographic grating. After the recording process of holographic grating, the photopolymer is needed to go through the UV curing and baking process to fully achieve the monomer curing.

Fig. 12. (a) The fabricated freeform mirrors. (b) The aperture stop with wedge frame structure.

Download Full Size | PDF

Fig. 13. (a) and (b) are the HOE1 and HOE2 after fabrication. (c) The holographic HUD prototype.

Download Full Size | PDF

The fabricated HOE1 and HOE2 are shown in Figs. 13(a) and 13(b). The prototype of the holographic HUD using the fabricated HOEs is shown in Fig. 13(c). To demonstrate the virtual and see-through light path image quality of the HUD system, a test is implemented. A commercial camera which is used to simulate the human eye is located at the center of the eyebox and the captured picture as well as the see-through outside scene is shown in Fig. 14. Visualization 1 shows the virtual image captured as the camera moves, which demonstrating good and uniform imaging performance and luminance across the eyebox.

Fig. 14. (a) The original image. (b) The virtual image captured at the center of the eyebox.

Download Full Size | PDF

4. Conclusion and discussions

In this paper, we proposed the design method of off-axis reflective imaging system based on freeform HOEs. The preliminary design of the holographic imaging system is obtained by using a point-by-point design method and optimization, which is then used for the preliminary design of the recording system of each HOE in the imaging system. The following joint optimization of the holographic imaging and recording systems considers the imaging performance of all sub-systems, the diffraction efficiency and the system constraints simultaneously. In addition, in the whole design process, the issues related to the system fabrication such as the stray lights elimination, the effect of the cover/substrate glasses of the actual HOEs are all considered. We designed and fabricated an off-axis reflective HUD system based on freeform HOEs as an example to validate the feasibility and effectiveness of the proposed method. Detailed design and development process of the prototype are demonstrated, which will promote the development of fields of the optical design, AR display and holographic optics. The virtual image with good imaging performance can be observed inside the system eyebox, meanwhile, realizing a good optical see-through viewing. The proposed generalized method can be used in the design and development of novel, compact and nonsymmetric freeform holographic imaging systems such as head-mounted display, HUD, and off-axis cameras and telescopes, as well as other related fields using holographic optics.

For the conventional HOE design and fabrication, the selected recording wavefronts are spherical wavefronts or plane wavefronts in general. No complex WCE is needed to be fabricated and the recording system setup is simple. However, the system specifications and imaging performance of the imaging system using conventional HOEs are limited. For our proposed method, extra WCEs are introduced to modulate the recording wavefronts in order to fabricate the freeform HOE with complicate phase function. In this way, the recording wavefronts of HOE are no longer simple spherical or plane wavefronts. The total fabrication cost and complexity for the whole recording system setup will be higher, but freeform HOE can be fabricated and the performance of the imaging system can be improved.

Some more discussions are given here to show the limitations and recommendations related to the proposed method. (1) The fabrication cost of WCE is relatively high, and its fabrication time may be long; (2) One WCE can be only used to fabricate one corresponding HOE. The fabrication flexibility can be higher if phase spatial light modulator (SLM) is used as the WCE; (3) Currently, the design is limited to single wavelength in this paper. For a multiple wavelengths system, the phase functions of HOEs as well as related recording systems corresponding to different wavelengths are generally different and should also be designed jointly; (4) Using WCEs in the recording system may introduce some stray lights, which may affect the fabrication process of HOE; (5) If the holographic imaging systems have very small FOV or aperture, the conventional HOE can be used due to simplicity. For the holographic imaging systems with relatively large FOV and aperture, our proposed method can achieve much better imaging performance.

Funding

National Key Research and Development Program of China (2021YFB2802100); Beijing Natural Science Foundation (1222026); National Natural Science Foundation of China (U21A20140, 61805012); Young Elite Scientist Sponsorship Program by CAST (2019QNRC001).

Acknowledgments

We thank Synopsys for the educational license of CODE V.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. H. Kogelnik, “Coupled wave theory for thick hologram gratings,” Bell Syst. Tech. J. 48(9), 2909–2947 (1969). [CrossRef]

2. Y. Liu, H. Liu, B. Wang, M. Wei, L. Li, and W. Wang, “Expansion of axial dispersion in a photopolymer-based holographic lensand its improvement for measuring displacement,” Appl. Opt. 59(27), 8279–8284 (2020). [CrossRef]

3. J. Yeom, J. Hong, and J. Jeong, “Projection-type see-through near-to-eye display with a passively enlarged eye-box by combining a holographic lens and diffuser,” Opt. Express 29(22), 36005–36020 (2021). [CrossRef]

4. A. Maimone, A. Georgiou, and J. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 1–16 (2017). [CrossRef]

5. C. Jang, K. Bang, G. Li, and B. Lee, “Holographic near-eye display with expanded eye-box,” ACM Trans. Graph. 37(6), 1–14 (2018). [CrossRef]

6. J. Yeom, K. Hong, Y. Jeong, C. Jang, and B. Lee, “Solution for pseudoscopic problem in integral imaging using phase-conjugated reconstruction of lens-array holographic optical elements,” Opt. Express 22(11), 13659–13670 (2014). [CrossRef]

7. K. Hong, J. Yeom, C. Jang, J. Hong, and B. Lee, “Full-color lens-array holographic optical element for three-dimensional optical see-through augmented reality,” Opt. Lett. 39(1), 127–130 (2014). [CrossRef]

8. S. Moon, C. Lee, S. Nam, C. Jang, G. Lee, W. Seo, G. Sung, H. Lee, and B. Lee, “Augmented reality near-eye display using Pancharatnam-Berry phase lenses,” Sci. Rep. 9(1), 6616 (2019). [CrossRef]

9. J. Su, X. Yan, Y. Huang, X. Jiang, Y. Chen, and T. Zhang, “Improvement of printing efficiency in holographic stereogram printing with the combination of a field lens and holographic diffuser,” Appl. Opt. 57(25), 7159–7166 (2018). [CrossRef]

10. J. Yeom, J. Jeong, C. Jang, K. Hong, S. Park, and B. Lee, “Reflection-type integral imaging system using a diffuser holographic optical element,” Opt. Express 22(24), 29617–29626 (2014). [CrossRef]

11. D. H. Close, “Holographic optical elements,” Opt. Eng. 14(5), 145408 (1975). [CrossRef]

12. D. Lisa, C. Kevin, and F. Thomas, “Holographic data storage: Coming of age,” Nat. Photonics 2(7), 403–405 (2008). [CrossRef]

13. Y. Awatsuji, T. Tahara, A. Kaneko, T. Koyama, K. Nishio, S. Ura, T. Kubota, and O. Matoba, “Parallel two-step phase-shifting digital holography,” Appl. Opt. 47(19), D183–D189 (2008). [CrossRef]

14. G. Rakuljic and V. Leyva, “Volume holographic narrow-band optical filter,” Opt. Lett. 18(6), 459–461 (1993). [CrossRef]

15. P. Asthana, G. Nordin, A. Tanguay, and B. Jenkins, “Analysis of weighted fan-out/fan-in volume holographic optical interconnections,” Appl. Opt. 32(8), 1441–1469 (1993). [CrossRef]

16. J. Piao, G. Li, M. Piao, and N. Kim, “Full Color Holographic optical element fabrication for waveguide-type head mounted display using photopolymer,” J. Opt. Soc. Korea 17(3), 242–248 (2013). [CrossRef]

17. Y. Zhang, X. Zhu, A. Liu, Y. Weng, Z. Shen, and B. Wang, “Modeling and optimizing the chromatic holographic waveguide display system,” Appl. Opt. 58(34), G84–G90 (2019). [CrossRef]

18. C. Draper, C. Bigler, M. Mann, K. Sarma, and P. Blanche, “Holographic waveguide head-up display with 2-D pupil expansion and longitudinal image magnification,” Appl. Opt. 58(5), A251–A257 (2019). [CrossRef]

19. C. Draper and P. Blanche, “Holographic curved waveguide combiner for HUD/AR with 1-D pupil expansion,” Opt. Express 30(2), 2503–2516 (2022). [CrossRef]

20. S. Kim and J. Park, “Optical see-through Maxwellian near-to-eye display with an enlarged eyebox,” Opt. Lett. 43(4), 767–770 (2018). [CrossRef]

21. C. Yoo, M. Chae, S. Moon, and B. Lee, “Retinal projection type lightguide-based near-eye display with switchable viewpoints,” Opt. Express 28(3), 3116–3135 (2020). [CrossRef]

22. C. Jang, K. Bang, S. Moon, J. Kim, S. Lee, and B. Lee, “Retinal 3D: augmented reality near-eye display via pupil-tracked light field projection on retina,” ACM Trans. Graph. 36(6), 1–13 (2017). [CrossRef]

23. B. Jackin, L. Jorissen, R. Oi, J. Wu, K. Wakunami, M. Okui, Y. Ichihashi, P. Bekaert, Y. Huang, and K. Yamamoto, “Digitally designed holographic optical element for light field displays,” Opt. Lett. 43(15), 3738–3741 (2018). [CrossRef]

24. H. Peng, D. Cheng, J. Han, C. Xu, W. Song, L. Ha, J. Yang, Q. Hu, and Y. Wang, “Design and fabrication of a holographic head-up display with asymmetric field of view,” Appl. Opt. 53(29), H177–H185 (2014). [CrossRef]

25. C. Mu, W. Lin, and C. Chen, “Zoomable head-up display with the integration of holographic and geometrical imaging,” Opt. Express 28(24), 35716–35723 (2020). [CrossRef]

26. R. C. Fairchild and J. R. Fienup, “Computer-originated aspheric holographic optical elements,” Opt. Eng. 21(1), 211133 (1982). [CrossRef]

27. R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik (Stuttg.) 35, 237–242 (1972).

28. G. Yang, B. Gu, and B. Dong, “Theory of the amplitude-phase retrieval in any linear transform system and its applications,” Int. J. Mod. Phys. B 07(18), 3153–3224 (1993). [CrossRef]

29. J. Mendes-Lopes, P. Benítez, J. C. Miñano, and A. Santamaría, “Simultaneous multiple surface design method for diffractive surfaces,” Opt. Express 24(5), 5584–5590 (2016). [CrossRef]

30. T. Yang, D. Cheng, and Y. Wang, “Design method of nonsymmetric imaging systems consisting of multiple flat phase elements,” Opt. Express 26(19), 25347–25363 (2018). [CrossRef]

31. C. Jang, O. Mercier, K. Bang, G. Li, Y. Zhao, and D. Lanman, “Design and fabrication of freeform holographic optical elements,” ACM Trans. Graph. 39(6), 1–15 (2020). [CrossRef]

32. P. Wissmann, S. Oh, and G. Barbastathis, “Simulation and optimization of volume holographic imaging systems in Zemax®,” Opt. Express 16(10), 7516–7524 (2008). [CrossRef]

33. J. David, B. Friedrich, D. François, F. Thomas, H. Rainer, H. Dennis, R. Thomas, W. Marc-Stephan, and V. Andy, “New recording materials for the holographic industry,” Proc. SPIE 7223, 72330K (2009). [CrossRef]

Parameter	Specification
Field of view (FOV)	20° × 14°
Exit pupil diameter (EPD)	40 mm
Effective focal length (EFL)	160 mm
Eye relief (ERF)	150 mm

Design of off-axis reflective imaging systems based on freeform holographic elements

Abstract

1. Introduction

2. Design method

2.1 Basic theory of holographic imaging system

2.2 Preliminary design of the holographic imaging system

2.3 Preliminary design of the recording systems

2.4 Joint optimization of the imaging system and the recording system

3. Design examples

4. Conclusion and discussions

Funding

Acknowledgments

Disclosures

Data availability

References

Supplementary Material (1)

Data availability

Cited By

Figures (14)

Tables (1)

Equations (20)

Optics Express