Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group
Journal Home About Issues in Progress Current Issue All Issues Feature Issues

Digital-optical co-design enables athermalization of hybrid optical systems

Shan Mao, Huaile Nie, and Jianlin Zhao

Open Access Open Access

Abstract

We proposed a digital-optical co-design that can effectively improve the image quality of refractive-diffractive hybrid imaging systems over a wide ambient temperature range. Diffraction theory was used to establish the degradation model and blind deconvolution image recovery algorithm was used to perform recovery for simulated images. The peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were used to evaluate the algorithm performance. A cooled athermalization dual-band infrared optical system with a double-layer diffractive optical element (DLDOE) was designed, and the results show an overall improvement for both PSNR and SSIM overall the whole ambient temperature range. This demonstrates the effectiveness of the proposed method for the image quality improvement of hybrid optical systems.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Diffractive optical elements (DOEs) have become increasingly popular in modern optical systems due to their distinctive achromatic and thermal characteristics. Opto-electronic instruments such as infrared optical systems and high-end commercial instruments, in particular, benefit from the capacity of DOEs in effectively improving the image quality of such systems while also achieving athermalization [1]. Traditional optical lens optimization [2], optical material matching [3], and wavefront encoding [4] provide some common methods for optical system athermalization, but these approaches are often based on aberration theory and principle of optical focal length distribution, making the optical systems complex, heavy, and bulky.

A hybrid optical system composed of DOEs and traditional lenses offers a promising solution, reducing complexity, material dependence, and simplifying the design optimization process [5]. Further, multi-layer DOEs (MLDOEs) can improve the diffraction efficiency over broad to multi-wavebands. However, they are highly sensitive to temperature alteration, which degrades the imaging quality of hybrid optical systems [6]. As a result, researchers seek for optimization designs of DOEs for hybrid optical system to ensure high diffraction efficiency across wide temperature range, yet this technology hasn’t been achieved.

In recent years, modern advances in computational imaging technology have made great progress in the field of DOEs. Y Peng proposed a diffraction achromatic imaging method which allowed for full-spectrum imaging using computationally optimized diffractive lenses [7]. F Heide created a coded DOE design for full-spectrum computational imaging which combined a coded DOE and self-calibrating imaging [8]. J Yang proposed an adaptive wiener filtering (AWN), which significantly improved the details of restored images and aided with the imaging of dark, weak targets [9]. X Dun created a learning rotationally symmetric diffractive achromatic lens design with concentric ring decomposition, reducing computational complexity and memory demands tenfold when compared with traditional end-to-end optimization [10]. Y Hu proposed an optical-digital joint design, centering the central wavelength for single-layer DOE in mid-wave infrared (MWIR) and imaging reconstruction with a constructed point spread function (PSF) model and then added to long-wave infrared (LWIR) [11]. P Wang proposed a chromatic aberration corrected diffractive lens for ultra-broadband focusing [12]. K Liu used networks to complete image restoration [13]. We combined optical design with image restoration, designing a dual-band infrared optical system with a MLDOE that utilized image restoration to improve the imaging quality of hybrid optical systems over wide ambient temperature range [14].

Although there have been researches concerning single-layer DOE, a digital-optical imaging method combining optical system optimization design and blind deconvolution restoration algorithms for hybrid optical systems with MLDOEs has yet to be explored, especially for athermalization need of hybrid optical system. Such a method can reduce the effect on diffraction efficiency reduction that occurs with ambient temperatures changing and improve imaging quality across a broader range of conditions. In this paper, we propose such a method and declare the potential importance for its applications for hybrid imaging systems athermalization realization that operate at wide temperature ranges, which can ultimately reduce the design difficulty and overall cost for athermalization optical system.

2. Design principle and method

2.1 Effect on diffraction efficiency of ambient temperature for MLDOE in dual-band

DLDOE is the most typical MLDOE consisting of two harmonic DOEs of substrate materials with different dispersion properties, and it is with the same period width but different micro-structure heights, illustrated in Fig. 1.

 figure: Fig. 1.

Fig. 1. Structure of DLDOE.

Download Full Size | PDF

For imaging optical systems, since surface micro-structure parameters DOEs are much larger than the incident wavelength, scalar diffraction theory can satisfy the requirements of design and accuracy. So, at room temperature of 20°C, diffraction efficiency and band-integral average diffraction efficiency (BIADE) of DLDOE based on scalar diffraction theory can be expressed as [15]

$${\eta _m}(\lambda ) = {\textrm{sinc}^2}\left\{ {m - \frac{{{H_1}[{1 - {n_1}(\lambda )} ]+ {H_2}[{{n_2}(\lambda ) - 1} ]}}{\lambda }} \right\}$$
$${\overline \eta _m} = {\overline \eta _m}({\lambda _{\min }},{\lambda _{\max }}) = \frac{1}{{{\lambda _{\max }} - {\lambda _{\min }}}}\int_{{\lambda _{\min }}}^{{\lambda _{\max }}} {{\eta _m}(\lambda )} \textrm{d}\lambda$$
where, λ1 and λ2 are the design wavelength pair, λmin and λmax are the minimum and maximum wavelengths overall working waveband, λ is the incident wavelength, H1 and H2 are the micro-structure heights of DLDOE, n1(λ) and n2(λ) are the refractive indices of the substrates etched by DLDOE.

To ensure high optical transform function (OTF) for the dual-band DLDOE, the design wavelength pair must be selected carefully to ensure high diffraction efficiency, where, it is ensured by two design wavelengths λ1 and λ2, as

$$\overline {{{\overline \eta }_m}({\lambda _1},{\lambda _2})} = \frac{1}{2}[{\overline {{\eta _1}} _m}({\lambda _1},{\lambda _2}) + {\overline {{\eta _2}} _m}({\lambda _1},{\lambda _2})]$$

When the maximum comprehensive BIADE (CBIADE) is obtained [16], the corresponding calculation can be performed and then the result is used for subsequent optical system design. The specific steps are as follows:

  • (1) set the diffraction order to 1for the dual-band and search the entire waveband range for the maximum $\overline {{{\overline \eta }_1}({\lambda _1},{\lambda _2})}$;
  • (2) select the design wavelength pair λ1 and λ2 when the CBIADE obtains to the maximum, and the two optimal micro-structure heights can be calculated;
  • (3) calculated the two diffraction orders corresponding to the DLDOE.

Considering environment temperature T into account, its real BIADE can be expressed as

$${\overline \eta _m}(T) = {\overline \eta _m}({\lambda _{\min }},{\lambda _{\max }},T) = \frac{1}{{{\lambda _{\max }} - {\lambda _{\min }}}}\int_{{\lambda _{\min }}}^{{\lambda _{\max }}} {{\eta _m}(\lambda ,T)} \textrm{d}\lambda$$

Then, CBIADE and diffraction efficiency can be expressed as

$$\overline {{{\overline \eta }_m}({\lambda ,T} )} = \frac{1}{2}[{{{\overline {{\eta_1}} }_m}({\lambda ,T} )+ {{\overline {{\eta_2}} }_m}({\lambda ,T} )} ]$$
and
$$\scalebox{0.9}{$\displaystyle{\eta _m}(\lambda ,T) = {{\textrm{sinc}} ^2}\left\{ {\left. {m - \frac{{{H_{\textrm{a1}}}[{{n_0} - {n_1}(\lambda )} ]+ {H_1}\left[ {\frac{{\textrm{d}{n_0}}}{{\textrm{d}T}} - \frac{{\textrm{d}{n_1}(\lambda )}}{{\textrm{d}T}}} \right] + {H_{\textrm{a2}}}[{{n_2}(\lambda ) - {n_0}} ]+ {H_2}\left[ {\frac{{\textrm{d}{n_2}(\lambda )}}{{\textrm{d}T}} - \frac{{\textrm{d}{n_0}}}{{\textrm{d}T}}} \right]}}{\lambda }} \right\}} \right.$}$$
where, Ha1 and Ha2 are the real micro-structure heights with ambient temperature changing, calculated by ${H_{\textrm{a1}}} = {H_1}(1 + {\alpha _{\textrm{g1}}} \cdot \Delta T)$ and ${H_{\textrm{a2}}} = {H_2}(1 + {\alpha _{\textrm{g2}}} \cdot \Delta T)$; dn1(λ)/dT and dn2(λ)/dT are refractive index coefficients of DLDOE substrate materials affected by temperature; n0 is the refractive index of the DLDOE intermediate media, usually with air gap of n0 = 1, and dn0/dT is its refractive index temperature coefficient; αg1 and αg2 are the thermal expansion coefficient of DLDOE substrate materials; ΔT is the difference between ambient and room temperature.

2.2 Image degradation model establishment

The image degradation is expressed as

$$g(x,y) = f(x,y) \otimes h(x,y) + n(x,y)$$
where, f(x,y) stands for the original image, h(x,y) is the degradation function, n(x,y) is the noise, and g(x,y) is degraded image. For infrared optical systems, its PSF acts as a degradation function and is affected by temperature, expressed as
$$g(x,y,T) = f(x,y) \otimes h(x,y,T) + n(x,y,T)$$

For hybrid optical systems with a DLDOE, diffraction orders of degraded images are imaged on different planes. And the images are then superimposed on image surface of the 1st order degraded image, resulting in blurred images. Therefore, the total degradation image for optical system is expressed as the product of degraded image at each diffraction order in primary plane and diffraction efficiency at each order. Secondly, the corresponding diffraction efficiency of each order is expressed as the average diffraction efficiency of each order. Additionally, the degraded image at each order during superposition is represented by PSF convolution corresponding to the original image and the diffraction order.

Based on the above, the image degradation model can be represented as

$$g(x,y,T) = \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{g_m}(x,y,T) \cdot {\eta _m}(x,y,T)} + n$$
where, g(x,y,T) is the degraded image affected by temperature, m is the diffraction order, mmin and mmax are the maximum and minimum diffraction orders, gm(x,y,T) is the degraded image of the 1st order affected by temperature, ηm(x,y,T) is the diffraction efficiency of the 1st order affected by temperature, and n is the total noise.

Specifically, we first obtain the restored image affected by environmental temperature, then calculate the spatially invariant PSF affected by temperature at each level, and finally obtain the synthesized PSF affected by temperature by superposition. When replacing ηm(x,y,T) in Eq. (10) by ${\overline \eta _m}(T)$, the image degradation model can be expressed as

$$g(x,y,T) \approx \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{g_m}(x,y,T) \cdot } {\overline \eta _m}(T) + n$$

The degenerate image of each order is represented as

$${g_m}(x,y,T) = f(x,y) \otimes PS{F_m}(x,y,T)$$

Substituting Eq. (11) into Eq. (10) yields to

$$g(x,y,T) \approx \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {f(x,y) \otimes PS{F_m}(x,y,T) \cdot } {\overline \eta _m}(T) + n$$

Equation (13) is the image degradation model affected by temperature used for simulation. When BIADE is used for the calculation, the degraded image can be expressed as

$$g(x,y,T) \approx \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {f(x,y) \otimes PS{F_m}(x,y,T) \cdot } {\overline \eta _m}({\lambda _{\min }},{\lambda _{\max }},T) + n$$

2.3 Establishment of image restoration model

From Eq. (13), it can be seen that two important factors affecting the degraded image, PSF and diffraction efficiency. They are both temperature related, and result in the degraded images being a comprehensive output by temperature. Image degradation is still a process of first convolution, then multiplication and superimposing spatially variable PSFs for all orders. Because of the complex nature of the deconvolution process of spatial changing, it is necessary to consider transforming the convolution of spatial change into a spatially invariant convolution.

The substitution in Eq. (12) indicates the temperature-affected restored image. It is replaced by a spatially invariant PSF representing the different orders affected by temperature. For superposition, it subsequently represents the composite PSF affected by temperature. Then, the above method is followed to obtain the restoration results from the temperature-affected images, modeled and then presented as

$$\left\{ \begin{array}{l} g(x,y,T) \approx \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {\widehat f(x,y,T) \otimes {{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}(T) + n} \\ \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {\widehat f(x,y,T) \otimes {{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}(T)} \textrm{ = }\widehat f(x,y,T) \otimes \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}} (T)\\ \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}(T)} \textrm{ = }\widehat {PSF}(x,y,T) \end{array} \right.$$

Equation (15) is the expression of the integrated PSF influenced by temperature, when BIADE is used for the calculation, it is presented as

$$\widehat {PSF}(x,y,T) = \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}({\lambda _{\min }},{\lambda _{\max }},T) }$$

Then, combining Eqs. (15) and (16) yields to

$$g(x,y,T) \approx \widehat f(x,y,T) \otimes \widehat {PSF}(x,y,T) + n$$

Equation (20) is the image degradation model used for recovery affected by temperature.

2.4 Image quality evaluation

Image quality evaluation methods consist of subjective evaluation and objective evaluation. Subjective evaluation is based on people’s subjective judgment to directly evaluate the image quality. It mainly evaluates the clarity, contrast of images, and won’t require complex calculations. The evaluation result will be affected by multiple factors, such as the color and grayscale of the image, which may cause visual illusions and lead to incorrect judgments. The evaluation result will also be affected by the subjective factors of the evaluator, such as the evaluator's evaluation criteria, mental state, mood, hobbies, etc., so subjective evaluation cannot achieve absolute consistency. Objective evaluation method uses the error value between the obtained image and the original image to judge the quality of the image. The image quality evaluation is a function that calculates the image with a certain functional relationship, which reflects the image quality. On the basis of the original clear image, the restored image is operated with the original clear image by a certain function, and the function value is used as an evaluation criterion to evaluate the quality of the restored image.

(1) PSNR [17]

PSNR is based on the mean squared error (MSE) definition. Given a raw image of size I and noise image K after noise is added, its MSE can be defined as

$$MSE = \frac{1}{{mn}}\sum\limits_{i = 0}^{m - 1} {\sum\limits_{j = 0}^{n - 1} {{{[{I(i,j) - K(i,j)} ]}^2}} }$$
and then the PSNR is defined as
$$PSNR = 10 \cdot {\log _{10}}\left( {\frac{{MAX_I^2}}{{MSE}}} \right) = 20 \cdot {\log _{10}}\left( {\frac{{MA{X_I}}}{{\sqrt {MSE} }}} \right)$$

In Eq. (19), MAXI is the maximum pixel value of the image, PSNR is calculated in dB. If each pixel is represented by 8-bit binary, its value is 28-1 = 255. However, note that this is a calculation method for grayscale images. For a color image, it can usually be calculated by the following steps: calculate the MSE value for each of the three channels of RGB images and then average it, finally search PSNR, and the larger PSNR value means the better image quality.

(2) SSIM [18]

The basic idea of SSIM is to evaluate the similarity of two images through three aspects, namely brightness l(x,y), contrast c(x,y), and structure s(x,y), and expressed as

$$l(x,y) = \frac{{2{\mu _x}{\mu _y} + {c_1}}}{{\mu _x^2 + \mu _y^2 + {c_1}}},\;\;\; c(x,y) = \frac{{2{\sigma _x}{\sigma _y} + {c_{_2}}}}{{\sigma _x^2 + \sigma _y^2 + {c_2}}},\;\;\;s(x,y) = \frac{{{\sigma _{xy}} + {c_3}}}{{{\sigma _x}{\sigma _y} + {c_3}}}$$
where, ${\mu _x}$ and ${\mu _y}$ stand for the mean value of x and y, $\sigma _x^2$ and $\sigma _y^2$ stand for the variance of x and y, ${\sigma _x}{\sigma _y}$ stands for the covariance for x and y, in addition, c1 = (k1L)2 and c2 = (k2L)2 are two covariance and c3 is usually defined as c3 = c1/2, L is for the pixel range defined as 2B-1. The default value of K1 and K2 is usually defined as 0.01 and 0.03, and SSIM is presented as
$$SSIM(x,y) = \frac{{(2{\mu _x}{\mu _y} + {c_1})(2{\sigma _x}{\sigma _y} + {c_{_2}})}}{{(\mu _x^2 + \mu _y^2 + {c_1})(\sigma _x^2 + \sigma _y^2 + {c_2})}}$$
where, SSIM takes the value range of [0, 1], larger value means the smaller image distortion.

3. Dual-band infrared optical system design and image simulation

3.1 Cooled dual-band infrared optical system design

We optimally design a cooled dual-band infrared optical system with a DLDOE for realizing athermation overall the wide temperature range. For simplification, here we use only two optical materials for system design, germanium (Ge) and zinc selenide (ZnSe), and the number of lenses is controlled within 5 pieces. The specific indicators of the optical system and its detector are presented in Table 1.

Tables Icon

Table 1. Design indicators of the infrared optical system

View Table | View all tables in this article

The solid model of the optical system is shown in Fig. 2, which consists of 5 lenses and 1 flat glass. It is structured from left to right with the 1st, 4th, and 5th lenses made of ZnSe, the 2nd and 3rd lenses made of Ge. In addition, there are two diffractive surfaces on the rear of the 3rd and front of the 4rd lens, and 8 aspherical surfaces. The flat glass is the protective glass of the Dewar flask with material of Ge. The cold stop is implemented as the aperture stop, which is placed between the imaging surface and optical system, with the exit pupil placed at the cold stop, and the exit pupil size is the same as the cold stop, so the optical system can achieve 100% cold stop efficiency. The focal plane of the cooled detector is located inside the Dewar. The structural parameters of the optical system are given in Table 2.

 figure: Fig. 2.

Fig. 2. Solid model of the cooled infrared optical system.

Download Full Size | PDF

Tables Icon

Table 2. Optical system parameters

View Table | View all tables in this article

The aspheric polynomial coefficients are determined by ai, represented as

$$z = \frac{{c{r^2}}}{{1 + \sqrt {1 - (1 + k){c^2}{r^2}} }} + {a^i}{r^2}^i$$
where, z is the sagittal height of aspheric surface, c is the curvature, k is the conic constant, and r is the radial coordinate.

The phase delay of diffractive surface can be represented as

$$\Phi = M\sum\limits_{i = 1}^N {{A_i}{\rho ^{2i}}}$$
where, M is the diffraction order, ρ is the radial radius of the DLDOE substrate, and Ai is the polynomial coefficient.

The aspheric and DOE parameters are presented in Table 3 and Table 4, respectively.

Tables Icon

Table 3. Parameters of aspheric surfaces

View Table | View all tables in this article
Tables Icon

Table 4. Parameters of DLDOE

View Table | View all tables in this article

MTFs for the infrared optical system at -40°C, 20°C and 60°C is shown in Fig. 3, and they are greater than 0.48 and 0.40 for MWIR and LWIR at the detector cut-off frequency of 17lp/mm, respectively. The GEO and RMS radii of the system for all fields at different temperatures are presented in Table 5. At -40°C, the maximum GEO radii of 27,500µm and 18,867µm for MWIR and LWIR, respectively, larger than the pixel size of the detector. At room temperature, the maximum GEO radii for MWIR and LWIR are 14.656µm and 13.631µm, respectively. At -40°C, the maximum RMS radii for MWIR and LWIR are 14.720µm and 11.402µm, respectively, smaller than the pixel size of the detector.

 figure: Fig. 3.

Fig. 3. MTF for optical system. (a) MWIR & -40°C; (b)LWIR & -40°C; (c) MWIR & 20°C; (d) LWIR & 20°C; (e) MWIR & 60°C; (f) LWIR & 60°C.

Download Full Size | PDF

Tables Icon

Table 5. The maximum value of the GEO and RMS radius at different temperatures

View Table | View all tables in this article

We can see that, at room temperature, the optical system exhibits good imaging quality in both dual-bands. However, when the temperature deviates from room temperature, the GEO radius changes drastically and the CBIADE decreases substantially. Additionally, the GEO radius of the non-1st-order diffraction plane is much larger than the pixel size, resulting in a blurred image and deterioration of imaging quality. Consequently, the imaging quality of the designed optical system is greatly affected by the change in ambient temperature. The DLDOE design result, based on CBIADE method, is presented in Table 6.

Tables Icon

Table 6. DLDOE design result

View Table | View all tables in this article

The CBIADE versus ambient temperature for the combination of Ge and ZnSe substrates and the BIADE versus ambient temperature for MWIR and LWIR is shown in Fig. 4.

 figure: Fig. 4.

Fig. 4. CBIADE and BIADE versus ambient temperature for the combination substrates of Ge-ZnSe. (a) CBIADE versus the dual-band; (b) CBIADE versus temperature; (c) and (d) BIADE for MWIR and LWIR.

Download Full Size | PDF

3.2 Dual-band infrared image simulation

According to the previous analysis, the image degradation model of the dual-band infrared optical system can be calculated based on BIADE under the influence of ambient temperature, where the degradation image is obtained by the image simulation function with ZemaxOpticStudio software.

The image simulation process is clarified as follows: first, calculate the BIADE for each diffraction order for MWIR and LWIR at different temperatures; second, input a clear infrared image into the optical system as a light source bitmap, and each diffraction order image is output at certain temperature; third, multiply the images at each diffraction order with its corresponding BIADE and superimposing, the resulting image is approximately a simulated image at the certain temperature.

The relationship between BIADE and ambient temperature at diffraction orders for MWIR and LWIR is shown in Fig. 5, with only 5 diffraction orders of BIADE being plotted here due to its diffraction efficiency distribution.

 figure: Fig. 5.

Fig. 5. BIADE versus ambient temperature at each diffraction order. (a) MWIR; (b) LWIR.

Download Full Size | PDF

The BIADE of MWIR and LWIR at different temperature are listed in Table 7. After calculation, the average of total BIADE at each temperature for MWIR and LWIR are 95.359% and 97.717%, respectively, which are relatively high and can be used for image simulation.

Tables Icon

Table 7. BIADE at different temperatures (%)

View Table | View all tables in this article

Then, we simulate the imaging by inputting a clear infrared image to the optical system, the size of the light source bitmap is set to 320*240 pixels, shown in Fig. 6.

 figure: Fig. 6.

Fig. 6. Light source bitmap.

Download Full Size | PDF

Images at diffraction orders are output for MWIR and LWIR, and comparisons are shown in Table 8, where the sub-graphs (a1) ∼(a6) represent 1st, 2nd, 3rd, 0, -1st diffraction order and simulation image, respectively.

Tables Icon

Table 8. Image comparison at different temperatures

View Table | View all tables in this article

It can be seen that the diffraction images at the 1st diffraction order are relatively clear at each temperature, indicating that the athermal effect on the optical system is the best. The images at other diffraction orders, however, are quite blurred because the non-primary diffraction images are strongly scattered in primary images. The LWIR non-primary diffraction images are in particular more blurred than MWIR images, because LWIR non-primary diffraction images are more diffused, which is also due to the fact that the absolute values of the diffraction orders of DLDOE are smaller in LWIR than in MWIR. Interestingly, despite this, the diffraction efficiency of the 1st order for LWIR is more severe at each temperature, making the LWIR simulation image clearer than the MWIR image. As the ambient temperature further deviates from the normal temperature, the blurriness of the image becomes more severe.

4. Image restoration and analysis

For PSF establishment of dual-band infrared optical system, taking working temperature of -20°C as an example, the PSF at different diffraction orders (${\widehat {PSF}_m}(x,y,T)$) and synthetic PSF ($\widehat {PSF}(x,y,T)$) outputs for MWIR and LWIR is shown in Fig. 7, where (a1)∼(a6) and (b1)∼(b6) stand for the 1st, 2nd, 3rd, 0, -1st and the comprehensive PSF, respectively.

 figure: Fig. 7.

Fig. 7. PSF at -20°C for MWIR and LWIR for different diffraction orders. (a) MWIR and (b) LWIR.

Download Full Size | PDF

Since the PSF for each pixel is small and the actual display is dark, Fig. 7 is magnified results of multiplying the actual value by a certain multiple, where the sub-graph Fig. 7(a1) is magnified 30 times, Figs. 7(a2) and (a4) are magnified 300 times, Figs. 7(a3) and (a5) are magnified 1000 times, and Fig. 7(a6) are magnified 100 times. In Fig. 7(b), Fig. 7(b1) is magnified by a factor of 2, Fig. 7(b2) and (b4) are magnified by a factor of 1000 and Fig. 7(b3), (b5) and (b6) are magnified by a factor of 2000. It can be seen that the 1st diffraction order has a large effect on the overall PSF, the 0th and 2nd diffraction order have a small effect on the overall PSF, and the -1st and 3rd order diffraction have a small effect on the overall PSF.

Then, without considering the image noise for image restoration, a blind deconvolution algorithm is used for image restoration. The simulated images of MWIR and LWIR at each temperature are restored and compared with the simulated images before restoration, shown in Fig. 8, where the sub-graphs (a1) and (a2) are simulation and restoration diagrams at -40°C; (a3) and (a4) are simulation and restoration diagrams at -20°C; (a5) and (a6) are simulation and restoration diagrams at 0°C; (a7) and (a8) are simulation and restoration diagrams at 40°C; (a9) and (a10) are simulation and restoration diagrams at 60°C. Additional, sub-graphs for LWIR is the same as the MWIR imaging.

 figure: Fig. 8.

Fig. 8. Comparison before and restoration of (a)MWIR and (b)LWIR.

Download Full Size | PDF

The restored image and the simulation image are compared with light source bitmap using PSNR and SSIM for calculation, respectively, and the results are shown in Table 9. It can be seen that the image quality is effectively improved within the ambient temperature range.

Tables Icon

Table 9. Result comparisons

View Table | View all tables in this article

5. Conclusion

We presented a digital-optical co-design for athermalization of a refractive-diffractive hybrid imaging system in a wide ambient temperature range with a DLDOE. An image degradation model of DLDOE was established by converting the spatially varying PSF into a spatially invariant PSF. An image degradation model for restoration was also put forward and presented. A cooled dual-band refractive infrared hybrid optical system with a DLDOE was then designed achieving athermalization with good satisfied image quality using only two common optical materials, Ge and ZnSe, for the wavebands of 3.7-4.8µm and 7.7-10µm, working temperature range of -40∼60°C, focal length of 105 mm, F-number of 1.5, and full field of view of 6°. The optical system was used for image simulation, with a blind reconstruction algorithm used for image restoration. Finally, PSNR and SSIM evaluation functions were used to evaluate its image quality. Results showed that the quality of the restored image was improved at each temperature in both dual-bands. For instance, for MWIR, when the ambient temperature was -20°C, the diffraction efficiency of the 1st order decreased to about 40%, with the restored image displaying the best quality improvement in terms of both the PSNR increased from 23.4926 to 27.2919, and the SSIM increased from 0.8154 to 0.9296. These results confirmed the effectiveness of the proposed method in improving the image quality over a wide ambient temperature range. Our method and result can help to promote the wide temperature application of DOE while enriching the application range of computational imaging technologies.

Funding

National Natural Science Foundation of China (61905195, 61927810); Northwestern Polytechnical University Central University Basic Research Business (310202011QD001); Key Laboratory of Spectral Imaging Technology, Chinese Academy of Sciences (LSIT202006W); Xi'an Young Talents Sponsorship Project (095920201316).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. T. H. Jamieson, “A thermalization of optical instruments from the optomechanical viewpoint,” Pro. SPIE 10265, 1026508 (1992). [CrossRef]  

2. T. Y Lim and S. CPark, “Achromatic and athermal lens design by redistributing the element powers on an athermal glass map,” Opt. Express 24(16), 18049–18058 (2016). [CrossRef]  

3. S. E. Ivanov and G. E. Romanova, “Optical material selection method for an apochromatic athermalized optical system,” J. Opt. Technol. 83(12), 729–733 (2016). [CrossRef]  

4. B. Feng, Z. L. Shi, H. Z. Liu, and Y. H. Zhao, “Effect of PSF position bias in wavefront coding infrared imaging system,” J. Opt. 22(2), 025703 (2020). [CrossRef]  

5. C. R. Bigwood, “New infrared optical systems using diffractive optics,” Pro. SPIE 4767, 1–12 (2002). [CrossRef]  

6. M. X. Piao, Q. F. Cui, H. Zhu, C. X. Xue, and B. Zhang, “Diffraction efficiency change of multilayer diffractive optics with environmental temperature,” J. Opt. 16(3), 035707 (2014). [CrossRef]  

7. Y. Peng, Q. Fu, F. Heide, and W. Heidrich, “The diffractive achromat full spectrum computational imaging with diffractive optics,” SIGGRAPH ASIA 2016 Virtual Reality meets Physical Reality: Modelling and Simulating Virtual Humans and Environments, 1–2 (2016).

8. F. Heide, Q. Fu, Y. Peng, and F. Heidrich, “Encoded diffractive optics for full-spectrum computational,” Sci. Rep. 6(1), 33543 (2016). [CrossRef]  

9. J. Yang, S. Wang, L. Wen, P. Yang, W. Yang, C. Guan, and B. Xu, “Experimental study on imaging and image deconvolution of a diffractive telescope system,” Appl. Opt. 58(33), 9059–9068 (2019). [CrossRef]  

10. X. Dun, H. Ikoma, G. Wetzstein, Z. Wang, X. Cheng, and Y. Peng, “Learned rotationally symmetric diffractive achromat for full-spectrum computational imaging,” Optica 7(8), 913–922 (2020). [CrossRef]  

11. Y. Hu, Q. F. Cui, L. D. Zhao, and M. X. Piao, “PSF model for diffractive optical elements with improved imaging performance in dual-waveband infrared systems,” Opt. Express 26(21), 26845–26857 (2018). [CrossRef]  

12. P. Wang, N. Mohammad, and R. Menon, “Chromatic-aberration-corrected diffractive lenses for ultra-broadband focusing,” Sci. Rep. 6(1), 21545 (2016). [CrossRef]  

13. K. Liu, X. Yu, Y. Xu, Y. Xu, Y. Yao, N. Di, Y. Wang, H. Wang, and H. Shen, “Computational imaging for simultaneous image restoration and super-resolution image reconstruction of single-lens diffractive optical system,” Appl. Sci. 12(9), 4753 (2022). [CrossRef]  

14. S. Mao, H. L. Nie, T. Lai, and N. Xie, “Computational imaging in dual-band infrared hybrid optical system with wide temperature range,” Sensors 22(14), 5291 (2022). [CrossRef]  

15. N. Takehiko and O. Hideki, “Research on multi-layer diffractive optical elements and their application to camera lenses,” Optical Society of America, (2002).

16. C. X. Xue, Q. F. Cui, T. Liu, L. L. Yang, and B. Fei, “Optimal design of a multilayer diffractive optical element for dual wavebands,” Opt. Lett. 35(24), 4157–4159 (2010). [CrossRef]  

17. P. Garg and R. R. Kishore, “Optimized color image watermarking through watermark strength optimization using particle swarm optimization technique,” J. Inform. Opt. Sci. 41(6), 1499–1512 (2020). [CrossRef]  

18. Z. Wang, A. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)
Fig. 1.
Fig. 1. Structure of DLDOE.

View in Article | Download Full Size | PDF

Fig. 2.
Fig. 2. Solid model of the cooled infrared optical system.

View in Article | Download Full Size | PDF

Fig. 3.
Fig. 3. MTF for optical system. (a) MWIR & -40°C; (b)LWIR & -40°C; (c) MWIR & 20°C; (d) LWIR & 20°C; (e) MWIR & 60°C; (f) LWIR & 60°C.

View in Article | Download Full Size | PDF

Fig. 4.
Fig. 4. CBIADE and BIADE versus ambient temperature for the combination substrates of Ge-ZnSe. (a) CBIADE versus the dual-band; (b) CBIADE versus temperature; (c) and (d) BIADE for MWIR and LWIR.

View in Article | Download Full Size | PDF

Fig. 5.
Fig. 5. BIADE versus ambient temperature at each diffraction order. (a) MWIR; (b) LWIR.

View in Article | Download Full Size | PDF

Fig. 6.
Fig. 6. Light source bitmap.

View in Article | Download Full Size | PDF

Fig. 7.
Fig. 7. PSF at -20°C for MWIR and LWIR for different diffraction orders. (a) MWIR and (b) LWIR.

View in Article | Download Full Size | PDF

Fig. 8.
Fig. 8. Comparison before and restoration of (a)MWIR and (b)LWIR.

View in Article | Download Full Size | PDF

Tables (9)

Tables Icon

Table 1. Design indicators of the infrared optical system

View Table | View all tables in this article
Tables Icon

Table 2. Optical system parameters

View Table | View all tables in this article
Tables Icon

Table 3. Parameters of aspheric surfaces

View Table | View all tables in this article
Tables Icon

Table 4. Parameters of DLDOE

View Table | View all tables in this article
Tables Icon

Table 5. The maximum value of the GEO and RMS radius at different temperatures

View Table | View all tables in this article
Tables Icon

Table 6. DLDOE design result

View Table | View all tables in this article
Tables Icon

Table 7. BIADE at different temperatures (%)

View Table | View all tables in this article
Tables Icon

Table 8. Image comparison at different temperatures

View Table | View all tables in this article
Tables Icon

Table 9. Result comparisons

View Table | View all tables in this article

Equations (22)

Equations on this page are rendered with MathJax. Learn more.

$${\eta _m}(\lambda ) = {\textrm{sinc}^2}\left\{ {m - \frac{{{H_1}[{1 - {n_1}(\lambda )} ]+ {H_2}[{{n_2}(\lambda ) - 1} ]}}{\lambda }} \right\}$$
$${\overline \eta _m} = {\overline \eta _m}({\lambda _{\min }},{\lambda _{\max }}) = \frac{1}{{{\lambda _{\max }} - {\lambda _{\min }}}}\int_{{\lambda _{\min }}}^{{\lambda _{\max }}} {{\eta _m}(\lambda )} \textrm{d}\lambda$$
$$\overline {{{\overline \eta }_m}({\lambda _1},{\lambda _2})} = \frac{1}{2}[{\overline {{\eta _1}} _m}({\lambda _1},{\lambda _2}) + {\overline {{\eta _2}} _m}({\lambda _1},{\lambda _2})]$$
$${\overline \eta _m}(T) = {\overline \eta _m}({\lambda _{\min }},{\lambda _{\max }},T) = \frac{1}{{{\lambda _{\max }} - {\lambda _{\min }}}}\int_{{\lambda _{\min }}}^{{\lambda _{\max }}} {{\eta _m}(\lambda ,T)} \textrm{d}\lambda$$
$$\overline {{{\overline \eta }_m}({\lambda ,T} )} = \frac{1}{2}[{{{\overline {{\eta_1}} }_m}({\lambda ,T} )+ {{\overline {{\eta_2}} }_m}({\lambda ,T} )} ]$$
$$\scalebox{0.9}{$\displaystyle{\eta _m}(\lambda ,T) = {{\textrm{sinc}} ^2}\left\{ {\left. {m - \frac{{{H_{\textrm{a1}}}[{{n_0} - {n_1}(\lambda )} ]+ {H_1}\left[ {\frac{{\textrm{d}{n_0}}}{{\textrm{d}T}} - \frac{{\textrm{d}{n_1}(\lambda )}}{{\textrm{d}T}}} \right] + {H_{\textrm{a2}}}[{{n_2}(\lambda ) - {n_0}} ]+ {H_2}\left[ {\frac{{\textrm{d}{n_2}(\lambda )}}{{\textrm{d}T}} - \frac{{\textrm{d}{n_0}}}{{\textrm{d}T}}} \right]}}{\lambda }} \right\}} \right.$}$$
$$g(x,y) = f(x,y) \otimes h(x,y) + n(x,y)$$
$$g(x,y,T) = f(x,y) \otimes h(x,y,T) + n(x,y,T)$$
$$g(x,y,T) = \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{g_m}(x,y,T) \cdot {\eta _m}(x,y,T)} + n$$
$$g(x,y,T) \approx \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{g_m}(x,y,T) \cdot } {\overline \eta _m}(T) + n$$
$${g_m}(x,y,T) = f(x,y) \otimes PS{F_m}(x,y,T)$$
$$g(x,y,T) \approx \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {f(x,y) \otimes PS{F_m}(x,y,T) \cdot } {\overline \eta _m}(T) + n$$
$$g(x,y,T) \approx \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {f(x,y) \otimes PS{F_m}(x,y,T) \cdot } {\overline \eta _m}({\lambda _{\min }},{\lambda _{\max }},T) + n$$
$$\left\{ \begin{array}{l} g(x,y,T) \approx \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {\widehat f(x,y,T) \otimes {{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}(T) + n} \\ \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {\widehat f(x,y,T) \otimes {{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}(T)} \textrm{ = }\widehat f(x,y,T) \otimes \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}} (T)\\ \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}(T)} \textrm{ = }\widehat {PSF}(x,y,T) \end{array} \right.$$
$$\widehat {PSF}(x,y,T) = \sum\limits_{m = {m_{\min }}}^{{m_{\max }}} {{{\widehat {PSF}}_m}(x,y,T) \cdot {{\overline \eta }_m}({\lambda _{\min }},{\lambda _{\max }},T) }$$
$$g(x,y,T) \approx \widehat f(x,y,T) \otimes \widehat {PSF}(x,y,T) + n$$
$$MSE = \frac{1}{{mn}}\sum\limits_{i = 0}^{m - 1} {\sum\limits_{j = 0}^{n - 1} {{{[{I(i,j) - K(i,j)} ]}^2}} }$$
$$PSNR = 10 \cdot {\log _{10}}\left( {\frac{{MAX_I^2}}{{MSE}}} \right) = 20 \cdot {\log _{10}}\left( {\frac{{MA{X_I}}}{{\sqrt {MSE} }}} \right)$$
$$l(x,y) = \frac{{2{\mu _x}{\mu _y} + {c_1}}}{{\mu _x^2 + \mu _y^2 + {c_1}}},\;\;\; c(x,y) = \frac{{2{\sigma _x}{\sigma _y} + {c_{_2}}}}{{\sigma _x^2 + \sigma _y^2 + {c_2}}},\;\;\;s(x,y) = \frac{{{\sigma _{xy}} + {c_3}}}{{{\sigma _x}{\sigma _y} + {c_3}}}$$
$$SSIM(x,y) = \frac{{(2{\mu _x}{\mu _y} + {c_1})(2{\sigma _x}{\sigma _y} + {c_{_2}})}}{{(\mu _x^2 + \mu _y^2 + {c_1})(\sigma _x^2 + \sigma _y^2 + {c_2})}}$$
$$z = \frac{{c{r^2}}}{{1 + \sqrt {1 - (1 + k){c^2}{r^2}} }} + {a^i}{r^2}^i$$
$$\Phi = M\sum\limits_{i = 1}^N {{A_i}{\rho ^{2i}}}$$
Select as filters
Select Topics Cancel
Top
       
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.
Privacy | Terms of Use