1. Introduction

To describe quasi-single frequency light propagation through an optical system, the paraxial approximation is often assumed [1,2]. That is, all the rays are supposed to be nearly parallel to the optical axis, forming a well collimated beam. However, this assumption is rarely fulfilled in real situations, so that the light characteristics after the optical system deviate from the ideal case of the paraxial approximation [1,3,4]. These deviations, known as monochromatic aberrations (MAs), lead directly to a reduction in the performance of the optical system. Therefore, rigorous analysis of MAs is essential for its better design and implementation [5–7].

It is well known that the wavefront can be expressed as a superposition of Zernike polynomials, and that its decomposition into Zernike polynomials and the identification of the coefficient of each Zernike polynomial can lead to a complete analysis of the MAs of an optical system [8–13]. Since the aberrations arise from phase distortion, the interferogram plays a crucial role in wavefront analysis [14]. The single-shot method, which uses the Fourier transform of the spatial frequency components, and the multi-step method, which measures fringe patterns with different optical path differences (OPDs), are well-known interferometric techniques for identifying the Zernike coefficients and thus the given aberrations [14–16]. In theory, it is quite straightforward to extract Zernike coefficients and to retrieve the phase distribution from the measured interferometric pattern; however, phase wrapping typically occurs in the decomposition process, so a suitable algorithm that can unwrap the phase with high speed and high accuracy is always required [11,15,17]. In addition, a large tilt angle required in the single-shot measurement, vibrations, and errors in determining the OPDs in the multi-step measurement can degrade the aberration reconstruction [14,15,18]. In this respect, improving the robustness of the measurement to noise without sacrificing the speed of measurement and analysis is a critical issue in phase retrieval problems.

Recently, several methods have shown that deep learning can significantly improve the performance of phase retrieval with a smaller number of measurements. For example, a method of training four interferometric patterns with corresponding aberrations has been proposed without the use of a phase unwrapping algorithm; however, it is not entirely free from the factors such as vibrations and OPD errors, which may ultimately reduce the aberration analysis performance. [16]. While the use of deep learning has the advantage of reducing the computational time and the number of measurements required compared to the methods based on unwrapping algorithms, it has a clear limitation in that it requires a large number of training samples, which are multiple interferometric patterns and corresponding Zernike coefficients. [16,19,20]. Synthetic patterns can be generated numerically to overcome the lack of training samples [16]; however, such a method can suffer from performance degradation if there are significant discrepancies between synthetic and real-world aberration patterns.

This study reports on a deep learning based aberration analysis method that exhibits excellent performance in terms of root mean square error (RMSE), peak signal-to-noise ratio (PSNR), and noise robustness. This algorithm requires only two measurements for phase retrieval by training a deep learning network to provide a direct prediction of the Zernike coefficients. To train a high-performance deep learning network, a novel synthetic data generation method is also proposed, which can probabilistically synthesize near-realistic aberration patterns by exploiting the distribution statistics of the real-world Zernike coefficients initially obtained from the conventional 4-step measurements under a variety of realistic conditions. Based on the proposed 2-step method, a synthetic dataset of 200,000 near-realistic aberrations is statistically generated from only 15 sets of real-world aberration patterns under the so-called asymptotic condition, with which the deep learning network is trained to cope with a variety of different aberrations occurring in real-world optical systems. Through a combinatorial investigation applying the proposed 2-step method to both numerically generated and experimentally measured aberration patterns, it is verified that the proposed 2-step method exhibits excellent operational performance over other existing 4-step iterative methods for Zernike decomposition and aberration reconstruction.

2. Background theory and ideas

2.1 MAs and Zernike polynomial representation

Figure 1 below illustrates the conceptual description of MAs. When the input light is well collimated and aligned, it should be focused at a single point after passing through the lens [see Fig. 1(a)]. However, in real situations the input light is often not perfectly collimated and aligned due to various reasons such as optical misalignments, collimation errors, imperfections in optical components, and material impurities. These non-parallel light rays cause unwanted distortions such as defocus, astigmatism, coma, field curvature, etc., as shown in Fig. 1(b).

 

Fig. 1. Conceptual description on the MAs: (a) Paraxial input light and MA-free focusing. (b) Non-paraxial input light induced MAs. Examples of Zernike polynomials and their corresponding aberrations: (c) Z₂: x-tilt, (d) Z₃: astigmatism, (e) Z₄: defocus, and (f) Z₂₀: pentafoil.

Download Full Size | PDF

In principle, the MAs can be represented by Zernike polynomials as given below:

For simplicity, the above double-index Zernike expression using n and m can be mapped to a single-index expression based on some specific indexing rules such as Noll’s indices, Wyant indices, Arizona indices, and OSA indices [18]. In this paper, we follow the convention of OSA indices. Since the Zernike polynomials are a complete set of orthogonal functions, any complex aberration Φ can be represented by their superposition, as given below:

2.2 Interferogram based Zernike decomposition and aberration reconstruction

As shown in Fig. 1, MAs arise from wavefront distortion, i.e., phase distortion. Therefore, interferometric measurement plays an important role in investigating and decomposing the wavefront into Zernike polynomials. Figure 2 below shows the experimental configuration based on a Michelson interferometer that was built for the wavefront measurement in this study. When an input collimated 1064-nm laser beam (QDBRLD-1064-150, Qphotonics) enters the 50:50 beam splitter (CCM1-BS014/M, Thorlabs), it is split into transmitted and reflected beams of equal power. The transmitted beam then passes through a test lens (LA1145-B, Thorlabs), and the reflected beam is used as a reference beam for a mirror on a piezo-transducer stage. After the lens, the transmitted beam is reflected by the concave spherical mirror (43-469, Edmund Optics) and combined with the reference beam at the beam splitter. The combined beam is focused after passing through the imaging lens (ACL3026-B, Thorlabs), and a pinhole is located at the focal length of the imaging lens for filtering. A charge-coupled device (CCD) camera (CS135MUN, Thorlabs) placed after the pinhole records the interferometric fringe patterns. The measurement is repeated for different positions of the movable mirror, which is varied by the piezo-transducer stage with a step of λ/4. As a result, the intensity recorded at the photodetector can be expressed by

 

Fig. 2. Experimental configuration based on a Michelson interferometer for the wavefront measurement.

Download Full Size | PDF

The use of fewer steps may reduce the time required for the measurement; however, since the length order required to induce the phase shift is typically a few hundred nm, it is quite difficult to control the length difference that corresponds exactly to the phase shift of integral multiples of π/2. By obtaining more fringes at different path differences, the error from the imprecise phase control mentioned above is reduced, resulting in a calibration effect. In particular, for the 5-step measurement, I_λ gives the same fringe pattern as that of I₀ as long as the path difference λ is perfectly matched to the wavelength of the input light. Then the aberration expression for the 5-step measurement has exactly the same form as that for the 4-step measurement. In general, more measurements can be helpful for fine calibration; however, they can also be subject to auxiliary errors due to small misalignments or vibrations. This aspect can undermine the advantages of multi-step methods, which may end up giving a completely wrong result [14]. Therefore, if there is an algorithm that has strengths in both measurement speed and accuracy, it should be preferred over other aberration analysis algorithms.

2.3. Deep-learning-based Zernike decomposition with 2-step measurements

The concept of inferring the output via the trained relationship between the input and output without performing any computation makes deep learning a favorable candidate for avoiding phase unwrapping during Zernike decomposition. In practice, studies using a convolutional neural network (CNN) to train the interferometric fringe patterns obtained from the 4-step measurement with the wrapped phases have shown acceptable operational performance in terms of computational speed and accuracy [16,21,22]. However, considering that at least thousands of aberration data should be used for the learning process, the fact that it still requires four sequential measurements for each aberration dataset can be a critical issue in terms of time consumption and errors caused by the large number of multiple measurements.

In this study, we use a CNN that can perform Zernike decomposition based on only two measurements, thus significantly reducing the number of measurements required. In particular, the CNN is made to learn an underlying nonlinear mapping between the two aberration patterns and corresponding Zernike coefficients from the training dataset. Existing studies that have used a CNN for Zernike decomposition have some limitations in that a large amount of training data is essential to achieve reasonable performance in real-world systems [21,22]. Since measuring aberration patterns from real-world systems is obviously a demanding and time-consuming task, synthetic aberration patterns can be exploited [16], which should somehow be generated massively and automatically such that they remain “near-realistic” compared to the real-world aberration patterns. Notwithstanding, the use of synthetic data for training still has a problem that the CNN trained on it can generally show poor performance on real aberration patterns if there are significant discrepancies between synthetic and real-world aberration patterns, as shown in Figs. 3(a) and 3(b), respectively. This consequence is mainly due to the fact that the synthetic data set is usually generated by randomly selecting Zernike polynomial orders that are too arbitrary, i.e., it is determined on the basis of a uniform probability distribution without taking into account any distribution statistics of the Zernike coefficients of the real-world aberration patterns [16], although in realistic experimental situations MAs usually arise from a few dominant sources among the various phase distorting factors that can occur when light passes through an optical system, and thus exhibit some specific distribution statistics in the corresponding Zernike coefficient strengths.

 

Fig. 3. Measured and generated interferometric aberration patterns: (a) Measured real-world aberration pattern, (b) synthetic aberration pattern randomly sampled from a uniform probability distribution, and (c) synthetic aberration pattern statistically generated by the near-realistic synthetic data generation under the asymptotic condition.

Download Full Size | PDF

Therefore, we propose a novel synthetic data generation method, the so-called near-realistic synthetic data generation method under the asymptotic condition, to generate “real-world-like” aberration patterns based on the statistics of the real-world fringe patterns of a finite number, the latter of which are obtained by means of direct 4-step measurements under a variety of experimental conditions. In fact, the aberration pattern shown in Fig. 3(c) represents an example pattern out of the 200,000 synthetic aberrations statistically generated based on the proposed 2-step method. Although it is not necessarily the same as the real-world aberration pattern shown in Fig. 3(a), it has a very similar format to the real-world one, indicating that it was generated under much more realistic conditions than that of simple random selection without considering the statistics of the real-world aberration patterns. This is indeed the main significance of the proposed 2-step method, which, to the best of our knowledge, has not yet been demonstrated before. The details of how the near-realistic synthetic data generation method is implemented together with the CNN to be used are discussed in the following sections.

2.3.1. Near-realistic synthetic data generation under the asymptotic condition

To generate the near-realistic synthetic data, the first step is to acquire a finite number of real-world aberration patterns under a wide range of measurement conditions and extract their relevant statistical distribution parameters, i.e., the mean and variance parameters of the corresponding Zernike coefficients. The whole process of collecting the Zernike coefficients of the real-world aberration patterns is summarized in Fig. 4.

 

Fig. 4. Collection of the Zernike coefficients from the real-world aberration patterns of a finite number.

Download Full Size | PDF

In order to obtain real-world aberration patterns of a finite number, repeated 4-step based aberration measurements are performed using the aforementioned Michelson interferometer. For each measurement, the corresponding Zernike coefficients are extracted using the 4-step methods to form ${{\boldsymbol a}_j}$, which is a $({d + 1} )$-dimensional vector representing the Zernike coefficients collected from the $j$-th 4-step measurement trial. N times measurements will result in a real-world dataset of $\{{{{\boldsymbol a}_j}} \}_{j = 1}^N$. Hereafter, a group of these Zernike coefficient vectors representing the corresponding aberration patterns is called as a dataset.

In addition, the Zernike coefficients are collected using the derivative fitting method by taking the first-order difference between the true phase and the wrapped phase as the cost function [15]:

The statistical distribution parameters of the measured real-world dataset are then estimated, assuming that it is sampled from the population as follows:

 

Fig. 5. Near-realistic synthetic data generation under the asymptotic condition: The black cross and shaded area indicate the mean and standard deviation range by the Gaussian density estimation, respectively.

Download Full Size | PDF

The real-world dataset should be obtained under a variety of different conditions so that the measured aberration patterns represent a wide range of realistic aberrations, the size of which should also be large enough to provide relevant statistical distribution parameters of themselves. To obtain the “true” distributions of the real-world Zernike coefficients, it may be necessary to take an infinite number of measurements, which is practically impossible or irrelevant. Nevertheless, the asymptotic behaviors of the statistical distribution parameters of the measured real-world dataset can be readily derived as its size increases to infinity. Since the mean and variance parameters of the measured real-world dataset can be viewed as those of a dataset sampled from the population, as N goes to infinity, they will eventually converge to those of the population. It is therefore necessary to find out this asymptotic behavior of the statistical distribution parameters of the measured real-world dataset as its size increases. The key to answering this question lies in the rotational symmetry of the Zernike polynomials and the rotational invariance of the statistical distribution parameters of the asymptotic real-world aberration patterns. Note that the asymptotic real-world dataset hereafter means a virtual dataset of real-world aberration patterns obtained with an infinite number of trials under a variety of realistic conditions.

It is worth noting that Zernike polynomials are periodic in the azimuthal direction and therefore have rotational symmetry. This implies that the statistics of any Zernike coefficient should eventually exhibit a circularly symmetric or rotational-invariant distribution as the number of real-world aberration patterns increases to infinity. That is, it should be pointless to choose a particular azimuthal orientation with the asymptotic real-world dataset, since there must be infinitely many aberration patterns with infinitely many different azimuthal orientations. Conversely, under the asymptotic condition, a single measurement has the same effect as measuring all the cases with the same aberration pattern but with different orientations. This means that the asymptotic real-world dataset can be obtained even with a single aberration pattern by simply varying its azimuthal orientation, although it may represent too specific a situation. The statistical distribution parameters of the asymptotic real-world dataset can therefore be derived from those of the real-world dataset obtained with a finite number of measurements.

The procedure for implementing this asymptotic condition is slightly different depending on whether the second index for the Zernike coefficients, denoted by m, is 0 or not. When m = 0, the mean and variance of the Zernike coefficients of the measured real-world dataset can be taken as those of the asymptotic real-world dataset, since the Zernike polynomials with m = 0 are obviously independent of azimuthal rotation. When m ≠ 0, the mean of the Zernike coefficients of an individual order of the asymptotic dataset should vanish, while the variance of them should remain identical for both even and odd cases, which is easily understood in that the Zernike polynomials with m ≠ 0 are periodic in the azimuthal direction and there must be an infinite number of different cases in terms of the azimuthal orientation in the asymptotic real-world dataset. Thus, when m ≠ 0, zero can simply be taken as the mean of the Zernike coefficients of the specific order of the asymptotic real-world dataset while the mean of the variances of the even and odd coefficients of the measured real-world dataset can be taken as the variance of the corresponding Zernike coefficients of the asymptotic dataset. Using these “updated” statistical distribution parameters of the asymptotic real-world dataset projected from the measured real-world dataset, a synthetic dataset of any size can be statistically generated together with the aforementioned Gaussian distribution estimation. Since this synthetic dataset contains a wide range of different aberrations spanned by the updated statistical distribution parameters, the aberration analysis capability of the network to be trained with the synthetic dataset is not limited only to the cases of the measured real-world aberrations. In fact, this broad coverage contributes to the general applicability of the proposed deep learning based aberration analysis method, making it a valuable tool for analyzing a wide range of optical systems beyond the ideally corrected ones.

2.3.2. Training method and inference

In deep learning, a CNN is one of the most popular networks that has consistently shown strong performances on image datasets [23]. Therefore, the favorable capability of a CNN is applied to the Zernike decomposition problem. The input of the CNN is a concatenation of fringe patterns obtained from the measurements by the Michelson interferometer measurements at different OPDs, and the output of the CNN is a vector representing the Zernike coefficients of the corresponding MAs. The number of aberration patterns required for inference is set to two, which is, in fact, the minimum number required to obtain proper aberration analysis without the conjugate phase retrieval problem [26]. The size of the output vector, i.e., the number of the Zernike coefficients considered for MA analysis is determined to a relevant number, denoted by d + 1, by inspecting the overall MA characteristics of the measured real-world aberration patterns, as mentioned above that in realistic experimental situations MAs usually arise from a few dominant sources among the various phase distorting factors. The CNN architectures can be separated into two subgroups as shown in Fig. 6. The first group is a feature extractor that plays an important role in extracting useful visual information from the input fringe patterns. The feature extractor consists of a combination of convolution layers and pooling layers. The second group consists of fully connected layers that predict the Zernike coefficients from the extracted visual features from the feature extractor. In fact, the detailed CNN architecture can have different configurations of the feature extractors and the fully connected layers. For instance, the size of the filters, the number of filters, and the stride of the convolution operations can be different depending on the type of the CNN architecture. For each set of Zernike coefficients two synthetic real-world fringe patterns are generated with two different OPDs. The CNN is then trained and optimized by minimizing the mean square errors with the generated synthetic real-world dataset together with the corresponding Zernike coefficients. In the test phase, the resulting CNN is to infer the Zernike coefficients from just two aberration patterns obtained from the Michelson interferometer measurements at two different OPDs.

 

Fig. 6. Architecture of the neural network for the proposed 2-step aberration analysis method.

Download Full Size | PDF

3. Results and discussion

It is worth noting that we used a computer system based on Intel Core i7-13700KF CPU (3.4 GHz), NVIDIA GTX3090 GPU, and 16 GB of memory for the entire computational work of this study, including data collection, training, and network testing.

3.1. Collection of real-world aberration patterns, generation of the near-realistic synthetic aberration patterns, and training

As a rule of thumb, 15 different fringe patterns were experimentally obtained under a variety of measurement conditions, i.e., N = 15, using the Michelson interferometer shown in Fig. 2. The asymptotic condition was then applied to the measured real-world aberration patterns to obtain the distribution statistics of their corresponding Zernike coefficients. As discussed in Section 2.3.1, the estimated results for ${\mu _{RW}}$ and ${\mathrm{\sigma }_{RW}}$ are shown in the inset of Fig. 7. In the case of m ≠ 0, the respective ${\mathrm{\sigma }_{RW}}$ values were first estimated for the even and odd cases, and then the average of the variances by these two values was taken as the asymptotic variances for both cases as shown in the main illustration of Fig. 7. It was observed that the Zernike coefficients of orders of 0 to 5 dominated, while those higher than order 5 usually remained close to zero. This observation clearly indicates that the lens used in our experiment mainly produced x-tilt, y-tilt, defocus, and astigmatism, so a pattern generated by a simple uniform sampling strategy [16] can be very different from a real-world one, as already shown in Fig. 3(b). On the contrary, our method has a strong advantage in that the synthetic patterns are generated from the distribution statistics quantified from the measured real-world aberration patterns, so they are inherently near-realistic, as already shown in Fig. 3(c). In the final step, 200,000 different sets of Zernike coefficients were synthesized, being sampled from the Gaussian distributions with ${\mu _{ARW}}$ and ${\mathrm{\sigma }_{ARW}}$, which denote the corresponding asymptotic distribution parameters, and numerically generating the corresponding aberration patterns for two different OPDs: 0 rad and π/4 rad matched to the 2-step measurement. This synthetic dataset was used to train a CNN used for the deep learning process.

 

Fig. 7. Zernike coefficients estimated by the 4-step derivative fitting method: Dots indicate estimated coefficients for each aberration pattern. Estimated asymptotic mean and standard deviation are marked with a green bold line and a shaded area, respectively. Inset shows the statistical information of the measured real-world aberration patterns.

Download Full Size | PDF

The input to the CNN was set to a concatenation of two 256 × 256 aberration patterns, since our method only requires two aberration patterns. The output of the CNN was 6 real values corresponding to the 6 Zernike coefficients of the given MAs, i.e., d = 5. In this study, two CNN architectures were tested as feature extractors: GoogLeNet and ResNet101. The CNN was trained by minimizing the mean square errors on the 200,000 synthetic aberrations and their corresponding Zernike coefficients, of which 180,000 and 20,000 aberrations were used for training and validation, respectively. The network was implemented using PyTorch. The Adam optimizer was used to optimize the network with a learning rate of 1e-3. Both networks were trained for 50 epochs. For instance, the training curve of ResNet101 is shown in Fig. 8. For each epoch, the RMSE loss was measured on both the training and validation datasets. The orange dashed and blue dash-dotted lines in Fig. 8 indicate the training and validation losses, respectively. It is worth noting that both training and validation curves tended to decrease with epochs and converged after ∼ 20 epochs. In the test phase, the resulting CNN inferred the Zernike coefficients with two aberration patterns with different OPDs.

 

Fig. 8. Training curve of ResNet101 in comparison with the validation counterpart.

Download Full Size | PDF

3.2. Zernike decomposition of numerically generated aberration patterns

We first investigated the operational performance of the proposed 2-step aberration analysis method on the numerically generated aberrations. 100 sets of aberration patterns and corresponding Zernike coefficients for the first 6 orders were arbitrarily generated based on the given Gaussian density function, and their Zernike decomposition performances in terms of computational speed and accuracy were compared between the proposed 2-step method and the existing 4-step iterative methods, including the derivative fitting [15], the transport of intensity equation (TIE) [17], and the robust TIE [24] methods. Note that for fair comparisons, these numerical test samples were newly generated and had nothing to do with the synthetic aberration patterns used to train the two CNNs. We added a certain amount of Gaussian noise to the generated aberration patterns in order to see the noise robustness of each method. It is worth noting that although the deep learning based methods, such as the proposed 2-step method, could achieve an improvement in noise resilience if the network is trained with additive random noise [16,19], we did not consider including additional random noise in the synthetic dataset for training for a fair comparison of the proposed 2-step method with the other non-learning based iterative methods. In addition, in this situation the ground truth phase information was fully available, so we could readily estimate the RMSEs of the reconstructed phase distributions and their corresponding peak signal-to-noise ratios (PSNRs) between the reconstructed and original aberrations. The results obtained by the proposed 2-step and existing 4-step iterative methods are plotted with respect to the added noise level in Fig. 9.

 

Fig. 9. Phase accuracy of the aberration analysis for 100 sets of numerically generated aberration patterns with respect to the added noise level: (a) RMSE and (b) PSNR.

Download Full Size | PDF

It is worth noting that the proposed 2-step method together with the use of ResNet101 showed a superior performance in terms of RMSE to all other decomposition methods when the noise level was increased above 0.01, as shown in Fig. 9(a). In particular, at the noise level of 0.05, the ResNet101 model outperformed the derivative fitting, TIE, and robust TIE methods by more than 79%, 35%, and 29%, respectively, in terms of RMSE. Unlike the ResNet101 model, the GoogLeNet model showed relatively poor performance even when the noise level was increased above 0.01. We believe that this result was caused by the difference in model capacity between GoogLeNet and ResNet101, as the latter has more parameters than the former, and therefore has the capacity to learn and remember a larger amount of data than the former [25]. Thus, the simulation result implies that the ResNet101 model equipped with the proposed method showed a robust performance against the different noise levels due to the large learning capacity of the neural network. A similar tendency can also be observed for the PSNR, as shown in Fig. 9(b). It is worth noting that the higher the PSNR, the better the reconstruction, as the PSNR is generally inversely proportional to the RMSE. Therefore, it is reasonable to conclude that ResNet101 performed best when the noise level was greater than 0.01. We also compared the inference times of all the methods by measuring the average inference time during the aberration analysis of the 100 sets of the numerically generated aberration patterns. As summarized in Table 1, the learning-based methods absolutely outperformed the existing 4-step iterative methods. In particular, the ResNet101 model showed the fastest inference time. As the noise level was increased, the TIE and robust TIE methods generally required a much larger number of iterations to complete the phase reconstruction. In contrast, the derivative fitting method showed the second best performance in terms of computation time. This is because the derivative fitting method only uses matrix inversion. However, this advantage cannot always be maintained, as the computational cost tends to increase with the number of pixels in the pattern being tested. It is worth noting that, unlike the conventional aberration analysis methods, the learning-based models do not require additional iterations to reconstruct the phases. In this respect, it is reasonable to conclude that the learning-based methods can consistently outperform the existing 4-step iterative methods in terms of robustness to noise and inference time. However, it should be noted that although the simulation results showed reasonable performance, the deep learning based approaches do have some limitations in that if the aberration patterns are significantly different from the training dataset, the trained models tend to perform poorly and to show increased prediction errors. In fact, this problem is known as the distribution shift problem in machine learning [27]. However, this can be solved by using the domain adaptation or few-shot learning methods [28,29], which can readily adapt the learned network to the new situations with only a few additional datasets.

Table 1. Comparison of the inference times among methods

View Table | View all tables in this article

3.3. Zernike decomposition of experimentally measured aberration patterns

We also investigated the operational performance of the proposed 2-step aberration analysis method on the 15 sets of the real-world aberration patterns that had been experimentally obtained by the Michelson interferometer measurements under a variety of conditions and used to extract the asymptotic statistical distribution parameters of the real-world aberration patterns. Note that although the statistical distribution parameters of these measured real-world aberration patterns were used to generate the synthetic aberration patterns, the measured fringe patterns themselves were in no way used to train the network. In addition, in this situation the ground truth phase information was fully unknown, so we just estimate the normalized RMSEs (NRMSEs) of the reconstructed aberration patterns relative to the experimentally measured aberration patterns. The results obtained by the proposed and existing 4-step iterative methods are summarized in Table 2, and three typical cases are graphically illustrated in Figs. 10, 11, and 12. Although the overall performance of these methods was similar quantitatively, the differences among them were found to be quite significant qualitatively.

 

Fig. 10. Results of the aberration analysis: (a) Real-world aberration pattern, reconstructed aberration patterns (b) with GoogLeNet, (c) with ResNet101, (d) with derivative fitting, (e) with TIE, and (f) with robust TIE methods, (g) obtained Zernike coefficients, reconstructed aberration phase distributions (h) with GoogLeNet, (i) with t ResNet101, (j) with derivative fitting, (k) with TIE, and (l) with robust TIE, respectively.

Download Full Size | PDF

 

Fig. 11. Results of the aberration analysis: (a) Real-world aberration pattern, reconstructed aberration patterns (b) with GoogLeNet, (c) with ResNet101, (d) with derivative fitting, (e) with TIE, and (f) with robust TIE methods, (g) obtained Zernike coefficients, reconstructed aberration phase distributions (h) with GoogLeNet, (i) with t ResNet101, (j) with derivative fitting, (k) with TIE, and (l) with robust TIE, respectively.

Download Full Size | PDF

Table 2. Reconstruction errors for measured real-world aberration patterns.

View Table | View all tables in this article

In this situation, as summarized in Table 2, the existing 4-step iterative method based on Robust TIE and the proposed 2-step method based on ResNet101 seemed to perform best and second best, respectively. However, an important observation is that the reconstruction errors, i.e., the NRMSEs, were relatively high for all methods compared to those of the numerical simulation results (see Fig. 9). This is due to the fact that the NRMSEs were estimated in terms of the intensity distributions rather than the phase distributions and that there were significant intensity variations in the measured real-world aberration patterns. The latter is well illustrated in Fig. 10(a). That is, since the NRMSEs were simply quantified on the basis of pixel-wise intensity differences, the tendency for the intensity to decrease at the edges of the measured pattern could not be fully accounted for, resulting in a significant increase in the NRMSEs, which ranged from 0.31 to 0.47 for all the methods, including the proposed 2-step methods. Therefore, in the case of the measured real-world aberration patterns, where the ground truth information was by no means known, a simple quantitative evaluation, such as an evaluation in terms of the NRMSE between the reconstructed and measured aberration patterns, was not sufficient to understand the effectiveness of each method in more detail. Therefore, it is necessary to consider a qualitative evaluation in addition, as described below.

Figures 10, 11, and 12 show three typical results obtained by the proposed and existing methods. For each set of figures, the sub-figure (a) illustrates the measured real-world aberration pattern, the sub-figures (b-f) illustrate the reconstructed aberration patterns by the methods based on GoogLeNet, ResNet101, the derivative fitting, TIE, and robust TIE, respectively, the sub-figure (g) illustrates the magnitudes of the corresponding Zernike coefficients of the reconstructed aberration patterns, and the sub-figures (h-l) illustrate the corresponding phase distributions, respectively.

The results shown in Fig. 10 represent the case where the measured real-world aberration pattern contained a relatively low level of noise, while only showing a tendency for the intensity to decrease at the edges of the measured pattern. In this case, all methods were able to achieve reasonable reconstruction performance, with no significant differences in reconstructed aberration patterns and phase distributions, although the existing 4-step iterative method based on TIE was slightly outperformed. In fact, the two proposed 2-step methods were able to demonstrate fully compatible performance compared to the existing iterative methods, even though the former required only two measurements of aberration patterns instead of four. The effectiveness of the proposed 2-step methods was further highlighted in Figs. 11 and 12, which represent the cases where the measured real-world aberration patterns contained relatively high levels of noise. The existing 4-step iterative methods produced rather peculiar or unrealistic results, causing discontinuities and drastic changes in the reconstructed aberration patterns and phase distributions. However, no such anomalies were observed in the results of the two proposed 2-step methods. In particular, the proposed 2-step method based on ResNet101 produced the reconstructed aberration patterns most similar to the original measured patterns, demonstrating its superiority over the other methods. It should also be noted that even though the existing 4-step iterative method based on the derivative fitting used to extract the statistical distribution parameters of the measured real-world aberration patterns could not satisfactorily reconstruct the aberration pattern and phase distribution, especially for the case of Fig. 12, this does not mean that its validity is questionable. This is because the original measured aberration pattern considered in Fig. 12 had a highly skewed beam intensity distribution together with a high level of noise, so that it could not perform the Zernike decomposition satisfactorily. The consequences can be readily seen in the large discrepancy between the reconstructed and measured aberration patterns. However, this rather inaccurate prediction could not completely disrupt the distribution statistics of the real-world aberration patterns, as it was only an outlier from a statistical point of view. In fact, such statistical consequences were fully taken into account when generating the synthetic fringe patterns under the asymptotic condition. This is the reason why the two proposed 2-step methods trained by the corresponding deep learning networks were able to produce reasonably well matched reconstructed patterns even when a highly noisy and skewed fringe pattern was tested. This is indeed a clear and unsurpassed advantage of the proposed 2-step methodology based on statistical analysis, apart from the unmatched fast inference time.

 

Fig. 12. Results of the aberration analysis: (a) Real-world aberration pattern, reconstructed aberration patterns (b) with GoogLeNet, (c) with ResNet101, (d) with derivative fitting, (e) with TIE, and (f) with robust TIE methods, (g) obtained Zernike coefficients, reconstructed aberration phase distributions (h) with GoogLeNet, (i) with t ResNet101, (j) with derivative fitting, (k) with TIE, and (l) with robust TIE, respectively.

Download Full Size | PDF

4. Conclusion

We have proposed a noise robust deep learning based 2-step aberration analysis method that requires only two aberration interference measurements. By statistically analyzing the experimentally measured fringe patterns and their Zernike coefficients using the Gaussian density estimation, a real-world-like synthetic dataset of 200,000 different aberrations were numerically generated under the asymptotic condition from 15 sets of real-world aberrations patterns obtained by Michelson interferometer measurements under a variety of measurement conditions using the 4-step derivative fitting method [15]. We used the two types of CNN architectures based on GoogLeNet and ResNet101 for the data training and inference, and verified that only two fringe measurements were sufficient for complete aberration analysis. Through the combinatorial analysis of the aberration decomposition and reconstruction using the proposed method in comparison with the existing 4-step iterative methods based on the derivative fitting, TIE, and robust TIE, we verified that the proposed deep learning based 2-step aberration analysis method was able to outperform the other algorithms in terms of computational speed and noise robustness in both numerical and experimental cases. To the best of our knowledge, there have been no noise robust algorithms that can reconstruct the aberration from only two interference patterns. Although in the present study we have only considered the aberrations represented by Zernike polynomials of the orders of up to six, it is noteworthy that in principle there is no limit to the number of the aberration orders that can be considered simultaneously. Of course, other algorithms, such as derivative fitting, TIE, and robust TIE, can be used to decompose an aberration with many higher-order components. However, TIE and robust TIE can have difficulty in computing the matrix inversion because the higher-order aberrations contribute to the formation of a more complicated fringe pattern, resulting in a more complex interference matrix. In addition, since the higher-order aberrations can cause a drastic intensity fluctuation in the fringe pattern, which requires a long time for the derivative fitting method in the computation, the proposed 2-step aberration analysis method based on deep learning can achieve excellent aberration decomposition and reconstruction performance that surpasses the existing 4-step iterative aberration analysis methods, even when the higher-order aberrations are extensively involved. We hope that our method will be used as an effective tool for comprehensive aberration analysis and expect that further studies will extend its usefulness and improve its operational performance in terms of algorithm compactness, noise robustness, and computational speed.