Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

One-shot phase retrieval method for interferometry using a hypercolumns convolutional neural network

Open Access Open Access

Abstract

In three dimensional profilometry, phase retrieval technique plays a key role in signal processing stage. Fringe images need to be transformed into phase information to obtain the measurement result. In this paper, a new phase retrieval method based on deep learning technique is proposed for interferometry. Different from conventional multi-step phase shift methods, phase information can be extracted from only a single frame of an interferogram by this method. Here, the phase retrieval task is regarded as a regression problem and a hypercolumns convolutional neural network is constructed to solve it. Firstly, functions and each component of the network model are introduced in details; Then, four different mathematical functions are adopted to generate the training dataset; training and validation strategies are also designed subsequently; Finally, optimization processing is performed to eliminate local data defects in initial results with the help of polynomial fitting. In addition, hardware platform based on point diffraction interferometer is fabricated to support this method. Concluded from the experiment section, the proposed method possesses a desirable performance in terms of phase retrieval, denoising and time efficiency.

© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Optical three-dimensional profilometry is widely used in different key industries [1], such as industrial manufacturing, reverse engineering, medical diagnosis and aerospace. It has the characteristics of non-contact, high efficiency and remarkable precision, which is considered as one of the most promising profilometry. According to the principle, optical profilometry can be further categorized into different methods: time-based method, structured light method, projection method and interferometry. In the process of interferometry [2], test beam is reflected by test object and interferes with reference beam, then forms an interferogram on the detector. This interferogram brings the surface profile information of test object and need to be further processed to obtain the final measurement result. The principle of projection method [3]: regular black-white fringe pattern is projected on the surface of test object by a projector; CCD cameras are used to acquire the object images which covered with fringe pattern; Distorted fringes in that image contains surface profile information; With the help of certain processing, three-dimensional measurement result can be acquired. These kind of processing methods that extract profile information from fringe patterns are called “phase retrieval”. It plays a key role in the signal processing stage of profilometry. In this field, different methods have been studied and attempted for the applications.

Multi-step phase shift method is a mainstream one in phase retrieval [4,5]. Time domain phase shift is applied to test objects and we can obtain a series of interferograms with certain stride. Then phase data of test profile can be extracted by using subtraction and arctangent operations to those images. The more steps are engaged, the higher processing accuracy will be achieved, but the lower efficiency has to be suffered. Besides, systematical error is likely to be accumulated in multiple tests. Arctangent operation ‘atan2’ is inevitable in this method, which will cause the phenomenon of phase wrapping in the range [-π, π]. Therefore, additional phase unwrapping algorithm is needed, like quality-guided [6] or Goldstein branch cut algorithms [7]. To realize the high-accuracy phase shift, at less one actuator (like Piezoelectric Transducer) should be equipped in the system. However, it is in a higher cost and limited moving accuracy. For moving targets, phase shift measurement will be more difficult. Therefore, one-frame based phase retrieval is meaningful. Fast Fourier transform (FFT) [8,9] can achieve this goal: Fringe pattern is transformed into a spectrogram by 2D FFT; Based on carrier frequency, the corresponding filtering operations are adopted to extract wrapped phase data; Then, one also need to use unwrapping algorithm to get the final result. However, this method will not perform well when faced with the images contain loop fringes. In recent years, deep learning technique attracts more attentions and exerts desirable effect in different applications: Image segmentation [10], signal denoising [11], phase unwrapping [12], etc. Daichi Kando adopts U-Net model to extract phase data from fringe pattern [13]. However, the method can just deal with the data with simple shapes. Van der Jeught proposes a phase retrieval method for structured light profilometry using a 10-layer convolutional network [14]. It can measure the target objects with different shapes in a higher accuracy. While the height of measured data should be limited in a mild range.

To overcome the drawbacks of conventional methods, a new phase retrieval method is proposed, which is based on the hypercolumns convolutional neural network (HCNN). Phase information can be extracted from single frame of interferogram. We regard phase retrieval task as a regression problem: Using a deep neural network to transform fringe patterns into phase information. Firstly, image processing and feature extraction are applied to interferograms and then the corresponding phase information will be predicted by HCNN. Subsequently, we perform optimization processing to filter the data defects (fault prediction) in initial results. In this stage, phase data is treated as the three-dimensional curved surface and smoothed by polynomial fitting. With proposed method, phase information can be extracted accurately and real-timely without further unwrapping. Besides, it shows robustness to noise situations. Finally, we can obtain the profile information of test optics.

Section 2 introduces the principle of phase retrieval and training scheme of deep neural network; Optimization processing for initial phase extraction result is described in Section 3; Hardware platform of point diffraction interferometer is presented in Section 4; In Section 5, a series of experiments are carried out including performance comparison, denoising and time efficiency; Conclusion of the article is given in Section 6.

2. Principle

Deep learning technique is widely used in different applications due to its outstanding performance. Target classification and regression are the two main problems to be solved by this technique. For continuous phase data retrieval, deep learning is a suitable method to construct an end-to-end model, which can predict the relationship between fringe distributions and phase information.

2.1 Hypercolumns convolutional neural network

In this paper, phase retrieval task is regarded as a regression problem. Fortunately, with the help of deep learning techniques, a hypercolumns convolutional neural network (HCNN) is constructed to process interferograms and predict the corresponding phase data. The structure of HCNN model is shown as Fig. 1:

 figure: Fig. 1.

Fig. 1. Model structure of Hypercolumns convolutional neural network

Download Full Size | PDF

The input data of network model is interferogram that under phase extraction. The dataset of the interferogram will be put into the input layer of model and they have the same dimension. The output of this network model is the predicted phase data. Here, the HCNN network model is comprised of basic components. Conv2D: two-dimensional convolution operation; Conv2DT: two-dimensional transpose convolution operation; BN: Batch Normalization processing; Elu: exponential activation function; MP: two-dimensional Max-pooling operation; US: two-dimensional up sampling operation; DP: Dropout method for neurons.

Interferograms are put into the input layer and then enter the multi-convolution layers for feature extraction. Specially, multi-level convolution layers are constructed in auto encoder-decoder structure: The input data are engaged in multiple convolutions, max-pooling and up sampling operations and its contained feature will be compressed and reconstructed. Then the model can extract feature information in different scales. Usually, feature information in shallow layers has limited ability of abstract expression, while deep semantics information has drawbacks in feature accurate localization. Therefore, the concept of hypercolumns is applied to the network model. By using this kind of structure, feature map from different scales and layers are combined together and a new multi-dimensional feature map is produced. Bharath Hariharan use hypercolumns classifier for object segmentation and obtain accurate results [15]. Liao process the MRI images and segment out human left ventricle ROI from medical images with hypercolumns fully convolutional network [16]. Again, convolution operation is applied to the new feature map: 1×1 convolutional kernel will join the calculation and execute prediction to feature map in pixel level. Finally, we can get the phase information from interferogram and complete the regression task. The parameters of HCNN network model are given in Table 1:

Tables Icon

Table 1. Parameters of HCNN network model

In HCNN network model, adaptive moment estimation (Adam) [17] is adopted as the optimizer and RMSE as the loss function. In order to and prevent the phenomenon of over-fitting, we add Batch normalization, Dropout and L2 regularization methods to the HCNN network model to solve the corresponding problems. In addition, before training, data shuffle method is also applied to training and validation dataset to further prevent over-fitting. By using proposed method, phase retrieval task can be fulfilled only based on single frame of interferogram without phase shift operation.

2.2 Training and validation

To make the HCNN network model operate in optimal status and exert excellent performance, large quantity of samples is needed to train it. The richer samples in training dataset, the better effect will be obtained. Most of the training data samples can be generated by mathematic functions and the other part is acquired by the hardware platform. Here, dataset is made up of 4 parts: interferograms set for training Images_Train, phase dataset for training Phase_Train, interferograms set for validation Images_Test and phase dataset for validation Phase_Test. Specifically, data in Images_Train and Phase_Train are sample-to-sample correspondence and share the same data dimension (N, W, H). Data in Images_Test and Phase_Test are used for network validation and they share the same data dimension (M, W, H). Where N=16000, M=2400 are the number of samples in dataset; W is the width of images and H is the height. Four kinds of mathematic function are used to generate dataset, which can improve the variety. They are presented in Table 2:

Tables Icon

Table 2. Mathematic functions for dataset generation

As for Zernike polynomials [18], n is the order of polynomial, k and m belong to integer set. Where polynomial ${Z_{nm}}$ can be further expressed as Eq. (1):

$$\begin{array}{l} {Z_{nm}} = \left\{ \begin{array}{l} R_n^l(\rho )\sin l\theta \textrm{ }l < 0\\ R_n^l(\rho )\cos l\theta \textrm{ }l \ge 0 \end{array} \right.\\ R_n^l(\rho ) = \sum\limits_{s = 0}^{n - m} {{{( - 1)}^s}\frac{{(2n - m - s)!}}{{s!(n - s)!(n - m - s)!}}{\rho ^{2(n - s) - m}}} \end{array}$$
Parameters in the functions above are limited in certain ranges and produced by random value generators, so variety of shapes can be created. The diversified dataset is benefit for network training. Based on the measured surface data $W(x,y)$, the corresponding phase data $\varphi (x,y)$ can be obtained by using the relationship: $\varphi (x,y)\textrm{ = }{{2\pi \cdot W(x,y)} / \lambda }$. Where λ is the wavelength of test laser in the measurement.

By using the four mathematic functions in Table 1, phase data Phase_Train and Phase_Test are produced. Then we can acquire the corresponding interferogram set Images_Train and Phase_Train from intensity distribution equation of fringe pattern [19] in Eq. (2):

$$I(x,y) = {I_0}(x,y)\{{1 + V(x,y)\cos [{\varphi (x,y) + Noise(x,y)} ]} \}$$
To enhance the training effect, interferograms should be generated in different qualities: $I(x,y)$ is the interferogram; ${I_\textrm{0}}(x,y)$ is the background intensity of interferogram and its random value is selected in the range [50, 127]; $V(x,y)$ is the fringe contrast in [0.8, 1.0]; $\varphi (x,y)$ is phase data generated by functions and $Noise(x,y)$ is denoted as Gaussian random noise in the range [-π, π]. Based on the rich dataset, we can carry out training task for network model. Training strategy is illustrated in Fig. 2:

 figure: Fig. 2.

Fig. 2. Training and Validation scheme of HCNN model

Download Full Size | PDF

Training stage: Interferograms in Image_Train set will be input into the network model batch by batch; Then the predicted phase data Pred_Train set is produced by the model; By comparison, loss value (RMSE) between predicted quantity and the ground truth can be calculated. Here, loss function is expressed as Eq. (3):

$$Loss({{x_i}} )= \sqrt {\frac{1}{m}\sum\limits_{i = 1}^m {{{({HCNN({x_i}) - {y_i}} )}^2}} }$$
Where HCNN(xi) stand for network model. According to the loss value, Gradient descent method is adopted to updated global parameters of network model. In this way, training task is executed epoch by epoch until the certain condition is reached loss <0.01. In the whole training stage, dynamic learning_rate and early_stop strategies are used to contribute model convergence and prevent the phenomenon of over fitting.

A larger learning rate can accelerate the progress of training at the beginning, but it may bring instability in later period. For tiny learning rate, network training may probably encounter the status of under fitting. Therefore, dynamic decaying mechanism is adopted, which can adjust learning rate automatically with the training epoch increasing. Here decaying method is denoted as LR=αepoch×LR0 (decaying efficiency α=0.99, initial learning rate LR0=0.1). Usually, over fitting, impropriate learning rate and other factors may lead to instable training effect. Instead of static epochs setting, early stopping method can prevent training degeneration when tend is happened. This method can stop training when accuracy (loss) value is no longer increase (decrease) in continuous n epochs (here n=10). At the end of each epoch, program will perform the corresponding judgment.

Validation stage: Save the optimal trained model as an independent estimator for phase prediction; then we can get the prediction result Pred_Test from Images_Test. In theory, Pred_Test and Phase_Test have a tiny loss gap by evaluation. Training record is illustrated in Fig. 3:

 figure: Fig. 3.

Fig. 3. Training record of HCNN model

Download Full Size | PDF

We can find that the network model converges rapidly in beginning period and reaches the goal of RMSE<0.01 in less than 100 epochs. Besides, validation loss is slightly better than training loss during the whole training period.

3. Optimization

By using deep learning technique, phase retrieval can be realized well. Though accuracy of phase prediction result is relatedly higher, there may be exist data defects or “fault prediction” in local areas. Therefore, post processing is needed to optimize the initial result.

The CCD camera has a limited resolution in interferogram acquisition: Adjacent pixels can show a “black band” and a “white band” respectively, which stand for a wavelength/period. It can be inferred that changing rate of neighbor phase data will not exceed 2π, because interferogram $I(x,y)$ is the cosine function of phase $\varphi (x,y)$. Equation (4) shows the relationship between phase data and test surface profile.

$$W(x,y) = \frac{\lambda }{{2\pi }}\varphi (x,y)$$
From the equation, we can conclude that changing rate of profile amplitude is smaller than one λ. Data distribution of test surface is a smooth curved surface in theory. So in global scope, “defect areas” can be extracted by scanning the jumping values larger than λ. Scanning method is denoted as Eq. (5).
$$DE(x,y) = \left\{ \begin{array}{l} 255\textrm{ , }|p(x,y) - q(x,y)|> 1\textrm{ , }p,q \in W ({\textrm{they}}\;{\textrm{are}}\;{\textrm{neighbor}}\;{\textrm{points}})\\ 0\textrm{ , }|p(x,y) - q(x,y)|< 1\textrm{ , }p,q \in W ({\textrm{they}}\;{\textrm{are}}\;{\textrm{neighbor}}\;{\textrm{points}}) \end{array} \right.$$
where $DE(x,y)$ is the contour edge of defect areas in binary map, p and q are two arbitrary adjacent points in initial result W. After scanning, all the defects have been marked in the global scale. To solve this problem, multi-order polynomial fitting method is adopted to filter those defects. Procedure of optimization is shown in Fig. 4:

 figure: Fig. 4.

Fig. 4. Procedure of optimization. (a) Initial result with defects; (b) Defect areas scanning; (c) 3D curved surface fitting by multi-order polynomial; (d) Error map of fitting area;(e) Optimization result

Download Full Size | PDF

Here, we regard surface data as the three-dimensional curved surface: directions of horizontal, vertical in surface data stand for X, Y axis and amplitude for Z axis respectively. Then we can perform curved fitting operation to defect areas. Detail procedure is given as follow:

  • 1) Extract all the contour edge of defects by using Eq. (5) global scanning.
  • 2) Set all the data points in contour edge to null ‘NaN’ and we can obtain a closed area with a ‘hole’;
  • 3) Closed area with the hole is a connection area and morphology dilation operation (dilation coefficient is denoted as d=10) should be applied to it. Then we can get the dilated defect area;
  • 4) Calculate the minimum envelope rectangle of dilated area and cut it out from global surface data set. The following optimization will on the basis of this cutting area.
  • 5) Perform linear curved fitting to the cutting area (with hole) by using 5-order polynomial. Then 3D curved surface after fitting can be acquired. The fitting polynomial is shown as Eq. (6):
    $$\begin{aligned} f(x,y) &= {p_{00}} + {p_{10}}x + {p_{01}}y + {p_{20}}{x^2} + {p_{11}}xy + {p_{02}}{y^2} + {p_{30}}{x^3} + \\ & \quad \textrm{ }{p_{21}}{x^2}y + {p_{12}}x{y^2} + {p_{03}}{y^3} + {p_{40}}{x^4} + {p_{31}}{x^3}y + {p_{22}}{x^2}{y^2} + {p_{13}}x{y^3} + \\ & \quad \textrm{ }{p_{04}}{y^4} + {p_{50}}{x^5} + {p_{41}}{x^4}y + {p_{32}}{x^3}{y^2} + {p_{23}}{x^2}{y^3} + {p_{14}}x{y^4} + {p_{05}}{y^5} \end{aligned}$$
In the process of fitting, least square algorithm is adopted as the optimization method and the target function is RMSE. After multiple iterative calculation, accurate fitting can be reached at the condition of RMSE < T. At this stage, we worked out a series of polynomial coefficients [${p_{00}},{p_{01}},\ldots \ldots ,{p_{05}}$] for Eq. (6) to realize curved fitting.
  • 6) Use the fitted surface Z’ data to fill the ‘hole’, then data outside the ‘hole’ in the cutting area will perform weighted mean operation with original one.
  • 7) Transform local coordinate of cutting area back to the global one in original surface map.
Optimization performance has the relationship with the sizes of fitting window and defect. Through the simulation, we can find that the fitting error reaches the valley at the area ratio (defect to fitting window) of 2.6. As we known, the more points join the calculation, the more processing time is needed.

Figure 5 shows the changing tendency of that relationship. For performance balancing, area ratio of defect to fitting window should be set to 2.6: fitting error is smaller than that of network prediction and time consumption is acceptable.

 figure: Fig. 5.

Fig. 5. Effect of curved surface fitting. (a) Area ratio of defect-fitting area vs fitting accuracy; (b)Time consumption

Download Full Size | PDF

4. Hardware platform

Spherical/aspherical optical components are widely used in many different fields. In the process of their manufacturing, precision metrology is a critical aspect, because the surface quality is closely related to their performance. Here, point diffraction interferometer (PDI) [20] is constructed to test the surface profile quality of spherical/aspherical components. Figure 6 shows the scheme of PDI system.

 figure: Fig. 6.

Fig. 6. Hardware platform of point diffraction interferometer

Download Full Size | PDF

Point diffraction interferometer can use optical diffraction phenomenon to generate reference spherical wavefront (Precision can reach $PV < \lambda /{10^4}$, $\lambda \textrm{ = 632}\textrm{.8}nm$) and perform relative measurement to the optical components [21]:

  • (1) Leaser source outputs the beam through diaphragm and attenuator to the expanding collimator lens. Then laser beam will be transformed into the parallel light;
  • (2) The parallel light is converged to a spot by micro objective and is projected on the pinhole plate;
  • (3) After accurate alignment [2], the light can be diffracted and spherical wavefront will be generated at the back side of pinhole plate;
  • (4) Diffraction wavefront is divided into two parts. Test path: test wavefront propagates to the test optics and is reflected again by lens and pinhole plate; Reference path: Now, test wavefront (contains the profile information) is interfered with reference one and forms an interferogram on the CCD detector;
  • (5) In a certain sequential, interferogram acquisition and phase shift operations will be working in a synchronous status by our control system;
  • (6) Finally, signal processing will be carried out in a designed computer program to work out the final result.
In conventional measurement methods, phase shift component piezoelectric transducer (PZT) should be equipped in the system to perform phase shift operation. As shown in Fig. 7, test optics is driven by PZT in a certain stride and multiple interferograms are acquired. Fortunately, with the help of proposed method, one-shot phase retrieval can be achieved easily on the interferometer. Detail configuration of hardware components are presented in Table 3.

 figure: Fig. 7.

Fig. 7. Phase shift component in the system. (a) Piezoelectric transducer (PZT); (b) Test optics driven by PZT

Download Full Size | PDF

Tables Icon

Table 3. Hardware configuration of PDI system

5. Experiment

To validate the performance of proposed method, multiple tests are carried out including phase retrieval accuracy, anti-noise performance and time efficiency. As shown in Fig. 6, PDI system is constructed and operated in the condition of normal temperature and humidity.

5.1 Performance evaluation

Firstly, we should examine the basic performance of proposed method. Different types of interferograms will under the test, which can also validate its generality. Figure 8 presents the phase retrieval results of selected samples: Original interferograms, retrieval results, ground truth and the error map are shown in 1st, 2nd, 3rd, 4th column respectively. From the phase retrieval results, it can be found that the produced results have a higher similarity with the ground truth and the amplitude of error map is very small. root mean square error (RMSE) between our results and ground truth is given in Table 4:

 figure: Fig. 8.

Fig. 8. Phase retrieval results from different interferograms. (a) Interferograms; (b) results produced by proposed method; (c) Ground truth; (d) Error map between our results and ground truth

Download Full Size | PDF

Tables Icon

Table 4. Root mean square error between produced results and ground truth

Then, the proposed method is put into the practical application: a mild aspherical optics is tested by PDI system and the corresponding interferogram is acquired by CCD camera. We regard this image as an input data and use the well-trained model to interpret it.

Figure 9(a) is acquired interferogram from PDI system and Fig. 9(b) is the corresponding phase data extracted by proposed method. Due to shape deviation between aspherical surface and spherical reference wavefront, the phase data looks like a “cup”. Best fit radii of curvature is only shown in a certain annular areas. The corresponding measurement result by interferometer is shown in Fig. 9(c). The testing data are given in Table 5.

 figure: Fig. 9.

Fig. 9. Phase retrieval for aspherical optics metrology. (a) Interferogram in test; (b) result produced by proposed method; (c) Measurement result from Zygo interferometer

Download Full Size | PDF

Tables Icon

Table 5. Experiment results of aspherical / spherical optics metrology

Subsequently, we perform the measurement experiment for spherical optics (D=50mm, R=258mm). Like the experiment above, we use CCD to capture interferogram [ Fig. 10(a)] and retrieval its phase data by proposed method. Surface measurement data will be obtained as Fig. 10(b). This is the direct measurement result without removing piston, XY tilt and Z defocus. Measurement result from Zygo interferometer is illustrated in Fig. 10(c).

 figure: Fig. 10.

Fig. 10. Phase retrieval for spherical optics metrology. (a) Interferogram in test; (b) result produced by proposed method; (c) Measurement result from Zygo interferometer

Download Full Size | PDF

Measurement data of spherical optics are also given in Table 5. From Fig. 9, Fig. 10 and Table 5, it can be concluded that the testing results show higher consistency in both phase shape and measured data. Using HCNN to retrieval phase data is effective in practical applications.

5.2 Comparison test

Finally, FFT and phase shift methods are also join the test for performance comparison. However, these methods cannot produce the continuous phase directly. Unwrapped algorithms are needed for further processing. Here, Goldstein branch cut algorithms [7] are adopted to help phase retrieval. Figure 11 shows the processing results by the three methods above. Test phase data with freeform surface [Fig. 11(b)] is generated by 36-terms Zernike polynomials and its corresponding interferogram is Fig. 11(a).

 figure: Fig. 11.

Fig. 11. Phase retrieval results produced by three methods. (a) Interferogram; (b) Ground truth; (c) result by phase shift; (d) result by Fourier transformation method; (e) result by proposed method

Download Full Size | PDF

The result produced by 4-step phase shift method is presented in Fig. 11(c). Its global appearance is similar to ground truth [Fig. 11(b)], while some deviations are left in local areas. Through calculation, RMSE value between ground truth and phase shift is 0.0976. Figure 11(d) shows the result by Fourier transformation method. Though it is a one-frame processing method, phase data with complicated shapes can hardly extracted accurately. RMSE of its result even exceed 16.72. From the Fig. 11(e) in tests above, we can believe the proposed method is capable in phase retrieval and shows the excellent performance. RMSE is only in 0.041.

PsPNet and ResNet are the two common network models in different applications, such as image segmentation, target classification. Meanwhile, we also use these models to perform phase retrieval task and check the corresponding difference in performance. Figure 12 shows the processing results of the three network models.

 figure: Fig. 12.

Fig. 12. Phase retrieval results by different network model. (a) Interferogram; (b) phase map ground truth; (c) PsPNet result; (d) ResNet result; (e) HCNN result

Download Full Size | PDF

Here 2D vision of phase map is used to check the difference of data distribution. Firstly, the result of PsPNet shows a lower resolution, which has a discrete distribution like Mosaic [Fig. 12(c)]. RMSE between ground truth and its result is 0.058rad. Secondly, ResNet fail to produce the accurate phase result [Fig. 12(d)]. There are large area of error data in it. RMSE = 1.45rad. By contrast, HCNN output the most accurate result in the test [Fig. 12(e)]. Its RMSE value is only 0.0068rad.During the experiments, we find that HCNN is the most efficient network model in both training period and convergence rate. Then follows PsPNet and ResNet. Only 6 epochs are needed for HCNN to reduce loss value to 0.5, while 17 for PsPNet and 45 for ResNet.

5.3 Anti-noise test

In practical applications, image noise is inevitable during the measurement. Thus, anti-noise performance is a basic requirement for signal processing methods. Gaussian noise and random noise are common cases in image acquisition. In this test, phase retrieval method will be confronted with the interferograms corrupted by noise. Phase retrieval results from noised interferograms are illustrated in Fig. 13.

 figure: Fig. 13.

Fig. 13. Phase retrieval results from noised interferograms. (a) Noise corrupted interferogram; (b) results produced by proposed method; (c) Ground truth; (d) Error map between results and ground truth

Download Full Size | PDF

Though original data is corrupted by heavy noise, the results are still in an acceptable extent. Table 6 presents the experiment data of anti-noise data. RMSE values between results and ground truth are 0.0165 in Gaussian noise test and 0.0082 in random noise test. At the approximate signal-to-noise ratio level, the method seems more sensitive to Gaussian noise. In general, the results show that our method possess a desirable noise robustness.

Tables Icon

Table 6. Results of anti-noise test

5.4 Time efficiency

Time efficiency is a critical character of phase retrieval methods. Good time efficiency can contribute to the real-time performance of interferometer. It is not only depending on hardware configuration of computer, but also on the network model itself. Here, computer configuration is presented as follow, CPU: i7-7700k 4.2GHz, GPU: RTX2080Ti 11GB, RAM: DDR4 3000MHz, SSD: 1TB. The time efficiency of proposed method is analyzed in steps and the results are given in Table 7:

Tables Icon

Table 7. Analysis on time efficiency in steps

To fulfil the phase retrieval task, three processing steps should be executed in proposed method. From the data in Table 7, it can be found that HCNN prediction stage only takes the minority of calculation time (less than 1 ms) and the total consumption is in an acceptable level (less than 40.76 ms). That is to say, the proposed method can achieve phase retrieval in, at least, 25 fps, which can meet the need of real time measurement. Specifically, time consumption of optimization depends on the quality of initial results: the more data defects are existed, the more processing is needed.

6. Conclusion

This article presents a new phase retrieval method based on deep learning technique. Here, phase retrieval task is regarded as a problem of regression and Hypercolumns Convolutional Neural Network is constructed to solve it. Before application, dataset generation and training/validation strategy are designed to improve its prediction effect. Subsequently, we propose an optimization method for initial prediction results by using three-dimensional polynomial curved fitting. It can eliminate local data defects and further improve the accuracy of final result. Then PDI system is constructed to test optical components and proposed method. In experiment, we validate our method in three aspects: 1) Phase retrieval accuracy: For different types of interferograms, the proposed method can output the results lower than 0.0073 in Root Mean Square Error (RMSE). Measurement results of spherical/aspherical show a higher consistency with that of Zygo interferometer (ΔPV=0.04λ, ΔRMS=0.002λ). When dealing with freeform phase data, our method (RMSE=0.041) performances much better than phase shift (RMSE=0.0976) and FFT method (RMSE=16.72); 2) Anti-noise performance: phase retrieval accuracy can still reach 0.0165(RMSE) under the condition of 16.183dB Gaussian noise; 3) Time efficiency: Up to 16.32ms (61.27fps) consumption when processing with interferograms in 128×128. Through the validation, it can be proved that the proposed method has the features of one-shot phase retrieval ability, no unwrapping assistance, real time processing and noise robustness. In future work, we will focus on dynamic resolution phase retrieval techniques.

Funding

National Natural Science Foundation of China (51875448).

Acknowledgement

We express our sincere gratitude to the journal's editor and anonymous reviewers for their help in revising the paper. In addition, I would like to appreciate Dr. Z Wang’s kind support in my life.

Disclosures

The authors declare no conflicts of interest.

References

1. M. Halioua and H. C. Liu, “Optical three-dimensional sensing by phase measuring profilometry,” Opt. Lasers Eng. 11(3), 185–215 (1989). [CrossRef]  .

2. Z. Zhao, B. Li, X. Kang, L. Chen, and X. Wei, “Precision optical path alignment system for point diffraction interferometer based on image information,” Appl. Opt. 58(14), 3703 (2019). [CrossRef]  .

3. J. Shi, X. Zhu, H. Wang, L. Song, and Q. Guo, “Label enhanced and patch based deep learning for phase retrieval from single frame fringe pattern in fringe projection 3D measurement,” Opt. Express 27(20), 28929 (2019). [CrossRef]  .

4. Y. Zhang, X. Tian, and R. Liang, “Random two-step phase shifting interferometry based on Lissajous ellipse fitting and least squares technologies,” Opt. Express 26(12), 15059 (2018). [CrossRef]  .

5. Y. Zhang, X. Tian, and R. Liang, “Random three-step phase retrieval approach based on difference map Gram–Schmidt orthonormalization and Lissajous ellipse fitting method,” Opt. Lasers Eng. 121(169), 11–17 (2019). [CrossRef]  .

6. H. Zhong, J. Tang, S. Zhang, and M. Chen, “An Improved Quality-Guided Phase Unwrapping Algorithm Based on Priority Queue,” IEEE Geosci. Remote Sens. Lett. 8(2), 364–368 (2011). [CrossRef]  .

7. Q. Huang, H. Zhou, S. Dong, and S. Xu, “Parallel Branch-Cut Algorithm Based on Simulated Annealing for Large-Scale Phase Unwrapping,” IEEE Trans. Geosci. Remote Sens. 53(7), 3833–3846 (2015). [CrossRef]  .

8. M. Takeda, H. Ina, and S. Kobayashi, “Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry,” J. Opt. Soc. Am. A 72(1), 156–160 (1982). [CrossRef]  .

9. K. E. Perry and J. McKelvie, “A comparison of phase shifting and fourier methods in the analysis of discontinuous fringe patterns,” Opt. Lasers Eng. 19(4-5), 269–284 (1993). [CrossRef]  .

10. V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation,” IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017). [CrossRef]  .

11. K. Yan, Y. Yu, T. Sun, A. Asundi, and Q. Kemao, “Wrapped phase denoising using convolutional neural networks,” Opt. Lasers Eng. 128, 105999 (2020). [CrossRef]  .

12. J. Zhang, X. Tian, J. Shao, H. Luo, and R. Liang, “Phase unwrapping in optical metrology via denoised and convolutional segmentation networks,” Opt. Express 27(10), 14903 (2019). [CrossRef]  .

13. D. Kando, S. Tomioka, N. Miyamoto, and R. Ueda, “Phase extraction from single interferogram including closed-fringe using deep learning,” Appl. Sci. 9(17), 3529 (2019). [CrossRef]  .

14. S. Van der Jeught and J. J. J. Dirckx, “Deep neural networks for single shot structured light profilometry,” Opt. Express 27(12), 17091 (2019). [CrossRef]  .

15. B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik, “Hypercolumns for object segmentation and fine-grained localization,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2015), 07-12-June, pp.447–456. [CrossRef]  

16. F. Liao, X. Chen, X. Hu, and S. Song, “Estimation of the Volume of the Left Ventricle From MRI Images Using Deep Neural Networks,” IEEE Trans. Cybern. 49(2), 495–504 (2019). [CrossRef]  .

17. D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (2015), 5, pp. 1–15.

18. J. Y. Wang and D. E. Silva, “Wave-front interpretation with Zernike polynomials,” Appl. Opt. 19(9), 1510 (1980). [CrossRef]  .

19. V. H. Flores Muñoz, N.-I. T. Arellano, D. I. Serrano García, A. Martínez García, G. Rodríguez Zurita, and L. García Lechuga, “Measurement of mean thickness of transparent samples using simultaneous phase shifting interferometry with four interferograms,” Appl. Opt. 55(15), 4047 (2016). [CrossRef]  .

20. K. Otaki, K. Ota, I. Nishiyama, T. Yamamoto, Y. Fukuda, and S. Okazaki, “Development of the point diffraction interferometer for extreme ultraviolet lithography: Design, fabrication, and evaluation,” J. Vac. Sci. Technol., B: Microelectron. Process. Phenom. 20(6), 2449–2458 (2002). [CrossRef]  .

21. Z. Zhao, B. Li, X. Kang, J. Lu, X. Wei, and T. Liu, “Accurate localization method for subaperture stitching interferometry in aspherical optics metrology,” Rev. Sci. Instrum. 91(7), 075114 (2020). [CrossRef]  .

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1.
Fig. 1. Model structure of Hypercolumns convolutional neural network
Fig. 2.
Fig. 2. Training and Validation scheme of HCNN model
Fig. 3.
Fig. 3. Training record of HCNN model
Fig. 4.
Fig. 4. Procedure of optimization. (a) Initial result with defects; (b) Defect areas scanning; (c) 3D curved surface fitting by multi-order polynomial; (d) Error map of fitting area;(e) Optimization result
Fig. 5.
Fig. 5. Effect of curved surface fitting. (a) Area ratio of defect-fitting area vs fitting accuracy; (b)Time consumption
Fig. 6.
Fig. 6. Hardware platform of point diffraction interferometer
Fig. 7.
Fig. 7. Phase shift component in the system. (a) Piezoelectric transducer (PZT); (b) Test optics driven by PZT
Fig. 8.
Fig. 8. Phase retrieval results from different interferograms. (a) Interferograms; (b) results produced by proposed method; (c) Ground truth; (d) Error map between our results and ground truth
Fig. 9.
Fig. 9. Phase retrieval for aspherical optics metrology. (a) Interferogram in test; (b) result produced by proposed method; (c) Measurement result from Zygo interferometer
Fig. 10.
Fig. 10. Phase retrieval for spherical optics metrology. (a) Interferogram in test; (b) result produced by proposed method; (c) Measurement result from Zygo interferometer
Fig. 11.
Fig. 11. Phase retrieval results produced by three methods. (a) Interferogram; (b) Ground truth; (c) result by phase shift; (d) result by Fourier transformation method; (e) result by proposed method
Fig. 12.
Fig. 12. Phase retrieval results by different network model. (a) Interferogram; (b) phase map ground truth; (c) PsPNet result; (d) ResNet result; (e) HCNN result
Fig. 13.
Fig. 13. Phase retrieval results from noised interferograms. (a) Noise corrupted interferogram; (b) results produced by proposed method; (c) Ground truth; (d) Error map between results and ground truth

Tables (7)

Tables Icon

Table 1. Parameters of HCNN network model

Tables Icon

Table 2. Mathematic functions for dataset generation

Tables Icon

Table 3. Hardware configuration of PDI system

Tables Icon

Table 4. Root mean square error between produced results and ground truth

Tables Icon

Table 5. Experiment results of aspherical / spherical optics metrology

Tables Icon

Table 6. Results of anti-noise test

Tables Icon

Table 7. Analysis on time efficiency in steps

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

Z n m = { R n l ( ρ ) sin l θ   l < 0 R n l ( ρ ) cos l θ   l 0 R n l ( ρ ) = s = 0 n m ( 1 ) s ( 2 n m s ) ! s ! ( n s ) ! ( n m s ) ! ρ 2 ( n s ) m
I ( x , y ) = I 0 ( x , y ) { 1 + V ( x , y ) cos [ φ ( x , y ) + N o i s e ( x , y ) ] }
L o s s ( x i ) = 1 m i = 1 m ( H C N N ( x i ) y i ) 2
W ( x , y ) = λ 2 π φ ( x , y )
D E ( x , y ) = { 255  ,  | p ( x , y ) q ( x , y ) | > 1  ,  p , q W ( they are neighbor points ) 0  ,  | p ( x , y ) q ( x , y ) | < 1  ,  p , q W ( they are neighbor points )
f ( x , y ) = p 00 + p 10 x + p 01 y + p 20 x 2 + p 11 x y + p 02 y 2 + p 30 x 3 +   p 21 x 2 y + p 12 x y 2 + p 03 y 3 + p 40 x 4 + p 31 x 3 y + p 22 x 2 y 2 + p 13 x y 3 +   p 04 y 4 + p 50 x 5 + p 41 x 4 y + p 32 x 3 y 2 + p 23 x 2 y 3 + p 14 x y 4 + p 05 y 5
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.