Open Access
Add to BibSonomy
Abstract
The digital twin of optical systems can imitate its response to outer environments through connecting outputs from data–driven optical element models with numerical simulation methods, which could be used for system design, test and troubleshooting. Data-driven optical element models are essential blocks in digital twins. It can not only transform data obtained from sensors in real optical systems to states of optical elements in digital twins, but also simulate behaviors of optical elements with real measurements as prior conditions. For ground based optical telescopes, the digital twin of atmospheric turbulence phase screens is an important block to be developed. The digital twin of atmospheric turbulence phase screens should be able to generate phase screens with infinite length and high similarities to real measurements. In this paper, we propose a novel method to build the digital twin of atmospheric turbulence phase screens. Our method uses two deep neural networks to learn mapping functions between the space of parameters and the space of phase screens and vice versa. Meanwhile, a forecasting deep neural network is proposed to generate parameters for the next phase screen according to parameters extracted from a previous phase screen. The method proposed in this paper could be used to directly produce phase screens with infinite length and of any temporal or spatial power spectral density that follows statistical distributions of real measurements, which makes it an appropriate block in digital twins of ground based optical systems.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Thanks to recent developments in high performance computing technologies and numerical simulation methods, we could simulate optical systems with finer scale, larger size and contain more realistic effects [1–4]. Meanwhile sensors, such as wavefront sensors, laser rangefinders or thermometers, have better performance and lower price. Now we are able to install more sensors in an optical system to obtain high accuracy measurements from outer environments or optical elements. With these measurements and machine learning algorithms, we could further develop data–driven optical element models to learn functions between measurements and states of optical elements. These data-driven optical element models could be integrated together and we could connect outputs from these models with simulation methods to build digital twins of the whole optical system. The digital twin of optical systems could reproduce their real responses to different outer environments, which can not only help us to design new algorithms and instruments, but also be used for troubleshooting after optical systems are commissioned [5–8].
For ground based optical telescopes, the atmospheric turbulence is the main limitation to their performance [9]. Several different methods have been proposed to reduce atmospheric turbulence induced aberrations, such as: adaptive optic systems [10], lucky imaging [11] and image restoration methods [12]. It would be necessary to build the digital twin of the whole system for performance investigation and faults analysis and the digital twin of atmosphere turbulence phase screen is the first block to be developed. Meanwhile, machine learning algorithms have invoked a lot of novel methods for adaptive optics systems [13], such as data driven control or wavefront prediction algorithms. These new algorithms require atmospheric turbulence phase screens as training set. Differences between training set and real data would affect their performance. Therefore, it would be necessary to develop a method to generate phase screens that satisfy real properties of atmospheric turbulence. According to these requirements, the digital twin of atmospheric turbulence phase screens should have the following features:
- 1. it could reproduce continuous phase screens with real measurements as seeds;
- 2. it could automatically obtain spatial and temporal statistical properties of phase screens from real measurements and generate infinite long phase screens according to these properties.
Currently there are two types of phase screen generation methods: the representation method and the extrapolation method. The representation method uses basis functions (such as Zernike polynomials or Fourier Series) to represent atmospheric turbulence phase screens and generate phase screens with vectors of parameters as weights of these basis functions [14–17]. The representation method could generate large phase screens with fast speed and high accuracy, but finite length. The extrapolation method has been proposed to generate infinite long phase screens for simulation of adaptive optics systems [18,19]. However, both of these methods are model–based forward computation methods: they require prior assumptions about statistical properties of atmospheric turbulence, such as spatial power spectral density of phase screens. Therefore, they often require manual intervention to obtain statistical properties from real measurements. Deviation from standard turbulence model would make it hard to develop phase screen generation methods timely from real measurements [20–23].
Deep neural networks (DNN) can model very complex functions directly from data [24]. A recent work [25] has addressed the issue of efficiently simulating the atmospheric turbulence effect. Their design demonstrate the feasibility of combining computational blocks with data-driven neural networks. Besides, contemporary DNNs are optimized with Graphics Processing Units to improve calculation speed. Thanks to these developments, it would be possible for us to build the digital twin of atmospheric turbulence phase screens with DNNs. Our method includes three different DNN parts:
- 1. the measurement decomposition part, which could decompose phase screens obtained from measurements or other simulation methods into sets of parameters under predefined basis functions;
- 2. the phase screen generation part, which would generate phase screens from sets of parameters under predefined basis functions;
- 3. the random number generation part, which would generate parameters of the next phase screen from given parameters, according to temporal statistical properties learned from data.
With these three DNNs, our method could generate phase screens directly according to simulated or real measurements of atmospheric turbulence phase screens, with minimal human intervention. Our method could generate infinite long phase screens with high accuracy, acceptable speed and high similarities to real measurements. We will introduce the structure of our method in Section 2. Since our method is a data–driven method, there would be multiple ways to use our method, according to the way we obtain the training data. Phase screens in the training dataset could be generated through ordinary phase screen generation methods, fluid dynamical simulation or reconstructed from real measurements. When simulated phase screens are used, we use phase screens and parameters that are used to generate phase screens as the training data. In this scenario, our method could generate infinite long phase screens with high accuracy and acceptable speed. When phase screens reconstructed from real measurements or obtained from fluid dynamical simulation are used, we use phase screens and their representations, such as coefficients of Zernike Polynomials or zonal slope measurements, as the training data. In this scenario, our method could obtain statistical properties of atmospheric turbulence phase screens automatically and generate infinite long phase screens accordingly. We will discuss these two application scenarios in Section 3. In Section 4, we will give our conclusions and anticipate our future works.
2. Principle of the digital twin of atmospheric turbulence phase screens
2.1 Basic structure of the digital twin of atmospheric turbulence phase screens
DNNs are normally used as end-to-end black box models in many different applications. However according to our experience, it would be easier to check the performance of a DNN based model, if we split it into several small scale models and each of them has independent functional abilities [26]. The digital twin of turbulence phase screens is developed under this philosophy and its basic structure is shown in Fig. 1. There are three independent parts: the measurement decomposition part (Rand-Net), the phase screen generation part (Phs-Net) and the random number generation part (Next-Net). These different parts are trained and checked independently during the development stage and could be used independently or integrated together for real applications. We will introduce these different neural networks in the following subsections.
Fig. 1. The structure of the DNN used in this paper. Phase screens will be firstly decomposed into vectors of parameters by the Rand-Net as seed. Then, the Next-Net will generate vectors of parameters according to the input. At last, the Phs-Net will generate new phase screens according to outputs from the Next-Net.
2.2 Phase screen decomposition neural network – Rand-Net
For most simulation tasks, phase screens are matrix with large size, around hundreds to thousands pixels times hundreds to thousands pixels. While measurements of phase screens, such as slopes obtained by Shack-Hartman wavefront sensors or strength of turbulence obtained by DIMMs or SLODARs, are sparse and contain little information. Therefore, if we use real measurements to generate phase screens, we need to firstly reconstruct or generate phase screens from these measurements [26]. Then to reduce computation burden, we could further transform reconstructed phase screens or simulated phase screens to a domain which is orthogonal and contains less parameters, such as the space of Zernike Polynomials [14] or Fourier Series [17]. The Rand-Net is used to learn the transformation between phase screens and parameters in a predefined orthogonal space as shown in Fig. 2.
Fig. 2. The structure of the Rand-Net. The Rand-Net would output a vector of parameters according to input phase screens based on predefined basis functions. The Rand-Net includes an encoder part in blue color and a decoder part in yellow color. The detailed structure of the Rand-Net could be found in table S1 in the Supplement 1.
Inputs of the Rand-Net are phase screens and outputs of the Rand-Net are vectors of parameters. In this paper, we assume the phase screen with size of $128 \times 128$ pixels and the vector of parameters with size of $100\times 1$ pixels. The size of parameter vectors and the size of phase screens would be arbitrary. In that case, we could just modify the size of input layer and that of other layers accordingly. To increase representation ability of the Rand-Net, we introduce 3 additional Residual blocks. The structure of the Residual block is shown in table S2 in the Supplement 1. The activation function in each layers of the Rand-Net is the Leaky Relu function with 0.2 as its slope for negative values and the last layer of the Rand-Net uses Sigmoid function as activation function. The loss function in the Rand-Net is defined in Eq. (1):
where $f_i$ is the vector of parameters extracted from phase screen $i$ and $r_i$ is the vector of parameters that are used to generate or reconstruct phase screens.2.3 Phase screen generation neural network – Phs-Net
The Phs-Net is used to generate phase screens from given vectors of parameters. The Phs-Net could be viewed as the inverse function of the Rand-Net. However, the size of outputs of the Phs-Net could be arbitrary, because the Phs-Net is a deep convolution generative adversarial network, whose loss function is used to classify images based on their similarities, regardless of their size. If the size of outputs of the Phs-Net is larger than that of phase screens in the training set, the Phs-Net will generate phase screens with finer details according to a predefined power spectral density function. The structure of the Phs-Net is shown in Fig. 3, which includes a discriminator and a generator [27]. The generator is used to generate new data, while the discriminator is used to classify between newly generated data and real data. The generator and the discriminator will be trained together: the generator would try to fool the discriminator and the discriminator would try to better classify between real and generated data.
Fig. 3. The structure of the Phs-Net, which includes a generator and a discriminator. Through training the discriminator and the generator together, we could obtain a generator to generate high fidelity phase screens according to vectors of parameters.
The generator in the Phs-Net is a convolution neural network as shown in Fig. 4, which will generate a phase screen according to a given set of parameters. In this figure, we set the dimension of the input is $100\times 1$ pixels and that of the phase screen is $128\times 128$ pixels. For phase screens with different shapes, we can modify the structure of the generator accordingly. The activation function for all layers except the output layer is the LeakyReLU functions with 0.2 as its slope for negative values. The activation function for the output layer is the ReLU function.
Fig. 4. The structure of the generator in the Phs-Net. It is a convolution neural network which would generate a phase screen with a vector of parameters as input. The detail structure of the generator could be found in table S3 in the Supplement 1.
The discriminator in the Phs-Net is a convolution neural network as shown in Fig. 5, with phase screens as inputs and results obtained by the loss function as outputs. The loss function has two contributions: Binary Cross Entropy loss (BCE) and Focal Frequency loss (FFL). The BCE is defined in Eq. (2):
where $x_n$ and $y_n$ are outputs of the discriminator through comparing between phase screens generated by the generator and phase screens in the training set.Fig. 5. The structure of the discriminator in the Phs-Net. It is a Convolution Neural Network which would output similarities between generated and reference phase screens. The detail structure of the discriminator could be found in table S4 in the Supplement 1.
The FFL is defined to confine the power spectral density of generated phase screens within theory prediction (such as Von Karman power spectral, Kolmogorov power spectral or power spectral fitted by real measurements). The FFL is defined in Eq. (3):
2.4 Random number generation neural network – Next-Net
With a trained Rand-Net, we could get vectors of parameters of any phase screens under predefined basis functions. Therefore, if we put temporal continuous phase screens into the Rand-Net, we could then obtain vectors of parameters of these phase screens. These vectors of parameters can be used to train a forecasting neural network. The forecasting neural network could predict vectors of parameters of the next phase screen. Based on this concept, we propose the Next-Net as the forecasting neural network. The structure of the Next-Net is shown in Fig. 6 and full details could be found in table S5 in the Supplement 1.
Fig. 6. The structure of the Next-Net. It is also an AutoEncoder, which would output a vector of parameters according to the input vector of parameters. The detail structure of the Next-Net could be found in table S5 in the Supplement 1.
The Next-Net is also an AutoEncoder with vectors of parameters from previous phase screens as inputs and those from next phase screens as outputs. The size of the input and that of the output in the Next-Net are $100\times 1$ pixels in this paper. The structure of the Next-Net could be modified according to the size of vectors of parameters. The activation function in each layers of the Next-Net is also Leaky ReLU function with 0.2 as its slope for negative values and the last layer of the Next-Net also uses Sigmoid function as activation function.
As a forecasting neural network, the training data limits the minimal prediction step and temporal statistical properties of the Next-Net. The Next-Net could also use simulated or real phase screens as training data. For simulated data, we could define minimal steps between phase screens as one or sub pixel and split large phase screens into continuous smaller phase screens with one or sub-pixel shift (interpolated phase screens) as the training data. For real measurements, we could directly send parameters obtained from reconstructed phase screens by the Rand-Net as the training data. However, it should be noted that, when the turbulence is strong or the temporal sampling rate is low, there is a risk that the Next-Net would not be able to converge during the training stage.
3. Training and test of the digital twin of atmospheric turbulence phase screens
In this section, we will show the performance of the digital twin of atmospheric turbulence phase screens in two different scenarios. In the first scenario, the neural network is trained with simulated phase screens. In the second scenario, we show the application of the neural network as a digital twin block. We use phase screens reconstructed from wavefront measurements to train the neural network and the trained neural network could generate new phase screens with wavefront measurements as seeds. The training and test steps discussed in this paper are carried out in a computer with one RTX 3090 GPU card.
3.1 Training and test of the neural network with simulated phase screens
3.1.1 Training of the neural network with simulated phase screens
In the first scenario, the neural network is trained with simulated phase screens. The Series addition method is selected to generate phase screens in the training set, because it could produce phase screens with high accuracy at the cost of long computation time [17]. We have generated two sets of phase screens to train different parts of the neural network:
- 1. 4000 phase screens with size of $128 \times 128$ pixels and 4000 corresponding vectors of parameters with size of $100 \times 1$ pixels are used as training data of the Rand-Net and Phs-Net;
- 2. 4000 groups of continuous phase screens (1000 phase screens in each group) extracted from large phase screens are used as training data of the Next-Net, as shown in Fig. 7.
Fig. 7. A large phase screens will be divided into several stripes firstly. Then Each stripe will be divided into small patches of phase screens. There is no overlap area for each stripes and each stripe is an independent data set. There is only one pixel shift between adjacent phase screens in one long stripe.
All these phase screens have a size of $128\times 128$ pixels. The physical aperture of each phase screen is 1 meter (0.078 meter/pixel) with $r_0$ of 6 centimeter. The spatial power spectral density of these phase screens satisfies the Kolmogorov power spectral density.
We firstly train the Phs-Net with 4000 phase screens and corresponding vectors of parameters (dataset 1). For the generator, we select the Adam algorithm [29] as the optimizer. For the discriminator, we select the SGD algorithm as the optimizer. The batch size is 40. The learning rate is 0.0001 and will multiply 0.1 after 200 epochs. The Phs-Net will be trained by 2000 epochs.
We train the Rand-Net with a slightly different strategy as shown in Fig. 8. We will pick up the generator in a trained Phs-Net, freeze its weights and connect it to the output layer of the Rand-Net. Phase screens will firstly be put into the Rand-Net to generate vectors of parameters. Then vectors of parameters will be put into the generator of the Phs-Net to generate phase screens. At last, we will calculate the mean square error (MSE) between phase screens generated by the Phs-Net and original phase screens as the loss function. 4000 phase screens and corresponding parameter vectors are used as the training set. The Adam algorithm is used as the optimizer [29]. The batch size is set as 20 and there are 200 batches in each epoch. We have trained the Rand-Net with 200 epochs.
Fig. 8. The training stage for the Rand-Net. We will connect the generator from the trained Phs-Net to the Rand-Net. Phase screens will firstly be decomposed by the Rand-Net into parameters. Then parameters will be put into the generator to generate phase screens. The MSE between these two phase screens will be used as loss function to train the Rand-Net.
At last, we will train the Next-Net. All groups of phase screens defined in dataset 2 will be processed by the trained Rand-Net to obtain random vectors of them. The Adam algorithm is used as the optimizer [29]. The batch size is set as 40 and there are 100 batches in each epoch. We will train the Rand-Net with 200 epochs. In total, training of the whole neural network would require 45.55 hours.
3.1.2 Test of the neural network with simulated phase screens
After training steps mentioned above, we could obtain the neural network as a continuous phase screen generator for real applications. A trained version of the neural network can be downloaded from [30]. In this section, we will test the performance of the neural network. A phase screen with size of $128\times 128$ pixels is generated as the seed screen. The seed screen is put into the Rand-Net to generate a vector of parameters as an representation of the seed phase screen in the Series space. Then the vector of parameters is put into the Next-Net to generate a vector as Series representation of the next phase screen. The new vector will be fed into the Next-Net to generate next new vector and the new vector will also be fed into the Phs-Net to generate continuous phase screens, as we show in Fig. 1.
We use the trained neural network to generate 2000 continuous phase screens with size of $128\times 128$ pixels. It would cost 32.7 seconds for our method to generate these phase screens, while it would cost 11.3 hours for the Series Addition method. The neural network has shown faster speed than the Series Addition Method. Besides, since the DNN is highly parallel, we could generate larger phase screens without much time increment. We have tested the requirement of GPU memory and time in generating phase screens. It would cost 6 GB memory in generating phase screens with size of $128\times 128$ pixels and around 24 GB with size of $1024\times 1024$ pixels. Meanwhile, it would cost almost the same time for generating phase screens: 9.22 seconds for phase screens with size of $128 \times 128$ pixels and 36.02 seconds with size of $1024 \times 1024$ pixels.
We have calculated the spatial power spectral density of phase screens generated by the neural network and those of phase screens generated by the Series Addition method. As shown in Fig. 9, the spatial power spectral density of phase screens generated by the neural network is quite close to that of phase screens generated by the Series Addition method. Besides, we have also calculated the temporal power spectral density of phase screens generated by the neural network. We can find that the neural network has a temporal power spectral density that is close to theoretical predictions. These continuous phase screens are saved as a movie in [31] (See Visualization 1).
Fig. 9. The spatial and normalized temporal power spectral density of phase screens generated by the neural network and that of phase screens generated by the Series Addition method. We can find that the spatial power spectral density of phase screen generated by the neural network is very close to that of phase screens generated by the Series addition method. The temporal power spectral density of phase screen generated by our method is close to theoretical predictions.
3.2 Training and test of the neural network as a block in the digital twin
3.2.1 Training of the neural network as a block in the digital twin
In this scenario, we would consider to use the neural network as a block in the digital twin of an optical telescope. The neural network would use phase screens reconstructed from wavefront measurements as the training data. After training, the neural network should be able to reproduce phase screens according to simulation requirements and properties of real measurements. We have built a Monte-Carlo model to simulate the whole system, which includes a telescope with 1 meter diameter and a Shack-Hartmann wavefront sensor with $10\times 10$ sub–apertures to measure wavefronts [32]. There is a single layer of atmospheric turbulence phase screen in front of the primary mirror of the telescope. The digital twin of atmospheric turbulence phase screens is required to produce infinite long phase screens with $128\times 128$ pixels. A schematic draw of this Monte-Carlo Model and the digital twin of the optical telescope is shown in Fig. 10.
Fig. 10. Schematic draw of the Monte Carlo model and the digital twin model of the telescope. Phase screens reconstructed from wavefront measurements will be used to train the digital twin of atmospheric turbulence phase screens. After training, the digital twin of atmospheric turbulence phase screens could be used to generate phase screens for the digital twin of the optical telescope.
In the Monte Carlo model, we have generated 100 groups of atmosphere turbulence phase screens with the autoregressive model (AR) [23], because the AR model could include ‘boiling’ effect with the autoregression coefficient $\alpha$. The coherent length of these phase screens in each group is a random number with equal distribution between 6 centimeter and 10 centimeter. The power spectral density of these phase screens satisfies the Kolmogorov power spectral density. The wind speed is uniform (6 m/s) and the wind direction is the same (from west to east). Simulated phase screens have a spatial resolution of 300 pixels/meter. The Shack-Hartmann wavefront sensor measures wavefronts at a frequency of 300 Hz. We assume there are no read out noise, dark noise or photon noise in the Shack-Hartmann wavefront sensor. With the Monte Carlo simulation model, we could obtain slope measurements with the center of gravity method. With zonal wavefront reconstruction method, we could reconstruct phase screens with size of $128\times 128$ pixels from wavefront measurements. The training data for the digital twin are 100 groups of reconstructed phase screens and their corresponding slope measurements (600 phase screens and slope measurements in each group).
Firstly, we select $10\%$ of all phase screens and slope measurements of these phase screens to train the Rand-Net and the Phs-Net (6000 phase screens and slope measurements in total). The optimizer, the batch size and the learning rate are the same as we mentioned in Section 3.1.1. After training, the generator of the Phs-Net has been connected to the output layer of the Rand-Net. With the same loss function, batch size and optimizer, we train the Rand-Net. The procedure of training of the Next-Net is almost the same as we discussed in Section 3.1.2, except that slope measurements are used instead of parameters obtained from the Series Addition method. All groups of phase screens ($100 \times 600$ phase screens) are transformed to slope measurements by the Rand-Net and then we apply the same strategy to train the Next-Net. In total, it costs us 131 hours to train the whole neural network.
3.2.2 Test of the neural network as a block in the digital twin
After training, the neural network could be used as a block in the digital twin of ground based optical telescopes. We have generated 2000 phase screens of $128\times 128$ pixels with the same procedure as discussed in Section 3.1.2 to check the performance of our method. It costs around 32 seconds to generate phase screens with the neural network. We have modified the pixel scale in the AR model and also generated 2000 phase screens with size of $128\times 128$ pixels. It costs around 8 seconds for the AR method. Firstly, we have calculated the spatial power spectral density as shown in the left figure of Fig. 11. We can find that the neural network could reproduce phase screens with spatial power spectral density that is close to that of phase screens generated by the AR method. It should be noted that this figure shows that our method could generate phase screens with high fidelity in high spatial frequency band, where the Shack-Hartmann wavefront sensor can not measure, due to its low spatial sampling rate. It is probably caused by the generator of the Phs-Net, which uses power spectral density as regularization condition to generate detail structures that can not be measured by the Shack-Hartmann wavefront sensor.
Fig. 11. The spatial and normalized temporal power spectral density of phase screens generated by the neural network and that of phase screens generated by the AR method. We can find that the spatial power spectral density of phase screens generated by the neural network is very close to that of phase screens generated by the AR method. For the temporal power spectral density, both of these two methods could produce phase screens with temporal power spectral densities that are close to theory predictions.
We have further calculated the temporal power spectral density of phase screens as shown in the right figure of Fig. 11. Since the temporal power spectral density is important for performance evaluation of adaptive optics and the AR could include temporal variation (‘boiling effect’) in simulations. We have also plotted the power spectral density of phase screens generated by the AR method. We could find that the temporal power spectral densities of the original phase screens and those of phase screens generated by the neural network are quite close to results predicted by theory. However, since the autoregression coefficient $\alpha$ in the AR model is fixed as 0.999, the peak of the power spectral density function of phase screens generated by both the AR method and the neural network has lower peak than that predicted by the theory.
Since there are only hundreds to thousands phase screens in the training set, we could only obtain phase screens with finite length to train our neural network. Therefore, it would be necessary to investigate the performance of our method in generating phase screens that are longer than phase screens in the training set. We have generated several patches of continuous phase screens (20 phase screens for each patch) to train the neural network. After training, we have generated 500 continuous phase screens to test the performance of our method. Results are shown in Fig. 12. We can find that although the number of phase screens in the training set is much smaller than that of generated phase screens, our method still could obtain effective results.
Fig. 12. The temporal power spectral density of phase screens generated by different methods. The temporal power spectral density of phase screens generated by our method (digital twin) is close to that of 500 continuous phase screens generated by the AR method and the theoretical prediction. The 20 AR method stands for 20 continuous phase screens in the training set, which shows deviation from theoretical predictions.
Besides, it should be noted that, although the Phs-Net uses generative model to generate phase screens which would introduce variability to the final results, the Next-Net could still predict parameters of phase screens under predefined basis. Therefore, it is interesting to investigate the performance of the Next-Net. We have generated 500 phase screens with the AR method and decompose these phase screens to slope measurements. Then we input these slope measurements to the Next-Net and predict parameters of the next 499 phase screens. These results are shown in Fig. 13. Ground Truth and Digital Twin stands for normalized wavefront error reconstructed from slope measurements. They are used for comparison. MSE stands for mean square error between ground truth phase screens and phase screens generated by our method. As we can find that the Next-Net could predict phase screens with error between $1.44\%$ and $0.2\%$ of the total wavefront error in each frame.
Fig. 13. Mean square error between ground truth phase screens and screens generated by the Next-Net. The Next-Net in our method could predict phase screens in a predefined basis.
4. Conclusions and future work
As more and more optical, mechanical and electronic elements are used in optical telescopes, the optical telescope will become more and more complex. It would become hard to design and maintain different parts of a modern optical telescope by limited human resources and funding. With recent developments in sensors and machine learning algorithms, we could obtain more and more data from elements in telescopes and transform these measurements to states of different elements. With high fidelity optical element models, we could carry out simulations to test and predict system performance. This concept, known as digital twin, would help us to design and maintain optical telescopes with high efficiency and low cost.
We propose to use deep neural networks to build digital twin of atmospheric turbulence phase screens. Our method could directly learn spatial and temporal statistical properties of phase screens and generate infinite long phase screens with high accuracy and acceptable speed. However, there are still some problems left to be solved. First of all, our method is an end-to-end data-driven method, which purely models atmospheric turbulence phase screens from the training data, regardless of statistical parameters, such as the coherent length ($r_0$), the outer scale ($L_0$) and the inner scale ($l_0$). These statistical parameters are learned as hidden parameters in the neural network, which makes it hard to generate phase screens with a particular set of statistical parameters. Therefore, we still need to further develop an appropriate data selection and generation method to confine statistical parameters of output phase screens.
Secondly, some variable properties, such as phase screens with variable spatial or temporal power spectral densities or boiling factors could be learned by the neural network with appropriate training data. In this paper, we use the same autoregression coefficient in the AR model to generate phase screens as the training data. Although results show that our method could successfully intimate this effect, there is a very tough challenge to obtain phase screens with the same autoregression coefficients. The challenge includes both developing of an appropriate measure instruments and an appropriate data generation and evaluation method. In the future, we will analyze these problems and use our method to build data driven simulator not only for 3D atmospheric turbulence phase screens but also for thermal, dome seeing and other outer environments for large and small telescopes, such as the Chinese Large Optical/Infrared telescope [33,34] and the Sitian Telescope Array [35].
Funding
National Natural Science Foundation of China (12173027, 12173062.); National Key Research and Development Program of China (2019YFA0706601); science research grants from the China Manned Space Project (CMS-CSST-2021-A01, CMS-CSST-2021-B12).
Acknowledgments
Authors would like to thank the anonymous referees for their constructive and insightful commentary. Peng Jia would like to thank Professor Xiangqun Cui from Nanjing Institute of Astronomical Optics & Technology and Professor Kaifan Ji from Yunnan Observatory who provide very helpful suggestions for this paper.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are available in Ref. [30].
Supplemental document
See Supplement 1 for supporting content.
References
1. L. Wang and B. Ellerbroek, “Fast end-to-end multi-conjugate ao simulations using graphical processing units and the maos simulation code,” in Proceedings of the Second AO4ELT Conference, (Citeseer, 2011).
2. M. Le Louarn, R. Clare, C. Béchet, and M. Tallon, “Simulations of adaptive optics systems for the e-elt,” in Adaptive Optics Systems III, vol. 8447 (International Society for Optics and Photonics, 2012), p. 84475D.
3. R. Conan, A. Bouchez, F. Quiros-Pacheco, and B. McLeod, “The gmt dynamic optical simulation,” in Adaptive Optics for Extremely Large Telescopes 4–Conference Proceedings, vol. 1 (2015).
4. B. Carlomagno, O. Absil, M. Kenworthy, G. Ruane, C. U. Keller, G. Otten, M. Feldt, S. Hippler, E. Huby, D. Mawet, C. Delacroix, J. Surdej, S. Habraken, P. Forsberg, M. Karlsson, E. Vargas Catalan, and B. R. Brandl, “End-to-end simulations of the E-ELT/METIS coronagraphs,” in Adaptive Optics Systems V, vol. 9909 of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference SeriesE. Marchetti, L. M. Close, and J.-P. Véran, eds. (2016), p. 990973.
5. E. Glaessgen and D. Stargel, “The digital twin paradigm for future nasa and us air force vehicles,” in 53rd AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics and materials conference 20th AIAA/ASME/AHS adaptive structures conference 14th AIAA, (2012), p. 1818.
6. Y. Zheng, S. Wang, Q. Li, and B. Li, “Fringe projection profilometry by conducting deep learning from its digital twin,” Opt. Express 28(24), 36568–36583 (2020). [CrossRef]
7. D. Wang, Z. Zhang, M. Zhang, M. Fu, J. Li, S. Cai, C. Zhang, and X. Chen, “The role of digital twin in optical communication: Fault management, hardware configuration, and transmission simulation,” IEEE Commun. Mag. 59(1), 133–139 (2021). [CrossRef]
8. Q. Zhuge, “Modeling, monitoring, and self-learning techniques for building an ai-driven digital twin optical system,” in Next-Generation Optical Communication: Components, Sub-Systems, and Systems X, vol. 11713 (International Society for Optics and Photonics, 2021), p. 117130M.
9. V. I. Tatarski, Wave propagation in a turbulent medium (Courier Dover Publications, 2016).
10. H. W. Babcock, “The possibility of compensating astronomical seeing,” Publ. Astron. Soc. Pac. 65, 229–236 (1953). [CrossRef]
11. N. M. Law, C. D. Mackay, and J. E. Baldwin, “Lucky imaging: high angular resolution imaging in the visible from the ground,” Astron. & Astrophys. 446(2), 739–745 (2006). [CrossRef]
12. J.-L. Starck, E. Pantin, and F. Murtagh, “Deconvolution in astronomy: A review,” Publ. Astron. Soc. Pac. 114(800), 1051–1069 (2002). [CrossRef]
13. Y. Guo, L. Zhong, L. Min, J. Wang, Y. Wu, K. Chen, K. Wei, and C. Rao, “Adaptive optics based on machine learning: a review,” Opto-Electronic Advances pp. 200082–1 (2022).
14. N. A. Roddier, “Atmospheric wavefront simulation and zernike polynomials,” in Amplitude and Intensity Spatial Interferometry, vol. 1237 (International Society for Optics and Photonics, 1990), pp. 668–679.
15. R. Lane, A. Glindemann, and J. Dainty, “Simulation of a kolmogorov phase screen,” Waves in random media 2(3), 209–224 (1992). [CrossRef]
16. M. Charnotskii, “Sparse spectrum model for a turbulent phase,” J. Opt. Soc. Am. A 30(3), 479–488 (2013). [CrossRef]
17. P. Jia, D. Cai, D. Wang, and A. Basden, “Simulation of atmospheric turbulence phase screen for large telescope and optical interferometer,” Mon. Not. R. Astron. Soc. 447(4), 3467–3474 (2015). [CrossRef]
18. F. Assémat, R. W. Wilson, and E. Gendron, “Method for simulating infinitely long and non stationary phase screens with optimized memory storage,” Opt. Express 14(3), 988–999 (2006). [CrossRef]
19. P. Jia and S. Zhang, “Simulation and fabrication of the atmospheric turbulence phase screen based on a fractal model,” Res. Astron. Astrophys. 12(5), 584–590 (2012). [CrossRef]
20. C. Rao, W. Jiang, and N. Ling, “Spatial and temporal characterization of phase fluctuations in non-kolmogorov atmospheric turbulence,” J. Mod. Opt. 47(6), 1111–1126 (2000). [CrossRef]
21. P. Martinez, J. Kolb, M. Sarazin, and A. Tokovinin, “On the difference between seeing and image quality: when the turbulence outer scale enters the game,” The Messenger 141, 5–8 (2010).
22. V. Lukin, E. Nosov, V. Nosov, and A. Torgaev, “Causes of non-kolmogorov turbulence in the atmosphere,” Appl. Opt. 55(12), B163–B168 (2016). [CrossRef]
23. S. Srinath, L. A. Poyneer, A. R. Rudy, and S. M. Ammons, “Computationally efficient autoregressive method for generating phase screens with frozen flow and turbulence in optical simulations,” Opt. Express 23(26), 33335–33349 (2015). [CrossRef]
24. I. Goodfellow, Y. Bengio, and A. Courville, Deep learning (MIT University, 2016).
25. A. Rodriguez, C. R. Cuellar, L. F. Rodriguez, A. Garcia, V. R. Gudimetla, V. K. Kotteda, J. A. Munoz, and V. Kumar, “Stochastic analysis of les atmospheric turbulence solutions with generative machine learning models,” in Fluids Engineering Division Summer Meeting, vol. 83716 (American Society of Mechanical Engineers, 2020), p. V001T01A001.
26. P. Jia, M. Ma, D. Cai, W. Wang, J. Li, and C. Li, “Compressive shack–hartmann wavefront sensor based on deep neural networks,” Mon. Not. R. Astron. Soc. 503(3), 3194–3203 (2021). [CrossRef]
27. A. Radford, L. Metz, and S. Chintala, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks,” arXiv e-prints arXiv:1511.06434 (2015).
28. L. Jiang, B. Dai, W. Wu, and C. C. Loy, “Focal frequency loss for image reconstruction and synthesis,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, (2021), pp. 13919–13929.
29. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).
30. J. Peng, W. Weihua, N. Runyu, and X. Xiaolei, “Code for digital twin of atmosspheric turbulence phase screens based on deep neural networks (doi: 10.12149/101110),” (2022).
31. J. Peng, W. Weihua, N. Runyu, and X. Xiaolei, “Visulization for digital twin of atmosspheric turbulence phase screens based on deep neural networks,” (2022).
32. A. G. Basden, N. Bharmal, D. Jenkins, T. Morris, J. Osborn, J. Peng, and L. Staykov, “The durham adaptive optics simulation platform (dasp): Current status,” SoftwareX 7, 63–69 (2018). [CrossRef]
33. X. Cui, Y. Zhu, M. Liang, D.-q. Su, X. Yuan, Z. Hu, H. Bai, and B. Gu, “Introduction on Chinese 12m optical/infrared telescope (LOT),” in Ground-based and Airborne Telescopes VII, vol. 10700 of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference SeriesH. K. Marshall and J. Spyromilio, eds. (2018), p. 107001P.
34. P. Jia, A. Basden, and J. Osborn, “Ground-layer adaptive-optics system modelling for the chinese large optical/infrared telescope,” Mon. Not. R. Astron. Soc. 479, 829–843 (2018). [CrossRef]
35. J. Liu, R. Soria, X.-F. Wu, H. Wu, and Z. Shang, “The SiTian Project,” An. Acad. Bras. Cienc. 93(suppl 1), 20200628 (2021). [CrossRef]
