Open Access
Add to BibSonomy
Abstract
Data-driven approaches have been proposed as effective strategies for the inverse design and optimization of photonic structures in recent years. In order to assist data-driven methods for the design of topology of photonic devices, we propose a topological encoding method that transforms photonic structures represented by binary images to a continuous sparse representation. This sparse representation can be utilized for dimensionality reduction and dataset generation, enabling effective analysis and optimization of photonic topologies with data-driven approaches. As a proof of principle, we leverage our encoding method for the design of two dimensional non-paraxial diffractive optical elements with various diffraction intensity distributions. We proved that our encoding method is able to assist machine-learning-based inverse design approaches for accurate and global optimization.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Over the past two decades, the advancement of photonics has been enabling vast approaches for manipulating light in the wavelength scale. By engineering the building blocks of the photonic materials and devices, the behavior of light, such as phase, amplitude, and polarization of transmitted light and near field responses of wave, can be accurately controlled. Diverse photonic devices ranging from diffractive optical elements (DOE) to metamaterials (MM) and metasurfaces (MS) [1,2], are designed for extensive applications such as virtual/augmented reality displays [3,4], miniaturized imaging systems [5,6], and quantum optics platforms [7]. However, the complex mechanisms of light-matter interaction prevents an intuitive strategy for the design of the building blocks in the photonic devices. As such, various inverse design and optimization algorithms have been developed for the expeditious design of photonic structures. For example, given some parameters that define the photonic device, adjoint methods [8,9] calculate the gradients of the parameters with respect to the objective function, and incrementally updates the structure by subtracting the gradients from the parameters. Metaheuristic optimizations [10,11], on the other hand, treat the physical system as a black box, and update the parameters following the manner inspired by physical and biological systems.
With the fast evolution of machine learning (ML) and deep learning (DL) techniques [12], data-driven methods are emerging as an alternative way to discover and design photonic structures and devices [13]. Fully connected neural networks (FCNs) are leveraged for the approximation of photonics systems with tens to hundreds of parameters, and have been utilized to successfully optimize photonic structures such as photonic crystals [14,15], waveguides [16,17], chiral metamaterials [18], and metasurfaces [19]. Deep reinforcement learning has also been proposed to design and optimize photonic structures for color generation [20] and perfect absorption [21] with potential applications in imaging, displays and bioengineering. When the degrees of freedom (DOF) of the photonic system grows to thousands and more, convolutional neural networks (CNN) are adopted for the accurate prediction of the physical responses with much lower computational complexity [22,23]. Photonic structures represented in pixelated images, for example, are usually processed by CNNs to reduce the DOF for further optimization. Additionally, generative models, such as variational autoencoders (VAEs) [24] and generative adversarial networks (GANs) [25,26], are utilized for the design of high DOF metasurface nanostructures in an expeditious way [27–29]. The stochastic nature of the generative models enables the exploration of the solution space in a global way. Consolidating traditional optimization techniques, GAN and VAE are able to discover the topology of nanostructures with improved efficiency and robustness [30–33].
With regard to the inverse design of photonic structures, optimizing the topology of a photonic structure with arbitrary shape has been a long-sought-after goal. Typically, the topology of photonic structures is represented in binary images. Because of the discretization and the high DOF of binary images, optimization is likely stuck in local minima. Although generative models are able to discover new topologies so as to approach the global minimum, the bias of the training dataset and the limited capacity of the network cause incompleteness of the solution space, i.e., the global minimum may not be included in the space defined by the training dataset. Here, we propose an encoding method that is able to transform the binary image to a continuous sparse representation. This encoding approach can be used for data generation and dimensionality reduction of photonic structures, augmenting the capacity of ML models for analyzing the dataset, and enhancing the likelihood of achieving the global minimum in the optimization problems. As a proof of principle, we consolidate the proposed encoding method and a DL-based optimization framework [28,34,35] to inversely design and optimize non-paraxial diffractive optical elements (DOE) with various diffraction intensity distributions. Traditionally, an iterative Fourier transform algorithm (IFTA) is used for the design of binary phase mask of the DOE with a small diffraction angle [36,37]. However, this approach does not take into account the physical process and loses its fidelity when the paraxial approximation is not valid. We proved that our encoding method is able to assist data-driven methods for the accurate and global optimization of DOE topologies.
2. Encoding the topology of photonic structures
Our goal is to encode the nanostructures represented in a 2D binary image into a continuous sparse representation so as to assist data-driven approaches such as ML and DL to analyze, discover, and optimize the topology of photonic structures. In the following discussion, we denote the binary image to be encoded as $f(x,y) \in \{0,1\}^N$, where $x$ and $y$ are coordinates of the image, and $N$ is the dimension of the image. Fourier transform (FT) can be used to transform the binary image to a sparse representation. However, when some operations such as filtering out high-frequency components, are applied to the transformed sparse representation, the inverse Fourier transform (IFT) of the sparse representation may not be a binary image anymore. Although a threshold can be used to binarize the image, different thresholds can transform a non-binary image to different binary images. An inappropriate choice of the threshold may result in the incorrect retrieval of the desired topology. Consequently, FT is not able to encode, decode, and manipulate arbitrary topologies of photonic structures for the general purpose of dimensionality reduction, data analysis, and device optimization.
Instead of simply applying FT to the binary image, we carry out FT to the level set function $\phi (x,y)$ that defines the topology of the structure. A level set function $\phi (x,y)$ is defined as a 3D surface parametrized by the coordinates $x$ and $y$. The topology of a photonic structure can be represented by the zero-level set of $\phi (x,y)$ as $\Gamma =\{(x,y)|\phi (x,y)=0\}$. Given a certain binary image as shown in Fig. 1(a), our encoding strategy is first to construct a level set function as illustrated in Fig. 1(b), and then to apply FT to the level set function so as to derive the sparse representation of the binary image as presented in Fig. 1(c). Inversing the whole process reconstructs a sparse representation to a binary image. The detailed procedure of encoding and decoding photonic structures is shown in Fig. 1(d). We first construct the level set function of the original image $f(x,y)$ through the transform:
Fig. 1. Description of the encoding method (a) Binary image representation of a nanostructure. (b) Illustration of a level set function $\phi (x,y)$. The topology shown in (a) (encircled by red line) is represented by the zero-level set. (c) Encoded sparse representation of the nanostructure shown in (a). (d) The outline of the encoding method. For the encoding process, a level set function is first constructed from the given binary image. The spare representation of the image is derived from the Fourier transform of the level set function. Decoding the binary image from the sparse representation is the inverse of the encoding process.
The exponential function maps the original image to the discrete value $\{-1,1\}^N$, so that the topology of the structure can be represented by $\Gamma =\{(x,y)|\phi _{e} (x,y)=0\}$. By carrying out FT to $\phi _{e}$, we can find the sparse representation of $\phi _{e}$ in the complex frequency space as
By the property of FT, $\hat {\phi }_{e}$ naturally satisfies the condition: where $\hat {\phi }_{e}^{\dagger}$ is the conjugate of $\hat {\phi }_{e}$. Thus, for an image with $N$ pixels, the DOF of its sparse representation is also $N$. The choice of the level set function $\phi _{e}(x,y)$ is not unique, as long as it is invertible, continuous, and bounded with respect to the image $f(x,y)$. The invertibility of the level set function ensures the decoding can be realized by inversing the encoding process, the continuity ensures the continuous topology variation by perturbing the latent vectors, and the boundedness guarantees the encoding vectors are finite for the manipulation of the topology. With the encoded representation $\phi _{e}(x,y)$, certain operations $P$ such as filtering out high-frequency components can be applied to change the topology of the structure $\hat {\phi }_{d} (x,y)=P\hat {\phi }_{e} (x,y)$. To ensure that the corresponding decoded image from $\phi _{d} (x,y)$ is also binary, $\hat {\phi }_{d}$ should meet the same condition $\hat {\phi }_{d} (k_{x},k_{y} ) = \hat {\phi }_{d}^{\dagger} (-k_{x},-k_{y} )$. To decode the image, we apply the encoding process in a reversed order. In detail, we first apply the inverse Fourier transform (IFT) to $\hat {\phi }_{d}$ to find the level set function that represents the photonic structure and then retrieve the binary image representation through the operation: where $\textbf {ang}(\cdot )$ is a function that calculates the phase of $\phi _{d} (x,y)$, and $1/\pi$ is the normalizer that ensures the retrieved image has binary values 0 and 1. Note Eq. (4) is identical to setting a threshold $\phi _{0}=0$ and retrieving the image by3. Properties of the encoding method
The encoding method illustrated above transforms a binary image to a continuous sparse representation, allowing incremental variation of the topology of the structures by perturbing $\hat {\phi }_{d} (k_{x},k_{y}$). We will illustrate several properties of this encoding method and show its advantages in representing photonic structures for data-driven photonic discovery. To be consistent with terminology in machine learning, we will call the space of the sparse representation a latent space and $\hat {\phi }_{d}$ will be a latent vector.
3.1 Dimensionality reduction
Suppose a photonic structure is represented in an image with $N$ number of pixels, and also the DOF of $N$. Performing inverse design and optimization in such a high-dimensional space is difficult. On the other hand, the topology of a structure usually presents some properties such as continuity and connectiveness for the purpose of proper simulation and fabrication. Thus, the available structures cluster in a small region in the $N$-dimensional image space. Machine learning algorithms such as VAEs and GANs have been used to reduce the dimensionality of the photonic structures. However, a trained ML model can only faithfully encode and decode the structures topologically similar to the training dataset. Our method, derived without the dependence of data, is able to process binary image data in a fast and general manner.
Figures 2(a) to 2(d) illustrate the process of encoding and decoding a photonic structure. The initial structure, as shown in Fig. 2(a), is represented in an image with $N=64\times 64$. Our method encodes the structure into its frequency representation $\hat {\phi }_{e}$, the norm of which is presented in Fig. 2(b). After cropping the high-frequency components as in Fig. 2(c), only dominant components are kept in $\hat {\phi }_{d}$. Recovering $\hat {\phi }_{d}$ into binary images through our method, we achieve the decoded image. Interestingly, as decoding the structure is essentially performing the IFT, the recovered images can have arbitrarily large DOF. The decoded example shown in Fig. 2(d) has a resolution of $128\times 128$. As cropping inevitably deletes a portion of the information in the latent space, our encoding method is an irreversible lossy compression. The higher order of terms kept in the latent space, the finer feature will be remained in the decoded images. Other dimensionality reduction approaches can be applied to the encoded latent vectors to further reduce the complexity of the inverse design problem.
Fig. 2. Properties of the encoding method (a) – (d) Dimensionality reduction using the proposed encoding method. Initial binary image (a) is encoded to the sparse representation (b). The low-dimensional representation (latent vector) can be achieved by deleting the high-frequency components as shown in (c). The latent vector can be recovered to the initial structure without substantial loss of information. (e) – (i) Continuously varying two topologies by linearly interpolating the latent vectors. (j) – (n) Generated samples with various geometric symmetries. The shown images are tiled unit cells of the generated patterns. (o) – (p) Adding fine features to initial pattern (o) by gradually expanding the dimensions of latent vectors from $7\times 7$ to $15\times 15$.
3.2 Continuity of the latent space
Since FT and IFT are uniformly continuous operators, the decoded image $f_{d} (x,y)$ can be incrementally varied by perturbing $\hat {\phi }_{d} (k_{x},k_{y})$ . Figures 2(e) to 2(i) show continuous topological variation from the first pattern shown in Fig. 2(e) to the last one in Fig. 2(i). The first and last patterns are randomly constructed with latent vectors $\hat {\phi }_{1},\hat {\phi }_{2}\in \mathbb {R}^{7\times 7}$. Figures 2(f) to 2(h) present the intermediate that are decoded from the linearly interpolated latent vector $\hat {\phi }_{d}=\lambda \hat {\phi }_{1}+(1-\lambda )\hat {\phi }_{2}$, where $\lambda \in (0, 1)$. As we can observe, the two distinct topologies can be smoothly transformed by linearly interpolating their latent vectors. This property is indispensable for the fast convergence when evolution strategy (ES) is utilized for the topology optimization [34]. It is noteworthy that linear interpolation does not always result in continuous topological transformation, especially when the latent space is high dimensional and the topologies of the two patterns are significantly distinct. In this situation, the geodesic, representing the shortest path between the two patterns in the latent space, should be computed for the smooth transformation.
3.3 Symmetry
Symmetry is a crucial geometric property that should always be considered in the design of photonic devices. Properly leveraging the symmetry of the photonic structure reduces the time of simulation and mitigates the difficulty of optimization. Our encoding method maintains the symmetry of binary images in the frequency spaces. Figures 2(j) to 2(n) display a few randomly generated patterns with various geometric symmetries. The dimension of the latent space we chose is $N=5\times 5$. Without any constraints on the latent space, the generated pattern shown in Fig. 2(j) does not present any symmetry. In order to generate a centrosymmetric pattern, we need to enforce the latent vector to be centrosymmetric, i.e., $\hat {\phi }_{d} (k_{x},k_{y} )=\hat {\phi }_{d} (-k_{x},-k_{y} )$. Combined with Eq. (3), this condition is equivalent to the latent vectors being real, reducing the DOF of the latent vectors to $\lfloor N/2 \rfloor +1=13$. Figure 2(k) is a randomly generated centrosymmetric pattern with such constrain. Similarly, if $\hat {\phi }_{d} (k_{x},k_{y} )=\hat {\phi }_{d} (k_{x},-k_{y} )$ and $\hat {\phi }_{d} (k_{x},k_{y} )=\hat {\phi }_{d} (-k_{x},k_{y} )$ are enforced, we can generate axisymmetric patterns such as the one shown in Fig. 2(l). The DOF in this case is reduced to $\lfloor \sqrt {N}/2+1\rfloor ^2=9$. By additionally constraining $\hat {\phi }_{d}(k_{x},k_{y} )=\hat {\phi }_{d} (k_{y},k_{x})$, axisymmetric patterns with axis of symmetric $y=x$ can be produced as shown in Figs. 2(m) and 2(n). In this circumstance, the DOF of the pattern is $(\lfloor {\sqrt {N}/2 \rfloor }+1) (\lfloor \sqrt {N}/2 \rfloor +2)/2=6$, indicating that only six variables are required to arbitrarily manipulate the topology of photonic structures. This unique feature of the encoding/decoding method can be leveraged for reducing the parameters in the inverse design of metasurfaces and photonic crystals with specific polarization requirements.
3.4 Multilevel optimization
When the DOF of a photonic structure is large, optimization techniques suffer from problems such as slow convergence and local minimum. In this situation, multilevel optimization [38] can be used for designing the structure and enhancing the performance. By the nature of FT, our encoding method allows the multilevel optimization of photonic structures by gradually modifying the corresponding latent vectors. Figures 2(o) to 2(s) present an example of adding finer features to the initial structure (Fig. 2(o)) through our encoding method. The initial structure is constructed from a $\hat {\phi }_{d}^{0} \in \mathbb {R}^{7\times 7}$. By attaching additional vectors to the latent vector, we can augment the latent vector to $\hat {\phi }_{d}^{1} \in \mathbb {R}^{9\times 9}$ in a higher dimension. Figure 2(p) presents the decoded image of the augmented latent vector $\hat {\phi }_{d}^{1}$. A few features such as the hole in the center appear. Repeating the augmentation process results in the incremental evolution of the structures with finer features as shown in Figs. 2(q) to 2(s). This unique property of the encoding method enables the consolidation of traditional optimization and multilevel optimization for the inverse design of high DOF photonic structures.
4. Designing non-paraxial diffractive optical elements (DOEs)
As a case study, in this section we will represent how the encoding method can be applied in the inverse design and optimization of binary DOEs. Traditionally, the design of DOEs have relied on an iterative Fourier transform algorithm (IFTA). The algorithm can generate binary phase masks whose FT is proportional to the required diffraction intensity distributions. However, IFTA does not take into account the actual physical process, resulting in the inaccuracy of the modeling for diffraction intensity. This inaccuracy prevents an effective design method for the non-paraxial diffractive beam splitters. To solve this problem, we consolidate the proposed encoding method and a hybrid inverse design algorithm [34] to design $3\times 3$ non-paraxial diffractive beam splitters with various diffraction intensity distributions.
The cross section of a common configuration of the DOEs is shown in Fig. 3(a). The grating and the substrate share the same material with a refractive index $n=1.56$, and the thickness of the grating pattern is $t=840$ nm. Monochromatic light with a wavelength of $\lambda _{0} = 940$ nm is incident from the substrate side. We set the period of the DOE as $p=2.83$ $\mu$m, so that the angle between the 1st and the $0^{th}$ order diffraction is about $21^{\circ }$. The objective of the design is to identify the DOE patterns that are able to accurately diffract the incident light into the 9 different directions with various required intensity distributions. We note that under the incident wavelength of 940 nm there are total $5\times 5$ diffraction orders generated from the DOE. We aim to optimize the intensity distribution of the central 9 orders, while treating the other orders as noises and suppress the intensities of them.
Fig. 3. Configuration of the DOE and the optimization method (a) The cross section of the DOE. The grating pattern and the substrate share the same material with a refractive index of $1.566$. The period of the DOE is $p=2.83$ $\mu$m and the thickness of the grating pattern is $t=840$ nm. Light with a wavelength $\lambda _{0} = 940$ nm is incident from the substrate side. Our aim is to optimize the grating pattern such that the central $3\times 3$ order diffractions present various intensity distributions. The angle between 0 and +1 order diffraction is $21^{\circ }$. (b) Architecture of neural network simulator for predicting the diffraction intensities of DOEs. The input is the encoded vectors of the DOEs, and the output is the vector containing normalized diffraction intensities and maximum intensity of all diffraction orders. The network is an eight-layer fully connected networks, and each hidden layer has 128 neurons. (c) Schematic of the evolution strategy. Randomly generated latent vectors are evaluated by the network simulator. Elites whose performance is closed to the design objectives are selected for subsequent reproduction and mutation. The algorithm iterates until some encoded vectors satisfy the design objectives or the maximum iteration is reached.
In order to leverage data-driven approaches such as DL for fast global optimization, we first generate sufficient DOEs that are able to diffract light to the desired directions. Since the encoding method we proposed is based on FT, it is sufficient to sample $\hat {\phi }_{d}$ from $[-1,1]^{3\times 3}$ as the sparse representation for the design of $3\times 3$ diffractive beam splitters. In practice, we can further simplify the representation of each DOE for the convenient training of a neural network model. In detail, we write $\hat {\phi }_{d}$ as:
Fig. 4. Training of the network and statistic of the optimized results (a) The variation of training (blue) and validation (orange) loss versus the training epoch. The validation loss reaches 0.03 after 100 epochs of training. (b) Uniformity errors of 150 designed DOE structures with the objective of all diffraction intensities being equal. The blue bars represent the distribution of $U_{err}$ calculated with the network simulator during the optimization, and the oranges bars are the evaluated $U_{err}$ using RCWA for validation. The final design is selected from the validated DOE with the minimum $U_{err}$.
In order to globally optimize the topology of the grating patterns, we adopt the modified evolution strategy (ES) [34] as shown in Fig. 3(c). The algorithm starts with sampling a population of random encoded vectors $v$. Each vector is regarded as an individual in the population. These vectors are simulated through the neural network simulator. Based on the simulated results, the population is subsequently evaluated by certain design objectives, and the elites of the population are selected for the following reproduction and mutation. The algorithm iterates until one of the individuals achieves the design criterion or the maximum iteration is reached. To design DOEs with various diffraction intensity distributions $\eta _{obj} \in [0,1]^{3\times 3}$, we define the uniformity error of a design as:
Figures 5(a) to 5(h) present eight examples of designed DOE with various intensity distribution $\eta _{obj}$. For each panel in Fig. 5, the left plot is the tiled unit cell of the designed DOE, the middle image shows the simulated diffraction efficiencies of the desired $3\times 3$ orders and other suppressed noises, and the right plot compares the desired diffraction intensities (blue) and RCWA simulated results of the designed DOE (orange). With the help of our encoding method, the hybrid framework successfully identified DOE structures with diffraction intensity distribution matching the design objectives. Quantitively, the uniformity errors $U_{err}$ of designed DOEs from Figs. 5(a) to 5(h) are 0.035, 0.045, 0.073, 0.068, 0.194, 0.036, 0.352, and 0.079, respectively. Since $U_{err}$ is calculated through scaled intensity $\eta _{scale}$ and objective intensity $\eta _{obj}$ is the denominator of $\eta _{scale}$, the error is extremely sensitive to the objective intensities with small values. In the examples shown in Figs. 5(f), 5(h) and 5(i), diffraction intensities in some orders are required to be small. In such cases a tiny disagreement of the actual diffraction and objective intensities induces large $U_{err}$. Nevertheless, the overall intensity distributions of all designs have excellent agreement with respect to the objectives, confirming the effectiveness of our encoding methods in the machine-learning-based inverse design approaches.
Fig. 5. Examples of designed DOEs with various diffraction intensity distributions. In each panel, the leftmost figure represents a tiled unit cell of the designed DOE. The middle image represents the simulated efficiencies of all the diffraction orders. The rightmost plot compares the objective intensities (blue) versus the RCWA simulated intensities (orange) of the design. All the designed DOEs are able to diffract light with intensity distributions essentially replicating the design objectives. The uniformity errors of the displayed designs are (a) 0.035, (b) 0.045, (c) 0.073, (d) 0.068, (e) 0.194, (f) 0.036, (g) 0.352, (h) 0.079, respectively. By the definition of Eq. (10), when the design objectives include diffraction orders with small intensities, a tiny disagreement of actual diffraction and objectives induces large uniformity errors. This leads to a large $U_{err}$ for the design shown in (e), (g), and (h).
5. Conclusion
In summary, we have proposed an encoding method that is able to transform the topology of a photonic structure represented in discrete, high-dimensional, and binary pixelated images into a continuous sparse representation. We explored properties of the encoding methods, such as the continuity of the latent space and geometric properties of the encoded topologies, and discussed the potential application of this method for dimensionality reduction and data generation for the data-driven photonics optimization. As a case study, we utilized the encoding method and a deep learning-based optimization framework to design $3\times 3$ DOEs with non-paraxial diffraction angle and various diffraction intensity distributions. The encoding method allows us to generate sufficient data for the optimization without exploring unnecessary solution space. The encoded DOE represented in the low dimension also enhances the accuracy of the network and, as a result, the fidelity of the design is increased.
Although the proposed encoding methods are aimed at assisting the data-driven inverse design of photonic structures, other derivative-free traditional optimizations [40] can also take advantage of the continuous low-dimensional representation of photonic structures. If global optimization is not required, local search algorithms can be applied to the latent vectors of the structures without generating a redundant dataset. In the future, we expect to explore the application of the encoding method with both traditional and data-driven optimization approaches for the discovery and design of other photonic media such as photonic crystals and metasurfaces, and anticipate to consolidating the encoding method with deep generative models to produce complex patterns for the general inverse design of photonic structures.
Disclosures
The authors declare no conflicts of interest.
References
1. N. Yu, P. Genevet, M. A. Kats, F. Aieta, J.-P. Tetienne, F. Capasso, and Z. Gaburro, “Light propagation with phase discontinuities: generalized laws of reflection and refraction,” Science 334(6054), 333–337 (2011). [CrossRef]
2. X. Luo, “Subwavelength artificial structures: opening a new era for engineering optics,” Adv. Mater. 31(4), 1804680 (2019). [CrossRef]
3. S. Lan, X. Zhang, M. Taghinejad, S. Rodrigues, K.-T. Lee, Z. Liu, and W. Cai, “Metasurfaces for near-eye augmented reality,” ACS Photonics 6(4), 864–870 (2019). [CrossRef]
4. G.-Y. Lee, J.-Y. Hong, S. Hwang, S. Moon, H. Kang, S. Jeon, H. Kim, J.-H. Jeong, and B. Lee, “Metasurface eyepiece for augmented reality,” Nat. Commun. 9(1), 4562 (2018). [CrossRef]
5. M. Khorasaninejad, W. T. Chen, R. C. Devlin, J. Oh, A. Y. Zhu, and F. Capasso, “Metalenses at visible wavelengths: Diffraction-limited focusing and subwavelength resolution imaging,” Science 352(6290), 1190–1194 (2016). [CrossRef]
6. D. Lin, P. Fan, E. Hasman, and M. L. Brongersma, “Dielectric gradient metasurface optical elements,” Science 345(6194), 298–302 (2014). [CrossRef]
7. K. Wang, J. G. Titchener, S. S. Kruk, L. Xu, H.-P. Chung, M. Parry, I. I. Kravchenko, Y.-H. Chen, A. S. Solntsev, Y. S. Kivshar, D. N. Neshev, and A. A. Sukhorukov, “Quantum metasurface for multiphoton interference and state reconstruction,” Science 361(6407), 1104–1108 (2018). [CrossRef]
8. S. Molesky, Z. Lin, A. Y. Piggott, W. Jin, J. Vucković, and A. W. Rodriguez, “Inverse design in nanophotonics,” Nat. Photonics 12(11), 659–670 (2018). [CrossRef]
9. J. S. Jensen and O. Sigmund, “Topology optimization for nano-photonics,” Laser Photonics Rev. 5(2), 308–321 (2011). [CrossRef]
10. C. Wang, S. Yu, W. Chen, and C. Sun, “Highly efficient light-trapping structure design inspired by natural evolution,” Sci. Rep. 3(1), 1025 (2013). [CrossRef]
11. P. R. Wiecha, A. Arbouet, C. Girard, A. Lecestre, G. Larrieu, and V. Paillard, “Evolutionary multi-objective optimization of colour pixels based on dielectric nanoantennas,” Nat. Nanotechnol. 12(2), 163–169 (2017). [CrossRef]
12. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015). [CrossRef]
13. K. Yao, R. Unni, and Y. Zheng, “Intelligent nanophotonics: merging photonics and artificial intelligence at the nanoscale,” Nanophotonics 8(3), 339–366 (2019). [CrossRef]
14. C.-X. Liu, G.-L. Yu, and G.-Y. Zhao, “Neural networks for inverse design of phononic crystals,” AIP Adv. 9(8), 085223 (2019). [CrossRef]
15. B. Wu, K. Ding, C. Chan, and Y. Chen, “Machine prediction of topological transitions in photonic crystals,” arXiv preprint arXiv:1907.07996 (2019).
16. T. Zhang, J. Wang, Q. Liu, J. Zhou, J. Dai, X. Han, Y. Zhou, and K. Xu, “Efficient spectrum prediction and inverse design for plasmonic waveguide systems based on artificial neural networks,” Photonics Res. 7(3), 368–380 (2019). [CrossRef]
17. M. H. Tahersima, K. Kojima, T. Koike-Akino, D. Jha, B. Wang, C. Lin, and K. Parsons, “Deep neural network inverse design of integrated photonic power splitters,” Sci. Rep. 9(1), 1368 (2019). [CrossRef]
18. W. Ma, F. Cheng, and Y. Liu, “Deep-learning-enabled on-demand design of chiral metamaterials,” ACS Nano 12(6), 6326–6334 (2018). [CrossRef]
19. I. Malkiel, M. Mrejen, A. Nagler, U. Arieli, L. Wolf, and H. Suchowski, “Plasmonic nanostructure design and characterization via deep learning,” Light: Sci. Appl. 7(1), 60 (2018). [CrossRef]
20. I. Sajedian, T. Badloe, and J. Rho, “Optimisation of colour generation from dielectric nanostructures using reinforcement learning,” Opt. Express 27(4), 5874–5883 (2019). [CrossRef]
21. T. Badloe, I. Kim, and J. Rho, “Biomimetic ultra-broadband perfect absorbers optimised with reinforcement learning,” Phys. Chem. Chem. Phys. 22(4), 2337–2342 (2020). [CrossRef]
22. T. Asano and S. Noda, “Optimization of photonic crystal nanocavities based on deep learning,” Opt. Express 26(25), 32704–32717 (2018). [CrossRef]
23. P. R. Wiecha and O. L. Muskens, “Deep learning meets nanophotonics: A generalized accurate predictor for near fields and far fields of arbitrary 3d nanostructures,” Nano Lett. 20(1), 329–338 (2020). [CrossRef]
24. D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114 (2013).
25. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems, (2014), pp. 2672–2680.
26. M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” arXiv preprint arXiv:1701.07875 (2017).
27. Z. Liu, D. Zhu, S. P. Rodrigues, K.-T. Lee, and W. Cai, “Generative model for the inverse design of metasurfaces,” Nano Lett. 18(10), 6570–6576 (2018). [CrossRef]
28. Z. Liu, D. Zhu, K.-T. Lee, A. S. Kim, L. Raju, and W. Cai, “Compounding meta-atoms into metamolecules with hybrid artificial intelligence techniques,” Adv. Mater. (2019).
29. W. Ma, F. Cheng, Y. Xu, Q. Wen, and Y. Liu, “Probabilistic representation and inverse design of metamaterials based on a deep generative model with semi-supervised learning strategy,” Adv. Mater. 31(35), 1901111 (2019). [CrossRef]
30. S. So and J. Rho, “Designing nanophotonic structures using conditional deep convolutional generative adversarial networks,” Nanophotonics 8(7), 1255–1261 (2019). [CrossRef]
31. J. Jiang, D. Sell, S. Hoyer, J. Hickey, J. Yang, and J. A. Fan, “Free-form diffractive metagrating design based on generative adversarial networks,” ACS Nano 13(8), 8872–8878 (2019). [CrossRef]
32. Z. A. Kudyshev, A. V. Kildishev, V. M. Shalaev, and A. Boltasseva, “Machine-learning-assisted metasurface design for high-efficiency thermal emitter optimization,” arXiv preprint arXiv:1910.12741 (2019).
33. J. Jiang and J. A. Fan, “Global optimization of dielectric metasurfaces using a physics-driven neural network,” Nano Lett. 19(8), 5366–5372 (2019). [CrossRef]
34. Z. Liu, L. Raju, D. Zhu, and W. Cai, “A hybrid strategy for the discovery and design of photonic nanostructures,” arXiv preprint arXiv:1902.02293 (2019).
35. R. S. Hegde, “Photonics inverse design: Pairing deep neural networks with evolutionary algorithms,” IEEE J. Sel. Top. Quantum Electron. 26(1), 1–8 (2020). [CrossRef]
36. F. Wyrowski and O. Bryngdahl, “Iterative fourier-transform algorithm applied to computer holography,” J. Opt. Soc. Am. A 5(7), 1058–1065 (1988). [CrossRef]
37. F. Roddier and C. Roddier, “Wavefront reconstruction using iterative fourier transforms,” Appl. Opt. 30(11), 1325–1327 (1991). [CrossRef]
38. A. Migdalas, P. M. Pardalos, and P. Värbrand, Multilevel optimization: algorithms and applications, vol. 20 (Springer Science & Business Media, 2013).
39. M. Moharam, E. B. Grann, D. A. Pommet, and T. Gaylord, “Formulation for stable and efficient implementation of the rigorous coupled-wave analysis of binary gratings,” J. Opt. Soc. Am. A 12(5), 1068–1076 (1995). [CrossRef]
40. J. Mockus, Bayesian approach to global optimization: theory and applications, vol. 37 (Springer Science & Business Media, 2012).
