Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Fourier ptychographic topography

Open Access Open Access

Abstract

Topography measurement is essential for surface characterization, semiconductor metrology, and inspection applications. To date, performing high-throughput and accurate topography remains challenging due to the trade-off between field-of-view (FOV) and spatial resolution. Here we demonstrate a novel topography technique based on the reflection-mode Fourier ptychographic microscopy, termed Fourier ptychograhpic topography (FPT). We show that FPT provides both a wide FOV and high resolution, and achieves nanoscale height reconstruction accuracy. Our FPT prototype is based on a custom-built computational microscope consisting of programmable brightfield and darkfield LED arrays. The topography reconstruction is performed by a sequential Gauss-Newton-based Fourier ptychographic phase retrieval algorithm augmented with total variation regularization. We achieve a synthetic numerical aperture (NA) of 0.84 and a diffraction-limited resolution of 750 nm, increasing the native objective NA (0.28) by 3×, across a 1.2 × 1.2 mm2 FOV. We experimentally demonstrate the FPT on a variety of reflective samples with different patterned structures. The reconstructed resolution is validated on both amplitude and phase resolution test features. The accuracy of the reconstructed surface profile is benchmarked against high-resolution optical profilometry measurements. In addition, we show that the FPT provides robust surface profile reconstructions even on complex patterns with fine features that cannot be reliably measured by the standard optical profilometer. The spatial and temporal noise of our FPT system is characterized to be 0.529 nm and 0.027 nm, respectively.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Topography measurement with a wide field-of-view (FOV), high resolution, and nanometer accuracy can provide useful information on material functionalities [1] and has wide applications in industry [2], such as defect detection, panel inspection, and surface characterization. In the past few decades, many advanced techniques [35] have been developed to perform topography. In general, they can be divided into two categories, including contact and non-contact techniques [3]. Contact techniques include stylus profilometry, scanning tunneling microscopy (STM) [6], and atomic force microscopy (AFM) [7], which use a small tip to interact with a surface and physically inspect the sample’s topography. However, high-speed inspection across a wide FOV using these techniques is challenging due to the slow scanning speed and potential damages to the sample.

Non-contact measurement techniques, such as X-ray scattering [8], scanning electron microscopy (SEM) [9], transmission electron microscopy(TEM) [10], and scatterometry [11,12] are widely used to provide accurate assessment over a broad range of resolutions and a large FOV [5]. In particular, non-contact optical inspection techniques for surface metrology [13] are appealing because of their non-destructive nature, high acquisition speed, and high accuracy. However, existing optical surface metrology systems are primarily based on the principle of optical interferometry, which is relatively bulky and sensitive to speckle noise.

Fourier ptychographic microscopy (FPM) [1416] is a computational imaging technique that enables increasing the imaging system’s space-bandwidth product (SBP) by synthesizing multiple low-resolution (LR) images into a high-resolution image across a wide FOV. The use of the partially coherent LED source makes the setup free from speckle artifacts. The ability to provide large-SBP reconstruction makes FPM a promising candidate for non-contact topography. Transmission-mode FPM systems [14,15] have been extensively developed and optimized in the past few years. To perform topography measurement, the reflection-mode geometry must be implemented. Although several reflection-mode FPM systems have been developed [1721], the accuracy of the reconstructed topography results have not been quantitatively evaluated [17,18,20,21]. In addition, a few designs suffer from limited synthetic NA [18,19].

Here, we demonstrate Fourier ptychograhpic topography (FPT) based on a new reflection-mode FPM system. Our FPT provides both a wide FOV and high resolution, and achieves nanoscale surface reconstruction accuracy. Our FPT system is based on a custom-built computational microscope consisting of programmable brightfield (BF) and darkfield (DF) LED arrays (Fig. 1(a)). The LED positions are optimized to provide efficient Fourier coverage (Fig. 1(b)). We develop an iterative algorithm to reconstruct the surface topography (Fig. 1(c)) by solving the Fourier ptychographic phase retrieval problem using the sequential Gauss-Newton method [15] with additional total variation regularization terms [22] to suppress unwanted phase artifacts. By capturing both BF and DF intensity measurements, we experimentally demonstrate a synthetic NA of 0.84 and diffraction-limited resolution of $750\;\textrm{nm}$, corresponding to a $3\times$ improvement over the native NA of the objective lens across a $1.2\times 1.2\;\textrm{mm}^2$ FOV. We demonstrate the FPT’s ability to provide wide-FOV, high resolution, and nanoscale accuracy topography reconstruction on a variety of reflective samples with different patterned structures. The accuracy of the reconstructed surface profile is benchmarked against high-resolution optical profilometry measurements. In addition, we show that the FPT provides robust surface profile reconstructions even on complex patterns with fine features that cannot be reliably measured by the high-resolution optical profilometer. The FPT system spatial and temporal noise are experimentally characterized to be $0.529\;\textrm{nm}$ and $0.027\;\textrm{nm}$, respectively. We believe this FPT technique may find many scientific and industry applications for material characterization and surface inspection.

 figure: Fig. 1.

Fig. 1. Fourier ptychographic topography. (a) Experimental setup of FPT. The BF LED illuminator is relayed to the back focal plane of the OL, where the outermost LED ring matches the OL’s NA. The DF LED illuminator is installed around the OL. (b) Example FPT measurement and reconstruction. Top left: example intensity images illuminated by a BF LED and a DF LED, respectively. Bottom left: the Fourier coverage of all the BF and DF LEDs shown in different colors, as well as the coverage from the corresponding intensity measurement indicated by the black circle. Right: FPT reconstruction for the zoom-in region.

Download Full Size | PDF

2. Method

2.1 FPT setup

Our FPT setup is shown in Fig. 1(a). The reflection-mode microscope is built on an 0.28NA, 10$\times$ objective lens (Edmund Optics, 378-803-3, Mitutoyo Plan Apo Infinity Corrected Long WD Objective) paired with a 200mm focal length tube lens (Edmund Optics, MT-1 Accessory Tube Lens). We place a brightfield (BF) LED array and relay it to the back focal plane of the objective lens (OL). The BF array consists of a central LED and two concentric rings. The radii of the two concentric rings are $4.2,$ and $8.4\;\textrm{mm}$, respectively, and the number of LEDs are $8$, and $16$, respectively. The relay system consist of a 4f system ($f_1=250\;\textrm{mm}, f_2=150\;\textrm{mm}$, Thorlabs ACT508-250-A, AC508-150-A) and a beam splitter (Thorlabs, CCM1-BS013). We select the lens pairs to project the outermost BF LED ring to the perimeter of the OL’s pupil aperture. Based on this design, the corresponding illumination NAs for the two rings are 0.14 and 0.28, respectively. We directly install a DF LED array around the OL. The array consists of three concentric rings with radii $15.74, 19.11,$ and $22.98\;\textrm{mm}$, respectively. The corresponding numbers of DF LEDs are $24, 28$, and $32$, respectively. We place the DF LED array approximately $34\;\textrm{mm}$ above the sample plane. The corresponding illumination NAs of the three LED rings are $0.42, 0.49,$ and $0.56$, respectively. The LED arrays are custom-built, including RGB LEDs (Kingbright APTF1616SEEZGQBDC, RGB LED, central wavelengths: 630$\textrm{nm}$, 515$\textrm{nm}$, 460$\textrm{nm}$) and LED drivers (Texas Instruments, TLC5955). The circuit is designed by Altium Designer 21. All the experiments reported in this work use only the red LED (central wavelength $\lambda =630\;\textrm{nm}$, bandwidth $20\;\textrm{nm}$). We place the sample on a multi-axis stage (consisting of Newport 37 series, Thorlabs NanoMax300, Thorlabs APY002) to manually fine-tune the focus. The images are capture by a camera (The Imaging Source, DMK38UX541, pixel size $2.74\;\mathrm{\mu}\textrm{m}$, $4504\times 4504$ pixels). To take the measurements, we light LEDs sequentially and capture the images. The exposure time is set at $500\;\textrm{ms}$ and the total acquisition time is around 2 minutes to capture 109 images in each experiment. We use an Arduino Teensy 3.2 to synchronize both the camera and the LED illumination. The FOV of the system is limited by the camera sensor size, and is approximately $1.2\times 1.2\;\textrm{mm}^2$. The maximum synthesized NA is $\textrm{NA}_{\textrm{illum}}+\textrm{NA}_{\textrm{obj}}=0.84$, where the $\textrm{NA}_{\textrm{illum}}$ and $\textrm{NA}_{\textrm{obj}}$ denote the OL’s NA and maximum illumination NA. The synthesized NA is thus 3$\times$ of the OL’s NA (0.28), corresponding to a diffraction-limited resolution of ${\lambda }/({\textrm{NA}_{\textrm{illum}}+\textrm{NA}_{\textrm{obj}}})={630}/{0.84}=750\;\textrm{nm}$.

2.2 Forward model of FPT

The forward model of FPT is similar to the transmission-mode FPM [15,23] based on two assumptions: (1) the incident plane wave is reflected by the highly reflective sample surface as if it incidents from the opposite direction mirrored by the sample; (2) the resulting phase shift induced by the sample is doubled by the “mirror” geometry.

The forward model (after proper normalization to account for the system magnification) that describe the intensity image from the $i^{\textrm{th}}$ LED illumination is

$$I_i(\textbf{r}) = |\mathcal{F}^{{-}1}_{[O(\textbf{k}-\textbf{k}_i)P(\textbf{k})]}(\textbf{r})|^2,$$
where $I_i(\textbf{r} )$ is the captured low-resolution intensity image for the LED $i=1,2,\ldots,N_{\textrm{LED}}$ with $N_{\textrm{LED}}$ representing the total number of LEDs. Example experimentally captured BF and DF intensity images are shown in Fig. 1(b). $|\cdot |$ takes the amplitude of the complex field, and $\textbf{r} =(x, y)$ denotes the lateral coordinates and $\mathcal {F}^{-1}$ is the inverse Fourier transform. $O(\textbf{k} )$ is the spectrum of reflected sample field $o(\textbf{r} )$, $\textbf{k} =(k_{x},k_{y})$ represents the spatial frequency, and $\textbf{k} _i=(k_{xi},k_{yi})=({\sin \theta _{xi}}/{\lambda }, {\sin \theta _{yi}}/{\lambda })$ is the spatial frequency of reflected plane wave, with $(\theta _{xi}, \theta _{yi})$ based on the illumination angle of the $i^{\textrm{th}}$ LED. The corresponding Fourier coverage from each LED determined by $O(\textbf{k }-\;\textbf{k }_i)P(\textbf{k} )$, as shown in the bottom left figure in Fig. 1(b) and $P(\textbf{k} )$ is the pupil function of the OL.

2.3 Inverse problem of FPT

2.3.1 Problem formulation

The inverse problem is formulated as a non-convex optimization problem that jointly estimates the sample’s spatial spectrum $O(\textbf{k} )$ and the pupil function $P(\textbf{k} )$ [15,22,23]:

$$\underset{O(\textbf{k}),P(\textbf{k})} {\operatorname{min}}\{\mathcal{D}\{O(\textbf{k}),P(\textbf{k})\}+\alpha_1\mathcal{R}_1\{\mathcal{F}^{{-}1}_{[O(\textbf{k})]}\}+\alpha_2\mathcal{R}_2\{\mathcal{F}^{{-}1}_{[O(\textbf{k})]}\}\},$$
where the $\mathcal {D}\{O(\textbf{k} ),P(\textbf{k} )\}$ is the data fidelity term, and $\mathcal {R}_1\{\mathcal {F}^{-1}_{[O(\textbf{k} )]}\}, \mathcal {R}_2\{\mathcal {F}^{-1}_{[O(\textbf{k} )]}\}$ are the regularization terms with $\alpha _1, \alpha _2$ controlling the strengths of regularization. $\mathcal {F}^{-1}_{[O(\textbf{k} )]}$ is the inverse Fourier transform of the spectrum $O(\textbf{k} )$ and is $o(\textbf{r} )$.

The data fidelity term is

$$\mathcal{D}\{O(\textbf{k}),P(\textbf{k})\}={\sum_{i=1}^{N_{\textrm{LED}}}\big\|\sqrt{I_i(\textbf{r})}-|\mathcal{F}^{{-}1}_{[O({\textbf{k}}-{\textbf{k}_i})P(\textbf{k})]}(\textbf{r})|\big\|^2_2},$$
where $\|\cdot \|_2$ is the $L_2$ norm.

The regularization term is

$$\alpha_1\mathcal{R}_1\{\mathcal{F}^{{-}1}_{[O(\textbf{k})]}\}+\alpha_2\mathcal{R}_2\{\mathcal{F}^{{-}1}_{[O(\textbf{k})]}\}=\alpha_1\||o(\textbf{r})|\|_{\textrm{TV}}+\alpha_2\|\arg(o(\textbf{r}))\|_{\textrm{TV}},$$
where the $\|\cdot \|_{\textrm{TV}}$ denotes the isotropic total variation (TV) operator [24]. The two regularization terms enforce smoothness constraints on both the amplitude and phase to suppress the high-frequency noise in the reconstruction [19,22].

The sample’s surface height $h(\textbf{r} )$ is calculated from the reconstructed high-resolution phase image $\arg (o(\textbf{r} ))$ by the following expression

$$h(\textbf{r})=\frac{\lambda}{4\pi}\arg(o(\textbf{r})).$$

2.3.2 Reconstruction algorithm

We develop an iterative algorithm to optimize Eq. (2) and estimate the sample’s surface topography. The flowchart of the reconstruction algorithm is shown in Fig. 2. First, we pre-process the raw intensity images by subtracting the background offset, normalizing the intensity, and calibrating the illumination angles. We subtract the estimated offset and set negative values from the subtraction to be 0. The background offset comes from dark currents of the CMOS camera and spurious reflections from the optical system. The offset for a given LED illumination is estimated by capturing an “dark” image without placing any sample. Next, we normalize each low-resolution image with the experimentally calibrated incident light intensity, which is measured by a power meter (Thorlabs PM100D). We put the power meter at the sample position to measure the illumination intensity from each LED for the calibration. The pre-processed image is used as the $I_i$ in Eq. (1). Finally, the BF illumination angle is calibrated using the algorithm in [25].

 figure: Fig. 2.

Fig. 2. Reconstruction algorithm flowchart of FPT. HR: high-resolution, LR: low-resolution.

Download Full Size | PDF

We start the iterative reconstruction algorithm by an initial estimate of $O(\textbf{k} )$ as the Fourier transform of the estimated object that is reconstructed using the quantitative differential phase contrast (DPC) method [26,27] and setting $P(\textbf{k} )$ to be a binary circle whose radius is set by the OL’s NA. Next, we solve the optimization by following [19,22]. In addition, we also optimize the pupil function for aberration correction. Our method is similar to the proximal splitting method [28,29].

The basic structure of each iteration is to first perform the sequential Gauss-Newton’s update [15,23] for the data fidelity term, followed by the proximal update for the TV regularization [22]. The sequential Gauss-Newton’s method updates the estimate of the object spectrum and pupil function incrementally from the images $i=1$ to $N_{\textrm{LED}}$. This process iterates until the stopping criteria are met. In the $l^\textrm{th}$ incremental update, $l= 1, 2,\ldots, N_{\textrm{LED}}$, we estimate the gradient of the data fidelity term by the following steps:

(1) We introduce an auxiliary function $\Psi _i^l(\textbf{k} )$ to represent the estimated field immediately after the pupil from the $i^\textrm{th}$ LED:

$$\Psi_i^l(\textbf{k})=O^l(\textbf{k}-\textbf{k}_i)P^l(\textbf{k}).$$
next, we impose an intensity constraint using the pre-processed intensity image $I_i$ and update the auxiliary function. We compute the real space function of the auxiliary function
$$\psi^l_i(\textbf{r})=\mathcal{F}^{{-}1}_{[\Psi^{l}_{i}(\textbf{k})]}(\textbf{r}).$$
and rescale the real space auxiliary function by an amplitude factor and return the rescaled auxiliary function back to the Fourier space:
$$\Psi_i^l(\textbf{k})'=\mathcal{F}_{[\psi^{l}_{i}(\textbf{r})']}(\textbf{k}), \textrm{with}\ \psi^{l}_{i}(\textbf{r})'=\sqrt{I_i(\textbf{r})}\frac{\psi^{l}_{i}(\textbf{r})}{|\psi^{l}_{i}(\textbf{r})|},$$
where $\mathcal {F}$ is the Fourier transform.

(2) We calculate the updated sample spectrum $O^{l+1}(\textbf{k} )$ and pupil function $P^{l+1}(\textbf{k} )$:

$$O^{l+1}(\textbf{k}) = O^{l}(\textbf{k}) + \frac{|P^{l}(\textbf{k}+\textbf{k}_i)|[P^{l}(\textbf{k}+\textbf{k}_i)]^{*}[\Psi_i^l(\textbf{k}+\textbf{k}_i)'-O^{l}(\textbf{k})P^{l}(\textbf{k}+\textbf{k}_i)]}{|P^{l}(\textbf{k})|_{\textrm{max}}(|P^{l}(\textbf{k}+\textbf{k}_i)|^2+\delta_1)},$$
$$P^{l+1}(\textbf{k}) = P^{l}(\textbf{k}) + \frac{|O^{l}(\textbf{k}-\textbf{k}_i)|[O^{l}(\textbf{k}-\textbf{k}_i)]^{*}[\Psi_i^l(\textbf{k})'-O^{l}(\textbf{k}-\textbf{k}_i)P^{l}(\textbf{k})]}{|O^{l}(\textbf{k})|_{\textrm{max}}(|O^{l}(\textbf{k}-\textbf{k}_i)|^2+\delta_2)},$$
where $\delta _1$ and $\delta _2$ are regularization constants to ensure numerical stability.

We repeat the above two steps and update the object spectrum and the pupil function using all the low-resolution images to obtain the updated spectrum $\hat {O}(\textbf{k} ),\hat {P}(\textbf{k} )$. Next, we transfer the updated spectrum to real space by taking the inverse Fourier transform: $\hat {o}(\textbf{r} )=\mathcal {F}^{-1}_{[\hat {O}(\textbf{k} )]}(\textbf{r} )$ and compute the proximal mapping of the TV regularization for the amplitude $|\hat {o}(\textbf{r} )|$ and the phase $\arg (\hat {o}(\textbf{r} ))$ by following [24,29]:

$$|o(\textbf{r})|=\mathop{\arg \min}_{\textbf{z}}{\big\||\hat{o}(\textbf{r})|-\textbf{z}\big\|_2^2}+\alpha_1\|\textbf{z}\|_\textrm{TV}$$
$$\arg(o(\textbf{r}))=\mathop{\arg \min}_{\textbf{z}}{\big\|\arg(\hat{o}(\textbf{r}))-\textbf{z}\big\|_2^2}+\alpha_2\|\textbf{z}\|_\textrm{TV}$$

The above iterative process is repeated until the convergence criterion or the maximum number of updates ($L_{\textrm{max}}$) is reached. Our convergence criterion is when the difference in the cost function between two consecutive iterations falls below a tolerance ($\varepsilon$). The final high-resolution (HR) reconstruction of the sample topography is calculated by first unwrapping the reconstructed phase using the algorithm in [30,31] and then transfer the phase to height using Eq. (5). Finally, the reconstructed height map is post-processed by subtracting the slow-changing background estimated by the morphological open operator.

In our experiment, we first set $\alpha _1,\alpha _2, \delta _1, \delta _2, \varepsilon, L_{\textrm{max}}$ to be 0.01, 0.01, 1, 10, $1\times 10^{-4}$, 50 respectively, and then manually fine-tune each parameter for different samples depending on the reconstruction result and computation time. Additional discussions on the effects of TV regularization and post-processing are in section 4.

3. Result

First, we image an amplitude-type high-resolution USAF test target (Ready Optics, 2015a USAF), as shown in Fig. 3. A full-FOV ($1.2\times 1.2\;\textrm{mm}^2$) low-resolution image under a BF LED illumination is shown in Fig. 3(a), with the magnified view in Fig. 3(b). A DF measurement for the same region is shown in Fig. 3(c). To demonstrate the performance of our FPT technique, we show two reconstruction results. First, our FPT reconstruction with only all the BF LEDs is shown in Fig. 3(d), which clearly resolves the features in Group 9 Element 5 ($1230\;\textrm{nm}$ period; $812.7$ lines per millimeter(lpm)) and matches well to the theoretical diffraction-limited resolution ($1125\;\textrm{nm}$ period) of a 0.56 (0.28+0.28) NA system. Next, we show further improved resolution by including the DF LEDs in the reconstruction. The FPT reconstruction using all the BF and DF LEDs is shown in Fig. 3(e), which clearly resolves the features in Group 10, Element 3 ($776\;\textrm{nm}$ period, $1290$lpm) and matches well to the diffraction-limited resolution ($750\;\textrm{nm}$ period) of a 0.84 ($0.28+0.56$) synthetic NA system. The contrast transfer function (CTF) from our reconstruction is further shown in Fig. S1 and additional discussion about the resolving power of our FPT system can be found in the Supplement.

 figure: Fig. 3.

Fig. 3. Reconstruction of amplitude-type resolution target. (a) Captured low-resolution full-FOV image of the target. Magnified image of the region containing the finest structures with (b) a BF and (c) a DF LED illumination. (d) The amplitude reconstruction with all the BF LEDs. The reconstructed region around the Element 5 in Group 9 is shown in the zoomed-in green box. (e) The amplitude reconstruction with all the BF and DF LEDs. The reconstructed region around the Element 3 in Group 10 is shown in the zoomed-in green box.

Download Full Size | PDF

Next, we image a phase-type sample (manufactured by Samsung) to quantify the performance of topography reconstruction by our FPT. The sample is made by etching $100\;\textrm{nm}$ height features with different sizes on the silicon wafer and then coat the wafer with a thin layer of ${{\textrm{SiO}}_2}$. In Fig. 4, the first and second groups contain equally spaced bars with line width (= spacing) of $1\;\mathrm{\mu}\textrm{m}$ and $480\;\textrm{nm}$, respectively. The reconstructed surface topology of the sample using our FPT is shown in Fig. 4(a–c). We also image the sample with an optical profilometer (Zygo NewView 9000, OL: 100$\times$, 0.85 NA) as shown in Fig. 4(d–f). By comparing the results from these two independent measurements, we conclude that our FPT can get an accurate height map of the sample and achieves a spatial resolution of $960\;\textrm{nm}$. As shown in the horizontal and vertical cutlines, FPT reconstructed height is close to that measured by the optical profilometer. Due to residual artifacts from our background removal method, the edges of the reconstructed map is slightly lower than the “ground truth”, which is further discussed in the section 4. Overall, this result shows that our FPT technique can provide accurate 3D topography reconstruction with high spatial resolution.

 figure: Fig. 4.

Fig. 4. Topography reconstruction of a high-resolution phase-type sample. (a) Reconstructed height map. (b,c) Height profiles from the magenta and red dash lines in (a), respectively. (d) Measurement from a high-resolution optical profilometer. (e,f) Height profiles from the magenta and red dash lines in (d), respectively.

Download Full Size | PDF

Next, we test our FPT system on three additional phase-type samples containing different patterns (manufactured by the same process by Samsung) in Fig. 5. The intensity images captured under the on-axis BF LED are shown in the first column. The reconstructed height maps are shown in the second column. The 1D height profiles (from the dashed red lines in the 2D maps) are shown in the third column. In Fig. 5(a), the sample contains “negative” patterns, in which the bars are etched into the substrate by $100\;\textrm{nm}$ deep. For the regions on the right containing equally spaced lines, the etched line width is $1 \;\mathrm{\mu}\textrm{m}$ and the pitch is $4 \;\mathrm{\mu}\textrm{m}$. For comparison, the reconstruction from a “positive” pattern sample is shown in Fig. 5(b). The sample contains bars that are $100\;\textrm{nm}$ tall above the substrate. For the regions on the right containing equally spaced lines, the lines are $1 \;\mathrm{\mu}\textrm{m}$ wide and the pitch is $4 \;\mathrm{\mu}\textrm{m}$. As shown in reconstructions of Fig. 5(a) and (b), the reconstructed pitch and height of both samples closely matches with the ground truth values. Finally, we show the result from a sample containing concentric rectangles with thin lines in Fig. 5(c). Our FPT system successfully reconstructs the line structures with the correct $100\;\textrm{nm}$ height. Notably, all of the three samples cannot be reliable measured by the high-resolution optical profilometer, as shown in Supplementary Fig. S2. We attribute the failure of interferometry-based optical profilometer measurements to the fine features that result in complex interference artifacts. In contrast, our FPT does not suffer from these artifacts by exploiting multiple measurements under different illumination angles and solving a phase retrieval algorithm. These results highlight that our FPT can robustly characterize samples containing different types of surface topographies.

 figure: Fig. 5.

Fig. 5. Phase-type samples with different patterns. First column: intensity images illuminated by the on-axis BF LED. Second column: the reconstructed height map. Third column: the height profiles from the dashed red lines in the corresponding height map. The ground-truth reference heights are denoted by the dashed magenta lines.

Download Full Size | PDF

4. Discussion

In this section, we analyze the noise of our FPT system, the effectiveness of the TV regularization, and our post-processing method.

We characterize the system noise by imaging a flat sample region and follow the procedure in [32], as shown in Fig. 6(a). First, we quantify the spatial noise by taking the standard deviation (std) of the reconstructed height map in the left image in Fig. 6(a). The histogram of the reconstructed height is shown as the inset. Next, we quantify the temporal noise by assessing the temporal std of the reconstructions from 30 sets of repeated measurements, as shown in the right image in Fig. 6(a). The amount of temporal noise is taken as the spatial average of the temporal std map. The histogram of the temporal std is shown as the inset. Based on the procedures, we find that the spatial and temporal noise are $0.529\;\textrm{nm}$ and $0.027\;\textrm{nm}$, respectively.

 figure: Fig. 6.

Fig. 6. (a) Characterization of our FPT system noise. Left: spatial noise; inset: histogram of the reconstructed height. Right: temporal noise; inset: histogram of the temporal std of the reconstructions from 30 sets of repeated measurements. (b) Effects of our post-processing method and the TV regularization. Left: the reconstructed height map with post-processing but without TV regularization; Right: the reconstructed height profiles with three different methods.

Download Full Size | PDF

We show the effectiveness of our post-processing method and the TV regularization in our algorithm in Fig. 6(b). Here, we reconstruct the phase-type sample in Fig. 5(a) using three different methods, including without post-processing nor TV regularization (“raw reconstruction”), with post-processing without TV regularization, and with both post-processing (fine-tuned) and TV regularization. The reconstructed sample height map with post-processing but without TV regularization is shown in the left image in Fig. 6(b). The height profiles from the dashed red line using the three methods are shown in the right figure in Fig. 6(b). The black line represents the reconstructed height profile from the raw reconstruction. A slowly varying background is observed in the reconstruction, although the relative height change of the line patterns appears correct. After the post-processing, the slowly varying background is removed as shown in the red line. This post-processing method may affect the reconstruction accuracy at the edges of the sample as shown in Fig. 6(b). The rightmost bar should be approximately $100\;\textrm{nm}$ tall, which is the case in the raw reconstruction; however, after the post-processing, the height is reduced to $50\;\textrm{nm}$. We attribute this artifact to the open operator used in our post-processing algorithm, which suffers from a trade-off between the extent of background removal and feature fidelity as controlled by the size of the erosion mask. In our experiment, the disk size is set to be 10 pixels and manually fine-tuned for different samples. For the sample shown in Fig. 6(b), we can fine tine the erosion mask size to get better result as shown in the blue line in Fig. 6(b). The rightmost bar is correctly post-processed.

Although our post-processing method can remove the slowly varying background, high-frequency noisy artifacts still exist in the reconstructed features, as shown in the red line in yellow inset of Fig. 6(b). To alleviate this issue, we add TV regularization in our reconstruction algorithm. The noise suppression by this method is shown in the blue line in yellow inset of Fig. 6(b).

5. Conclusion

We presented a FPT system to enable high-resolution topography reconstruction across a wide FOV. We achieved $750\;\textrm{nm}$ resolution across a $1.2\times 1.2\;\textrm{mm}^2$ FOV. We demonstrated FPT on both amplitude- and phase-type samples containing different surface structures. In addition, we show that the FPT provides robust surface profile reconstructions even on complex patterns with fine features that cannot be reliably measured by the widely used high-resolution optical profilometry system. The spatial and temporal system noise are quantified to be $0.529\;\textrm{nm}$ and $0.027\;\textrm{nm}$, respectively.

There are still some challenges that need to be overcome in the future. First, accurate angle calibration of the DF LED is difficult, which may influence the accuracy of the reconstruction; Second, spatially varying aberration estimation is needed to achieve high-quality reconstruction across the full FOV. Despite these challenges, we have demonstrated topography reconstruction with high-resolution, wide-FOV, and high accuracy using our FPT prototype. We envision that our technique can find many applications in scientific and industry applications, such as surface characterization, metrology, panel inspection, and defect detection.

Funding

Samsung Global Research Outreach (GRO) program.

Disclosures

H.W., J.Z., J.S., S.P., W.K., M.L. and L.T. submit a patent application related to this work (Korean patent application code: 5-2004-036531-2). J.S., S.P., W.K., M.L.: Samsung (F, E, P)

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Supplemental document

See Supplement 1 for supporting content.

References

1. H. Assender, V. Bliznyuk, and K. Porfyrakis, “How surface topography relates to materials’ properties,” Science 297(5583), 973–976 (2002). [CrossRef]  

2. D. J. Whitehouse, Handbook of surface metrology (CRC Press, 1994).

3. C. Y. Poon and B. Bhushan, “Comparison of surface roughness measurements by stylus profiler, afm and non-contact optical profiler,” Wear 190(1), 76–88 (1995). [CrossRef]  

4. D. J. Whitehouse, “Surface metrology,” Meas. Sci. Technol. 8(9), 955–972 (1997). [CrossRef]  

5. N. G. Orji, M. Badaroglu, B. M. Barnes, C. Beitia, B. D. Bunday, U. Celano, R. J. Kline, M. Neisser, Y. Obeng, and A. Vladar, “Metrology for the next generation of semiconductor devices,” Nat. Electron. 1(10), 532–547 (2018). [CrossRef]  

6. G. Binnig, H. Rohrer, C. Gerber, and E. Weibel, “Surface studies by scanning tunneling microscopy,” Phys. Rev. Lett. 49(1), 57–61 (1982). [CrossRef]  

7. G. Binnig, C. F. Quate, and C. Gerber, “Atomic force microscope,” Phys. Rev. Lett. 56(9), 930–933 (1986). [CrossRef]  

8. K. Gaffney and H. N. Chapman, “Imaging atomic structure and dynamics with ultrafast x-ray scattering,” Science 316(5830), 1444–1448 (2007). [CrossRef]  

9. J. I. Goldstein, D. E. Newbury, J. R. Michael, N. W. Ritchie, J. H. J. Scott, and D. C. Joy, Scanning electron microscopy and X-ray microanalysis (Springer, 2017).

10. D. B. Williams and C. B. Carter, “The transmission electron microscope,” in Transmission electron microscopy, (Springer, 1996), pp. 3–17.

11. S. Kwon, J. Park, K. Kim, Y. Cho, and M. Lee, “Microsphere-assisted, nanospot, non-destructive metrology for semiconductor devices,” Light: Sci. Appl. 11(1), 32 (2022). [CrossRef]  

12. C. Yoon, G. Park, D. Han, S.-i. Im, S. Jo, J. Kim, W. Kim, C. Choi, and M. Lee, “Toward realization of high-throughput hyperspectral imaging technique for semiconductor device metrology,” J. Micro/Nanopatterning, Materials, Metrol. 21(02), 021209 (2022). [CrossRef]  

13. A. G. Marrugo, F. Gao, and S. Zhang, “State-of-the-art active optical techniques for three-dimensional surface metrology: a review,” J. Opt. Soc. Am. A 37(9), B60–B77 (2020). [CrossRef]  

14. G. Zheng, R. Horstmeyer, and C. Yang, “Wide-field, high-resolution fourier ptychographic microscopy,” Nat. Photonics 7(9), 739–745 (2013). [CrossRef]  

15. L. Tian, X. Li, K. Ramchandran, and L. Waller, “Multiplexed coded illumination for fourier ptychography with an led array microscope,” Biomed. Opt. Express 5(7), 2376–2389 (2014). [CrossRef]  

16. G. Zheng, C. Shen, S. Jiang, P. Song, and C. Yang, “Concept, implementations and applications of fourier ptychography,” Nat. Rev. Phys. 3(3), 207–223 (2021). [CrossRef]  

17. K. Guo, S. Dong, and G. Zheng, “Fourier ptychography for brightfield, phase, darkfield, reflective, multi-slice, and fluorescence imaging,” IEEE J. Sel. Top. Quantum Electron. 22(4), 77–88 (2016). [CrossRef]  

18. S. Pacheco, G. Zheng, and R. Liang, “Reflective fourier ptychography,” J. Biomed. Opt. 21(2), 026010 (2016). [CrossRef]  

19. G.-J. Choi, J. Lim, S. Jeon, J. Cho, G. Lim, N.-C. Park, and Y.-P. Park, “Dual-wavelength fourier ptychography using a single led,” Opt. Lett. 43(15), 3526–3529 (2018). [CrossRef]  

20. H. Lee, B. H. Chon, and H. K. Ahn, “Reflective fourier ptychographic microscopy using a parabolic mirror,” Opt. Express 27(23), 34382–34391 (2019). [CrossRef]  

21. H. K. Ahn and B. H. Chon, “Reflective fourier ptychographic microscopy using segmented mirrors and a mask,” Curr. Opt. Photonics 5(1), 40–44 (2021).

22. D. Ren, E. Bostan, L.-H. Yeh, and L. Waller, “Total-variation regularized fourier ptychographic microscopy with multiplexed coded illumination,” in Imaging and Applied Optics 2017 (3D, AIO, COSI, IS, MATH, pcAOP), (Optica Publishing Group, 2017), p. MM3C.5.

23. L.-H. Yeh, J. Dong, J. Zhong, L. Tian, M. Chen, G. Tang, M. Soltanolkotabi, and L. Waller, “Experimental robustness of fourier ptychography phase retrieval algorithms,” Opt. Express 23(26), 33214–33240 (2015). [CrossRef]  

24. A. Beck and M. Teboulle, “Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems,” IEEE Trans. on Image Process. 18(11), 2419–2434 (2009). [CrossRef]  

25. R. Eckert, Z. F. Phillips, and L. Waller, “Efficient illumination angle self-calibration in fourier ptychography,” Appl. Opt. 57(19), 5434–5442 (2018). [CrossRef]  

26. L. Tian and L. Waller, “Quantitative differential phase contrast imaging in an led array microscope,” Opt. Express 23(9), 11394–11403 (2015). [CrossRef]  

27. J. Chung, H. Lu, X. Ou, H. Zhou, and C. Yang, “Wide-field fourier ptychographic microscopy using laser illumination source,” Biomed. Opt. Express 7(11), 4787–4802 (2016). [CrossRef]  

28. P. L. Combettes and J.-C. Pesquet, “Proximal splitting methods in signal processing,” in Fixed-point algorithms for inverse problems in science and engineering, (Springer, 2011), pp. 185–212.

29. N. Perraudin, D. Shuman, G. Puy, and P. Vandergheynst, “UNLocBoX A matlab convex optimization toolbox using proximal splitting methods,” ArXiv e-prints arXiv:1402.0779 (2014). [CrossRef]  

30. D. C. Ghiglia and L. A. Romero, “Robust two-dimensional weighted and unweighted phase unwrapping that uses fast transforms and iterative methods,” J. Opt. Soc. Am. A 11(1), 107–117 (1994). [CrossRef]  

31. Y. Xue, S. Cheng, Y. Li, and L. Tian, “Reliable deep-learning-based phase imaging with uncertainty quantification,” Optica 6(5), 618–629 (2019). [CrossRef]  

32. C. Lee, Y. Baek, H. Hugonnet, and Y. Park, “Single-shot wide-field topography measurement using spectrally multiplexed reflection intensity holography via space-domain kramers–kronig relations,” Opt. Lett. 47(5), 1025–1028 (2022). [CrossRef]  

Supplementary Material (1)

NameDescription
Supplement 1       supplement

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1. Fourier ptychographic topography. (a) Experimental setup of FPT. The BF LED illuminator is relayed to the back focal plane of the OL, where the outermost LED ring matches the OL’s NA. The DF LED illuminator is installed around the OL. (b) Example FPT measurement and reconstruction. Top left: example intensity images illuminated by a BF LED and a DF LED, respectively. Bottom left: the Fourier coverage of all the BF and DF LEDs shown in different colors, as well as the coverage from the corresponding intensity measurement indicated by the black circle. Right: FPT reconstruction for the zoom-in region.
Fig. 2.
Fig. 2. Reconstruction algorithm flowchart of FPT. HR: high-resolution, LR: low-resolution.
Fig. 3.
Fig. 3. Reconstruction of amplitude-type resolution target. (a) Captured low-resolution full-FOV image of the target. Magnified image of the region containing the finest structures with (b) a BF and (c) a DF LED illumination. (d) The amplitude reconstruction with all the BF LEDs. The reconstructed region around the Element 5 in Group 9 is shown in the zoomed-in green box. (e) The amplitude reconstruction with all the BF and DF LEDs. The reconstructed region around the Element 3 in Group 10 is shown in the zoomed-in green box.
Fig. 4.
Fig. 4. Topography reconstruction of a high-resolution phase-type sample. (a) Reconstructed height map. (b,c) Height profiles from the magenta and red dash lines in (a), respectively. (d) Measurement from a high-resolution optical profilometer. (e,f) Height profiles from the magenta and red dash lines in (d), respectively.
Fig. 5.
Fig. 5. Phase-type samples with different patterns. First column: intensity images illuminated by the on-axis BF LED. Second column: the reconstructed height map. Third column: the height profiles from the dashed red lines in the corresponding height map. The ground-truth reference heights are denoted by the dashed magenta lines.
Fig. 6.
Fig. 6. (a) Characterization of our FPT system noise. Left: spatial noise; inset: histogram of the reconstructed height. Right: temporal noise; inset: histogram of the temporal std of the reconstructions from 30 sets of repeated measurements. (b) Effects of our post-processing method and the TV regularization. Left: the reconstructed height map with post-processing but without TV regularization; Right: the reconstructed height profiles with three different methods.

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

I i ( r ) = | F [ O ( k k i ) P ( k ) ] 1 ( r ) | 2 ,
min O ( k ) , P ( k ) { D { O ( k ) , P ( k ) } + α 1 R 1 { F [ O ( k ) ] 1 } + α 2 R 2 { F [ O ( k ) ] 1 } } ,
D { O ( k ) , P ( k ) } = i = 1 N LED I i ( r ) | F [ O ( k k i ) P ( k ) ] 1 ( r ) | 2 2 ,
α 1 R 1 { F [ O ( k ) ] 1 } + α 2 R 2 { F [ O ( k ) ] 1 } = α 1 | o ( r ) | TV + α 2 arg ( o ( r ) ) TV ,
h ( r ) = λ 4 π arg ( o ( r ) ) .
Ψ i l ( k ) = O l ( k k i ) P l ( k ) .
ψ i l ( r ) = F [ Ψ i l ( k ) ] 1 ( r ) .
Ψ i l ( k ) = F [ ψ i l ( r ) ] ( k ) , with   ψ i l ( r ) = I i ( r ) ψ i l ( r ) | ψ i l ( r ) | ,
O l + 1 ( k ) = O l ( k ) + | P l ( k + k i ) | [ P l ( k + k i ) ] [ Ψ i l ( k + k i ) O l ( k ) P l ( k + k i ) ] | P l ( k ) | max ( | P l ( k + k i ) | 2 + δ 1 ) ,
P l + 1 ( k ) = P l ( k ) + | O l ( k k i ) | [ O l ( k k i ) ] [ Ψ i l ( k ) O l ( k k i ) P l ( k ) ] | O l ( k ) | max ( | O l ( k k i ) | 2 + δ 2 ) ,
| o ( r ) | = arg min z | o ^ ( r ) | z 2 2 + α 1 z TV
arg ( o ( r ) ) = arg min z arg ( o ^ ( r ) ) z 2 2 + α 2 z TV
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.