Three-dimensional space optimization for near-field ptychography

An Pan; Baoli Yao

doi:10.1364/OE.27.005433

1. Introduction

Ptychography is a coherent diffractive imaging technique that produces wide field-of-view (FOV), quantitative phase images at a resolution limited only by the angular extent of the detector [1–4]. Different from traditional optical imaging techniques, there are no lenses for imaging in the optical setup. By recording overlapped diffraction patterns, the overlapping regions would result in abundant data redundancy, which can be used to recover the complex-valued information of object with the popular extended ptychographic iterative engine (ePIE) algorithm [5,6], to reconstruct the three-dimensional (3D) thick samples with the well-known 3ePIE approach [7–9], to decompose the coherent states with the latest mixed state algorithm [10–12], and to calibrate the system errors by advanced optimization algorithms [13–17]. This lensless method has been successfully demonstrated in the visible light [18,19] and electron regimes [20,21], and has earned considerable popularity in the x-ray community [22,23], where high-quality, high-resolution optics are challenging to manufacture.

Compared with the ePIE algorithm, 3ePIE approach solves the multiple scattering problems of thick samples, which is crucial to biomedical imaging and in situ studies [7]. By splitting the object into axial sections, the complex transmission function of each slice can be recovered with the identical data of conventional 2D ptychography. Different from tomography [23–26], no angle scanning or rotation and multiple measurements at each angle, or a reference beam are required. However, the specific thickness of a 3D sample is not known, while a rough thickness may be known. So it is not clear how many slices need to be divided into. Second, the distance between each layer is also unclear, while these two parameters are related and cannot be treated separately, and are sensitive and crucial to the reconstructions. The traditional method is to separate the sample with the same interval, for example, according to the depth of field (DOF), and to test the number of sections for the best reconstructions, for example, with the minimization of the error matrix [8]. This method is reasonable for the continuous samples and has achieved great results [7–9]. But this kind of segmentation approach may not be scientific enough for those discrete samples with an uneven spatial distribution, such as the microbeads solution, blood, etc. The two inaccurate parameters may yield the algorithm to diverge or generate artifacts and the empty slices may reduce the reconstruction quality. Besides, repeatedly testing the number of slices is a tedious work even for a continuous sample. Though the gradient-based methods [24] can retrieve the intervals of discrete objects, the number of layers is fixed. Herein, we modified the 3ePIE approach with the genetic algorithm, termed GA-3ePIE, to retrieve both parameters and applied it in the near-field ptychography. Since near-field ptychography provides advantages over far-field ptychography that large FOV can be imaged with fewer diffraction images, and with weaker requirements on the detector dynamic range and beam coherence [27–30]. Additionally, the Fresnel numbers are significantly higher than in those far-field ptychography [30], which is brought into focus recently. The validity and practicability of our method are verified by both simulations and experiments. The maximum number of sections that can be resolved is also investigated in numerical analysis, which is associated with the sampling and overlap rate in spatial domain. GA-3ePIE algorithm can be also promoted to image thick samples with coherent X-rays and in the electron regime. And our method can be also used in other computational imaging techniques such as Fourier ptychography [31–33]. The limitations of our method are discussed.

2. Methods

Figure 1 shows the schematic of 3D near-field ptychography with thick and discrete specimen of uneven spatial distribution. In order to provide a basis for the following discussion, we introduce the 3ePIE algorithm [7] first.

Fig. 1 Schematic of 3D near-field ptychography with thick specimen of uneven spatial distribution.

Download Full Size | PDF

Step 1. By splitting object into axial sections, each section is described by a complex transmission function O_s(r), s = 1, 2, …, N, where N is the number of sections.

Step 2. The first section is illuminated by a patch of radiation (termed probe), described by the 2D stationary complex function P(r), where r(x, y) is a Cartesian coordinate. The probe is shifted by a positional array R_c = (R_x,c, R_y,c) and c = 1,2,…,w, where w is the number of captured patterns. Then the exit wave is

E_{e, 1} (r) = P (r - R_{c}) \cdot O_{1} (r)

This wavefront is then propagated to, and becomes the incident wavefront for the second section, creating a subsequent incident wave:

E_{i, 2} (r) = ρ_{d_{1}} [E_{e, 1} (r)]

where

ρ_{d_{1}}

is an angular spectrum propagator over distance d₁ and the pixel size is invariant. And then the exit wave is:

E_{e, 2} (r) = E_{i, 2} (r) \cdot O_{2} (r)

Step 3. Continue this multiply-propagate process through subsequent slices until the exit-wave for the final slice.

\begin{array}{l} E_{i, N} (r) = ρ_{d_{N - 1}} [E_{e, N - 1} (r)] \\ E_{e, N} (r) = E_{i, N} (r) \cdot O_{N} (r) \end{array}

Propagate this wavefront to the detector to give E_c(u), which can use our previous adaptive-window angular spectrum (AWAS) algorithm [34] for arbitrary propagation distance and variable sampling windows.

E_{c} (u) = ρ_{d_{N}} [E_{e, N} (r)]

Replace the modulus of E_c(u) according to the measured diffraction pattern I_c(u) to give

E_{c}^{'} (u) = \sqrt{I_{c} (u)} \frac{E_{c} (u)}{| E_{c} (u) |}

Step 4. Back-propagate the corrected wavefront to the plane of the N_th object slice with our AWAS algorithm as well, given a revised estimate of $E_{e, N}^{'} (r)$ . Update each slice of the current object estimate and the incident wavefront.

\begin{array}{l} O_{N}^{'} = O_{N} + α \frac{E_{i, N}^{*}}{{| E_{i, N} |}_{\max}^{2}} (E_{e, N}^{'} - E_{e, N}) \\ E_{i, N}^{'} = E_{i, N} + β \frac{O_{N}^{*}}{{| O_{N} |}_{\max}^{2}} (E_{e, N}^{'} - E_{e, N}) \end{array}

where α and β are the step size of algorithm, which are usually set to 1. Repeat Steps 1-4 until all the recorded diffraction patterns have been addressed, which completes a single iteration of the algorithm. The convergence is monitored via a normalized sum of the square deviations

S S E = \sum_{c, u} \frac{{| | E_{c} (u) | - \sqrt{I_{c} (u)} |}^{2}}{I_{c} (u)}

If the SSE is small enough, we can deem that it reaches convergence and acquires very high image quality. Analyzing this 3ePIE algorithm, there are two sensitive parameters, the number of sections N and the distance between each slice d_s. Besides, the distance from the last slice to the detector d_N can be measured but sometimes it is not accurate adequately. To this end, we build our cost function as follows.

S S E = \min f (N, d_{1}, d_{2}, ..., d_{N})

where f is the 3ePIE algorithm. Considering that the optimization of spacing and the number of slices are discrete and nonlinear, we use the concept of genetic algorithm (termed GA-3ePIE approach) to solve this problem including the following steps.

Step 1. Initialization. Randomly generate multiple sets of individuals (solutions) and calculate their cost function respectively. An individual is a combination of N + 1 chromosomes (input). Both in our simulations and experiments 40 individuals are randomly generated. For the sake of quick search, the j_th individual termed a_j, and the v_th input termed a_jv is restricted from a certain bound [a_jv,min, a_jv,max], where v = 1, 2, …, N + 1, j = 1, 2, …, M, M is the total amount of individuals and N is integer. The more individuals we set, the quicker the algorithm converges, but the more time it costs for each iteration.

Step 2. Calculation of fitness. Fitness function is used to evaluate the advantages and disadvantages of individuals. The smaller the cost function is, the larger the fitness is, and therefore such individual would have stronger adaptability than others.

F = \frac{1}{f}

Step 3. Selection. Selection operation is to choose some outstanding individuals from the previous group with a certain probability and form new species to reproduce the next generation. Here we use the widely used roulette method [35,36], namely based on the proportion of fitness. The probability of each individual to be selected is defined by

P_{j} = \frac{F_{j}}{\sum_{1}^{M} F_{j}}

Step 4. Crossover. Crossover operation refers to two individuals randomly selected from the population according to the probabilities of Step 3. Through a combination of two chromosomes exchange, excellent traits of the substring inherit from parent string. Specifically, the j_th solution named a_j and the l_th solution named a_l will exchange their v_th input, and the operation method is shown below.

\begin{array}{l} a_{j v} = a_{j v} (1 - b) + a_{l v} b \\ a_{l v} = a_{l v} (1 - b) + a_{j v} b \end{array}

where b is a random number distributed in [0, 1].

Step 5. Mutation. The main purpose of the mutation operation is to maintain the population diversity, so that the algorithm can jump out of local minimum and try to obtain the optimal solution. Specifically, an individual from the population is selected according to the probabilities of Step 3, for example, the h_th individual termed a_h, and the v_th input termed a_hv is mutated as follows:

a_{h v} = {\begin{cases} a_{h v} + (a_{\max} - a_{h v}) g (k), r_{1} \geq 0.5 \\ a_{h v} + (a_{\min} - a_{h v}) g (k), r_{1} < 0.5 \end{cases}

where r₁ is a random number distributed in [0, 1]. And

g (k) = r_{2} {(1 - \frac{k}{k_{\max}})}^{2}

where r₂ is a random number distributed in [0, 1], k is the number of evolutions (iterations). With the increasing of k, the g(k) will close to zero. Steps 2-5 complete a single iteration of the algorithm. Generally the SSE reaches convergence within 30 iterations so we set k_max to 30 both in simulations and experiments. In simulations we also use the normalized mean square error (MSE) to evaluate the image quality of the recovery. The normalized MSE between two images G(x, y) and Q (x, y) can be defined as

M S E = \frac{\sum {[G (x, y) - Q (x, y)]}^{2}}{m n}

where m and n are the number of image pixels in the x and y coordinates, respectively. The summary of notations is listed in Table 1.

Table 1. Summary of notations

View Table | View all tables in this article

3. Simulations

A numerical simulation is executed to investigate the performance of the proposed GA-3ePIE algorithm by evaluating the convergence and the reconstruction quality with three slices. Figures 2(a1)-2(a6) present the amplitude and phase of each slice as for the ground truth respectively. All of them are 256 × 256 pixels (2.4μm pixel pitch). A pinhole with the physical radius of 163μm is placed at 0.2mm away from the object’s first slice. The distance between each slice and the distance from the last slice to the detector are d₁ = 10mm, d₂ = 20mm, d₃ = 50mm respectively, 5 × 5 diffraction patterns are captured by a 2000 × 2000 pixels camera with the pixel pitch of 6.5μm and a 632.8nm laser. Then the pixel size of object is given by [37,38]

Δ x_{1} = \frac{λ d_{N}}{L}

where L is the length of detector and λ is the wavelength. In this case N is equal to 3 and ∆x₁ is 2.4μm. The step length of translation of probe is 60μm with the overlap rate of 76%, which can be precisely given by [39]

R_{overlap} = \frac{1}{π} (2 arc \cos \frac{S_{t}}{2 r} - \frac{S_{t}}{r^{2}} \sqrt{r^{2} - \frac{S_{t}^{2}}{4}})

where S_t is the step length of translation, r is the radius of probe. All parameters are chosen to fit the experiment conditions.

Fig. 2 Reconstructions with different cases. (a1)-(a6) The simulated images served as the ground truth with d₁ = 10mm, d₂ = 20mm and d₃ = 50mm. (b1)-(b6) The ideal reconstructions with precise parameters. (c1)-(c6), (d1)-(d6), (e1)-(e4) and (f1)-(f8) The reconstructions using 3ePIE algorithm with four typical and unknown cases. Case 1: d₁ = 15mm, d₂ = 15mm and d₃ = 50mm; Case 2: d₁ = 10mm, d₂ = 10mm and d₃ = 50mm; Case 3: d₁ = 30mm and d₂ = 50mm; Case 4: d₁ = 10mm, d₂ = 10mm, d₃ = 10mm and d₄ = 50mm. (g1)-(g6) The reconstructions with our GA-3ePIE approach. The numbers beneath each image: the MSE compared with the ground truth.

Download Full Size | PDF

The calculation window of the probe depends on the scattering degree of the sample and it should be tailored without information loss. Generally, we tailor it as small as possible for fast calculation. The probe can be mapped into 156 × 156 pixels with the radius of 68 pixels (2.4μm pixel pitch). If every parameter is known and precise, the reconstructions of ideal case with 3ePIE approach are shown in Figs. 2(b1)-2(b6) respectively and cost 115s. All the simulations would converge and iterate 300 times for fair comparisons. The recovered probe’s amplitude and phase are presented at the bottom right. The MSEs of reconstructions compared with the ground truth are shown at the bottom of each result. Our reconstructions are performed using MATLAB (Version R2016a, Math Works, Natick, Massachusetts) on a desktop computer equipped with a central processing unit (Intel Core i7-7700), window 10 64-bit operating system and 64GB of random-access memory.

However, as discussed above, the number of slices N and the distance between layers d_s are unknown generally. So we assume four typical and unknown cases. In the unknown case 1, we assume that N is still equal to 3, the total distance is invariant. But with the same interval separation approach, i.e., d₁ = 15mm, d₂ = 15mm, d₃ = 50mm, the reconstructions are shown in Figs. 2(c1)-2(c6). Since the positions of first and third slices are correct, parts of their amplitudes can be resolved. But the results become bad compared with the ideal case due to the wrong place of the second slice. Then, considering the specific thickness is unknown, in the unknown case 2 we still use the same interval separation approach but we assume that d₁ = 10mm, d₂ = 10mm and d₃ = 50mm. And the reconstructions are shown in Figs. 2(d1)-2(d6). The unknown case 2 becomes worse than the case 1 when comparing the MSEs and the probe is distorted completely, since the place of the last slice is the only right position. The computation time of both the cases 1 and 2 are the same as ideal case with 115s. Next, considering the number of slices is unknown and we assume that N is equal to 2 but we keep the total distance invariant, namely d₁ = 30mm and d₂ = 50mm. The reconstructions are presented in Figs. 2(e1)-2(e4), which are terrible and cannot be resolvable completely and are worse than the case 1 due to the lack of second slice. The computation time is 77s. Finally, we assume the unknown case 4 that N is equal to 4 with the same interval of 10mm, namely d₁ = 10mm, d₂ = 10mm, d₃ = 10mm, d₄ = 50mm. The reconstructions are presented in Figs. 2(f1)-2(f8) and cost 152s. The unknow case 4 is quite close to the truth as shown in Fig. 3(b), which can also be explained from the perspective of physics. But with an empty slice the reconstruction quality would degenerate a little due to the increased uncertainty and require more computation time and data redundancy. Note that the unknown case 4 has ignored the potential seek time for parameters. To sum up, if there is a little inaccuracy of these two important parameters in 3ePIE approach for a discrete sample with uneven spatial distribution, the reconstructions deteriorate severely, especially the phase will be unresolvable extremely. But with our GA-3ePIE approach as shown in Figs. 2(g1)-(g6), the retrieved parameters and the reconstructions are the same as the ideal case. The range of d₁, d₂,…,d_N_-1 is restricted from 1 to 40mm, while d_N is the same as other simulations and is equal to 50mm, and N is restricted from 1 to 5. The total computation time with GA-3ePIE approach is around 42h. Combined with known information (prior knowledge), setting boundaries for each input reasonably will reduce computation time efficiently, which will be discussed in section 5. The cost function of GA-3ePIE algorithm versus the iterations is shown in Fig. 3(a) and the algorithm would converge at around 11 iterations. The computation time with 11 iterations is around 14h and each iteration contains 40 × 300 iterations of 3ePIE algorithm. In fact, due to the random initial guesses, the GA-3ePIE would converge at different iterations in different cases but Fig. 3(a) reflects the trend of our algorithm. Generally, the genetic algorithm would converge to a second-best solution quickly and converge to the optimal solution with more iterations [35,36]. Note that the value of y-axis is very close to zero within 10 iterations and would converge to 2e-6. If we use a constant interval, e.g. 10mm to retrieve the number of slices via our algorithm, the results are the same as Figs. 2(f1)-2(f8) and the computation time will reduce tremendously with 7h and the algorithm converges at 6 iterations, since there is only one unknown input. And the corresponding SSEs versus the iterations with different cases are shown in Fig. 3(b). Both the ideal case and GA-3ePIE have the same results and are close to zero, since they share the same parameters. But the unknown cases would not be better than the case with GA-3ePIE procedure.

Fig. 3 Results of three slices in simulations. (a) The cost function SSE versus the iterations with GA-3ePIE approach. (b) The SSEs versus the iterations with different cases.

Download Full Size | PDF

4. Experiments

In order to validate the effectiveness of GA-3ePIE procedure experimentally, we first compare the reconstructions of two overlapped USAF targets. The photograph of the optical setup is shown in Fig. 4. The He-Ne laser with the wavelength of 632.8nm is used in our experiment, 5 × 5 diffraction patterns are captured by a camera with 2160 × 2560 pixels (6.5μm pixel pitch), while the other two wavelengths would be used for future work. The physical radius of pinhole is 0.5mm. The step length of translation is 100μm with the overlap ratio of 87.6%. The pinhole is around 8.5mm far from the object, and the distance between each layer and from the last slice to detector are around d₁ = 20mm, d₂ = 70mm and then ∆x₁ is 3.15μm.

Fig. 4 Photograph of optical setup.

Download Full Size | PDF

The probe will be mapped into 338 × 338 pixels with the numerical radius of 159 pixels (3.15μm pixel pitch). A typical diffraction pattern is presented in Fig. 5(g). Note that a data preprocessing operation is implemented first to remove the background noise in order to keep the consistency of data [40,41], which is important to the recovery of each slice in 3D ptychography [34]. And the reconstructions with different cases are shown in Fig. 5 with 100 iterations for fair comparisons. If we treat it as the only one slice as shown in Fig. 5(a), the second slice would be resolved partially but a defocus artifact from the first slice would generate, which is similar to result of mechanical focusing. The reconstructions cost 62s. When we set the number of slices N to 2, two different cases are shown in Figs. 5(b1)-5(b2) and 5(c1)-5(c2). Especially when we set d₁ = 20mm and d₂ = 70mm according to our measurements (actually d₁ cannot be measured as a thick sample), the reconstructions are still bad, since the measurements may not be very precise and the two parameters are sensitive to 3D ptychography. The reconstructions cost 103s. When we set the number of slices N to 3 with d₁ = 10mm, d₂ = 10mm and d₃ = 70mm, there would be an empty slice and the reconstructions become closer to the truth as shown in Fig. 6(b) and cost 158s, which is coincident with the simulations. However, because the parameters are not correct, the object cannot be resolved. While with our GA-3ePIE approach, the reconstructions become clear and resolvable as shown in Figs. 5(e1)-6(e2) and 5(f1)-6(f2) and cost around 32h. The range of d₁, d₂,…,d_N_-1 is restricted from 1 to 40mm, while d_N is limited from 60 to 80mm, and N is restricted from 1 to 4. The precise results with our GA-3ePIE approach is d₁ = 26.5mm, d₂ = 72.3mm. The cost function versus iterations is shown in Fig. 6(a) and our algorithm will convergence at around 13 iterations, which costs around 13h. The SSEs versus iterations with different cases are shown in Fig. 6(b) and our algorithm has the lowest SSE than the other cases.

Fig. 5 Intensity reconstructions of USAF target with different parameters. (a) Intensity reconstructions with only one slice. (b1)-(b2) and (c1)-(c2) Intensity reconstructions with two slices but with different distance between layers. (d1)-(d3) Intensity reconstructions with three slices including one empty slice. (e1)-(e2) and (f1)-(f2) Intensity reconstructions with GA-3ePIE approach and their close-ups. (g) A typical diffraction patterns.

Download Full Size | PDF

Fig. 6 Results of USAF target. (a) The cost function SSE versus the iterations with GA-3ePIE approach. (b) The SSEs versus the iterations with different cases.

Download Full Size | PDF

In conventional ptychography, if there is a little inaccuracy about the distance from object to the detector, usually a digital refocusing is required to achieve clear results. However, it would be extremely difficult in the 3D ptychography. Only if all the parameters are correct, there would not be any artifacts stemmed from other slices compared Fig. 5 with Fig. 7. Based on the prior knowledge of the specific distance, the digital refocusing has been done as shown in Fig. 7. Note that to keep the d₁ invariable, both slices need to be propagated with the same distance as shown in Figs. 7(a), 7(b) and 7(c). Therefore, it would be nearly impossible to do the refocusing process manually especially with the increasing of the number of slices. Besides, the specific number of slices is also unknown.

Fig. 7 Digital refocusing results.

Download Full Size | PDF

In addition, we also compare the reconstructions of two overlapped biological samples, bee wings and fly wings as shown in Fig. 8. The physical radius of pinhole is 4.6mm due to the size of samples. The step length of translation is 0.67mm with the overlap ratio of 85% and 3 × 3 diffraction patterns are captured, which is the minimum amount for recorded images in near-field ptychography. The pinhole is around 22.6mm far from the object, and the distance between each layer and from the last slice to detector are around d₁ = 55.5mm, d₂ = 57.7mm. A typical diffraction pattern is presented in Fig. 8(e). Similar to the results of USAF target in the Fig. 5, only if all the parameters are precise, there would not be any artifacts stemmed from other slices as shown in white arrows. The computation time of tradition 3ePIE algorithm is 278s. While with our GA-3ePIE approach, the reconstructions become distinguishable as shown in Figs. 8(d1)-8(d2). The precise results with our GA-3ePIE approach is d₁ = 60.2mm, d₂ = 62.1mm. The range of d_s is restricted from 30 to 70mm and N is restricted from 1 to 4. The computation time is 96h. The cost function versus iterations is shown in Fig. 9(a) and our algorithm will convergence at around 14 iterations, which costs 48h. The SSEs versus iterations with different cases are shown in Fig. 9(b) and our algorithm has the lowest SSE than the other cases.

Fig. 8 Phase reconstructions of biological samples with different parameters. (a1) and (a2) Photographs of biological samples. (b1)-(b2) and (c1)-(c2) Phase reconstructions with two slices but with different distance between layers. (d1)-(d2) Phase reconstructions with GA-3ePIE approach. (e) A typical diffraction patterns.

Download Full Size | PDF

Fig. 9 Results of biological samples. (a) The cost function SSE versus the iterations with GA-3ePIE approach; (b) The SSEs versus the iterations with different cases.

Download Full Size | PDF

5. Discussions

As a quantitative phase imaging technique, the phase is crucial to the label-free biomedical imaging. Sequentially, we test the sensitivity of the distance d_s in 3ePIE algorithm quantificationally and set the average MSEs of all slices’ phase as the assessment criteria, since the SSE is too small to reflect the reconstruction quality compared Figs. 2 and 3. The ground truth is still fixed as shown in Figs. 2(a1)-2(a6) with three slices and d₁ = 10mm, d₂ = 20mm and d₃ = 50mm, while 20 groups of random results with different deviation ratios are shown in Fig. 4(a). The deviation ratio is defined as follows.

d e v i a t i o n s = \frac{Δ d_{s}}{d_{s}} \times 100 %

where the △d_s is differential between the input d_s and ground truth. The average MSE of these 20 groups of results is shown in the blank solid line of Fig. 4(a), while the shades of color from red to white represent the density distribution of error from dense to sparse. When there is no deviation, which means all the 20 groups of input are correct, they would converge with the MSE of 0.006. With the increasing of deviation ratio, the error increases severally. Even if there is only 5% of deviation, the error increases from 0.006 to 0.06, which is up tenfold. Therefore, the phase is very sensitive to the accuracy of the distance between each layer. More importantly, the number of slices is fixed. The error would be more severe when the number of slices is not known.

As for a thin 2D sample, the 3ePIE or GA-3ePIE approach will degenerate to ePIE algorithm, since both the 3ePIE and GA-3ePIE are still based on the ePIE algorithm with the multi-slice idea. However, if we can make sure that the sample is a thin 2D object with prior knowledge, then directly using the ePIE algorithm would be much efficient.

Furthermore, how thick we can image is also crucial to biomedical studies. The 3ePIE algorithm can image continuous samples with the thickness of 1mm theoretically [7]. And our method would be able to image the same samples and retrieve the number of slices, since it is based on the 3ePIE algorithm. But for discrete samples such as the microbeads solution and blood, using the resolvable number of slices would be more reasonable, since the interval between each layer is homogeneous medium without too much scattering. The number of slices that can be resolved is associated with the reductant information, namely the overlap rate and sampling in the spatial domain. The more number of slices requires, the more reductant information we need. Although ptychography does not require oversampling [37], more sampling would result in more redundancy. Similar sampling criteria are also shown in Fourier ptychography [39]. In numerical analysis, we keep the FOV, 5 × 5 scanning and the distance from the last slice to the detector d_N of 50mm unchanged but the size of probe and the step size will change accordingly. We set d_s to 2mm except for d_N. Generally, around 60% overlap rate is necessary for ptychography [2]. To achieve 58%, 67%, 76%, 83%, 89% and 93% overlap rate respectively, the radius probe is 120μm, 139μm, 163μm, 187μm, 211μm and 235μm, respectively and the step size is 81.6μm, 72μm, 60μm, 48μm, 36μm and 24μm, respectively. A diffuser is necessary to scatter the speckles when the number of slices is over 5 in our tests, which is generally used in ptychography [7–9]. Different images are used for simulations. Because the complexity of images is not the same, so we add an error tolerance in Fig. 10(b). The maximum number of slices that can be resolved is around 8 without objective. But with a 10 × objective, around 16 slices can be resolved, which is nearly double than without an objective. While with 20 × objective Godden et al. achieved the recovery of 34 sections [8].

Fig. 10 (a) The average MSEs of phase of layers versus the deviations of the distance between layers. (b) The number of slices can be resolved versus the overlap rate and sampling in spatial domain.

Download Full Size | PDF

Although our method can retrieve the parameters and obtain satisfied reconstructions, the cost of time is inevitable, which would increase with the number of slices to be resolved. To reduce the computation time, first is to set a reasonable boundary for different input. For example, the distance from the last slice to detector d_N can be measured and the tolerance should be around ± 10mm or less according to the specific experimental conditions. The thickness of sample can be measured so that the number of slices and the distance between each layer can have estimations. Generally, we will set the tolerance around ± 20mm for d_s. The smaller the tolerance is, the faster the algorithm converges. Second is to set a moderate number of individuals M which are tested in Table 2. We keep using the ground truth and parameters in Fig. 2 and tracing the computation time when our algorithm converges. And we find that M = 10(N + 1) is the most efficient case. Finally, genetic algorithm can be operated by parallel computing completely with the GPU acceleration [42,43], which would reduce the computation time tremendously.

Table 2. Computation time with different number of individuals

View Table | View all tables in this article

6. Conclusions

In this paper, a genetic algorithm based 3ePIE approach, termed GA-3ePIE method is proposed to retrieve the crucial and sensitive parameters for 3ePIE approach, i.e., the number of slices and the distance between each layer. The performance is verified by both simulations and experiments. The maximum number of sections that can be resolved is also investigated, which is associated with the sampling and overlap rate in spatial domain. Although our method can retrieve the parameters and obtain satisfied reconstructions, the cost of time is inevitable, which would increase with the number of slices. But luckily, the test task does not request any man-made operation. The future work may try to embed our GA-3ePIE method into the GPU acceleration for parallel processing to reduce the computation time.

Funding

National Natural Science Foundation of China (NSFC) (81427802, 61377008).

Acknowledgments

An Pan thanks Dr. Peng Li (University of Sheffield, UK) and Prof. Shiyi Shi (University of Chinese Academy of Sciences, China) for help and guidance in establishing the experimental apparatus, and Dr. Yuege Xie (University of Texas at Austin, USA) for support and encouragement all the time.

Disclosures

The authors have no relevant financial interests in this article and no potential conflicts of interest to disclose.

References

1. J. M. Rodenburg and R. H. T. Bates, “The theory of super-resolution electron microscopy via Wigner-distribution deconvolution,” Phil. Trans. R. Soc. Lond. A 339(1655), 521–553 (1992).

2. H. M. L. Faulkner and J. M. Rodenburg, “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93(2), 023903 (2004). [CrossRef] [PubMed]

3. J. M. Rodenburg and H. M. Faulkner, “A phase retrieval algorithm for shifting illumination,” Appl. Phys. Lett. 85(20), 4795–4797 (2004). [CrossRef]

4. J. M. Rodenburg, A. C. Hurst, A. G. Cullis, B. R. Dobson, F. Pfeiffer, O. Bunk, C. David, K. Jefimovs, and I. Johnson, “Hard-x-ray lensless imaging of extended objects,” Phys. Rev. Lett. 98(3), 034801 (2007). [CrossRef] [PubMed]

5. P. Thibault, M. Dierolf, O. Bunk, A. Menzel, and F. Pfeiffer, “Probe retrieval in ptychographic coherent diffractive imaging,” Ultramicroscopy 109(4), 338–343 (2009). [CrossRef] [PubMed]

6. A. M. Maiden and J. M. Rodenburg, “An improved ptychographical phase retrieval algorithm for diffractive imaging,” Ultramicroscopy 109(10), 1256–1262 (2009). [CrossRef] [PubMed]

7. A. M. Maiden, M. J. Humphry, and J. M. Rodenburg, “Ptychographic transmission microscopy in three dimensions using a multi-slice approach,” J. Opt. Soc. Am. A 29(8), 1606–1614 (2012). [CrossRef] [PubMed]

8. T. M. Godden, R. Suman, M. J. Humphry, J. M. Rodenburg, and A. M. Maiden, “Ptychographic microscope for three-dimensional imaging,” Opt. Express 22(10), 12513–12523 (2014). [CrossRef] [PubMed]

9. P. Li, D. J. Batey, T. B. Edo, and J. M. Rodenburg, “Separation of three-dimensional scattering effects in tilt-series Fourier ptychography,” Ultramicroscopy 158, 1–7 (2015). [CrossRef] [PubMed]

10. P. Thibault and A. Menzel, “Reconstructing state mixtures from diffraction measurements,” Nature 494(7435), 68–71 (2013). [CrossRef] [PubMed]

11. P. Li, T. Edo, D. Batey, J. Rodenburg, and A. Maiden, “Breaking ambiguities in mixed state ptychography,” Opt. Express 24(8), 9038–9052 (2016). [CrossRef] [PubMed]

12. X. Dong, X. Pan, C. Liu, and J. Zhu, “Single shot multi-wavelength phase retrieval with coherent modulation imaging,” Opt. Lett. 43(8), 1762–1765 (2018). [CrossRef] [PubMed]

13. M. Guizar-Sicairos and J. R. Fienup, “Phase retrieval with transverse translation diversity: a nonlinear optimization approach,” Opt. Express 16(10), 7264–7278 (2008). [CrossRef] [PubMed]

14. P. Thibault and M. Guizar-Sicairos, “Maximum-likelihood refinement for coherent diffraction imaging,” New J. Phys. 14(6), 063004 (2012). [CrossRef]

15. A. M. Maiden, M. J. Humphry, M. C. Sarahan, B. Kraus, and J. M. Rodenburg, “An annealing algorithm to correct positioning errors in ptychography,” Ultramicroscopy 120, 64–72 (2012). [CrossRef] [PubMed]

16. C. Zuo, J. Sun, and Q. Chen, “Adaptive step-size strategy for noise-robust Fourier ptychographic microscopy,” Opt. Express 24(18), 20724–20744 (2016). [CrossRef] [PubMed]

17. A. Pan, Y. Zhang, T. Zhao, Z. Wang, D. Dan, M. Lei, and B. Yao, “System calibration method for Fourier ptychographic microscopy,” J. Biomed. Opt. 22(9), 1–11 (2017). [CrossRef] [PubMed]

18. A. M. Maiden, J. M. Rodenburg, and M. J. Humphry, “Optical ptychography: a practical implementation with useful resolution,” Opt. Lett. 35(15), 2585–2587 (2010). [CrossRef] [PubMed]

19. J. Marrison, L. Räty, P. Marriott, and P. O’Toole, “Ptychography--a label free, high-contrast imaging technique for live cells using quantitative phase information,” Sci. Rep. 3(1), 2369 (2013). [CrossRef] [PubMed]

20. A. M. Maiden, M. C. Sarahan, M. D. Stagg, S. M. Schramm, and M. J. Humphry, “Quantitative electron phase imaging with high sensitivity and an unlimited field of view,” Sci. Rep. 5, 14690 (2015). [CrossRef] [PubMed]

21. M. J. Humphry, B. Kraus, A. C. Hurst, A. M. Maiden, and J. M. Rodenburg, “Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging,” Nat. Commun. 3(1), 730 (2012). [CrossRef] [PubMed]

22. P. Thibault, M. Dierolf, A. Menzel, O. Bunk, C. David, and F. Pfeiffer, “High-resolution scanning x-ray diffraction microscopy,” Science 321(5887), 379–382 (2008). [CrossRef] [PubMed]

23. M. Dierolf, A. Menzel, P. Thibault, P. Schneider, C. M. Kewish, R. Wepf, O. Bunk, and F. Pfeiffer, “Ptychographic X-ray computed tomography at the nanoscale,” Nature 467(7314), 436–439 (2010). [CrossRef] [PubMed]

24. E. H. R. Tsai, I. Usov, A. Diaz, A. Menzel, and M. Guizar-Sicairos, “X-ray ptychography with extended depth of field,” Opt. Express 24(25), 29089–29108 (2016). [CrossRef] [PubMed]

25. H. Öztürk, H. Yan, Y. He, M. Ge, Z. Dong, M. Lin, E. Nazaretski, I. K. Robinson, Y. S. Chu, and X. Huang, “Multi-slice ptychography with large numerical aperture multilayer Laue lenses,” Optica 5(5), 601–607 (2018). [CrossRef]

26. K. Shimomura, M. Hirose, T. Higashino, and Y. Takahashi, “Three-dimensional iterative multislice reconstruction for ptychographic X-ray computed tomography,” Opt. Express 26(24), 31199–31208 (2018). [CrossRef] [PubMed]

27. M. Stockmar, P. Cloetens, I. Zanette, B. Enders, M. Dierolf, F. Pfeiffer, and P. Thibault, “Near-field ptychography: phase retrieval for inline holography using a structured illumination,” Sci. Rep. 3(1), 1927 (2013). [CrossRef] [PubMed]

28. M. Stockmar, I. Zanette, M. Dierolf, B. Enders, R. Clare, F. Pfeiffer, P. Cloetens, A. Bonnin, and P. Thibault, “X-ray near-field ptychography for optically thick specimens,” Phys. Rev. Appl. 3(1), 014005 (2015). [CrossRef]

29. M. Stockmar, M. Hubert, M. Dierolf, B. Enders, R. Clare, S. Allner, A. Fehringer, I. Zanette, J. Villanova, J. Laurencin, P. Cloetens, F. Pfeiffer, and P. Thibault, “X-ray nanotomography using near-field ptychography,” Opt. Express 23(10), 12720–12731 (2015). [CrossRef] [PubMed]

30. R. M. Clare, M. Stockmar, M. Dierolf, I. Zanette, and F. Pfeiffer, “Characterization of near-field ptychography,” Opt. Express 23(15), 19728–19742 (2015). [CrossRef] [PubMed]

31. G. Zheng, R. Horstmeyer, and C. Yang, “Wide-field, high-resolution Fourier ptychographic microscopy,” Nat. Photonics 7(9), 739–745 (2013). [CrossRef] [PubMed]

32. A. Pan, Y. Zhang, K. Wen, M. Zhou, J. Min, M. Lei, and B. Yao, “Subwavelength resolution Fourier ptychography with hemispherical digital condensers,” Opt. Express 26(18), 23119–23131 (2018). [CrossRef] [PubMed]

33. L. Tian, Z. Liu, L. Yeh, M. Chen, J. Zhong, and L. Waller, “Computational illumination for high-speed in vitro Fourier ptychographic microscopy,” Optica 2(10), 904–911 (2015). [CrossRef]

34. A. Pan, M. Zhou, Y. Zhang, J. Min, M. Lei, and B. Yao, “Adaptive-window angular spectrum algorithm for near-field ptychography,” Opt. Commun. 430, 73–82 (2019). [CrossRef]

35. G. M. Morris, D. S. Goodsell, R. S. Halliday, R. Huey, W. E. Hart, R. K. Belew, and A. J. Olson, “Automated docking using a Lamarckian genetic algorithm and an empirical binding free energy function,” J. Comput. Chem. 19(14), 1639–1662 (1998). [CrossRef]

36. D. B. Conkey, A. N. Brown, A. M. Caravaca-Aguirre, and R. Piestun, “Genetic algorithm optimization for focusing through turbid media in noisy environments,” Opt. Express 20(5), 4840–4849 (2012). [CrossRef] [PubMed]

37. T. Edo, D. Batey, A. Maiden, C. Rau, U. Wagner, Z. Pesic, T. Waigh, and J. Rodenburg, “Sampling in x-ray ptychography,” Phys. Rev. A 87(5), 053850 (2013). [CrossRef]

38. D. Claus and J. M. Rodenburg, “Pixel size adjustment in coherent diffractive imaging within the Rayleigh-Sommerfeld regime,” Appl. Opt. 54(8), 1936–1944 (2015). [CrossRef] [PubMed]

39. J. Sun, Q. Chen, Y. Zhang, and C. Zuo, “Sampling criteria for Fourier ptychographic microscopy in object space and frequency space,” Opt. Express 24(14), 15765–15781 (2016). [CrossRef] [PubMed]

40. C. Wang, Z. Xu, H. Liu, Y. Wang, J. Wang, and R. Tai, “Background noise removal in x-ray ptychography,” Appl. Opt. 56(8), 2099–2111 (2017). [CrossRef] [PubMed]

41. Y. Zhang, A. Pan, M. Lei, and B. Yao, “Data preprocessing methods for robust Fourier ptychographic microscopy,” Opt. Eng. 56(12), 123107 (2017). [CrossRef]

42. K. Wang and Z. Shen, “A GPU-based parallel genetic algorithm for generating daily activity plans,” IEEE Trans. Intell. Transp. Syst. 13(3), 1474–1480 (2012). [CrossRef]

43. S. Marchesini, H. Krishnan, B. J. Daurer, D. A. Shapiro, T. Perciano, J. A. Sethian, and F. R. N. C. Maia, “SHARP: a distributed GPU-based ptychographic solver,” J. Appl. Cryst. 49(4), 1245–1252 (2016). [CrossRef]

Notation	Definition	Notation	Definition
O_s	Complex object function	Subscript e	Exit wave
P	Probe function	Subscript i	Incident wave
r	Cartesian coordinate	Subscript s	A certain section of object
N	Number of sections	Subscript c	A certain position of probe
R_c	Positional array	u	Coordinate at detector plane
w	Number of recorded patterns	d_s	Distance between each layer
E	Light wave field	$ρ_{d}_{_{s}}$	Angular spectrum operation
I_c	Measured diffraction pattern	f	3ePIE algorithm
α, β	Step size of 3ePIE algorithm	F	Fitness
SSE	Sum of the square error	a_j	A certain individual (solution)
M	Total amount of individuals	P_j	Probability of selection
MSE	Mean square error	a_jv	A certain chromosome (input)
m, n	Number of image pixels along x- and y- axis	a_jv,min, a_jv,max	Bound for a certain chromosome
k	number of evolutions (iterations) in GA-3ePIE	r₁, r₂	random number distributed in [0, 1]

Notation	Definition	Notation	Definition
O_s	Complex object function	Subscript e	Exit wave
P	Probe function	Subscript i	Incident wave
r	Cartesian coordinate	Subscript s	A certain section of object
N	Number of sections	Subscript c	A certain position of probe
R_c	Positional array	u	Coordinate at detector plane
w	Number of recorded patterns	d_s	Distance between each layer
E	Light wave field	$ρ_{d}_{_{s}}$	Angular spectrum operation
I_c	Measured diffraction pattern	f	3ePIE algorithm
α, β	Step size of 3ePIE algorithm	F	Fitness
SSE	Sum of the square error	a_j	A certain individual (solution)
M	Total amount of individuals	P_j	Probability of selection
MSE	Mean square error	a_jv	A certain chromosome (input)
m, n	Number of image pixels along x- and y- axis	a_jv,min, a_jv,max	Bound for a certain chromosome
k	number of evolutions (iterations) in GA-3ePIE	r₁, r₂	random number distributed in [0, 1]

Three-dimensional space optimization for near-field ptychography

Abstract

1. Introduction

2. Methods

3. Simulations

4. Experiments

5. Discussions

6. Conclusions

Funding

Acknowledgments

Disclosures

References

Cited By

Figures (10)

Tables (2)

Equations (18)

Optics Express

Number of slices N	Number of individuals M	Iteration times k of GA-3ePIE approach in convergence	Total iterations of 3ePIE algorithm	Computation time (h)
2	20	13	20 × 300 × 13	5.5
	30	7	30 × 300 × 7	4
	40	7	40 × 300 × 7	6
	50	6	50 × 300 × 6	6
3	30	18	30 × 300 × 18	17
	40	11	40 × 300 × 11	14
	50	11	50 × 300 × 11	17.5
	60	10	60 × 300 × 10	18.5
4	40	23	40 × 300 × 23	38
	50	17	50 × 300 × 17	35
	60	16	60 × 300 × 16	40
	70	16	70 × 300 × 16	46