Method for extending the depth of focus in X-ray microscopy

Fahu Li; Yong Guan; Ying Xiong; Xiaobo Zhang; Gang Liu; Yangchao Tian

doi:10.1364/OE.25.007657

1. Introduction

Transmission X-ray microscopy (TXM) is a promising technique that could provide detailed insight into the three-dimensional (3D) structure of matter and has been used for imaging energy, life, and materials science samples [1–3]. In high-resolution X-ray imaging, a zone plate (ZP) is the key element since the width of the outermost zone determines the spatial resolution of the resulting images [4]. The Rayleigh resolution of a ZP is $1.22 Δ r_{N}$ which corresponds to a depth of focus (DOF) of $4 Δ r_{N}^{2} / λ$ , where $Δ r_{N}$ denotes the width of the outermost zone and $λ$ denotes the wavelength of the incident X-ray [5]. The DOF clearly decreases as the spatial resolution of the microscopic system increases, so the sizes of samples that can be imaged by high-resolution microscopic systems are limited, and the DOF of high-resolution TXM is often shorter than the sample is thick, challenging the validity of the projection assumption [6]. In addition, the photon energy, i.e., the wavelength of incident X-ray, also affects the DOF. According to the expression $4 Δ r_{N}^{2} / λ$ , a large wavelength of incident X-ray corresponds to a short DOF. For instance, when a ZP with $Δ r_{N} = 40 n m$ is applied in soft TXM, in which the wavelength is 2.34 nm (531 eV) −4.38 nm (283 eV) in the water window region between absorption edges of carbon and oxygen [7–9], the DOF of the microscope is about 1.461-2.735 μm, which is quite small for imaging large samples. Therefore, the DOF must be extended for soft TXM in particular. When high-resolution TXM images of large samples are collected [10, 11], this DOF limitation becomes even more significant. Therefore, it is of great importance to extend the DOF of high-resolution TXM [6, 12, 13].

Selin et al. employed a reconstruction method based on the back-projections of focus-stacked image data to extend the DOF of a microscopic system. The focus-stacked back-projection (FSBP) algorithm was developed to enable focus-stacked data to be utilized to improve the resolutions and qualities of tomographic reconstructed TXM image [6]. However, the fused back-projections obtained via that method contained some defocused information and a rigorous proof of the proper operation of the FSBP method could not be provided. Liu et al. proposed a focus-stacked image fusion algorithm based on a fast discrete curvelet transform to extend the DOF of TXM. A completely in-focus image was generated by analyzing a stack of images taken in the Z-scanning a sample along the optical axis [13]. This method was effective for the relative sparse objects.

In this paper, an algorithm that can be used to extend the DOF of TXM by fusing image slices obtained from 3D reconstructions is described. Simulated and diatom frustule projections are presented that verify the effectiveness of the proposed method. The method proposed in this paper could deal with complex samples well.

2. Principles

The analytical method presented in this paper consists two stages: computed tomography (CT) reconstruction and sliced image fusion. Each projection at each tilt angle is collected by rotating the sample and a set of projection data is acquired. Then moving the ZP along the optical axis, another set of projection data with different focus distance is obtained. Last, these sets of projections are reconstructed by applying a filtered back-projection (FBP) reconstruction algorithm [14] to obtain 3D reconstruction data $R_{1} (x, y, z)$ , $R_{2} (x, y, z)$ , $R_{3} (x, y, z)$ , … $R s (x, y, z)$ with $Δ z_{1}$ , $Δ z_{2}$ , $Δ z_{3}$ , …, $Δ z_{s}$ respectively, where $Δ z$ is the defocus distance and $s$ is the number of 3D reconstruction data. $R_{1} (x, y, z)$ , $R_{2} (x, y, z)$ , $R_{3} (x, y, z)$ ,…, $R s (x, y, z)$ contain different in-focus and defocused information.

A series of 2D slices that coincide with the x-y plane can be obtained from a 3D reconstruction directly using TXM controller software of Zeiss X-ray Microscopy Company, and the same location slices from the slice data with different defocus distances are selected and treat as focus-stacked slice data. The image fusion algorithm starts with the slices in one stack of 3D reconstruction data $s l i c e_{1}$ , $s l i c e_{2}$ , $s l i c e_{3}$ ,…, $s l i c e_{n}$ with different $Δ z$ , where $s l i c e$ is a slice in a stack of 3D reconstruction data. If the sample is larger than the DOF, some of the object’s 3D reconstruction data will be out of focus. In other words, $s l i c e_{1}$ , $s l i c e_{2}$ , $s l i c e_{3}$ ,…, $s l i c e_{n}$ are only partially in-focus. Thus, the goal is to determine in-focus pixel on which to focus in the slice stack. To achieve this objective, an image fusion based discrete wavelet transform (DWT) [15–19] is applied to fuse the sliced image and obtain a fully in-focus image. If the all of the slices of 3D reconstruction data are well-focused, complete and fully focused 3D in-focus structural information can be obtained.

The main steps of the sliced image fusion algorithm are illustrated in Fig. 1. The same location slices of 3D reconstruction data with different distances are treated as slice focus stack. Then each slice in the slice stack is decomposed into a series of wavelet coefficients using a 2D DWT function,

{(c_{j} (x, y))}_{1, 2, ..., s} = D W T (s l i c e_{1, 2, ..., s} (x, y)),

where

c_{j}

is the wavelet coefficient of the slice,

s

denotes the index of the slice, and

j

is the index of wavelet coefficient of the slice. The local variance

V_{n} (x, y)

of every coefﬁcient

{(c_{j} (x, y))}_{1, 2, ..., s}

at each pixel within a neighborhood of

5 \times 5

pixels is deﬁned as

V_{n} (x, y) = \frac{{\sum_{i = - 2}^{2} \sum_{j = - 2}^{2} (I_{n} (x + i, y + j) - \bar{I} (x, y))}^{2}}{{(\bar{I} (x, y))}^{2}},

\bar{I} (x, y) = \frac{1}{25} \sum_{i = - 2}^{2} \sum_{j = - 2}^{2} I_{n} (x + i, y + j),

where

I_{n} (x, y)

denotes a pixel in wavelet coefficient,

n

denotes the pixel number of wavelet coefficient,

\bar{I} (x, y)

denotes the local mean of

5 \times 5

pixels. Then the coefﬁcients with highest variance, i.e., the sharpest contours [20] are selected and combined to obtain new coefficients

c_{m j}

, as follows:

c_{m j} = \underset{n}{\arg m a x} (V_{n} (x, y)) .

Once the coefficients are merged, the final fused slice is achieved through

s l i c e_{i n - f o c} (x, y) = | I D W T (\sum_{j} c_{m j} (x, y)) |,

where

I D W T

is the inverse 2D fast DWT function and

s l i c e_{i n - f o c}

is the output fully focused output sliced image. At last, based on the fully focused sliced image

s l i c e_{i n - f o c}

, complete and fully focused 3D structural information is obtained.

Fig. 1 Schematic of the proposed fusion framework. ${(I_{θ})}_{Δ z_{1}}$ , ${(I_{θ})}_{Δ z_{2}}$ , ${(I_{θ})}_{Δ z_{3}}$ : projection data in different $Δ z$ ; $θ_{0}$ , $θ_{1}$ ,…, $θ_{n}$ : the rotation angle of sample; $Δ z_{1}$ , $Δ z_{2}$ , $Δ z_{3}$ : the different distance of the sample between the ZP; FBP: filtered back-projection reconstruction algorithm; $R_{1}$ , $R_{2}$ , $R_{3}$ : 3D reconstruction data in different $Δ z$ ; $D W T$ : 2D DWT; $I D W T$ : inverse 2D DWT; $R_{i n - f o c u s}$ : fused in-focus 3D reconstruction data.

Download Full Size | PDF

3. Simulation and experiment

To verify the feasibility of the method proposed in this paper, a simulation and experimental analysis were performed. The tomographic data set was collected at the HZB Transmission X-ray Microscope in Berlin [21], which has an energy range of is 250-800eV (λ = 1.55nm-4.96nm) and a 40-nm ZP.

3.1 Simulation

The coherence parameter of soft X-ray transmission microscope in BESSY II is 0.67 [6]. Thus it is a partially coherent imaging case. For this imaging case it is sometimes useful to consider an apparent transfer function (ATF), defined by $H_{A} (f_{x}, f_{y}) = {\tilde{I}}_{o u t} (f_{x}, f_{y}) / {\tilde{I}}_{i n} (f_{x}, f_{y})$ , where ${\tilde{I}}_{o u t} (f_{x}, f_{y})$ and ${\tilde{I}}_{i n} (f_{x}, f_{y})$ are the Fourier transforms of the input and output intensity distributions, respectively [22–24]. In this work, the simulation method based on ATF is given and used to simulate the projections of the sample. The sample is decomposed into multiple slices along the optical axis, and each slice convoluted with corresponding apparent transfer function to obtain the image plane Fourier frequency. Then all the Fourier frequency is superposed together to obtain the projection in a tilt angle. Based on the ATF and the parameters of HZB Transmission X-ray Microscope in Berlin, λ is set to 3.2 nm, 180 projections in 1° steps between 0° and 179° are collected and three sets of projection data in three different positions, $Δ z_{1}$ , $Δ z_{2}$ , and $Δ z_{3}$ , are acquired by moving ZP along the optical axis. These three different positions is defined as three defocused distance of one sample. The X-ray microscope is depicted schematically in Fig. 2.

Fig. 2 Schematic of X-ray transmission microscope. Condenser: focuses a hollow central cone of X-rays onto the sample through the inner reflecting surface. Rotation axis: Sample rotates around the axis so that projections at different angles can be obtained. Objective ZP: used as an objective lens for TXM. Charge-coupled device (CCD): serves as a detector [12].

Download Full Size | PDF

The 8-μm-long, 2.4-μm-diameter cylinder that is depicted in Fig. 3(a) was employed as the sample, and TXM was performed with λ = 3.2 nm and DOF = 2 μm. Thus, the cylinder was much longer than the DOF. Figures 3(b)–3(d) present the projected images of the cylinder at three defocused positions −3 μm,0 μm and 3 μm. The white circles as shown in Fig. 3(b)–3(d) are the projections of the cylinder at different defocus distances. The local, enlarged views of projections in Figs. 3(e)–3(g) show that the boundaries of the projections are blurry, which indicates that parts of the sample are out of focus.

Fig. 3 (a) Cylinder with a length 8 μm, i.e., longer than the DOF, which was used as the sample in the simulation; (b) Projected image at 0°of the sample at 0 μm defocus distance; (c) Projected image at 0°of the sample at 3 μm defocus distance, and (d) Projected image at 0°of the sample at −3 μm defocus distance; (e), (f), and (g) Local, enlarged views of the projected images in (b), (c), and (d), respectively.

Download Full Size | PDF

As is well known, TXM is a transmission imaging method [23], which is different from visible light reflection imaging. Thus, the projection of a sample contains all of information about the sample in X-ray transmission imaging. If the sample is thicker than the DOF is long, the defocused information is superimposed with the focused information in the projection, which causes the 3D reconstruction data to contain defocused information. If the projected image is dense and superimposed, it is difficult to identify the defocused information to make it become focused. The current methods based on projection processing algorithms [6, 13] seem unable to deal with this issue. Thus, we developed a novel fusion method based on 2D stacks of sliced reconstruction data rather than projected images in this study. The reconstruction data-based method is not restricted by the defocused or focused parts of projected images.

The 3D reconstruction data were obtained at −3 μm, 0 μm, and 3 μm using the FBP reconstruction algorithm. Figure 4(a) depicts the 3D reconstruction of the cylinder that was acquired at 0 μm, and Fig. 4(b) is one slice of the reconstructed data. The left and right sides of the reconstructed cylinder are out of focus in both Figs. 4(a) and 4(b), which is in accordance with the cognitive pattern and simulated conditions. Figures 4(c) and 4(e) present the 3D reconstructions of the cylinder that were acquired at 3 μm and −3 μm, respectively, and Figs. 4(d) and 4(f) depict slices of the reconstructions in Figs. 4(c) and 4(e), respectively. The reconstructed slices of cylinder at 3 μm and −3 μm as shown in Figs. 4(c) and 4(e), respectively, are more blurry and twisty. When the cylinder is located at 3 μm or −3 μm, the defocused part of cylinder contains the defocused information from the 3D reconstruction data. The method proposed in this paper was used to deal with the defocused parts of the cylinder based on the reconstructed slices. Then, the fusion method was applied to obtain in-focus slices. The fused reconstruction and a slice of it are shown in Figs. 4(h) and 4(i), respectively. The fused results are all in focus, and the boundary of the cylinder is clear. These results indicate that the fusion method is effective and reliable. It should be noted that the fringing is clearly visible at the corners of the fusion reconstructed cylinder in Fig. 4(i). This fringing is caused by the FBP reconstruction algorithm. The projection along the boundary of the rectangle is not continuous, and the FBP reconstruction algorithm will produce singularity in filtering [25].

Fig. 4 (a) 3D reconstruction of the cylinder at 0 μm and (b) a slice of (a); (c) 3D reconstruction of the cylinder at 3 μm and (d) a slice of (c); (e) 3D reconstruction of the cylinder at −3 μm and (f) a slice of (e); (h) fully focused 3D reconstruction of the cylinder obtained using the proposed fusion method and (i) a slice of (h). The insets in each sub-figure show a magnified view of one part of reconstructed slice.

Download Full Size | PDF

To verify the performance of our method further, the multiple spatially distributed objects that are depicted in Fig. 5(a) were used as the sample to be simulated. Parts of the spheres and cubes are out of focus in this image. Projections were collected at 0 μm, 2.5 μm, and −2.5 μm and are presented in Figs. 5(b)–5(d), respectively. The sample is not entirely in focus in any of the projected images. It is difficult to distinguish between the focused and defocused parts since the data from those parts are projected together.

Fig. 5 (a) Sample used to illustrate the problem that occurs if the DOF is shorter than the sample is thick; (b) Projected image at 0°of the sample at 0μm defocus distance; (c) Projected image at 0°of the sample at 2.5 μm defocus distance, and (d) Projected image at 0°of the sample at −2.5 μm defocus distance. The insets in each sub-figure show a magnified view of two parts of projected image.

Download Full Size | PDF

Figure 6(a) depicts the 3D reconstruction of the multiple-object sample at 0 μm, and Fig. 6(b) is a slice of this reconstruction. The objects that are out of the focus are blurry. Moving the ZP 2.5 μm forward or backward along the optical axis caused the objects that were out of the focus to become more blurry, as shown in the 3D reconstructions in Figs. 6(c) and 6(e) and their respective slices in Figs. 6(d) and 6(f). Then, the reconstructed-slice-stack fusion method was applied to deal with the defocused information. Figures 6(h) and 6(i) present the fused result and its slice, respectively, and reveal that the previously blurry objects are clear, indicating that the fused result is entirely in focus.

Fig. 6 (a) 3D reconstruction of multiple objects at 0 μm and (b) a slice of (a); (c) 3D reconstruction of multiple objects at 2.5 μm and (d) a slice of (c); (e) 3D reconstruction of multiple objects at −2.5 μm and (f) a slice of (e); (h) fused 3D reconstruction of multiple objects and (i) a slice of (h). The insets in each sub-figure show a magnified view of one part of reconstructed slice.

Download Full Size | PDF

3.2 Experiment

To confirm the effectiveness of our method, a full tomographic data set including a focused stack at each tilt angle was acquired from the HZB Transmission X-ray Microscope in Berlin [21]. The sample was a pair of overlapping diatom frustules occupying an area of ~13μm × 8μm perpendicular to the rotation axis. The experimental conditions can be found in [6]. A total of 106 projections were collected at tilt-angles −52° to 53° at 1° increments with 1 s exposures. All of the images were corrected based on a reference image with a flat field intensityand aligned to the rotation axis. The stacks of projections data were collected along the optical axis in 1 μm increasement over a 10 μm range extending from −5 μm to 5 μm.

The 3D reconstructions were performed using FBP using the images with Δz = −5~5 μm (1 μm) that were obtained at each tilt angle. The 3D reconstructions with different Δz contained different focused and defocused information. Following the image fusion algorithm, the slices with the same location were first organized into stacks $i m g_{1}$ , $i m g_{2}$ , $i m g_{3}$ , …, $i m g_{s}$ of 3D reconstruction data and used as input. Each in-focus slice image $i m g_{i n - f o c u s}$ was obtained by using the image fusion algorithm. A series of slices of the frustule rib volume that was reconstructed by applying FBP perpendicular to the rotation axis is shown in Figs. 7(a)–7(c), and Fig. 7(d) presents the fused slice image. Because the reconstructions were derived from data obtained at limited angles, they contain many artifacts. Specifically, the artifacts resulted from the fact that the experimental tomographic data set included no data from a 74° wedge. Nevertheless, the fused slice image in Fig. 7(d) clearly reveals the raphe structures of the sample (indicated by the red and blue arrows) and offers a significant improvement in reconstruction quality over the original slices in Figs. 7(a)–7(c). Furthermore, Fig. 8 (Visualization 1) presents the 3D reconstruction of the diatom frustules that was obtained from the fused slice images and clearly depicts their raphe and pore structures.

Fig. 7 Sliced images viewed along the rotation axis that were obtained from the reconstructions of the diatom frustules at (a) −5 μm, (b) 0 μm, and (c) 5 μm. (d) Fused slice image viewed along the rotation axis. The each sub-figure (rigth) show a magnified view of one part of reconstructed slice.

Download Full Size | PDF

Fig. 8 The 3D reconstruction of the diatom frustules that was obtained from the fused slice images (see Visualization 1).

Download Full Size | PDF

4. Conclusion

In this paper, we proposed a method based on stacks of sliced images obtained along the optical axes of 3D reconstructions that can extend the DOF of TXM. Since this method is based on reconstructed data rather than on projections, it can be used to image complex samples, such as dense and superimposed objects. Both the theoretical and experimental data demonstrate that the proposed method can solve the limited-DOF problem in soft TXM. The possibility and limitations of the proposed method can be used to image large biological cell were discussed.

Funding

National Key Research and Development Program of China (2016YFA0400902); National Natural Science Foundation of China (NSFC) (11475175, 11405175, 11275204).

Acknowledgments

This work thanks HZB Transmission X-ray Microscope in Berlin for allocation of synchrotron radiation beam time and thanks data courtesy of Mårten Selin and the Biomedical and X-Ray Physics in Stockholm experimental data support.

References and links

1. A. Sakdinawat and D. Attwood, “Nanoscale X-ray imaging,” Nat. Photonics 4(12), 840–848 (2010). [CrossRef]

2. P. F. Guo, Y. Guan, G. Liu, Z. T. Liang, J. H. Liu, X. B. Zhang, Y. Xiong, and Y. C. Tian, “Modeling of gas transport with electrochemical reaction in nickel-yttria-stabilized zirconia anode during thermal cycling by Lattice Boltzmann method,” J. Power Sources 327, 127–134 (2016). [CrossRef]

3. Y. Guan, W. Li, Y. Gong, G. Liu, X. Zhang, J. Chen, J. Gelb, W. Yun, Y. Xiong, Y. Tian, and H. Wang, “Analysis of the three-dimensional microstructure of a solid-oxide fuel cell anode using nano X-ray tomography,” J. Power Sources 196(4), 1915–1919 (2011). [CrossRef]

4. M. Peuker, “High-efficiency nickel phase zone plates with 20 nm minimum outermost zone width,” Appl. Phys. Lett. 78(15), 2208–2210 (2001). [CrossRef]

5. D. Attwood, Soft X-Rays and Extreme Ultraviolet Radiation: Principles and Applications (Cambridge University, 2007).

6. M. Selin, E. Fogelqvist, S. Werner, and H. M. Hertz, “Tomographic reconstruction in soft x-ray microscopy using focus-stack back-projection,” Opt. Lett. 40(10), 2201–2204 (2015). [CrossRef] [PubMed]

7. S. Vogt, G. Schneider, A. Steuernagel, J. Lucchesi, E. Schulze, D. Rudolph, and G. Schmahl, “X-ray microscopic studies of the Drosophila dosage compensation complex,” J. Struct. Biol. 132(2), 123–132 (2000). [CrossRef] [PubMed]

8. C. Jacobsen, J. Kirz, and S. Williams, “Resolution in soft x-ray microscopes,” Ultramicroscopy 47(1-3), 55–79 (1992). [CrossRef]

9. C. Jacobsen, “Soft x-ray microscopy,” Trends Cell Biol. 9(2), 44–47 (1999). [CrossRef] [PubMed]

10. J. C. Andrews, E. Almeida, M. C. van der Meulen, J. S. Alwood, C. Lee, Y. Liu, J. Chen, F. Meirer, M. Feser, J. Gelb, J. Rudati, A. Tkachuk, W. Yun, and P. Pianetta, “Nanoscale X-ray microscopic imaging of mammalian mineralized tissue,” Microsc. Microanal. 16(3), 327–336 (2010). [CrossRef] [PubMed]

11. G. B. Kim, Y. J. Yoon, T. J. Shin, H. S. Youn, Y. S. Gho, and S. J. Lee, “X-ray imaging of various biological samples using a phase-contrast hard X-ray microscope,” Microsc. Res. Tech. 71(9), 639–643 (2008). [CrossRef] [PubMed]

12. Z. Liang, Y. Guan, S. Chen, and Y. Tian, “Whole cells imaged by hard X-ray transmission microscopy,” in Advanced Microscopy in Mycology (Springer, 2015), pp. 89–107.

13. Y. Liu, J. Wang, Y. Hong, Z. Wang, K. Zhang, P. A. Williams, P. Zhu, J. C. Andrews, P. Pianetta, and Z. Wu, “Extended depth of focus for transmission x-ray microscope,” Opt. Lett. 37(17), 3708–3710 (2012). [CrossRef] [PubMed]

14. Z. Liang, Y. Guan, G. Liu, R. Bian, X. Zhang, Y. Xiong, and Y. Tian, “Reconstruction of limited-angle and few-view nano-CT image via total variation iterative reconstruction,” Proc. SPIE 8851, 885113 (2013). [CrossRef]

15. H. Li, B. Manjunath, and S. K. Mitra, “Multisensor image fusion using the wavelet transform,” Graph. Models Image Proc. 57(3), 235–245 (1995). [CrossRef]

16. Y. Liu, S. P. Liu, and Z. F. Wang, “A general framework for image fusion based on multi-scale transform and sparse representation,” Inf. Fusion 24, 147–164 (2015). [CrossRef]

17. V. S. Petrović and C. S. Xydeas, “Gradient-based multiresolution image fusion,” IEEE Trans. Image Process. 13(2), 228–237 (2004). [CrossRef] [PubMed]

18. G. Pajares and J. Manuel de la Cruz, “A wavelet-based image fusion tutorial,” Pattern Recognit. 37(9), 1855–1872 (2004). [CrossRef]

19. A. Mansoor and A. B. Mansoor, “On image compression using digital curvelet transform,” in 2005 Pakistan Section Multitopic Conference (IEEE, 2005), pp. 1–4.

20. V. S. Petrović and C. S. Xydeas, “Gradient-based multiresolution image fusion,” IEEE Trans. Image Process. 13(2), 228–237 (2004). [CrossRef] [PubMed]

21. G. Schneider, P. Guttmann, S. Heim, S. Rehbein, F. Mueller, K. Nagashima, J. B. Heymann, W. G. Müller, and J. G. McNally, “Three-dimensional cellular ultrastructure resolved by X-ray microscopy,” Nat. Methods 7(12), 985–987 (2010). [CrossRef] [PubMed]

22. J. Otón, C. O. Sorzano, R. Marabini, E. Pereiro, and J. M. Carazo, “Measurement of the modulation transfer function of an X-ray microscope based on multiple Fourier orders analysis of a Siemens star,” Opt. Express 23(8), 9567–9572 (2015). [CrossRef] [PubMed]

23. C. Jacobsen, S. Williams, E. Anderson, M. T. Browne, C. J. Buckley, D. Kern, J. Kirz, M. Rivers, and X. Zhang, “Diffraction-limited imaging in a scanning transmission x-ray microscope,” Opt. Commun. 86(3-4), 351–364 (1991). [CrossRef]

24. J. W. Goodman, Statistical Optics (Wiley, 2000).

25. J. Frikel and E. T. Quinto, “Characterization and reduction of artifacts in limited angle tomography,” Inverse Probl. 29(12), 125007 (2013). [CrossRef]

Method for extending the depth of focus in X-ray microscopy

Abstract

Corrections

1. Introduction

2. Principles

3. Simulation and experiment

3.1 Simulation

3.2 Experiment

4. Conclusion

Funding

Acknowledgments

References and links

Supplementary Material (1)

Cited By

Figures (8)

Equations (5)

Optics Express