Holographic deconvolution microscopy for high-resolution particle tracking

Lisa Dixon; Fook Chiong Cheong; David G. Grier

doi:10.1364/OE.19.016410

In-line holographic video microscopy offers time-resolved information regarding the three-dimensional distribution of matter that scatters a beam of laser light [1, 2]. The scattered light interferes with the unscattered portion of the beam in the focal plane of an otherwise conventional microscope. A video camera then records the intensity of the magnified interference pattern with time resolution set by the exposure time and frame rate of the camera. Each holographic snapshot in the resulting video stream contains, in principle, comprehensive information regarding the three-dimensional distribution of matter.

Variants of this technique differ principally in how the holographic snapshots are analyzed. Fitting recorded holograms to results based on the exact theory of light scattering [3, 4] has been used to track colloidal spheres with nanometer resolution [3–5], to characterize individual spheres’ sizes, refractive indexes [3, 4, 6, 7], and porosities [8], and to detect molecular-scale coatings on micrometer-scale colloidal substrates [4, 6]. This wealth of information comes at a price. The nonlinear least-squares fits are computationally intensive, and must be targeted specifically for the kinds of samples being analyzed.

In-line holograms also may be analyzed more generally and more efficiently by numerically reconstructing the three-dimensional light field [1, 2, 9] with scalar diffraction integrals [10]. Such volumetric reconstructions do not commonly account for the physical properties of the scattering medium. Rather, the medium is treated as an isotropic and homogeneous dielectric. Bright features in the reconstruction then are identified with discrete objects in the sample [1,2, 5], particularly those that are smaller than the wavelength of light [5]. Dynamical measurements based on tracking such features through holographic video sequences have demonstrated 10 nm resolution for three-dimensional location of colloidal spheres [5] and nanorods [11], albeit with large systematic axial offsets in some cases [5].

Features in holographic reconstructions can be emphasized by deconvolving the reconstructed light distribution with the volumetric point-spread function for the reconstruction process [12, 13]. Here, we demonstrate and quantitatively assess the tracking resolution of holographic deconvolution microscopy using the Rayleigh-Sommerfeld diffraction integral both to reconstruct the volumetric intensity distribution and also to deconvolve it. For micrometer-scale colloidal spheres, the results suggest typical in-plane resolution approaching 1 nm and axial resolution of 10 nm. Three-dimensional tracking of metal-oxide nanorods yields comparably good results for locating the center of mass, and 1° orientation resolution in three dimensions.

Our in-line holographic microscope has been described previously [2]. It consists of a commercial inverted microscope stand (Nikon TE2000U) outfitted with a high-numerical-aperture oil-immersion objective (Nikon Plan-Apo, 100×, NA 1.4). The conventional illuminator is replaced with the collimated beam from a fiber-coupled diode laser (Coherent Cube) operating at a vacuum wavelength of 445 nm. The linearly polarized beam has a total power of 15 mW spread uniformly over 9 mm². Holographic images are captured by a low-noise gray-scale video camera (NEC TI 324A-II) and are recorded as an uncompressed digital video stream at 30 frames per second with a total system magnification of 135 nm/pixel. The camera’s 0.5 ms exposure time is fast enough to avoid measurable effects of motion blurring [4, 14–16].

The incident plane wave,

E_{0} (r, z) = E_{0} (r) e^{i k z} {\hat{ɛ}}_{0},

is assumed to propagate along ẑ with an real-valued amplitude E ₀(r) that may depend on position r = (x,y) in the transverse plane, and uniform polarization $\hat{ɛ}$ ₀. The scattered wave,

E_{S} (r, z) = E_{S} (r, z) \hat{ɛ} (r, z)

propagates in three dimensions with complex amplitude E_S(r, z) and spatially varying polarization $\hat{ɛ}$ (r, z). Their superposition in the focal plane yields the interference pattern

I (r) = {| E_{0} (r, 0) + E_{S} (r, 0) |}^{2}

= E_{0}^{2} (r) + 2 R e {E_{0} (r) E_{S} (r, 0) {\hat{ɛ}}_{0}^{*} \cdot \hat{ɛ} (r, 0)} + {| E_{S} (r, 0) |}^{2} .

Deliberately placing the scatterer well above the focal plane ensures both that polarization rotations are small,

{\hat{ɛ}}_{0}^{*} \cdot \hat{ɛ} (r, 0) \approx 1

, and also that the scattered wave is substantially less intense than the illumination. Normalizing by the illumination’s intensity distribution

I_{0} (r) = E_{0}^{2} (r)

then yields

b (r) \equiv \frac{I (r)}{I_{0} (r)} - 1 \approx 2 R e {E_{R} (r, 0)},

where the reduced scattered field is E_R(r, z) = E_S(r, z)/E ₀(r). Dropping |E_R(r, 0)|² from the definition of b(r) simplifies the analysis that follows at the cost of ignoring interference due to multiple scatterers. It therefore limits the complexity of the samples to which this formalism may be applied. In practice, the background image, I ₀(r), can be obtained either by moving the sample out of the field of view, or by taking a running median filter of a time-evolving sample.

If the complex scattered field were completely specified in the focal plane, it could be reconstructed at height z as the convolution

E_{R} (r, z) = E_{R} (r, 0) \otimes h (r, - z)

of the scattered amplitude in the focal plane with the Rayleigh-Sommerfeld propagator [10]

h (r, - z) = \frac{1}{2 π} \frac{\partial}{\partial z} \frac{e^{i k R}}{R},

where R ² = r ² + z ². The sign convention for z accounts for the object’s position upstream of the focal plane. Equation (6) may be rewritten with the Fourier convolution theorem as

{\tilde{E}}_{R} (q, z) = {\tilde{E}}_{R} (q, 0) H (q, - z),

where

{\tilde{E}}_{R} (q, z) = \int_{- \infty}^{\infty} E_{R} (r, z) e^{- i q \cdot r} d^{2} r

is the in-plane Fourier transform of E_R(r, z) and where

H (q, - z) = e^{i z {(k^{2} - q^{2})}^{\frac{1}{2}}}

is the Fourier transform of h(r, –z) [10, 17, 18]. To use this formalism to reconstruct the scattered field, we note that the Fourier transform of b(r) is

B (q) \approx {\tilde{E}}_{R} (q, 0) + {\tilde{E}}_{R}^{*} (q, 0) .

From this,

B (q) H (q, - z) = {\tilde{E}}_{R} (q, z) + {\tilde{E}}_{R}^{*} (q, - z)

may be recognized as the superposition of the scattered field at height z above the focal plane and a spurious field due to the object’s mirror image in the focal plane, which is known as the twin image. The twin image’s influence on the reconstructed field may be reduced by moving the sample away from the focal plane.

In the additional approximation that the illumination is uniform, the reconstructed field is

E_{R} (r, z) \approx \frac{e^{- i k z}}{4 π^{2}} \int_{- \infty}^{\infty} B (q) H (q, - z) e^{i q \cdot r} d^{2} q .

The associated intensity, I_R(r, z) = |E_R(r, z)|², is an estimate for the scattered light’s intensity at height z above the focal plane. This reconstruction differs from a numerically refocused image [1] because it comprises only the scattered field and not its superposition with the incident field.

The reconstruction also is not simply a representation of the object responsible for the scattered field, or even of the scattered field itself. The projection operation in Eq. (6) is formulated for light propagating through a homogeneous medium, and so does not account for the optical properties of any objects in the medium. The reconstructed field thus includes artifacts upstream of each scattering center in the sample. These artifacts and the twin image of the reconstructed field are both clearly visible in the volumetric reconstructions of colloidal spheres and a copper-oxide nanorod [11, 19] in Figs. 1(a) and 1(b), respectively. These artifacts, together with out-of-focus images tend to obscure features of interest in the reconstructed volume. Neither the nature nor the number of the scattering objects is immediately evident from the reconstructions in Fig. 1(a) or Fig. 1(b). These sources of blurring therefore hinder efforts to identify and track colloidal particles.

Fig. 1 (a) Normalized holographic video image of a three-dimensional arrangement of nine colloidal silica spheres in water together with the volumetric reconstruction obtained with Eq. (13). (b) Hologram and reconstruction of a copper-oxide nanorod diffusing freely in water. (c) Deconvolved reconstruction of the hologram in (a), obtained with Eq. (14). (d) (Media 1) Deconvolved reconstruction of the nanorod in (b). The same color and opacity table is used to visualize all four reconstructions.

Download Full Size | PDF

Most of these distractions can be removed by deconvolving the reconstructed data with the point spread function used for the reconstruction. Previous reports [12, 13] have used point-spread functions for particular objects in an effort to optimize detection resolution. Our deconvolution scheme instead is motivated by the observation that the Rayleigh-Sommerfeld formalism transforms an ideal point-like focal caustic [20] at height z_p above the focal plane into the three-dimensional intensity distribution K(r, z – z_p) = |h(r, z – z_p)|². The caustic’s twin image, similarly, is projected into K(r, z + z_p). Deconvolving with K(r, z) therefore should reduce the full three-dimensional scattering pattern reconstructed from the hologram to the set of focal caustics created by the sample. Twin images also will be reduced to focal caustics, but on the other side of the effective focal plane, which will not be reconstructed, and so will not be seen. Using the Rayleigh-Sommerfeld point-spread function in this way also eliminates the need for calibration measurements [12, 13].

Deconvolving I_R(r, z) with K(r, –z) is most easily implemented with the Fourier convolution theorem:

{\tilde{I}}_{D} (ρ) = \frac{{\tilde{I}}_{R} (ρ)}{\tilde{K} (ρ) + χ},

where Ĩ_R(ρ) is the three-dimensional Fourier transform of I_R(r, z), K̃(ρ) is the Fourier transform of K(r, –z), and χ is a small factor chosen for numerical stability. The deconvolved intensity, I_D(r, z) is obtained as the inverse Fourier transform of Ĩ_D(ρ). Figures 1(c) and 1(d) (Media 1) were obtained in this way from the data in Figs. 1(a) and 1(b), respectively, and are rendered with the same color and transparency tables. Blurring from artifacts and out-of-focus images is strongly suppressed in the deconvolved volumes, leaving distinct and well-resolved bright features that we associate with the focal caustics created by the samples. For example, the sample in Fig. 1(c) now can be recognized to consist of 9 colloidal spheres arranged in a body-centered cubic lattice, while that in Fig. 1(d) is quite clearly an inclined rod. These results were obtained with χ = 0.01, and comparably good contrast and resolution were obtained for 0.005 ≤ χ < 0.05.

The three-dimensional positions of the focal caustics can be recovered by analyzing moments of the three-dimensional intensity distribution [21]. For objects with dimensions comparable to the wavelength of light, furthermore, focal caustics should lie substantially on the objects themselves. This suggests that bright features in the deconvolved reconstruction also track the shape, size, position and orientation of such objects [5, 11]. A similar analysis is used to track objects in a volume deconvolved with a targeted point-spread function [12]. This approach requires the point-spread function to measured for the target object, or else to be computed based on a priori knowledge.

Tracking Brownian objects as they diffuse enables us to measure the accuracy and resolution attainable with holographic deconvolution microscopy [5, 11, 14, 15, 21]. The apparent mean-square displacement of a colloidal sphere freely diffusing along r̂_j,

Δ r_{j}^{2} (τ) = {〈 {[r_{j} (t + τ) - r_{j} (t)]}^{2} 〉}_{t} = 2 D_{j} τ + 2 ɛ_{j}^{2}

increases linearly with time according to the familiar Einstein-Smoluchowski result, but is off-set by the measurement error ɛ_j for centroid location along that axis [21]. In this, we assume that the camera’s shutter is fast enough that motion blurring may be ignored [14–16]. The data in Fig. 2(a) were obtained for a 0.8 μm diameter colloidal polystyrene sphere (Bangs Laboratories, lot number 912) diffusing in water at room temperature, T = 296 K. The three single-axis diffusion coefficients, D_j = 0.615 ± 0.157 μm²/s, (j = x,y,z) are consistent with each other and also are consistent with the Stokes-Einstein result D = k_BT/(6πηa_p) given η = 0.89 cP and using the radius a_p = 0.428 ± 0.001 μm obtained from Lorenz-Mie characterization [3]. Consistency among the three diffusion coefficients confirms that the sphere was far enough from the walls of the sample container that hydrodynamic coupling may be ignored. The fit values for the centroid tracking errors, ɛ_x = 6 ± 1 nm, ɛ_y = 7 ± 1 nm and ɛ_z = 10 ± 1 nm, are a factor of two smaller than the corresponding errors obtained with Rayleigh-Sommerfeld reconstruction without deconvolution [5], and are a factor of two larger than those obtained with Lorenz-Mie fits to the same holograms [5].

Fig. 2 (a) Mean-square displacement of a colloidal silica sphere obtained with holographic deconvolution tracking, together with a least-squares fit to Eq. (15). The inset shows the particle’s measured trajectory over 3 minutes. (b) Mean-squared displacement of the orientational unit vector of the diffusing copper-oxide nanorod from Fig. 1, together with a least-squares fit to Eq. (16). The inset shows the positions visited by ŝ(t) on the unit sphere, and are colored by time according to the rainbow color table. (c) Mean-squared displacement of the nanorod’s center of mass along (||) and transverse (⊥) to its instantaneous orientation, together with the predicted Einstein-Smoluchowski dependence on lag time τ. The inset (Media 2) shows 3 minutes of the nanorod’s diffusion as a ribbon tracing out r(t) and oriented along ŝ(t). Colors track orientation. (d) Hologram and deconvolved reconstruction of a lithographically patterned colloid in the form of the letter “X”. Inset: Conventional bright-field image of a similar particle lying flat in the focal plane.

Download Full Size | PDF

Although both the Rayleigh-Sommerfeld reconstruction and the subsequent deconvolution are numerically intensive, they can be less costly than Lorenz-Mie fitting, particularly when multiple particles are in the field of view. Our implementation in the IDL programming language requires 30 seconds to locate the 9 spheres in Fig. 1(c) through fits to the Lorenz-Mie theory [5] on a multi-core processor running at 3 GHz, but only 20 seconds for the present approach. Unlike fitting to Lorenz-Mie theory, holographic deconvolution does not yield high-resolution measurements of sphere radius or refractive index. The estimated axial position, furthermore, includes a systematic offset that depends on the size and composition of the object [5]. This complicates measurements of three-dimensional separations. Any such concerns are offset, however, by the generality of holographic deconvolution microscopy, which requires no foreknowledge of an object’s shape or composition.

For instance, the nanorod’s deconvolved reconstruction in Fig. 1(d) (Media 1) features a roughly cylindrical volume of length L = 5.0 ± 0.3 μm, whose length remains unchanged through a sequence of 5,000 snapshots as the nanorod rotates in three-dimensions. It is tempting to identify the center of this cylinder with the nanorod’s position, and the orientation of its axis with the nanorod’s orientation. Figure 2(b) shows the mean-squared displacement

Δ s^{2} (τ) = {〈 {| \hat{s} (t + τ) - \hat{s} (t) |}^{2} 〉}_{t}

of the orientational unit vector ŝ(t) obtained in this way. The nanorod’s apparent orientation and center-of-mass position r(t) are extracted from the deconvolved reconstruction using a peak-tracking skeletonization algorithm originally developed [11] to analyze volumetric reconstructions of the type presented in Fig. 1(b). The solid curve in Fig. 2(b) is a fit to

Δ s^{2} (τ) = 2 [1 - (1 - ɛ_{s}^{2}) exp (- 2 D_{r} τ)]

for the rod’s rotational diffusion coefficient D_r = 0.094 ± 0.009 s⁻¹ and the measurement error ɛ_s = sin Δθ = 0.036 ± 0.020 in the nanorod’s three-dimensional orientation. The latter figure is consistent with an error of Δθ = 1° in the rod’s orientation, which improves on previous results by more than a factor of two.

The nanorod’s translational diffusion is difficult to analyze in the laboratory frame because its viscous drag coefficient depends on its orientation, which is always changing [22]. In the co-rotating frame, however, its mean-squared displacement should evolve linearly in time,

Δ r_{| |}^{2} (τ) = 2 D_{| |} τ + 2 ɛ_{| |}^{2} and

Δ r_{⊥}^{2} (τ) = 4 D_{⊥} τ + 4 ɛ_{⊥}^{2},

when projected along and transverse to the measured orientations, respectively. The results plotted in Fig. 2(c) are consistent with this interpretation, and yield D _|| = 0.411 ± 0.003 μm²/s and D _⊥ = 0.241 ± 0.012 μm²/s.

The three diffusion coefficients for a freely diffusing rod,

D_{r} = \frac{3 k_{B} T}{π η L^{3}} [ln (\frac{L}{σ}) - γ],

D_{| |} = \frac{k_{B} T}{2 π η L} [ln (\frac{L}{σ}) - γ] and

D_{⊥} = \frac{k_{B} T}{4 π η L} [ln (\frac{L}{σ}) + γ],

depend on the rod’s length L and diameter σ, and also on a geometric factor γ [23]. From these, we obtain L = 5.12 ± 0.025 μm, which is consistent with the optical measurement, σ = 241 ± 12 nm, and γ = 0.25. The associated tracking error for the rod’s centroid is 10 nm in the plane and 50 nm along the optical axis, averaged over orientations.

The success of holographic deconvolution microscopy for tracking colloidal spheres and nanorods with nanometer resolution does not necessarily extend to more general objects, particularly those with dimensions larger than the wavelength of light. Equation (14) ignores interference due to light emanating from multiple sources. Provided the scattering centers are separated by multiple wavelengths of light, however, interference effects manifest themselves principally in the dimmer regions of the reconstructed intensity distribution, well away from interesting intensity maxima. When these conditions are not met, however, Eq. (14) can introduce artifacts of its own. The data in Fig. 1(d), for example, show the reconstruction of a lithographically defined colloidal particle in the form of the letter “X” [24]. Crafted from polymethylmethacrylate, this dielectric particle has features roughly one micrometer across, and thus scatters the incident plane wave illumination into a pattern that depends strongly on the particle’s orientation. The deconvolved volumetric reconstruction emphasizes caustics in that scattering pattern, rather than the shape itself, and so its true shape is not readily apparent. The same will be true of more highly structured samples such as biological cells. The neglect of the scattered intensity in the derivation of Eq. (5) also may have contributed to a loss of definition.

When applied with appropriate care, deconvolving the Rayleigh-Sommerfeld reconstruction of an in-line hologram with the Rayleigh-Sommerfeld point-spread function yields precise and accurate information for three-dimensional particle tracking. This technique excels for highly symmetric colloids such as spheres whose caustics precisely track the particle’s position, albeit with an axial offset [5]. Objects such as nanorods that have dimensions smaller that the wavelength of light similarly can be tracked with nanometer-scale resolution. Because the projected caustic coincides with the physical position of the object in such cases, the measurement’s accuracy can approach its precision. For all such systems, deconvolution with the Rayleigh-Sommerfeld point-spread function greatly simplifies three-dimensional tracking by de-emphasizing extraneous features of the three-dimensional scattering pattern in favor of its singularities. The measurements on model samples that we have presented confirm that deconvolution can substantially enhance tracking resolution compared with reconstruction alone [5].

Acknowledgments

We are grateful to Tom Mason for providing a sample of LithoParticles. This work was supported in part by the QORS Program of DARPA and in part by the National Science Foundation through grant DMR-0922680.

References and links

1. J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt. 45, 3893–3901 (2006). [CrossRef] [PubMed]

2. S.-H. Lee and D. G. Grier, “Holographic microscopy of holographically trapped three-dimensional structures,” Opt. Express 15, 1505–1512 (2007). [CrossRef] [PubMed]

3. S.-H. Lee, Y. Roichman, G.-R. Yi, S.-H. Kim, S.-M. Yang, A. van Blaaderen, P. van Oostrum, and D. G. Grier, “Characterizing and tracking single colloidal particles with video holographic microscopy,” Opt. Express 15, 18275–18282 (2007). [CrossRef] [PubMed]

4. F. C. Cheong, B. Sun, R. Dreyfus, J. Amato-Grill, K. Xiao, L. Dixon, and D. G. Grier, “Flow visualization and flow cytometry with holographic video microscopy,” Opt. Express 17, 13071–13079 (2009). [CrossRef] [PubMed]

5. F. C. Cheong, B. J. Krishnatreya, and D. G. Grier, “Strategies for three-dimensional particle tracking with holographic video microscopy,” Opt. Express 18, 13563–13573 (2010). [CrossRef] [PubMed]

6. F. C. Cheong, K. Xiao, and D. G. Grier, “Characterization of individual milk fat globules with holographic video microscopy,” J. Dairy Sci. 92, 95–99 (2009). [CrossRef]

7. K. Xiao and D. G. Grier, “Multidimensional optical fractionation with holographic verification,” Phys. Rev. Lett. 104, 028302 (2010). [CrossRef] [PubMed]

8. F. C. Cheong, K. Xiao, D. J. Pine, and D. G. Grier, “Holographic characterization of individual colloidal spheres’ porosities,” Soft Matter (to be published), DOI:. [CrossRef]

9. B. Rappaz, P. Marquet, E. Cuche, Y. Emery, C. Depeursinge, and P. J. Magistretti, “Measurement of the integral refractive index and dynamic cell morphometry of living cells with digital holographic microscopy,” Opt. Express 13, 9361–9373 (2005). [CrossRef] [PubMed]

10. J. W. Goodman, Introduction to Fourier Optics, 3rd ed. (McGraw-Hill, 2005).

11. F. C. Cheong and D. G. Grier, “Rotational and translational diffusion of copper oxide nanorods measured with holographic video microscopy,” Opt. Express 18, 6555–6562 (2010). [CrossRef] [PubMed]

12. T. Latychevskaia, F. Gehri, and H.-W. Fink, “Depth-resolved holographic reconstructions by three-dimensional deconvolution,” Opt. Express 21, 22527–22544 (2010). [CrossRef]

13. Y. Cotte, M. F. Toy, N. Pavillon, and C. Depeursinge, “Microscopy image resolution improvement by deconvolution of complex fields,” Opt. Express 18, 19462–19478 (2010). [CrossRef] [PubMed]

14. T. Savin and P. S. Doyle, “Role of finite exposure time on measuring an elastic modulus using microrheology,” Phys. Rev. E 71, 041106 (2005). [CrossRef]

15. T. Savin and P. S. Doyle, “Static and dynamic errors in particle tracking microrheology,” Biophys. J. 88, 623–638 (2005). [CrossRef]

16. L. Dixon, F. C. Cheong, and D. G. Grier, “Holographic particle-streak velocimetry,” Opt. Express 19, 4393–4398 (2011). [CrossRef] [PubMed]

17. G. C. Sherman, “Application of the convolution theorem to Rayleigh’s integral formulas,” J. Opt. Soc. Am. 57, 546–547 (1967). [CrossRef] [PubMed]

18. U. Schnars and W. P. O. Jüptner, “Digital recording and reconstruction of holograms,” Meas. Sci. Technol. 13, R85–R101 (2002). [CrossRef]

19. T. Yu, C. H. Sow, A. Gantimahapatruni, F. C. Cheong, Y. W. Zhu, K. C. Chin, X. J. Xu, C. T. Lim, Z. X. Shen, J. T. L. Thong, and A. T. S. Wee, “Patterning and fusion of CuO nanorods with a focused laser beam,” Nanotechnology 16, 1238–1244 (2005). [CrossRef]

20. J. F. Nye, Natural Focusing and Fine Structure of Light (Institute of Physics Publishing, 1999).

21. J. C. Crocker and D. G. Grier, “Methods of digital video microscopy for colloidal studies,” J. Colloid Interface Sci. 179, 298–310 (1996). [CrossRef]

22. Y. Han, A. M. Alsayed, M. Nobili, J. Zhang, T. C. Lubensky, and A. G. Yodh, “Brownian motion of an ellipsoid,” Science 314, 626–630 (2009). [CrossRef]

23. M. Doi and S. F. Edwards, The Theory of Polymer Dynamics (Clarendon Press, 1986).

24. C. J. Hernandez and T. G. Mason, “Colloidal alphabet soup: Monodisperse dispersions of shape-designed LithoParticles,” J. Phys. Chem. 111, 4477–4480 (2007).

Holographic deconvolution microscopy for high-resolution particle tracking

Abstract

Acknowledgments

References and links

Supplementary Material (2)

Cited By

Figures (2)

Equations (22)

Optics Express