1. Introduction

Holograms of colloidal spheres obtained with holographic video microscopy [1,2] can be interpreted with predictions of the Lorenz-Mie theory of light scattering [3] to track each particle in three dimensions, and to measure its size and refractive index [4]. State-of-the-art implementations [4–7] can locate a sphere and resolve its radius both to within a few nanometers, and can determine its refractive index to within a part per thousand [8–10]. The cost of this powerful technique is the computational burden of fitting each hologram pixel-by-pixel to theoretical predictions [4, 11]. Here, we demonstrate that techniques of machine learning can reduce the processing time by a factor of a thousand, yielding real-time performance.

Our approach to fast holographic characterization, depicted schematically in Fig. 1, employs the support vector machine (SVM) algorithm [12] to compare experimental measurements with pre-computed predictions of the Lorenz-Mie theory [3, 4, 13]. Whereas nonlinear fitting typically requires more than a second on a 1 Gflop computer, a trained SVM can estimate the size, refractive index or axial position of a micrometer-scale sphere in under a millisecond on the same hardware.

 

Fig. 1 Colloidal characterization by holographic microscopy and machine learning. Colloidal spheres flowing down a microfluidic sample scatter light from a collimated laser beam to form an in-line hologram. Features in the beam are identified, and their radial profiles presented to support vector machines (SVMs) that compare them with a library of training data to estimate each spheres’ radius a_p and refractive index n_p. The scatter plot shows results for 2,500 spheres drawn at random from a mixture of four different types of spheres. Each point is colored by the local density of data points, ρ(a_p, n_p).

Download Full Size | PDF

2. Fast holographic characterization with machine learning

The in-line holographic microscope used for these studies [2,4,10] illuminates the sample with a linearly polarized collimated laser beam (Coherent Cube, 20 mW) at a vacuum wavelength of λ = 447nm. The fluence of the 3 mm-diameter beam is comparable to that of a conventional microscope illuminator. Optical forces and light-induced heating therefore are negligible. Light scattered by a sphere propagates to the focal plane of a custom-built video microscope [10] where it interferes with the undiffracted portion of the original beam. The microscope magnifies this interference pattern onto the detector of a greyscale video camera (NEC TI 324AII), which records its intensity with a system magnification of 135 nm/pixel. Each snapshot in the video stream constitutes a hologram of the particles in the channel.

The electric field of the incident beam at position r in the focal plane may be modeled as a plane wave with spatial dependence

A particle located at r_p relative to the center of the focal plane scatters the incident illumination, E₀(r_p), to the focal plane as

Previous implementations of Lorenz-Mie microscopy [4] fit Eq. (4) to measured holograms using a_p, n_p and r_p as adjustable parameters. These fits are exquisitely sensitive to errors in the particle’s in-plane position, and so must be performed over the entire two-dimensional intensity distribution [4]. Here, we instead use Eq. (4) to train support vector machines, which then are able to estimate a_p, n_p and z_p from a hologram’s one-dimensional radial profile. We obtain these profiles from measured holograms by averaging around centers of rotational symmetry [13] with single-pixel resolution, yielding 100-point data vectors. The microscope’s focal plane is adjusted so that the interference fringes in a typical sphere’s hologram extend to roughly this scale. Averaging over angles to obtain a radial profile reduces the dimensionality of the analysis problem and accounts in part for our method’s computational efficiency.

Our SVMs are implemented with scikit-learn, an open-source machine learning software package [15] that builds upon the libsvm library of Support Vector Machine algorithms [16, 17]. Each SVM computes one output parameter from an input vector consisting of a radial profile, b(r), that has been digitized into 100 single-pixel bins. Characterizing and tracking a colloidal particle therefore requires three SVMs, one for each of a_p, n_p and z_p. Figure 1 schematically represents this process for estimating a_p and n_p.

An SVM computes its output by comparing b(r) with sets of training data, b_n(r), that are obtained from Eq. (4) over a range of values of a_p, n_p and z_p. Each training set constitutes one support vector in the space spanned by these parameters. To facilitate these comparisons, we construct SVMs with radial basis functions [12],

To prevent overfitting, the weights ω_n are constrained to have magnitudes less than a maximum value, which conventionally is denoted by C [12]. Optimizing an SVM with larger values of C improves its ability to recognize its training data, but renders it less able to interpolate smoothly between its support vectors when presented with novel or noisy inputs. Some candidate support vectors may be assigned small weighting factors in optimizing s̃(b) over a corpus of training data; these are automatically eliminated from the SVM [12]. The choice of γ and C thus determines which support vectors are included in the SVM, and their relative importance for computing the output. Because this process is nonlinear, optimal values are obtained by exhaustive search over the range 0.1 ≤ γ ≤ 10 and 0.1 ≤ C ≤ 1, 000. Statistically indistinguishable results are obtained in the present application for values of γ and C that vary from their optimal values by up to fifty percent.

We trained SVMs with a 5,000-member training set of radial profiles, b_n(r), computed with Eq. (4) using the Lorenz-Mie theory. Parameters for these computed profiles were evenly distributed over a volume in the three-dimensional space spanned by 13.5μm ≤ z_p ≤ 75μm, 0.4μm ≤ a_p ≤ 1.75μm, and 1.4 ≤ n_p ≤ 1.8 at a resolution of 1.35 μm in z_p, 0.1 μm in a_p and 0.1 in n_p. Training time increases dramatically with the number of training sets, and also with larger values of C and γ. Once trained, however, an SVM translates input data vectors into output parameter estimates extremely rapidly.

The quality of a trained SVM can be assessed by presenting it with independently computed cross-validation data. Optimal values for C and γ minimize differences between estimated parameters and the inputs. Using a 500-member cross-validation set, we obtained best performance for estimating z_p with C = 100 and γ = 1, best performance for n_p with C = 10 and γ = 0.5, and best performance for a_p with C = 10 and γ = 0.6.

Sampling the entire parameter space accessible to holographic characterization with resolution comparable to the precision realized with nonlinear fits [4] would require more than 10¹⁰ training sets. If, however, the system of interest is characterized by a more modest range of parameters, then results from an initial SVM analysis can be used to generate a comparatively small set of training data spanning the relevant range. This specialized training proceeds rapidly and yields substantial improvements in precision.

3. Characterization of colloidal mixtures

The data plotted in Fig. 1 are SVM estimates for the radii and refractive indexes of 2,500 colloidal spheres flowing down a 20 μm-deep microfluidic channel formed by bonding the edges of a glass microscope cover slip to the surface of a glass microscope slide. The peak flow speed of 1 mm/s transports a sphere across the field of view in no fewer than two video frames, ensuring that every particle in the flow has a chance to be analyzed. Anisotropic blurring due to a sphere’s 100 nm motion during the camera’s 0.1 ms exposure time suppresses contrast along the direction of motion, but does not appreciably influence the azimuthal average, b(r) [18]. Spurious results arising when multiple spheres’ interference patterns overlap contribute outliers to the observed distribution of particle sizes and refractive indexes. Such artifacts are minimized by diluting the sample to a volume fraction less than 10⁻³ so that no more than three particles are present in any frame.

The sample was prepared by dispersing roughly equal proportions of four types of colloidal spheres in water: 1 μm-diameter and 2 μm-diameter spheres made of polystyrene (Thermoscientific Catalog No. 5100A and Bangs Laboratories Catalog No. SS04N, respectively) and silica (Duke Standards Catalog Nos. 8100 and 4202A, respectively). Each of these four populations is monodisperse, with a standard deviation in the radius of less than 5%. This four-component mixture was flowed through the observation volume during a 12 min interval, and analyzed particle-by-particle. Each data point in Fig. 1 corresponds to an individual sphere, and is colored by the local density of measurements.

SVM-mediated holographic characterization clearly identifies the four populations of particles and provides estimates for their relative abundances. The mode values of the refractive indexes, n_p ≈ 1.4 and 1.6 are consistent with values for silica and polystyrene, respectively. Values for the radii, a_p ≈ 0.5μm and 1 μm are consistent with manufacturers’ specifications. Characterizing multicomponent dispersions is a unique capability of holographic particle analysis, and can be performed with SVMs as fast as particles’ holograms can be acquired. This performance is all the more noteworthy because the distribution of particle properties is built up one particle at a time and so does not rely on a priori assumptions.

Neither the instrument nor the analytical technique requires extensive calibration. The wavelength of the laser and the effective magnification can be calibrated once and used for all samples. The refractive index of the medium is the only free parameter, and often can be obtained separately. These parameters are used to train the SVMs in advance, after which they can be used to analyze arbitrary samples dispersed in the medium.

4. Tracking and assessment of precision

Tracking a single colloidal sphere as it sediments and diffuses provides insights into the precision and accuracy of SVM-mediated holographic characterization. The data in Fig. 2 were obtained with a 1.59 μm-diameter polystyrene sphere (Duke Scientific, catalog 4016A) diffusing as it sedimented through deionized water near the midplane of a 120 μm-deep channel. Figure 2(a) shows the time-resolved trajectory, z_p(t), obtained from a sequence of 4,500 video frames recorded at 29.97 frames/s using iterative SVM training.

 

Fig. 2 Tracking and characterizing a single colloidal sphere. (a) The estimated axial position z_p(t) relative to the focal plane of the microscope of a single polystyrene sphere sedimenting through water as it diffuses. The line is a least-squares fit. Insets show the sphere’s hologram at the beginning and end of the trajectory. (b) The radius a_p and refractive index n_p estimated from each hologram in the same sequence, colored by time. Each dot corresponds to values obtained from a single hologram. (c) The mean-squared displacement, $\mathrm{\Delta }{z}_p^2(\tau )$ as a function of lag time τ computed from the data in (a), including statistical error bars. The superimposed line is a fit to Eq. (7).

Download Full Size | PDF

Because polystyrene is roughly 5 percent more dense than water, the sphere sediments more that 10 μm over the course of the experiment. The insets to Fig. 2(a) show how markedly the hologram’s appearance changes from the beginning of the trajectory to the end. Despite these changes, the SVMs’ estimates for the radius and refractive index plotted in Fig. 2(b) remain clustered around the mean values a_p = 0.79 ± 0.02μm and n_p = 1.600 ± 0.006.

Uncertainties in estimated parameters are computed as standard deviations of the distribution of results plotted in Fig. 2(b). These should be interpreted with care because errors in SVM estimates need not be independent or normally distributed. Data points in Fig. 2(b) cluster around different values as the particle descends, which suggests that different support vectors dominate the estimates for a_p and n_p when the sphere is at different axial positions. Systematic errors in the individual parameters therefore may vary with changes in any of the parameters’ values. Even so, the averages of the SVM estimates are consistent with the manufacturer’s specifications, and differ only slightly from those obtained with a full Lorenz-Mie analysis of the same data set [10], which yields a_p = 0.805 ± 0.001μm and n_p = 1.5730 ± 0.0006. Nonlinear fitting offers ten times better precision and accuracy [10]. SVM analysis is a thousand times faster.

The mean sedimentation speed, v_p = 66 ± 1nm/s, estimated from the slope of z_p(t) is somewhat smaller than the value measured with fits to the Lorenz-Mie theory [10] of 75 ± 1nm/s. This discrepancy further suggests that the SVM estimate for a parameter’s value may depend on the value itself. If we nevertheless assume that errors in z_p are normally distributed with a root-mean-square value ε_z, then the diffusing particle’s mean-squared displacement should evolve over time interval τ as

5. Conclusions

The results presented here are typical of the performance of SVMs for characterizing and tracking colloidal spheres of a wide variety of compositions and sizes over a large axial range. The speed and precision of SVM characterization is ideal for monitoring, feedback control and quality assurance in any industrial process involving colloidal spheres. In applications where the greatest precision is required, parameters estimated with SVMs can be used to initialize nonlinear least-squares fits. Starting from such reasonable estimates speeds convergence and reduces the likelihood of failed fits. Being able to resolve multimodal distributions by quickly amassing single-particle measurements avoids ambiguities inherent in population-averaging methods such as dynamic light scattering. Extracting the refractive index as well as the size offers insights into sample composition that otherwise would not be available. SVM-accelerated tracking can be used for real-time three-dimensional particle-tracking velocimetry [8]. For applications such as microrefractometry [9], the medium’s refractive index, n_m, can be estimated instead of the particle’s.

This combination of capabilities enables new applications. For example, the distribution of properties in colloidal mixtures could serve as fingerprints for complex fluid systems, with the sizes, refractive indexes and relative abundances encoding information that can be accessed with SVM-mediated holographic characterization.

Such applications can be realized with comparatively simple instruments [10] conveying image data to low-power computers. Although training SVMs can be computationally intensive, the data comprising a set of trained SVMs occupies less than 100 Mbyte. Pre-computed SVMs therefore can be archived and rapidly retrieved when needed. This approach lends itself to implementation on embedded computers for integration into low-cost analytical instruments.

Other machine-learning techniques also might be effective for analyzing holograms of colloidal particles. Artificial neural networks, for instance, can be trained in the same manner as the present SVM implementation to interpret radial profiles of experimental holograms. SVMs have the advantage that their training process proceeds deterministically, and therefore tends to be faster. Once successfully trained, however, artificial neural networks are generally more computationally efficient. Regardless of implementation, the present results demonstrate that machine-learning methods facilitate fast and precise measurements of colloidal properties.