1. INTRODUCTION

Particle image velocimetry (PIV) is a well-known method for the experimental determination of fluid velocity fields [1]. It relies on tracking tracer particle positions over time, thereby reconstructing the local velocity fields. Over the past 25 years, different techniques have been developed to implement PIV in microfluidic devices, where µPIV refers to a microscopic adaptation of established macroscopic PIV techniques [2]. Classical PIV is an in-plane measurement technique and allows the reconstruction of two-dimensional velocity fields. Several methods have been proposed to retrieve three-dimensional information. For example, in [3] or [4], traditional confocal microscopy [2] techniques have been adapted to change the focal plane rapidly. Such techniques, however, only provide two-dimensional velocity components in the three-dimensional volume [5]. Stereoscopic PIV and tomography [6–8] were successfully used for µPIV (e.g., [9,10]). Those techniques remain difficult to implement in microfluidic applications and only apply to a very limited depth of field (DoF) [5]. In addition, digital holographic microscopy [11] (DHM) methods were developed for µPIV [12] and proved to be very accurate [13,14], typically allowing tracking the positions of 1 µm particles with submicrometer accuracy, and can be applied to a relatively large depth of field (of about 1 mm) [15]. DHM is generally used in transmission, which is unsuitable for the microfluidic samples we want to characterize. First, the refractive index changes of the liquid can distort the acquired holographic data. Moreover, most of our samples have nontransparent walls in the optical path, leading to multiple internal reflections in the device and inducing strong pervasive detected interference patterns. This made detecting the relatively weak signal emanating from the particles with DHM difficult, so we targeted a different approach. Alternate techniques record the optical patterns of fluorescent particles. The use of fluorescence is classical in µPIV applications. In macro PIV, the illumination consists of a light sheet. In µPIV, one is interested in obtaining measurements with micrometric out-of-plane resolution. Forming a micrometric thick light sheet in a microchannel would be extremely difficult, and broad field illumination is preferred [16,17]. Fluorescence allows a filter to isolate the signal emanating from the particles from the scattered light from channel walls, significantly improving the signal-to-noise ratio [8]. This technique was recently applied for the characterization of 3D flows in defocusing µPIV [18] and general defocusing particle tracking (GDPT) [19]. The principle of this technique is to evaluate the position of the particles along the optical axis by comparing the normalized cross-correlation function of the target particles with reference calibration image stacks. This technique presents several key advantages: it is experimentally robust (w.r.t. variations of the investigated fluid’s refractive index and the optical system’s alignment) and applies to particles of arbitrary shapes. In [18], the authors performed a detailed characterization of the uncertainty of the method. They showed that the particle positioning accuracy was below 0.1 µm in plane and about 2 µm along the optical axis, using 2.24 µm particles and a ${10\times }$ magnification lens in a measurement volume of ${1510}\;\unicode{x00B5}{\rm m}\;{\times}\;{1270}\;\unicode{x00B5}{\rm m}\;{\times}\;{160}\;\unicode{x00B5}{\rm m}$. GDPT is very accurate over a DoF where defocusing patterns with sufficient intensity can be acquired. Extending it would require using larger particles. However, in some microfluidic systems, using larger particles is not appropriate. In the present contribution, we propose a novel method to determine 3D flow profiles to overcome the limitations of the above-mentioned methods. Our methodology is based on scanning–defocusing particle tracking. The scanning procedure consists of fixing the target sample to an electrodynamic magnet, allowing it to continuously oscillate the microfluidic substrate in and out of the focal plane of a fixed optical system. The shaking procedure allows for larger magnification optics. Our method relies on a very limited calibration dataset and a simple data analysis procedure, still giving access to accurate tracking of particles over an extended sample depth. The present paper describes our experimental setup and the data processing pipeline. We report the results of a set of experiments to verify that the shaking process does not influence the hydrodynamics in the samples. In addition, we present a 10-s 3D trajectory of a particle in an acoustically generated vortex with micrometric accuracy.

2. METHODOLOGY AND EXPERIMENTAL SETUP

A. Optical Setup

Our setup was designed to characterize flows in microfluidic devices, and the typical volume of interest (VOI) dimensions are ${0.5}\;{\rm mm}\;{\times}\;{0.5}\;{\rm mm}\;{\times}\;{1}\;{\rm mm}$ (${\rm length} \times {\rm width} \times {\rm depth}$). A schematic of the optical system is presented in Fig. 1. An excitation continuous blue laser beam (Coherent, OBIS FP 488LS) is used at the maximum output power of about 60 mW. The beam is pulsated through a chopper wheel (Thorlabs, MC2000B-EC) and diffused through a laser speckle reducer (Optotune LSR 3000). Fluorescent particles are imaged with a magnification objective (depending on the experiments, we use an Edmund Optics ${50\times}$ ULWD, with NA 0.55, or a ${100} \times$ ULWD, with NA 0.55). The images are recorded at 1000 fps with a camera (PCO dimax HS4 with a CMOS sensor with resolution $2000 \times 2000$ and pixel size $11\,\,\unicode{x00B5}{\rm m} \times 11\,\,\unicode{x00B5}{\rm m}$). The camera is equipped with a macro objective (Samyang 100 mm F2.8) to adjust the desired FOV to the camera sensor. The emitted light from the particles is passed through an emission filter (Thorlabs, MF525-39) to reduce the background noise.

 

Fig. 1. Optical system. An excitation continuous blue laser (1) beam is shaped through a set of lenses (2), pulsated with a chopper wheel system (3), and diffused (4) to the volume of interest (VOI) of a microfluidic chip (5). The liquid is seeded with fluorescent particles imaged through a magnification objective (6), and the data are recorded with a high-speed camera (7), equipped with a bandpass filter (8) and macro objective (9) to adjust the desired FOV to the CMOS sensor. The position of the sample is periodically varied by a mechanical shaker (10) perpendicular to the optical axis, effectively displacing the focal plane and scanning the entire VOI and monitored by a capacitive sensor (11).

Download Full Size | PDF

B. Scanning Setup

The microfluidic substrate is oscillated with an electrodynamic magnet (Brüel & Kjær, Mini-shaker type 4810) perpendicular to the optical axis, effectively displacing the focal plane and scanning the entire VOI. A sinusoidal signal sent to the shaker by a computer is amplified with a signal amplifier (Brüel & Kjær, type 2719). The position of the sample is monitored by a capacitive sensor (Lion Precision, C18). The typical scanning frequency ranges from 10 to 20 Hz. All subsystems of the setup are operated by custom software and synchronized by a DAQ system (National Instruments 779675-01) with temporal resolution below $2 \cdot {10^{- 5}}\; {\rm s}$. The position sensor voltage readings were calibrated over the entire range of interest (${\rm typically}\;\pm {400}\;{\unicode{x00B5}{\rm m}}$) by manually displacing the sample along the optical axis by 10 µm steps with a translation stage. This calibration was performed multiple times, and from the dispersion of the data, the resolution on the relative position (displacement) of the sample was better than 0.1 µm.

C. Microfluidics

The characteristics of the microfluidic chips used in this study are depicted in Fig. 2. The microchannel was etched in a silicon wafer, and the top and bottom of the channel were sealed with borosilicate glass. The internal section (${\rm width} \times {\rm depth}$) of the channels is $0.375\; {\rm mm} \times 0.525\;{\rm mm} $, and the channel length is 20 mm (Fig. 2C). The channels are equipped with three inlets and outlets for infusion and evacuation of liquids and sealed by bonding 200 µm internal diameter Polymicro Technologies (Phoenix, AZ, USA) glass capillaries with dual-cure epoxy sealant; cf. Fig. 2(B). All connections are realized with IDEX connectors. One of the topics of interest in our research concerns the handling of particles and crystals by inducing acoustic streaming, for which a piezo-ceramic actuator ($15\; {\rm mm} \times 20\; {\rm mm} \times 1\;{\rm mm} $, APC International, Mackeyville, PA USA) with an eigenfrequency of around 2.0 MHz was placed at the back of the chip (Fig. 2(A)]. A Tektronix AFG1062 function generator is used to apply a sinusoidal voltage to the piezo element. An RF power amplifier (Electronics & Innovations, 210 L) amplified the applied voltage with a maximal total output power of 10 W. Active temperature control of the sample to $\pm {0.1}\;{^\circ{\rm C}}$ is implemented with a Peltier element to limit heating with the supplied acoustic energy. The chip’s position is measured through a metallic position-measuring target, as a distance of several centimeters is required to avoid electric interference between the piezo element and the position sensor. All the components of the microfluidic assembly are incorporated in a custom PMMA holder.

 

Fig. 2. Microfluidic assembly. The internal section of the channels is $0.375 \;{\rm mm} \times 0.525\;{\rm mm} $, and the channel length is 20 mm. The chip is equipped with a piezo element to achieve acoustic streaming. The temperature of the sample is controlled with a Peltier element. The position of the chip is measured through a metallic position-measuring target to avoid electric interferences between the piezo element and the position sensor.

Download Full Size | PDF

 

Fig. 3. Data processing pipeline. The top row of the figure shows how the calibration stack is computed. A reference template stack is made from an analytical particle model fitted on calibration video of stationary particles. The bottom row shows the processing of experimental videos, detecting 3D particle positions in every frame and converting these into 3D particle trajectories (cf. Supplement 1 for more detailed diagrams).

Download Full Size | PDF

D. Particles

Aqueous suspensions of fluorescently labeled polystyrene particles (PS-FluoGreen, microParticles GmbH) were used. Different sizes of particles were tested (diameters of 0.45, 1, 2, and 5 µm) and found to be suitable for our setup. Larger particles are preferred whenever possible as they result in larger fluorescence signals and allow for shorter camera exposure times. Particles of 2 µm diameter (PS-FluoGreen-2.0, microParticles GmbH) were selected for this study as these are the largest possible particles for our applications. For larger particles, acoustic radiation dominates the drag forces of the acoustic streaming, leading to a focusing of the particles in the center of the channel [20]. This allows for a 3D vortex profile to be analyzed with our newly introduced technique. Particles are spherical and monodisperse ($2.07 \pm 0.3\,\,\unicode{x00B5}{\rm m}$), with absorption and emission frequencies of 502/518 nm, and present a density close to water ($1.05\; {\rm g}/{{\rm cm}^3}$).

E. Algorithms and Data Processing

The principle of the particle detection and positioning method is based on the analysis of the characteristics of the observed optical patterns as a function of the distance of the particles to the focal plane; it requires comparing the experimental images with calibration images [18], as shown schematically in Fig. 3. For the calibration pipeline, we use videos where the particles are stationary. The video is scanned for in-focus particles using filtered thresholding, serving as anchor points. From there, stacks of defocused particles at these positions over time can be extracted with known relative depths. Directly using those images as templates has shown to be unreliable because of noise and asymmetries; averaging did not yield satisfactory results due to the imperfect alignment of different instances. Instead, we use an analytical parametric particle model that fits the calibration data described in Supplement 1. The radius of the fitted model templates has a near-linear relationship with the particle depth position, confirming the adequacy of the model. We generate an image calibration stack from this model, quantizing the axial dimensions and returning representative templates for local depth levels.

 

Fig. 4. Shaking profile and particle depth calibration. A few particles were injected, and defocusing patterns were recorded over the 0.525 mm depth of the channel by applying sinusoidal voltages to the shaking device. 2 µm particles were used. The radii of the patterns were measured with the Hough transform, and a linear relation between the radius of the pattern and the distance to focus of the particles was established, allowing for positioning particles along the microscope’s optical axis.

Download Full Size | PDF

In the main pipeline, the video frames are preprocessed to eliminate various noise sources, such as residual fluorescence signals due to reflections or scattering by the microchannel walls, and unused frame regions are removed to speed up the remainder of the pipeline. We apply a weighted normalized autocorrelation function for each frame for every template image, using GPU acceleration. In every pixel, the strongest response and corresponding template index are kept. Neighboring pixels may return similar detections emanating from the same particle, which are grouped using morphological operators before thresholding. The centroids of the detections, the template index, and the current shaker position combined will return 3D particle position detections for a given frame. More details can be found in Supplement 1.

This cloud of 3D positions can now be converted to particle trajectories. Based on all detections’ relative spatial and temporal distances, we use a greedy matching algorithm, i.e., pairing detections into segments with minimal length, provided their mutual distance is below some threshold. In this phase, we can also easily eliminate occasional spurious detections from the previous phase since they will typically manifest as isolated detections in space and time far from all others. We can obtain short path segments across the whole video; the complete trajectories still have periodic gaps since particles will always be too out of focus to be detectable in some frames of the shaking cycle when the scanning depth is large. Neighboring segments in time and space are combined into the final uninterrupted trajectories. After those processing steps, we can obtain the trajectories of the particles within the liquid volume.

3. EXPERIMENTS AND RESULTS

A. Calibration Experiments

Preliminary calibration experiments were performed to build a calibration reference dataset of the optical patterns of the particles versus the position along the optical axis. The solution of microparticles was prepared and injected into the channel with a syringe pump; the dilution with milliQ water was adjusted to obtain a few particles in the field of view. Closing valves stopped the flow in the channel, and, a few seconds later, the liquid in the channel was immobile.

The sample was shaken at low frequencies (1 and 2 Hz) to collect small variations of the defocusing pattern with depth, and frame sequences were recorded at 1000 fps. A typical scanning profile obtained over a 525 µm range at a scanning frequency of 2 Hz is shown in Fig. 4. The first sinusoid period was deformed, and, after a few cycles, the scanner stabilized, and reproducible profiles were obtained. The defocused particles typically produce ring-shaped patterns, with an additional bright central spot as particles get more in focus. Conceptually, a quasi-linear relation between the pattern radius and the particle position can be established (Fig. 4), allowing for retrieving the distance to focus from the pattern of a defocused particle. Therefore, the depth of any given particle could be retrieved as a function of the shape and size of the defocused pattern.

B. Static Particles Experiments

A series of experiments was performed to optimize the detection algorithms and to investigate the influence of the scanning on the hydrodynamics in the microchannel. For those experiments, all experimental parameters were set identically to the ones used for the calibration experiments, except for the scanning frequency: three experiments were performed with scanning frequencies of 5, 10, and 20 Hz. In each experiment, the particles were injected into the channel, and the flow was stopped using valves. After a few seconds (10–20 s), once the residual pressure gradients were equilibrated (i.e., when the particles appeared static by the naked eye), the image acquisition was started first, and the shaking was started right after.

 

Fig. 5. Measurement of the particle positioning accuracy in 3D. The figure shows the position history of 2 µm particles in a $525\,\,\unicode{x00B5}{\rm m} \times 375\,\,\unicode{x00B5}{\rm m}$ microchannel in the absence of flow during a 10-s scanning sequence at different shaking frequencies. Three particles were imaged through three independent experiments, and the results were aggregated: the blue, red, and pink particles correspond to experiments with shaking frequencies of 5 Hz, 10 Hz, and 20 Hz, respectively. In the figure, the solid green lines represent the walls of the microchannel, and the dashed green lines the field of view. The data shows that the particles are propelled at the scanner’s start. The data corresponding to the first two periods of a scan are displayed in yellow. After removing the yellow data points and averaging the positions over one cycle of the scanner, the experimental accuracy of the positioning of a particle is better than $\pm {1}\;{\unicode{x00B5}{\rm m}}$ in each of the three dimensions.

Download Full Size | PDF

For each frame, ${F_i}$, the image is scanned and used to generate an array of relative coordinates of detected particles ${D_{{\rm rel},i}}$:

All detections are displayed in Fig. 5(A): the particles appear static because all the computed coordinates are superimposed at the plot scale. An experimental evaluation of the accuracy of the setup can be derived from the four zoomed plots of the figure. Several sources contribute to the error in the retrieved particles’ positions. Some sources are associated with experiments, such as slight variations of the alignment of the optical system (cleaning microfluidic chips in between experiments sometimes requires disassembling the microfluidic system out of the microscope), small temperature variations in the room of the microscope, or the difficulty of defining a satisfying criterion for experimentally immobilizing a particle in a liquid. Other sources are related to the technique itself. First, the particles appear to move just after the scanner has started. This is a consequence of the large amount of energy required to initiate the oscillation of the shaker. The density difference between the particles and the base fluid is small, and this displacement is limited to a few micrometers only. In addition, the zoom plots in Figs. 5(D) and 5(E) clearly allow identifying oscillations in the measured trajectories, indicating the influence of the shaking. The amplitude of the scanning is large, and the noise on the measurement of the scanner’s position contributes to the signal through Eq. (2). The particle’s $(x,y)$ position is determined by computing the centroid of the optical pattern. Although the particles are fixed in the microchannel, they move with respect to the camera, which results in motion blur; as a result, the scanning amplitude and frequency influence the recorded patterns. Camera exposure times were kept as low as possible, below 1 ms, to limit this effect. The contributions of these different aspects on the accuracy of the individual measured positions of each particle at each time step are merged and difficult to assess or improve individually. From the experimental data, we evaluate the accuracy of individual detection to $\pm {2}\;{\unicode{x00B5}{\rm m}}$ in each direction, depending on experimental parameters, such as the camera exposure time or the scanning amplitude and frequency. Moreover, many images are acquired in each experiment, and each trajectory involves several hundred detections; the accuracy can be improved by averaging the subsequently detected positions of a particle. The effect of averaging the positions of each particle over a temporal window corresponding to a period of the scanner is illustrated in Fig. 5(C). The positioning accuracy of a particle at each time depends on the scanning frequency. After averaging, the accuracy is about 0.5 µm and 1 µm for scanning frequencies up to 10 Hz and 20 Hz, respectively. Overall, these results indicate that the perturbation of the hydrodynamics in the channel by the scanning is very limited and that its impact on the trajectories of the particles is below the data processing noise.

 

Fig. 6. Measurement of the particle positioning accuracy in 3D. The figure shows the position history of 2 µm particles in a $525\,\,\unicode{x00B5}{\rm m} \times 375\,\,\unicode{x00B5}{\rm m}$ microchannel under a constant flow during a 10-s scanning sequence at 10 Hz. After retrieving the positions of the detected particles, segments were identified and sorted to obtain trajectories over the entire VOI. In this experiment, the final resolution after processing is about 1 µm.

Download Full Size | PDF

C. Flow Experiments

In this section, we report the results of the next experiment, measuring the trajectories of particles flowing along the microchannel. The microparticle solution was diluted to obtain about five particles in the field of view. The syringe pump (WPI 78-9100) continuously infused liquid at about one ml/h through the channel using a Hamilton gastight #1725 syringe. For this experiment, we used a scanning frequency of 10 Hz. Images were processed as previously, and after building the dataset of the absolute coordinates of the particles ${D_{{\rm abs},i}}$ the detections were sorted into trajectories, as shown in Fig. 6(A). The top plot of the figure displays all the detected positions of the particles during the experiment. The reconstructed trajectories appear segmented because, with our setup, the defocusing only allows detecting particles over a depth of focus of about $\pm {30}\;{\unicode{x00B5}{\rm m}}$. In flow experiments, additional error sources contribute to the signals: the motion of the particles slightly increases the motion blur. Moreover, the illumination is not perfectly homogeneous over the FOV, and distortion of the optical patterns is observed. The distortion varies according to the position in the microscope’s field of view (in both the $xy$ plane and the $z$ direction). This error could be mitigated by refining the calibration data, particularly by recording different calibration stacks for different subvolumes of the volume of interest, as performed in [18]. This approach, however, requires tedious experimental work as it implies renewal of the calibration dataset after each modification of the microfluidic system, which turned out to be impractical. In this experiment, for particles in the central part of the channel (i.e., with the maximal velocity in the Poiseuille flow), the time required to cross the entire field of view was about 1.5 s, and the typical number of data (detection) points per trajectory was about 400. The reconstructed trajectories obtained after averaging and interpolating are shown in Figs. 6(B) and 6(C), and the accuracy of the 3D tracking is about 1 µm in each direction.

D. Investigation of Acoustic Streaming

This section reports the last experiment’s results, measuring the trajectory of a particle in an acoustically generated vortex [21]. Acoustic streaming is known to be very sensitive to the experimental adjustments of the microfluidic system. For a 375 µm wide channel, acoustic resonance is expected to be obtained at an oscillation frequency of the piezo of 2 MHz. In practice, however, stationary waves along the width of the channel are obtained for certain specific frequencies. The applied signal’s frequency was adjusted to a resonance frequency with a precision of up to 1 kHz. Experimentally, this step was performed by infusing fluorescent particles and by monitoring the efficiency of the acoustic streaming as a function of frequency. When an optimal frequency was reached, stable vortices were obtained, and the motion of a particle was captured for 12 s. After image processing, the number of data points was about 4000 detections. The obtained trajectory is shown in Fig. 7. The evaluation of the accuracy of the particle tracking for complex flows such as acoustic flows is a delicate question. The experimental implementation or realistic modeling of acoustic flows with micrometric reproducibility is impossible. The tracking accuracy may be slightly decreased in the case of flows with a velocity component along the optical axis. The experimental data indicate that our technique allows robust tracking of the particle motion in three dimensions with micrometric accuracy.

 

Fig. 7. Trajectory of a 2 µm particle undergoing acoustic vortices. This figure shows the 3D trajectory of a 2 µm particle during 12 s in an acoustically generated flow. Due to the scanning, only about 30% of the acquired images contain a particle. The noise around the trajectory is the result of the limitations of the overall image processing sequence. These results indicate that our technique allows robust tracking of the particle motion in three dimensions.

Download Full Size | PDF

4. CONCLUSION

We developed a scanning technique for the tracking of flows in microfluidic applications. Our method is inspired by defocusing particle tracking and extends the depth of the volume of interest by shaking the target sample along the microscope’s optical axis. An electromagnet performed the scanning and allowed the use of high magnification optics for the samples over 500 µm in depth. We reported the results of several experiments. Tracking of static particles indicates that the technique allows tracking particles with micrometer accuracy. The technique was implemented to reconstruct the trajectories of particles in flows in microchannels. We used a continuous Poiseuille flow to illustrate the tracking of particles in 3D over the entire target volume. In addition, we report the characterization of the trajectory of a particle in an acoustic flow. The experimental results show that the technique is very robust and can be used to characterize 3D flows in demanding microfluidic applications.