1. Introduction

The optical forces exerted by a few milli-Watts of laser light are negligible at the macro-scale, yet they can dominate the dynamics of nano- and micro-particles. Optical tweezers exploit these forces to precisely manipulate fine dielectric particles without mechanical contact [1], enabling delicate mechanical measurements at the molecular [2, 3] and colloidal [4] scales. A number of strategies have emerged for creating large numbers of optical traps from a single laser beam [5, 6, 7], but have not yet matched the broad impact of relatively simple systems with one or two traps. While open loop multiplexed systems show promise for separations [8, 9], emerging applications in materials science, cell biology and possibly quantum computing [10, 11] require complex coordinated movement of large numbers of traps that demand closed loop control. A number of “human-in-the-loop” control systems have been developed in recent years [12, 13, 14, 15] These graphical user interfaces require a human operator to direct the motion of traps. While this approach is quite robust, it cannot be efficiently extended to applications requiring precise manipulations or large numbers of particles.

In this paper, we outline and implement a strategy for automatically and precisely manipulating matter at the microscale. We remove the operator from the feedback loop and demonstrate automated trapping, sorting, and assembly of colloidal silica microparticles, based on holo-graphic optical tweezers (HOT) [6, 16]. We outline our hierarchical approach to the control of optical tweezers in Fig. 1(a). Physical optics image and manipulate our sample using optical microscopy and optical tweezers. Optical analysis algorithms extract features from images and calculate laser wavefronts. A process control system enables users to define processes and seamlessly determines the manipulations to realize them.

2. Physical optics

Microscopy. We image colloidal suspensions in bright field with an inverted microscope (Nikon TE-2000) with a N.A. 1.4 100× oil immersion objective lens, labeled OL in Fig. 1(b). A dichroic mirror, labeled DM, couples the trapping wavelength into the objective lens while transmitting the rest of the visible spectrum to a CCD camera (Hitachi KP-M32N). A frame grabber (Data Translation DT3155) digitizes the images in real time for feature recognition.

Micro-manipulation. A diode-pumped solid state laser (Coherent Verdi V5) outputs up to 5W CW at a wavelength of 532nm. A Keplerian telescope expands the beam to fill the face of a spatial light modulator (Holoeye LC-R-2500). The spatial light modulator (SLM) realizes computer-generated holograms in real time [17, 18] by independently modulating the phase of reflected linearly polarized light across 1024×768 pixels from 0 to 2π with 256 levels. The phase levels of the SLM can be updated at rates up to 75 Hz. A second Keplerian telescope images the SLM onto the back aperture of the objective lens. The first-order diffraction pattern is only 1.5× brighter than the zeroth-order diffraction spot. Therefore, a 500 µm diameter ball bearing, labeled BB, mounted in the intermediate focal plane of the second telescope blocks the zeroth-order diffraction spot. The ball bearing blocks a 3µm² area in the focal plane.

 

Fig. 1. Closed Loop HOT. (a) A block diagram of the control system. (b) A schematic diagram of the experimental setup.

Download Full Size | PDF

3. Optical analysis

Feature recognition. Particle centers are identified in a three step process usingMATLAB [19]. First, digitized images are processed with a spatial bandpass filter to remove pixel-level noise and long wavelength intensity variations. We then identify all local intensity maxima above a minimum threshold as preliminary particle centers. This step locates particles in the focal plane to a resolution of a single pixel - about 100 nm. If further resolution is required, we calculate the first three moments of the intensity distribution around the preliminary particle centers to obtain the integrated intensity, refined particle position and radius of each particle image [20]. This step locates particles to sub-pixel resolution - typically about 10 nm. The essential idea here is that while diffraction enforces a lower limit on the size of a particle’s image, our ability to locate its centroid is not diffraction-limited. For low-noise images, the spatial resolution, σ_x ≈ε/w, where e is the effective width of a pixel in the focal plane of the objective lens and w is number of pixels spanning the diameter of the particle’s image.

Wavefront design. We sculpt the wavefront of the trapping laser in the back aperture of the objective lens to create discrete intensity maxima in its focal plane. A number of methods have been developed to calculate phase-only holograms. For the current results, we primarily employed a simplified version of the Gerchberg-Saxton algorithm [21, 16]. Here, the phase profile at the back aperture, Φ(r⃗), is given by a single inverse Fourier transform:

where δ ⁽²⁾(ρ⃗) is the two-dimensional Dirac delta function, ρ⃗ ₁…ρ⃗ _N are the locations of the N traps in the focal plane and $A_j^2$ is the intensity of the jth trap. The phase factor, θ_j , is a randomly assigned to each trap. This random phase factor improves diffraction efficiency and trap uniformity, especially for highly symmetric configurations of traps [22]. Identical results can be obtained using the method of prisms and lenses (PL) [17]. The two methods differ only in their numerical procedures. The number of computations for an M-pixel hologram scales like MlnM for the FFT, while it scales as MN for PL. In our MATLAB implementation, we found that the FFT algorithm outperformed PL when the number of traps is more than about three. This is because MATLAB is highly-optimized for the efficient computation of FFTs and its interpreted nature is not well suited to the rapid execution of loops.

 

Fig. 2. Avoiding Collisions between Identical Particles. (a) (0.7MB) Twenty-five one-micron silica particles are transformed from “Y” to “LUX” in 3 sec using HOT. (c) Particle trajectories extracted from bright field images using feature recognition methods described in the text. Note that particles move in straight lines unless collisions are detected. [Media 1]

Download Full Size | PDF

4. Process control

Path planning. This module assigns trajectories, ρ⃗_j (t), to particles to execute the desired process. Two primary physical limitations constrain the trajectories. First, the finite spatial extent of the optical trap limits the step size, s, between successive trap locations. We have found a step size of about one particle diameter, s≈2a, to be optimal. This limits the speed of trapped particles to about sf, where f is the refresh rate of the feedback look. In our current system, this gives maximum velocities of 2–10 µm/s. With faster feedback loops, the maximum velocity will ultimately be determined by the balance of viscous and optical forces on the trapped particles. Second, trajectory assignment must avoid particle collisions. When trapped particles pass within a critical distance, comparable to the particle diameter, they can be knocked out of their respective traps or can both fall into the same trap.

We have developed a simple set of traffic rules for particle movements that satisfies these constraints while efficiently transforming an arbitrary initial configuration of trapped particles, ${\overrightarrow{\rho }}_j^i$ , into the desired configuration, ${\overrightarrow{\rho }}_j^f$ . First, we assign destinations to particles by temporarily ignoring potential collisions and iteratively minimizing the length of the longest particle trajectories. Once each particle has been assigned a destination, we assign a new position for each particle in the next timestep according to

Essentially, each particle makes a step of size s directly toward its destination. We have two sets of rules for resolving collisions. If colliding particles are identical, we simply exchange their destinations. If particle identity is important, as in a sorting process, then the colliding particles take a cue from pedestrians on a sidewalk and each take a step to their right

 

Fig. 3. Assembly and Sorting. (a) (0.9MB) Automated assembly of 1.0 µm colloidal silica into a 5×5 grid. (b) (2.2MB) Automated sorting of colloidal silica by size. Scale bar represents 5 µm. Movies are accelerated 4×.

Download Full Size | PDF

where ẑ is the unit vector normal to the focal plane. In either case, if these modifications do not resolve the collision, then the particles remain stationary for the timestep. If particles cannot make a collision-free move after two timesteps, then they are each assigned a random step. The algorithm used for identical particles is demonstrated in Fig.2. In order to achieve the rapid manipulations demonstrated in this example, all holograms were calculated in advance. Thus, the refresh rate for the particle positions is limited by the drive electronics of the SLM.

In this paper, we have restricted particle trajectories to 2-D. However, the implementation of 3-D trajectories is straight-forward and would significantly decrease the frequency of collisions between particles. Furthermore, our current control algorithm ignores the presence of a small area near the optical axis, where the trapping light is blocked by the zeroth-order spatial filter. Consequently, particle trajectories passing through this zone may lead to lost particles. However, since this zone represents <0.1% of the total area addressable by the SLM, such events are rare. When necessary, this problem can easily be circumvented by treating this zone as a fixed “virtual particle” in the path planning module.

Process definition. At the highest level of the control system, accessed from the command line in the MATLAB environment, the user specifies the type of process to be executed. Here we demonstrate assembly and sorting.

Our assembly algorithm has two steps. First, the desired number of particles are automatically trapped. In real-time, the algorithm locates particles in the field of view and calculates a hologram to trap them in place. It is essential that our control system responds rapidly so that traps are created at the observed particle locations before they diffuse out of reach. For particles larger than the wavelength of light, the in-plane spatial extent of the optical tweezers is about one particle diameter, 2a. Therefore, we can estimate the time for a particle to diffuse beyond the tweezers’ region of influence using the ensemble-averaged solution to the Langevin equation,

where η is fluid viscosity and k_BT is the thermal energy scale. In the present system of 1 µm diameter particles in water (η=0.001 Pa s) at room temperature, τ_D =0.6 s. The assembly of a 5×5 square grid of traps is demonstrated in Fig. 3(a). First, the control system automatically traps the desired number of particles. Next, the path planning module calculates the trajectories for all of the particles. Finally, the holograms are calculated and displayed in real-time to realize particle trajectories. In this example, the assembly took 35 seconds and resulted in one vacancy. The algorithm can easily be re-executed to remove such defects.

Our sorting algorithm is very similar to the assembly algorithm, as shown in Fig. 3(b). Here, we separate a mixture of 1.0 µm and 1.9µm diameter silica particles. The algorithm first traps all of the particles in the field of view. Then, it uses the feature recognition module to discriminate particles of different sizes by their integrated intensity. Next, it assigns trajectories to the particles, placing the larger particles to the top of the field of view and the smaller particles to the bottom of the field of view. This path planning stage takes about 16 seconds. Next, holograms are calculated and displayed in real-time, achieving an initial 94%accurate sorting within another 45 seconds. At this point, the algorithm detected one large particle trapped among the small particles and repeated the separation to achieve perfect sorting after an additional 40 seconds.

In these examples, holograms were calculated in real-time and this calculation limited the refresh rate of the feedback loop to f<3 Hz.

5. Conclusions

We have demonstrated the feasibility of automated trapping, assembly and sorting of colloidal particles at the microscale using closed loop holographic optical tweezers. While HOT has been chosen for the physical method of optical micromanipulation, time-shared optical tweezers [5] and optically-addressed dielectrophoresis [23] can also be used with similar results. The implementation of the optical analysis module on graphics processing units will enable an increase in the feedback speed of about an order of magnitude [24]. These high-speed systems will enable exciting applications for the sorting and assembly of colloidal and biological materials at the microscale. Additionally, such fast systems will enable the implementation of multiple force clamps for the exploration of the mechanotransduction in live cells.