High-speed scanning interferometric focusing by fast measurement of binary transmission matrix for channel demixing

Xiaodong Tao; Dare Bodington; Marc Reinig; Joel Kubby

doi:10.1364/OE.23.014168

1. Introduction

With the incomparable advantages of high-resolution imaging of live organisms, optical microscopy has become an indispensable tool for biological research. However the performance of even the most advanced microscope is still compromised due to the inhomogeneities in biological tissues. The improvement of the imaging depth for optical microscopy is still in its infancy. As light propagates through biological tissues, it can be refracted, scattered and absorbed, limiting the imaging resolution and depth. To correct the refractive aberration, adaptive optics (AO) has been extensively investigated for applications in optical microscopy [1–10]. The aberrations for ballistic light through biological tissues are measured directly or indirectly and then corrected by an optical phase modulator, such as a deformable mirror (DM) or a liquid crystal spatial light modulator (LC-SLM). Recent results have shown its great potential to correct spatial and temporal aberration in imaging of live samples [10–13]. As the imaging depth increases, AO becomes less effective for focusing light in the sample. Multiple scattering becomes a dominant factor limiting the image depth. However the amplitude of elastic scattering loss is noted to be an order of magnitude or more than that of absorption [14]. Overcoming the elastic scattering can dramatically extend the light penetration depth.

Thanks to the deterministic process of scattering, there exists a linear relationship between the input modes and the output modes of the wavefront, which can be described by a transmission matrix (TM) [15, 16]. To achieve a desired electric field on the output modes, there always exists a corresponding input wavefront. A straightforward way is to record the electric field directly using interferometric techniques and achieve the desired optical field using optical or digital phase conjugations [17, 18]. However the illumination light needs to be directed in opposite directions for imaging and recording, respectively. In addition, coherent light from the target is often required for the wavefront measurement. Another way to generate the desired optical field through scattering media is to measure the TM using a LC-SLM and a full-field interferometric measurement. Then the light transmission can be completely controlled and the scattering media can act as a lens to transfer the image [19, 20]. To focus the beam through scattering tissue at a single or multiple output channels, iterative optimization methods have been demonstrated [21–23]. To obtain optimized focusing, thousands of degrees of freedom of the incident wavefront need to be modulated and measured. Currently the main limiting factor is the speed of the optical light modulator. Most of the existing systems using a LC-SLM take minutes for one optimization [19–24]. The slow focusing rate cannot follow the speckle change in live tissues with decorrelation times on the scale of milliseconds. The refresh time can be further decreased to several seconds by using an expensive segmented deformable mirror (DM) [25,26]. A more complex system based on spatial frequency modulation can achieve 400 milliseconds for one correction [27]. All optical feedback has also been applied to focus light through turbid media at a submicrosecond time scale by the process of field self-organization inside a laser cavity [28]. The sample must be put in a specially designed cavity and coherent light is required for the feedback.

Recently another high speed light modulator, the digital micromirror device (DMD), has been used to compensate scattering in biological tissue [29–33]. It contains millions of fast switchable micromirrors to modulate the intensity of the light based on its two states. A commercially available DMD has frame rates up to 22 kHz and could have more than one million of pixels. Yet, it is much less expensive than a segmented DM due to its mass production for digital light projectors. The first demonstration of binary intensity modulation using a DMD was made by Mosk’s group, where a sequential iterative algorithm was used to focus light through turbid media [29]. The lower efficiency of intensity modulation compared with phase modulation can be compensated by the large number of available channels on the DMD. Phase modulation on a DMD using a hologram was also proposed at the expense of phase resolution loss [30,31]. The resolution can be further improved by a superpixel-based spatial amplitude and phase modulation [32]. A genetic algorithm has also been used in binary intensity modulation to improve the performance of the optimization process [33].

In this paper, we demonstrate a fast binary intensity modulation based on the measurement of the binary TM. Although it is possible to measure the full phase of each element of the TM using an interferometer, the information is redundant for the binary intensity modulation [34]. Since the DMD can only switch on or off each channel, the state of each channel depends on whether the relative phases of connected elements in the TM are in the range of π. Here we demonstrate a simple and effective way to measure binary information in the TM, which is particularly suitable for binary intensity modulation. For each correction, the binary TM was calculated based on measurements of the intensity change at the target with a series of input masks. After preloading the measurement masks, the DMD can run at full speed during measurement. Compared with the optimization method, no feedback information is needed during the measurement. The proposed method only requires one measurement for each input mode. The total time for a single correction is only 75ms for 1024 input modes. The phase modulation using an off-axis digital holograph made with a DMD can achieve higher enhancement with higher speed. However the light efficiency is often low (10%). The direct intensity modulation used in this paper has much higher light efficiency (60%) [30,31]. This is more suitable for applications that are sensitive to optical power loss, such as multi-photon imaging. The low diffraction efficiency could dramatically decrease the probability of two photon or three photon absorption during measurement. Higher diffraction efficiency gives higher signal to noise ratio (SNR) at the target, which can increase the accuracy of the transmission matrix measurement. To avoid the low SNR during the measurement caused by the low intensity of the reference field, the reference optimization method by scanning of the speckles on the target is demonstrated. It provides more stable focusing, especially for focusing through a fast changing media or two dimensional scanning through a slowly changing scattering media. This benefit can be seen in the stable focusing shown in the video clip included in our manuscript as Media 1. The stability of this focus should be compared to the stability of the focus shown in Fig. 4 in [31]. The proposed method was tested on both stationary and moving samples with different decorrelation times. Finally we demonstrate focusing light through a highly dynamic scattering sample, a live drosophila embryo.

2. Materials and methods

2.1 Measurement of the binary TM

Light transport through scattering medium can be characterized by a TM, which connects electric fields from the incoming and outgoing channels [15]. In this paper, we only consider a single focus spot. To focus the light through scattering medium at one output channel, the electric field at the output channel, $E_{o u t}$ , can be considered as the sum of the contributions from all the input channels as shown in Fig. 1(a), which is given by,

E_{o u t} = K E = \sum_{n} k_{n} e_{i n}^{n}

E = {(\begin{matrix} e_{i n}^{1} & \dots & e_{i n}^{N} \end{matrix})}^{T}

is the input electric field vector, where

e_{i n}^{n}

is the electric field at the n^th input channel.

K = (\begin{matrix} k_{1} & \dots & k_{N} \end{matrix})

is the TM for a single output channel, where k_n connects the n^th input channel and the output channel. The complex phasor k_n follows a circular Gaussian distribution [22]. The phase of k_n follows a uniform distribution between –π and π [29]. By measuring the complex value of each element k_n, the light at the input channels can be modulated and a constructive interference focus point can be generated at the output channel. However the full phase measurement is redundant for the binary amplitude modulation. When using a binary amplitude modulator, the goal is to open the input channels which can generate constructive interference at the target and block the other channels, as shown in Fig. 1(b). The only information required is the binary state of each element in the transmission matrix, i.e. whether their relative phases are in the range of π.

Fig. 1 Principle of binary intensity modulation. (a) Focusing through scattering samples without modulation. A reference electric field E_Ref at the target is the sum of electric fields coming from all channels. (b) After measuring the binary transmission matrix (TM), an enhanced focus at the target is achieved by blocking the channels which have destructive interference with E_Ref. (c) The combination of the reference and Hadamard basis is displayed on the DMD. The amplitude at the output, ${| E_{R H n} |}^{2}$ , can be used to calculate the binary TM (Section 2.1). Higher reference intensity gives a more accurate measurement. By scanning the speckles around the target, the maximum reference intensity is achieved at the target before measurement of the binary TM.

Download Full Size | PDF

When a plane wave $e_{0}$ reflects from a binary amplitude modulator, such as the DMD, the electric field after the modulator E_B, can be written as

E_{B} = A e_{0}

where

A = {[\begin{matrix} a_{1} & \dots & a_{N} \end{matrix}]}^{T}

is the binary modulation vector. Therefore the electric field at the output can be written as

E_{o u t} = K E_{B} = K A e_{0} .

We first define a reference electric field at the output channel, when all channels are turned on as shown in Fig. 1(a),

E_{R e f} = K A_{R e f} e_{0}, where A_{R e f} = {[\begin{matrix} 1 & \dots & 1 \end{matrix}]}^{T} .

Then to achieve constructive interference at the output, we need to block the channels which have destructive interference with E_Ref. That is to open the channels with output phase in the range of (φ_Ref–π/2, φ_Ref + π/2) and block the other channels as shown in Fig. 1(b), where φ_Ref is the phase of the reference. The resulting electric field at the output then becomes

E_{o u t} = K A_{K} e_{0} .

Since the input electric field e₀ is a plane wave, the binary modulation vector A_K is also a binary TM which contains binary information of the TM based on its relative phase to the reference electric field. For simplicity, we align the reference phasor E_Ref along the real axis. Therefore the elements of A_K, can be calculated as

a_{n} = {\begin{matrix} 1 & Re (k_{n} e_{0}) \geq 0 \\ 0 & Re (k_{n} e_{0}) < 0 \end{matrix}

where Re() is the real part of a complex vector. To obtain Re(k_ne₀), we chose the Hadamard basis as the input basis because of its orthogonal property [35]. The output electric field for different Hadamard modes is given by

[\begin{matrix} E_{H 1} & \dots & E_{H N} \end{matrix}] = K [\begin{matrix} H_{1} & \dots & H_{N} \end{matrix}] e_{0}

where E_Hn is the output electric field for the n^th Hadamard mode. H_n is the n^th Hadamard mode defined as a Nx1 vector, where N is the number of channels. Therefore Re(Ke₀) can be calculated by

Re (K e_{0}) = \frac{1}{N} [\begin{matrix} Re (E_{H 1}) & \dots & Re (E_{H N}) \end{matrix}] {[\begin{matrix} H_{1} & \dots & H_{N} \end{matrix}]}^{T}

where []^T is the transpose of the matrix. Although it is hard to directly measure Re(E_Hn), it can be estimated by the intensity of the sum of E_Hn and E_Ref at the output channel. Because values of the elements in the Hadamard matrix

H_{n}

are either −1 or 1, the addition of A_ref and a Hadamard mode can be obtained after the DMD,

E_{B I} = \frac{1}{2} (A_{R e f} + H_{n}) e_{0}

where the elements of

\frac{1}{2} (A_{R e f} + H_{n})

are either 0 or 1, which can be perfectly modulated on the DMD. The output electric field is the summation of the reference electric field E_Ref and the electric field for a Hadamard mode E_Hn at the output channel as shown in Fig. 1(c).

E_{R H n} = E_{H n} + E_{R e f}

Then the relationship between Re(E_Hn) and

| E_{R H n} |

can be calculated accordingly as

Re (E_{H n}) = β (\frac{{| E_{R H n} |}^{2}}{{| E_{R e f} |}^{2}} - \frac{{| E_{H n} |}^{2}}{{| E_{R e f} |}^{2}} - 1)

where

β = \frac{1}{2} | E_{R e f} |

is a constant. The derivation of Eq. (11) is presented in detail in the Appendix. If the intensity of E_Ref is sufficiently large that

{| E_{R e f} |}^{2} ≫ {| E_{H n} |}^{2}

then

Re (E_{H n}) ≅ β ({| E_{R H n} |}^{2} / {| E_{R e f} |}^{2} - 1) .

From Eq. (8) and Eq. (13), a vector J based on the intensity measurement is obtained, which can be used to determine the final binary TM, A_K.

J = [\begin{matrix} {| E_{R H 1} |}^{2} & \dots & {| E_{R H N} |}^{2} \end{matrix}] {[\begin{matrix} H_{1} & \dots & H_{N} \end{matrix}]}^{T} ≅ γ Re (K e_{0}) .

where

γ = \frac{N {| E_{R e f} |}^{2}}{β}

is a constant and N is the number of the channels. Since half of the channels are turned on, the threshold T can be selected so that the sum of matrix elements A_K is equal to N/2. The final result can be calculated as

a_{n} = {\begin{matrix} 1 & j_{n} \geq T \\ 0 & j_{n} < T \end{matrix}

This calculation is based on the assumption that the intensity of E_Ref is sufficiently large (12). However the relatively low intensity of the reference field in the real situation introduces errors to Eqs. (13)–(15). To evaluate the effect of the reference intensity on the performance of the binary intensity modulation, a Monte Carlo simulation was carried out by assuming a circular complex Gaussian distribution for the output electric field of each Hadamard mode. We generated 1000 groups of electric fields. Each group contained 1024 electric fields which correspond to 1024 Hadamard modes. The final binary patterns in each group are calculated by using Eq. (11) and Eq. (13) separately. To quantify the accuracy of the binary modulation, we compare these two binary patterns and calculate the estimation error of the final pattern, M, as

M = N_{e r r o r} / N_{t o t a l}

where N_error is the number of error channels. N_total is the total number of the channels. Different reference intensity levels are tested using the same procedure. The simulation results shown in Fig. 2 give the relationships between M and

{| E_{R e f} |}^{2} / 〈 I 〉

, where

〈 I 〉

is the mean of the speckle pattern. When the modulation is applied on a dark spot of the reference speckle pattern,

{| E_{R e f} |}^{2} / 〈 I 〉

is close to zero. Half the channels will get the wrong modulation. The contributions from the correct channels will be cancelled by those from the wrong channels.

Fig. 2 The relationship between the estimation error and the reference intensity.

Download Full Size | PDF

To overcome this issue, a reference optimization process is performed by using a scanner in the system to steer the beam along one axis on the sample when all the micromirrors on the DMD are on as shown in Fig. 1(c). The speckle pattern at the back of the scattering sample shifts with the scanning angle of the galvanometer. The intensity variation on the target location is monitored by a photomultiplier tube (PMT). The probability density function of the intensity is given as [36]

P (I) = \frac{1}{〈 I 〉} \exp (- \frac{I}{〈 I 〉})

Where

〈 I 〉

is the mean intensity. In this way, the brightest speckle is positioned onto the PMT by the scanning mirror for a reliable estimation of A_K. The probability that the intensity exceeds a given threshold I_t, is given as [36]

P (I > I_{t}) = \exp (- \frac{I_{t}}{〈 I 〉})

If we define the number of samples captured as m, the maximum intensity of those samples as I_max = max(I₁, …, I_m), the probability that I_max exceeds a given threshold I_t is given by

P (I_{\max} > I_{t}) = 1 - {(1 - \exp (- \frac{I_{t}}{〈 I 〉}))}^{m} .

The expectation value of I_max is given as follows,

Ε (I_{\max} / 〈 I 〉) = \int_{0}^{\infty} 1 - {(1 - \exp (- \frac{I_{t}}{〈 I 〉}))}^{m} d (\frac{I_{t}}{〈 I 〉}) .

Figure 3 shows the relationship between

Ε (I_{\max} / 〈 I 〉)

and m. With 10 samples on the scanning line,

Ε (I_{\max} / 〈 I 〉)

is around 2.9. From Fig. 2, the error of the final pattern would be around 12.6%. Increasing the samples during scanning can further decrease the error. By applying this method, we can dramatically improve focusing through a scattering sample. Here the pattern shift could be caused by the memory effect. If the speckle patterns are totally uncorrelated after mirror scanning, the intensity of the speckles still obeys the negative exponential statistics as defined in Eq. (17). The above analysis is still valid. In applications for laser scanning imaging systems, the existing galvanometric mirrors in the system can be utilized for reference optimization, which can reduce the cost and complexity of the system.

Fig. 3 Relationship between the sample number, m, and the expectation value of the maximum reference intensity $Ε (I_{\max} / 〈 I 〉)$ .

Download Full Size | PDF

2.2 System setup

The experimental setup is shown in Fig. 4. A DMD (DLi4130, 0.7” XGA, Digital Light Innovations) with 1024x768 mirrors and 22.727 kHz frames per second was used as a binary intensity modulator. A HeNe laser at 633nm (25-LHP-991, CVI Melles Griot) was employed as a light source in the experiment. In order to make the intensity more uniform across the aperture of the DMD, a telescope composed of lenses L1 (20x/0.40, Newport) and L2 (f = 150mm, AC254-150-A, Thorlabs) expands the beam by 16.7 times. The output beam from the telescope with a 1/e² diameter of 10.8 mm covers the exit pupil of the DMD. An iris diaphragm I1 was mounted after the telescope for avoiding artifacts from the edge of the window aperture coating on the DMD. The laser beam was directed to the DMD by a folding mirror. The incident angle of the beam on the DMD was adjusted carefully to achieve the blaze condition when the reflected beam from the micro mirror lines up with the sixth order of the diffraction grating from the DMD. The majority of the energy is directed into the blazed order. The other orders were blocked by the iris diaphragm I2. The efficiency of the DMD, defined as the ratio of the reflected light to the incident light on the DMD, is around 60% when all mirrors are in the full on-state. The DMD is conjugated with the galvanometer by lenses L3 (f = 150mm, AC254-150-A, Thorlabs) and L4 (f = 50mm, AC254-50-A, Thorlabs), which have a clear aperture of 4mm. Lenses L5 (f = 50mm, AC254-50-A, Thorlabs) and L6 (f = 150mm, AC254-150-A, Thorlabs) further conjugate the aperture of the galvanometer to the 10.8mm diameter rear pupil of a 10x, NA 0.3 objective O1 (PL FLUOTAR 10/0.3, Leitz). The sample, mounted on a three axis nanopositioning stage (NanoMax301, MELLES GRIOT), was placed in front of the objective. The light that is scattered when going through the sample was collected by another 10x, NA 0.25 objective O2 (PL 10/0.25, Leitz), focused by an imaging lens L7 (f = 200, AC254-200-A, Thorlabs) and then split into two orthogonal paths with a 10/90 beam splitter SB (BNP26K05510/90, RMI). 90 percent of light was fed into a PMT (H7422-20, Hamamatsu) for fast intensity measurement. A 20μm diameter pinhole (PH) was placed in front of the PMT to collect light from a 1.63 μm diameter area on the object plane. Another 10 percent of the light was captured by a CCD camera (M1400, Dalsa) for monitoring the change of the speckle pattern with a 5ms exposure time. To adjust the power of the laser, a polarizer was mounted on the output port of the laser, which is not shown in the figure.

Fig. 4 Experimental setup for interferometric focusing by binary measurement of the transmission matrix. The laser output from a HeNe laser (wavefront length λ = 633) is expanded by the lenses L1 and L2 and limited by an iris (I1). The beam covers the whole aperture of the DMD and is relayed by lenses L3 and L4 to a scanner. The unwanted high order beam is blocked by another iris (I2). The beam is further relayed by lenses L5 and L6 and focused on the sample by an objective lens (O1). The diffuse light after the sample is collected by another objective lens (O2) and focused on the PMT and a CCD camera by lens L7. The beam is divided by a 10/90 beam splitter (SB). A pinhole (PH) is installed in front of the PMT to collect the light only from the target.

Download Full Size | PDF

2.3 Operation principle

The existing intensity modulation method using a DMD requires updating the new mask on the DMD after the measurement for each of the input channels [29]. The time required for transferring the image from the computer to the DMD driver board does not allow for following the fast dynamic changes in live biological tissues or for scanning a focused beam. Furthermore, the sequential method for updating the state of one channel at a time based on the DMD-pixel basis introduces a low signal to noise ratio and slows the initial convergence. In the proposed method, the sequence of binary masks, based on the Hadamard basis set, is generated and preloaded onto the DMD driver board before performing the intensity modulation. For simplicity, all masks are square inside the pupil of the microscope objective. The intensity modulation process includes the four following steps:

(1) The first step is the reference intensity optimization. A galvanometer steers the beam with the target point at the center of the travel when all mirrors are in the full on-state. The intensity at the target point is monitored by the PMT during scanning and m samples evenly distributed on the scan line are recorded by a data acquisition card (DAQ) (PCIe-6363, NI). To obtain uncorrelated samples, the distance between two neighboring samples should be larger than the size of one speckle, which is determined by the optical resolution of the objectives. In the proposed system, the setting of 5μm works well for the specimens studied here. After obtaining the m samples, the galvanometer steers the beam to the location where the brightest speckle is on the target.
(2) The second step is the measurement of the binary TM. A sequence of preloaded Hadamard basis images are displayed on the DMD at the full speed of 22.7 kHz. The intensity change is monitored by the PMT. The DAQ synchronizes the DMD and PMT and records the measurement data.
(3) In the third step, the binary TM is calculated by Eqs. (14)–(15) and the data on the final mask is transferred to the DMD driver board.
(4) In the final step, the new mask is updated on the DLP and the exposure of the camera is triggered.

To automate the above process, a customized program running on a personal computer (Dell Precision T3610) was developed in C + + . The OpenCV library, optimized for Intel multi-core processors, was utilized to minimize the calculation time. When the 1024 Hadamard modes are applied during measurement and the exposure time of the camera is set as 5ms, the system can operate at 80 ms/frame as shown in Fig. 5(a). Figure 5(b) shows the time graph for one correction. The first 22ms is spent on the reference intensity optimization. It is followed by an interval of 45 ms for the measurement of the binary TM. Then the calculation and data transfer takes another 8 ms. The total time for one correction before the camera exposure is 75ms. In Section 3.1, different numbers of modes are tested. In Section 3.2 and 3.3, 1024 Hadamard modes are applied.

Fig. 5 Timing graph of the signal from PMT during system operation (a) and an enlarged time graph for one correction (b).

Download Full Size | PDF

2.4 Sample preparation

To validate the proposed method, a series of scattering specimens were prepared. First, we used a 220 grit ground glass diffuser (DG10-120, Thorlabs) as an artificial specimen. Then a fixed brain slice from a YFP-M line transgenic mouse was prepared. A brain coronal sample section with a thickness of 300μm was cut with a microtome. The sample was kept in Phosphate Buffered Saline (PBS) solution and stored at 4 °C until use. Custom-made chambers were made on glass slides for mounting the thick tissue. The sample was mounted in a chamber with 5% agarose. The experiments were performed within 10 hours of mounting. Another scattering biological sample is a section of 150μm thick mouse skull mounted on a glass slide with anti-fade reagents (Invitrogen) and covered with a coverslip. For the live biological specimen, GFP-Histone flies were reared on Formula 4-24 Drosophila Medium (Carolina Biological Supply Co.). Embryos were collected for one hour on grape juice agar then aged at room temperature for three hours. The embryos were adhered to the coverslips with heptane glue without dechorionation.

3. Experimental results

3.1 Focusing light through fixed samples

The proposed system was tested first using a 220 grit ground glass diffuser as the scattering medium. The diffuser is mounted in front of the objective L1. The scattered light at the back side of the diffuser is collected by the objective L2. Before applying the focusing, the random speckles are captured by the CCD camera as shown in Fig. 6(a). The aperture of the DLP is divided into 1024 channels. Before measurement, the reference intensity optimization is performed when all mirrors are in the full on-state. Then 1024 Hadamard binary patterns are displayed on the DLP. The final pattern is calculated where the half of the channels giving negative contribution of the focusing are turned off as shown in Fig. 6(b). This binary pattern generates a single sharp spot at the output channel as shown in Fig. 6(c).

Fig. 6 Focusing through a ground glass diffuser. (a) The speckle pattern on the image plane without intensity modulation. (b) The binary pattern projected on the DLP for intensity modulation. (c) The focus in the image plane after intensity modulation. (d) Intensity enhancement for different mode settings and comparison with the theoretical enhancement calculated in [29]. Error bars represent the standard error over 10 measurements. Scale bars, 5 µm.

Download Full Size | PDF

To quantify the improvement after focusing, the intensity enhancement is defined by the ratio of the intensity at the focus to the background intensity [26]. When ten measurements were collected for different positions on the diffuser, an enhancement of 68 ± 0.86 was achieved. To compare the intensity enhancements for different configuration modes, we performed the experiments using 256, 1024, 2304 and 4096 modes. The intensity enhancement for different modes settings is shown in Fig. 6(d). The enhancement increases linearly with the number of the modes used. In Fig. 6(d), the experimental results are also compared with the theoretical enhancement calculated in [29]. Although it is smaller than the ideally expected enhancement because of the measurement noise, this result is close to the experimental results reported in [29].

3.2 Scanning the focus through scattering media

The reference intensity optimization process can dramatically enhance the focus when the reference intensity is low. This can give a stable two dimensional (2D) scan through a scattering media with the DMD. Here the ground glass diffuser is tested. We applied the modulation at each pixel in a region of 40x40 pixels on the CCD camera. Then the stored masks can be replayed on the DMD at the rate of 22.727 kHz for a 2D scan. Here the image is captured by the CCD camera with a 5ms exposure time after each mask refreshment (Media 1).

Figure 7(a) shows the speckle pattern before the modulation. The enhancement map in a 2D plane without the reference intensity optimization is shown in Fig. 7(b). As can be seen, the dark areas in the reference speckle pattern get little improvement because of the inaccurate estimation of the binary TM according to Eq. (13). When the optimization is applied, much higher enhancement was achieved in the dark areas as shown in Fig. 7(c). The histogram of the enhancement in these two cases is shown in Fig. 7(d). The enhancement mean increases from 42.7 to 63.6. The standard deviation decreases from 19.1 to 11.1. By increasing the sampling number during the optimization, more stable enhancement can be achieved. In this experiment, because the images from the camera are used for calculation of the TM, the whole measurement takes 5.5 seconds. It can be used for a dynamic sample with a long decorrelation time.

Fig. 7 Scanning the focus through a ground glass diffuser. Without modulation, the speckle pattern is shown at the back of the ground glass (a). After applying the binary intensity modulation at each pixel on the image plane, the improvement at each pixel depends on the reference intensity (b). By applying the reference optimization, the improvement becomes more uniform throughout the field (c). The histogram of the enhancement in these two cases is shown in (d). Scale bars, 5μm. (Media 1)

Download Full Size | PDF

3.3 Focusing light through dynamic turbid media

To evaluate the ability of our system for focusing light through dynamic turbid media with high speed intensity modulation, three samples were tested, which includes a 220 grit ground glass diffuser, a fixed 300μm thick mouse brain slide and a 150μm thick mouse skull. The samples were mounted on a nanopositioning stage to mimic the situations where different decorrelation times are applied. The decorrelation time of a dynamic turbid media is the decay time of the autocorrelation of the speckle pattern, which shows the temporal stability of the sample [37]. To calculate the decorrelation times, a series of speckle images were captured by the CCD camera when the sample was moving. The normalized cross-correlations (NCC) between the images at the original position and other positions were calculated for different samples as shown in Fig. 8. After fitting a two-term Gaussian model, the half-width at half-maximum (HWHM) was calculated as the decorrelation radius of the sample, which is 3.02 µm, 2.63 µm and 3.29 µm for the ground glass diffuser, the mouse brain tissue and skull tissue, respectively. By moving the samples with an appropriate speed, different decorrelation times can be achieved. Since a single iteration time of the system with the 1024 modes measurement is 80ms, the systems can achieve a reliable focus when the decorrelation time of the sample is longer than 80ms.

Fig. 8 Determination of the decorrelation distance of the samples. The normalized cross-correlations (NCC) between the images of the samples at the original position and the ones at different distances are calculated for mouse brain tissue with a thickness of 300μm, the mouse skull with a thickness of 150μm, and a ground glass diffuser. The solid curves are two-term Gaussian model fitting. The decorrelation radius is calculated as the HWFW of the measurement.

Download Full Size | PDF

We first tested the ground glass diffuser with a decorrelation time of 80ms. The sample is under a reciprocating motion at a speed of 37.75 µm/s as shown in Fig. 9(a). Figure 9(b) shows the intensity change at the target without correction (blue curve), with a single correction at the starting point A (green curve) and with a dynamic correction (red curve). The corresponding images captured by a CCD camera at the four locations indicated in Figs. 9(a) and 9(b) are shown in Figs. 9(c)–9(f). Without intensity modulation, the speckle pattern shifts when the sample moves as shown in the bottom row in Figs. 9(c)–(f). The low intensity fluctuation is detected at the target point which is shown as the blue curve in Fig. 9 (b). When the binary intensity modulation is applied, a sharp focus is obtained at the target. Figures 9(c)–(e) show the images after the first three iterations. An enhanced intensity at the target is achieved during operation as shown in the red curve in Fig. 9(b). A single correction at the starting location cannot follow the dynamic change of the sample. After 0.16 second, the intensity at the target drops to a half as shown in Fig. 9(b). After 0.24 second, the focus is hard to differentiate from the background speckles as shown in Fig. 9(e). The focus comes back when the sample moves back to the location A (or D) as shown in Fig. 9(f).

Fig. 9 Focusing light though a moving diffuser. (a) The sample is under a reciprocating motion at a speed of 37.75 µm/s. (b) Three experiments were performed with dynamic modulation, single modulation and no modulation. The intensity changes for the first 1.5 seconds are shown as red, green and blue curves respectively. (c) The images captured from the CCD camera at 0.08, 0.16, 0.24 and 1.04 seconds are shown. These three situations are shown in the top, middle and bottom rows respectively. The corresponding phase is shown in the left corner of each image. Scale bars, 5μm. (Media 2)

Download Full Size | PDF

Then the proposed system was tested with different decorrelation times for different samples. 12 experiments were performed for the ground glass diffuser (Media 2), mouse skull tissue (Media 3), and brain tissue (Media 4), with decorrelation times of 40ms, 80ms, 160ms and 240ms, which is 0.5x, 1x, 2x and 3x of the system refresh time respectively. The 15 second videos for these three samples are recorded with a frame rate of 80ms. Figure 10 shows the mean and standard deviation of the enhancement for different conditions. When the decorrelation time is set at the refresh time (80ms), the proposed system can achieve enhancements of 51.57 ± 0.50, 52.45 ± 0.69 and 41.69 ± 0.48 with standard deviations of 6.92, 9.50 and 6.56 for the diffuser, mouse skull tissue and brain tissue, respectively. It can be seen that longer decorrelation time gives higher intensity enhancement and less variation. For the samples with decorrelation times smaller than the system refresh time, the TM measurement may not follow the fast changes of the media, but it can still get around 30 times improvement at the decorrelation time of 40ms.

Fig. 10 The image enhancement for the diffuser, mouse skull and brain tissue. The decorrelation time is set at 40ms, 80ms, 160ms and 240m, which is 0.5x, 1x, 2x and 3x of the system refresh time. The error bar represents the standard deviation over 190 samples during 15 seconds. (Media 3 for the mouse skull tissue, Media 4 for the mouse brain tissue)

Download Full Size | PDF

3.4 Focusing light through live biological sample

The ability to measure the binary TM quickly is essential to compensate dynamic scattering in live biological samples. Here, a three hour old Drosophila embryo without dechorionation was mounted on a slide with heptane glue. The focus plane was set at the back side of the embryo. While the egg-shell was the major source of static scattering, the rapidly changing inhomogeneous structures inside the embryo generated dynamic scattering. The decorrelation time was measured using the method discussed in Section 3.3. The speckle patterns were monitored by the CCD camera with a frame rate of 30Hz for 18 seconds (Media 5). The NCC between each frame with the first frame was calculated as shown in Fig. 11(a), which decays to one half at 3.5 seconds after measurement. The speckle patterns at 0 seconds and 3.5 seconds are shown in Figs. 11(b) and 11(c), which show a big difference between these two images. After that, the amplitude decayed slowly and went down to around 0.3 after 18 seconds. Figure 11(d) shows the image at 17 seconds. Similar patterns indicated by arrows can be observed in Figs. 11(b)–11(d). This could be caused by static scattering from the egg-shell.

Fig. 11 Measurement of decorrelation time for a drosophila embryo. (a) The NCC of the images captured during the first 18 seconds is calculated. The decorrelation time is 3.5 seconds. The images captured by the CCD camera at 0, 3.5 and 17 seconds are shown in (b), (c) and (d) respectively. Similar patterns are indicated by arrows. Scale bars, 5μm. (Media 5)

Download Full Size | PDF

To focus light through the embryo, two experiments were performed in series (Media 6). First, dynamic modulation was applied. The mask is refreshed every 80ms. Then images were recorded with an exposure time of 5ms for about 15 seconds. Next, we kept the same configuration but the images were captured with the same mask obtained from the first modulation. The enhancement with both dynamic modulation (blue) and single correction (red) during the first 15 seconds is shown in Fig. 12(a). The enlarged plot during the first 4 seconds is shown in Fig. 12(b). Figures 12(c) and 12(d) show the images at 0.8s, 1.28s, 3.52s and 5.04s for dynamic modulation and single modulation, respectively. The intensity mask is also shown at the lower left corner of each image. After turning on the binary intensity modulation, a sharp focus spot was achieved at the back side of the embryo after the first correction, as shown in the first image from the left in Fig. 12(c). With a single modulation, the static mask cannot compensate the dynamic change of the inner structure of the embryo. At the decorrelation time of 3.5s, the enhancement decays to almost one half as shown in Figs. 12(b) and 12(d). However with dynamic modulation, a stable focus is achieved with the enhancement around 50, as shown in Fig. 12(c). During the operation time of 15 seconds, the system can keep the enhancement at the mean of 50.3 and standard deviation of 6.36. Although the sample has a decorrelation of 3.5 seconds, the fast modulation ability can still benefit the intensity enhancement as shown in the images at 1.28s in Figs. 12(c) and 12(d).

Fig. 12 Focusing light though a live drosophila embryo. (a) The enhancement for single and dynamic modulation during the first 15 seconds is shown as the red and blue curves respectively. The enlarged view during the first 4 seconds is shown in (b). The images from the CCD camera at 0.8, 1.28, 3.52 and 5.04 seconds is shown in (c) and (d) with dynamic and single modulations, respectively. The lower left corner of the images shows the corresponding mask on the DMD. Scale bars, 5µm. (Media 6)

Download Full Size | PDF

4. Conclusion

We report a high-speed interferometric focusing method to compensate dynamic scattering in live biological tissue based on the fast measurement capability of the binary TM. Compared with the conventional intensity modulation method, the proposed method can directly measure the binary TM based on the measurement of the intensities from targets. Compared with the previously demonstrated binary amplitude modulation method using feedback information [29, 33, 34], the proposed method directly measures the binary TM after displaying preloaded measurement masks on the DMD. By using the Hadamard basis, the interference between the optical fields from the reference and each basis element is achieved by displaying the summation of a matrix with all ones and the Hadamard basis elements on the DMD. To overcome the estimation error caused by the low intensity of the reference field at the target, a reference optimization method is demonstrated which can give much more stable focusing during two dimensional scanning. By using a fast DMD as the binary intensity modulator, the proposed method can achieve a 75ms measurement time and an 80ms system refresh time. Although the proposed system is slower than the phase modulation method using an off-axis digital holograph made with a DMD [30, 31], the higher diffraction efficiency of the proposed method would give a higher SNR during interferometric measurement for multi-photon florescence imaging.

The advantages of high speed focusing through dynamic scattering tissue have been tested by a moving diffuser, mouse brain tissue and skull tissue. The experiments with a live drosophila embryo show its advantage for manipulation of light in live biological tissue. The ability to use the intensity from the target for calculation of the binary TM makes it suitable for fluorescence imaging and targeting. As a fast, simple, power-efficient and low-cost solution to deliver light through biological tissue, it has potential for a wide range of applications from basic biological research to clinical investigations.

Appendix

From the Fig. 13, the relationship between the imaginary part and real part of the optical field E_Hn is given by

Re {(E_{H n})}^{2} + Im {(E_{H n})}^{2} = {| E_{H n} |}^{2}

The similar relationship for the optical field E_RHn is given by

{(Re (E_{H n}) + | E_{R e f} |)}^{2} + Im {(E_{H n})}^{2} = {| E_{R H n} |}^{2}

where Im() is the imaginary part of a complex vector. Since we are not interested in

Im {(E_{H n})}^{2}

, Eq. (21) and (22) can now be written as

Re (E_{H n}) = \frac{1}{2 | E_{R e f} |} (\frac{{| E_{R H n} |}^{2}}{{| E_{R e f} |}^{2}} - \frac{{| E_{H n} |}^{2}}{{| E_{R e f} |}^{2}} - 1)

Here the amplitude of the reference,

| E_{R e f} |

, is constant at the target point if both the incoming beam and the sample are fixed. Then we can define a constant,

β = \frac{1}{2} | E_{R e f} |

. Equation (23) can be rewritten as

Re (E_{H n}) = β (\frac{{| E_{R H n} |}^{2}}{{| E_{R e f} |}^{2}} - \frac{{| E_{H n} |}^{2}}{{| E_{R e f} |}^{2}} - 1)

If

{| E_{R e f} |}^{2} ≫ {| E_{H n} |}^{2}

, then the intensity of E_RHn measured by the photodetector has a linear relationship with

Re (E_{H n})

as indicated in Eq. (13).

Fig. 13 The complex plane for the optical field of E_Hn, E_Ref and E_RHn.

Download Full Size | PDF

Acknowledgments

The results presented herein were obtained at the W.M. Keck Center for Adaptive Optical Microscopy (CfAOM) at University of California Santa Cruz. The CfAOM was made possible by the generous financial support of the W.M. Keck Foundation. This material is based upon work supported by the National Science Foundation under Grant Numbers 1353461 and 1429810. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. We would like to thank Prof. Yi Zuo and Dr. Eric Chen of MCD Biology at UCSC for the mouse tissue samples that were used in these experiments; Prof. William Sullivan, Dr. Shaila Kotadia and Dr. Frederic Landmann of MCD Biology, UC Santa Cruz, for the Drosophila embryo samples and useful conversations. We also acknowledge the technical support from Ben Abrams in the UCSC Life Sciences Microscopy Center.

References and links

1. J. A. Kubby, ed., Adaptive Optics for Biological Imaging (CRC, 2013).

2. M. J. Booth, M. A. Neil, R. Juskaitis, and T. Wilson, “Adaptive aberration correction in a confocal microscope,” Proc. Natl. Acad. Sci. U.S.A. 99(9), 5788–5792 (2002). [CrossRef] [PubMed]

3. P. Marsh, D. Burns, and J. Girkin, “Practical implementation of adaptive optics in multiphoton microscopy,” Opt. Express 11(10), 1123–1130 (2003). [CrossRef] [PubMed]

4. M. J. Booth, “Adaptive optics in microscopy,” Philos Trans A Math Phys Eng Sci 365(1861), 2829–2843 (2007). [CrossRef] [PubMed]

5. M. Rueckel, J. A. Mack-Bucher, and W. Denk, “Adaptive wavefront correction in two-photon microscopy using coherence-gated wavefront sensing,” Proc. Natl. Acad. Sci. U.S.A. 103(46), 17137–17142 (2006). [CrossRef] [PubMed]

6. D. Débarre, E. J. Botcherby, T. Watanabe, S. Srinivas, M. J. Booth, and T. Wilson, “Image-based adaptive optics for two-photon microscopy,” Opt. Lett. 34(16), 2495–2497 (2009). [CrossRef] [PubMed]

7. N. Ji, D. E. Milkie, and E. Betzig, “Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues,” Nat. Methods 7(2), 141–147 (2010). [CrossRef] [PubMed]

8. C. Wang, R. Liu, D. E. Milkie, W. Sun, Z. Tan, A. Kerlin, T. W. Chen, D. S. Kim, and N. Ji, “Multiplexed aberration measurement for deep tissue imaging in vivo,” Nat. Methods 11(10), 1037–1040 (2014). [CrossRef] [PubMed]

9. X. Tao, B. Fernandez, O. Azucena, M. Fu, D. Garcia, Y. Zuo, D. C. Chen, and J. Kubby, “Adaptive optics confocal microscopy using direct wavefront sensing,” Opt. Lett. 36(7), 1062–1064 (2011). [CrossRef] [PubMed]

10. X. Tao, J. Crest, S. Kotadia, O. Azucena, D. C. Chen, W. Sullivan, and J. Kubby, “Live imaging using adaptive optics with fluorescent protein guide-stars,” Opt. Express 20(14), 15969–15982 (2012). [CrossRef] [PubMed]

11. N. Olivier, D. Débarre, and E. Beaurepaire, “Dynamic aberration correction for multiharmonic microscopy,” Opt. Lett. 34(20), 3145–3147 (2009). [CrossRef] [PubMed]

12. X. Tao, A. Norton, M. Kissel, O. Azucena, and J. Kubby, “Adaptive optical two-photon microscopy using autofluorescent guide stars,” Opt. Lett. 38(23), 5075–5078 (2013). [CrossRef] [PubMed]

13. K. Wang, D. E. Milkie, A. Saxena, P. Engerer, T. Misgeld, M. E. Bronner, J. Mumm, and E. Betzig, “Rapid adaptive optical recovery of optimal resolution over large volumes,” Nat. Methods 11(6), 625–628 (2014). [CrossRef] [PubMed]

14. W. Cheong, S. A. Prahl, and A. J. Welch, “A review of optical properties of biological tissues,” IEEE J. Quantum Electron. 26(12), 2166–2185 (1990). [CrossRef]

15. S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13(12), 123021 (2011). [CrossRef]

16. A. P. Mosk, A. Lagendijk, G. Lerosey, and M. Fink, “Controlling waves in space and time for imaging and focusing in complex media,” Nat. Photonics 6(5), 283–292 (2012). [CrossRef]

17. Z. Yaqoob, D. Psaltis, M. S. Feld, and C. Yang, “Optical phase conjugation for turbidity suppression in biological samples,” Nat. Photonics 2(2), 110–115 (2008). [CrossRef] [PubMed]

18. M. Cui and C. Yang, “Implementation of a digital optical phase conjugation system and its application to study the robustness of turbidity suppression by phase conjugation,” Opt. Express 18(4), 3444–3455 (2010). [CrossRef] [PubMed]

19. S. M. Popoff, G. Lerosey, R. Carminati, M. Fink, A. C. Boccara, and S. Gigan, “Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media,” Phys. Rev. Lett. 104(10), 100601 (2010). [CrossRef] [PubMed]

20. S. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Image transmission through an opaque material,” Nat. Commun. 1(6), 81 (2010). [CrossRef] [PubMed]

21. I. M. Vellekoop, A. Lagendijk, and A. P. Mosk, “Exploiting disorder for perfect focusing,” Nat. Photonics 4(5), 320–322 (2010). [CrossRef]

22. I. M. Vellekoop and A. P. Mosk, “Focusing coherent light through opaque strongly scattering media,” Opt. Lett. 32(16), 2309–2311 (2007). [CrossRef] [PubMed]

23. O. Katz, E. Small, Y. Bromberg, and Y. Silberberg, “Focusing and compression of ultrashort pulses through scattering media,” Nat. Photonics 5(6), 372–377 (2011). [CrossRef]

24. I. M. Vellekoop, E. G. van Putten, A. Lagendijk, and A. P. Mosk, “Demixing light paths inside disordered metamaterials,” Opt. Express 16(1), 67–80 (2008). [CrossRef] [PubMed]

25. J. Tang, R. N. Germain, and M. Cui, “Superpenetration optical microscopy by iterative multiphoton adaptive compensation technique,” Proc. Natl. Acad. Sci. U.S.A. 109(22), 8434–8439 (2012). [CrossRef] [PubMed]

26. C. Stockbridge, Y. Lu, J. Moore, S. Hoffman, R. Paxman, K. Toussaint, and T. Bifano, “Focusing through dynamic scattering media,” Opt. Express 20(14), 15086–15092 (2012). [PubMed]

27. M. Cui, “A high speed wavefront determination method based on spatial frequency modulations for focusing light through random scattering media,” Opt. Express 19(4), 2989–2995 (2011). [CrossRef] [PubMed]

28. M. Nixon, O. Katz, E. Small, Y. Bromberg, A. A. Friesem, Y. Silberberg, and N. Davidson, “Real-time wavefront shaping through scattering media by all-optical feedback,” Nat. Photonics 7(11), 919–924 (2013). [CrossRef]

29. D. Akbulut, T. J. Huisman, E. G. van Putten, W. L. Vos, and A. P. Mosk, “Focusing light through random photonic media by binary amplitude modulation,” Opt. Express 19(5), 4017–4029 (2011). [CrossRef] [PubMed]

30. D. B. Conkey, A. M. Caravaca-Aguirre, and R. Piestun, “High-speed scattering medium characterization with application to focusing light through turbid media,” Opt. Express 20(2), 1733–1740 (2012). [CrossRef] [PubMed]

31. A. M. Caravaca-Aguirre, E. Niv, D. B. Conkey, and R. Piestun, “Real-time resilient focusing through a bending multimode fiber,” Opt. Express 21(10), 12881–12887 (2013). [CrossRef] [PubMed]

32. S. A. Goorden, J. Bertolotti, and A. P. Mosk, “Superpixel-based spatial amplitude and phase modulation using a digital micromirror device,” Opt. Express 22(15), 17999–18009 (2014). [CrossRef] [PubMed]

33. X. Zhang and P. Kner, “Binary wavefront optimization using a genetic algorithm,” J. Opt. 16(12), 125704 (2014). [CrossRef]

34. J. Jang, J. Lim, H. Yu, H. Choi, J. Ha, J. H. Park, W. Y. Oh, W. Jang, S. Lee, and Y. Park, “Complex wavefront shaping for optimal depth-selective focusing in optical coherence tomography,” Opt. Express 21(3), 2890–2902 (2013). [CrossRef] [PubMed]

35. A. Hedayat and W. D. Wallis, “Hadamard matrices and their applications,” Ann. Stat. 6(6), 1184–1238 (1978). [CrossRef]

36. J. W. Goodman, Statistical Properties of Laser Speckle Patterns (Springer-Verlag, 1975), Chap. 2, 8–75.

37. I. M. Vellekoop and C. M. Aegerter, “Focusing light through living tissue,” Proc. SPIE 7554, 755430 (2010). [CrossRef]

High-speed scanning interferometric focusing by fast measurement of binary transmission matrix for channel demixing

Abstract

1. Introduction

2. Materials and methods

2.1 Measurement of the binary TM

2.2 System setup

2.3 Operation principle

2.4 Sample preparation

3. Experimental results

3.1 Focusing light through fixed samples

3.2 Scanning the focus through scattering media

3.3 Focusing light through dynamic turbid media

3.4 Focusing light through live biological sample

4. Conclusion

Appendix

Acknowledgments

References and links

Supplementary Material (6)

Cited By

Figures (13)

Equations (24)

Optics Express