Open Access
Add to BibSonomy
Abstract
Fluorescence lifetime imaging microscopy (FLIM) has been widely used in the field of biological research because of its high specificity, sensitivity, and quantitative ability in the sensing cellular microenvironment. The most commonly used FLIM technology is based on time-correlated single photon counting (TCSPC). Although the TCSPC method has the highest temporal resolution, the data acquisition time is usually long, and the imaging speed is slow. In this work, we proposed a fast FLIM technology for fluorescence lifetime tracking and imaging of single moving particles, named single particle tracking FLIM (SPT-FLIM). We used feedback-controlled addressing scanning and Mosaic FLIM mode imaging to reduce the number of scanned pixels and the data readout time, respectively. Moreover, we developed a compressed sensing analysis algorithm based on alternating descent conditional gradient (ADCG) for low-photon-count data. We applied the ADCG-FLIM algorithm on both simulated and experimental datasets to evaluate its performance. The results showed that ADCG-FLIM could achieve reliable lifetime estimation with high accuracy and precision in the case of a photon count less than 100. By reducing the photon count requirement for each pixel from, typically, 1000 to 100, the acquisition time for a single frame lifetime image could be significantly shortened, and the imaging speed could be improved to a great extent. On this basis, we obtained lifetime trajectories of moving fluorescent beads using the SPT-FLIM technique. Overall, our work offers a powerful tool for fluorescence lifetime tracking and imaging of single moving particles, which will promote the application of TCSPC-FLIM in biological research.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Tracking the movement of biological particles could promote the understanding of the life activities in the cell [1–3]. For example, many macromolecules such as hormones, neurotransmitters and enzymes are transported in and out of cell membranes through vesicles [4]. However, the regulation mechanism of intracellular vesicles transport has not been fully revealed until now. Direct observation of the interaction of moving particles in living cells will be undoubtedly helpful for understanding the motion mechanism [5]. In 2013, Balint et al. showed the transport of lysosomes on live cell microtubules driven by motor proteins [6] using single particle tracking (SPT) technology and stochastic optical reconstruction microscopy (STORM). However, because the trajectory and structure obtained by this method were based on fluorescence intensity, the interaction information of lysosomes during transport could not be directly provided. Compared with fluorescence intensity, fluorescence lifetime measurement makes it easier to monitor the intermolecular interactions.
Fluorescence lifetime imaging microscopy (FLIM) [7] is widely used to study intracellular life activities by measuring the lifetime of fluorophores. Fluorescence lifetime, referred to as the average residence time of the fluorophore in the excited state, is independent of excitation light intensity, fluorophore concentration and photobleaching. It changes sensitively with the interaction between fluorescent molecules and other molecules or the variation of surrounding microenvironmental parameters such as pH, ion concentration, viscosity and refractive index [8–11]. Therefore, fluorescence lifetime is often used to monitor molecular interactions or quantitatively measure the microenvironmental parameters [12–16]. However, the current FLIM technology is limited on real-time tracking and imaging of moving particles because of the relatively slow imaging speed.
FLIM experiments can be conducted either in time- or frequency- domain [12]. In time-domain approaches, time-correlated single photon counting (TCSPC) is the most commonly used technique to detect fluorescence lifetime [17]. TCSPC-FLIM performs single photon counting over every effective excitation cycle, and then estimates fluorescence lifetime by fitting the final histogram of the photon count distribution for each pixel. The advantages of TCSPC-FLIM usually includes high temporal resolution, high sensitivity, and large dynamic range [18,19]. However, sufficient photon counts are required in the lifetime analysis process, while low counting rate is crucial during detection to avoid histogram distortion. This contradiction is resolved by repeat scanning to obtain a fluorescence lifetime image, which results in a slow imaging speed however, and makes TCSPC-FLIM difficult to adapt to the tracking and imaging of moving particles [20–23]. To solve this problem, researchers have tried to shorten the data acquisition time in TCSPC-FLIM through both hardware and software optimizations. For example, Hirvonen et al. implemented wide-field TCSPC-FLIM with an SPAD array integrated with time-to-digital converters, greatly reducing the data acquisition time and increasing the imaging speed [24]. However, this method has some shortcomings such as low photon detection efficiency, and has not reached the stage of mature application. Koenig et al. used a TCSPC module with ultra-short dead time and a hybrid PMT to reduce the system dead time to about 1 ns, thus increasing its counting capability [25]. Sorrells et al. proposed a method for calculating single photon counts from directly sampled time-domain FLIM data allowing accurate fluorescence lifetime and intensity measurements and can be imaged up to two times faster [26]. Margara et al. have built a high performance, multi-channel TCSPC acquisition system that uses dedicated cameras and parallel processing of data via GPUs to greatly speed up processing [27].
The frequency-domain FLIM can be divided into wide-field mode and scanning mode, depending on the chosen microscopic imaging method. The advantages of wide-field frequency-domain FLIM include the fast imaging speed, the ease of modulating a continuous laser, the lack of convolution and the ability to obtain information from all pixels simultaneously. However, it is hard to distinguish two very different lifetimes, is susceptible to photobleaching and sample motion, has poor chromatography capabilities, and has poor temporal resolution and signal-to-noise ratio. In 2012, Zhao et al. constructed an all-solid-state camera with on-chip phase-sensitive detection [28], and in 2016 Raspe et al. used the camera to develop a single-image fluorescence lifetime microscope (siFLIM) [29], which is faster and could minimize photobleaching and eliminates the effects of sample movement. The frequency-domain FLIM in scanning mode has a strong layer analysis capability and high signal-to-noise ratio, but the imaging speed is slow and photobleaching is severe. In 2021, Zhang et al. built an instant FLIM system to present the first in vivo 4D FLIM of microglial dynamics in intact and injured zebrafish and mouse brains for up to 12 h [30].
In order to improve the imaging speed of existing TCSPC-FLIM equipment to obtain fluorescence lifetime information of moving targets in biological samples, we proposed a single particle tracking FLIM (SPT-FLIM) technique based on feedback-controlled addressing scanning and Mosaic FLIM mode imaging. Additionally, a compressed sensing lifetime analysis algorithm based on alternating descent conditional gradient (ADCG) was developed and applied to analyze the TCSPC-FLIM data of really low photon count (such as 100 and below). Our method reduces the size of the scanning area, the data readout time and the required photon count per pixel, greatly improving the lifetime imaging speed and realizing the fluorescence lifetime tracking and imaging of moving particles using TCSPC.
2. Method
2.1 Optical setup and feedback-controlled addressing scanning
FLIM was performed on a customized two-photon excited inverted fluorescence microscope, which used two orthogonal acousto-optic deflectors (AODs) instead of galvanometers to conduct addressing scanning [31]. As shown in Fig. 1(a), a Ti: Sapphire laser (Chameleon Ultra II, Coherent) provides the excitation light with a repeat frequency of 80 MHz and a wavelength of 800 nm. A pair of AODs (DTSXY-400-640, A-A Optoelectronic) are controlled by the digital signals from a DAQ card (USB-6353, National Instruments). Addressing scanning is realized by loading specific acoustic frequencies on the acousto-optic crystals to determine the deflection angle of the diffraction light. The scanning rate is 10 kHz. A dispersion prism is used prior to the AODs to pre-compensate both spatial and temporal dispersions resulting from the crystals [32]. The scanning beam arrives at the sample plane through an 100×/1.45 NA objective (Nikon), and the fluorescent emission is collected with the same objective. The fluorescence is split and simultaneously detected by a PMT (PMC-100-0, Becker & Hickl) with a TCSPC card (SPC-150, Becker & Hickl GmbH) for lifetime imaging and an EMCCD (DU897, Andor) for intensity imaging.
Fig. 1. Diagram of the optical setup (a) and the idea of feedback-controlled addressing scanning and Mosaic FLIM mode collecting (b) for fluorescence lifetime tracking and imaging of single moving particles.
A way to achieve fluorescence lifetime tracking and imaging of moving targets is to combine SPT and feedback-controlled addressing scanning method with TCSPC-FLIM technology. The basic idea is to select an appropriate small area for scanning and imaging the moving particle, and use the real-time localization results to feedback control the movement of the scanning area, realizing the tracking scanning of moving objects. As illustrated in Fig. 1(b), we first look for the moving target of interest in the large pre-selected imaging field of view (FOV), and then select a scanning region containing the target as small as possible (determined according to particle size and motion). While the TCSPC module obtains the fluorescence lifetime data of this area, the EMCCD camera collects the fluorescence intensity image and a SPT program is used to locate the target from the image. The AOD scanning area is updated using the centroid of the particle as the center. This cycle continues until the target is outside of the FOV of the EMCCD. In this way, the scanning area could follow the target movement as small as possible, which reduces the single frame data acquisition time and improves the imaging speed.
Besides, we introduced the Mosaic FLIM mode [33] provided by Becker & Hickl GmbH to reduce the time required for data readout. Mosaic FLIM is an acquisition mode used to record an array of FLIM images. The “elements” of the mosaic may represent lifetime images for different lateral offset, sample depth, or wavelength channels. Here, we used them to represent the lifetime data collected at different times. Instead of recording one single frame of FLIM image into an individual dataset, this mode records the entire mosaic into one single, large photon distribution dataset, so that no time for readout operations has to be reserved between each two elements. In this way, it can reduce the total time required for data readout and is suitable for fast FLIM imaging.
2.2 ADCG-FLIM analysis algorithm
Besides the number of pixels in the scanning area and the data readout time, the number of photon count accumulated in each single pixel is also a key factor that could affect the time of single frame lifetime image acquisition. Because the photon counting histogram collected by the TCSPC method is a discrete sampling of the fluorescence decay curve, the commonly used fluorescence lifetime analysis methods usually require sufficient sample size (i.e., photon count) to obtain accurate lifetime estimation. For instance, the most commonly used least-square (LS) fitting method performs well when the photon count is more than 1000. The maximum likelihood estimate (MLE) method can reduce the photon count requirement to about 300 with a relatively high signal-to-noise ratio (SNR). However, in order to avoid distortion caused by photon accumulation and count loss, the photon count rate per pulse cycle in TCSPC-FLIM imaging is usually less than 1. Therefore, repeat scanning is generally needed to accumulate enough photon counts for each pixel, which is the main reason for the relatively slow imaging speed in TCSPC-FLIM. Some of the research in nowadays also devoted to lifetime estimation in the low photon number case. Chen et al. used a deep learning approach based on generative adversarial networks to estimate lifetimes relatively quickly under low photon count conditions [34]. Fazel et al. proposed a Bayesian non-parametric framework to provide a means to deconvolve lifetime maps from direct photon arrival analysis with subpixel spatial resolutions with limited photon counts [35]. These studies have been able to estimate lifetimes at as low as 50 photon counts without considering noise. However, fast FLIM data typically implies low photon counts with low SNRs. Taimori et al. proposed a game theoretic model to estimate fluorescence lifetimes with single-exponential decay quickly and robustly at low SNRs, and demonstrated the stability of lifetime estimation at a photon count level of 350 [36].
In order to further improve the imaging speed and optimize the FLIM technique for moving particle tracking and imaging, we here developed a fluorescence lifetime analysis algorithm, ADCG-FLIM algorithm [37], to analyze fast TCSPC-FLIM data with both low photon counts (≤100) and low SNRs, by taking into account the background noise in the detection model for a compressed sensing method. Since the number of fluorescent lifetime components is sparse, a compressed sensing algorithm is suitable for the estimation. The basic idea of compressed sensing is to obtain the source signal (decay parameters including lifetime) from the measured data (photon count distribution) according to the detection model (TCSPC-FLIM), and this is a sparse inverse problem [38,39]. There is an approach to solving parametric sparse inverse problems via an augmented version of the classical conditional gradient method, which is the ADCG method [40]. The ADCG algorithm employs both the rapid local convergence of nonlinear programming algorithms and the stability and global convergence guarantees associated with convex optimization [37]. Here we use ADCG method to estimate lifetime according to the TCSPC-FLIM detection model.
In TCSPC-FLIM, the measured histogram ƒ(t) (photon counts over arriving times) is the convolution of fluorescence exponential decay function g(t) and the instrument response function (IRF) XIRF(t) of the system,
where τ is the lifetime and b(t) is the background noise. Generally, the XIRF(t) can be approximated as a Gaussian distribution,Therefore, the convolution in Eq. (1) can be:
The detection model can be also expressed in matrix form as:
where y is the measured signal (photon count histogram); Φ is the observation matrix, which describes the lifetime detection process in TCSPC-FLIM. Specifically, it is the matrix form of the convolution results in Eq. (5). b is the background noise, and x represents the decay parameters, including lifetime τ and total photon count w. When x is sufficiently sparse, it can be exactly recovered by computing the minimum solution of a loss function l(Φx, y) [41]:According to the above explanation, we wrote a Python program and realized the lifetime analysis on TCSPC-FLIM data. We termed the algorithm as ADCG-FLIM. The main flow of the algorithm is shown in Fig. 2.
2.3 Simulations
In order to evaluate the performance of the ADCG-FLIM algorithm for analyzing TCSPC lifetime data of low photon count with different noise levels, the simulation program Laine [42] was used to generate simulated TCSPC-FLIM datasets under different conditions. For simplicity, the decay model is single exponential. The number of time bins in TCSPC detection is 256, and the image size is 150 × 150 pixels. Referring to the parameters of the real system, σIRF is set to be 0.05 ns. Both Poisson noise and background noise are added to the datasets. Fig. S1 shows the intensity (in terms of photon count) images of two simulated datasets. In dataset I (Fig. S1a), the three blocks from left to right in each row correspond to different fluorescence lifetimes (1.0 ns, 2.0 ns and 3.0 ns, respectively), while the background noise is uniformly 10% of the total photon count. In dataset II (Fig. S1b), the fluorescence lifetimes are uniformly 2.0 ns, while the five blocks from left to right in each row correspond to different background noise levels (10%, 20%, 30%, 40% and 50%, respectively). For both datasets, the three rows from top to bottom correspond to different total photon count per pixel (50, 100 and 1000, respectively).
2.4 Sample preparation
We used the solution of Rhodamine B with a concentration of 1 mmol/L as the homogeneous experimental sample for evaluating the ADCG-FLIM algorithm. Then we used fluorescent beads (F8888, Et/Em: 505/515, Molecular Probes Inc.) with a diameter of 200 nm as a motion sample for SPT-FLIM tracking and imaging. They were dissolved in a 2:1 mixture of glycerol and water.
3. Results and discussion
3.1 Evaluation of ADCG-FLIM algorithm on simulated data
The ADCG-FLIM algorithm was used to analyze the simulated datasets described above, and the estimation results were compared with those obtained with the traditional LS fitting algorithm (termed LS-FLIM here) and MLE algorithm (termed MLE-FLIM here). Fig. 3 shows the comparisons of the reconstructed lifetime images and estimated lifetime histograms, and Fig. 4 shows the lifetime estimation errors and precisions, respectively, for each situation. It is worth noting that the MLE algorithm sometimes generated obvious anomalies (e.g., lifetime values greater than 100 ns), especially in low photon count data with high noise levels. These estimated lifetimes were then replaced with 0 ns (as shown by red color pixels on the middle panels in Fig. 3(a) and (c)) for displaying the reconstructed lifetime image completely. They were also eliminated before we carried out statistics to avoid statistical errors.
Fig. 3. Reconstructed fluorescence lifetime images (a, c) and corresponding estimated lifetime histograms (b, d) of the simulated dataset I (a-b) and II (c-d) analyzed by ADCG-FLIM (left panels), MLE-FLIM (middle panels) and LS-FLIM (right panels) algorithms, respectively. Note: MLE-FLIM algorithm sometimes generated obvious outliers (e.g., lifetime values greater than 100 ns). These estimated lifetimes were replaced with 0 ns (shown by red color pixels) for displaying the reconstructed lifetime image completely.
Fig. 4. Statistical results for the estimated lifetimes of the simulated dataset I (a) and II (b) obtained with ADCG-FLIM, MLE-FLIM and LS-FLIM algorithms, respectively. The mean absolute error (MAE, left panels) between the estimated lifetime and the true value over the pixels in each block illustrates the lifetime estimation accuracy. The standard deviation (St. Dev., right panels) of the distribution of estimated lifetimes describes the estimation precision. Note: Outliers produced by MLE-FLIM algorithm were eliminated before statistics.
Fig. 3(a)-(b) and Fig. 4(a) show the results for dataset I, from which we could evaluate the performance of the ADCG-FLIM algorithm for TCSPC-FLIM data with different lifetimes and photon counts. For the 1000-photon-count data, it is obvious that ADCG-FLIM, LS-FLIM and MLE-FLIM (despite several abnormal pixels shown by the white arrows on the middle panel in Fig. 3(a)) are able to estimate the lifetimes accurately, as shown in both the reconstructed lifetime images (the bottom rows in Fig. 3(a)) and the histograms (the red lines in Fig. 3(b)). Not surprisingly, all three algorithms perform worse when the photon count decrease to a low level of 50 and 100 (the top and middle rows in Fig. 3(a), and the green and blue lines in Fig. 3(b), respectively). However, ADCG-FLIM is found to do much better than LS-FLIM does in these cases, especially for short lifetime data (1.0 ns, the left peaks in Fig. 3(b)). To quantitatively evaluate and compare the performance of the different methods, we calculated the mean absolute error (MAE) between the estimated lifetime and the true value over the 50 × 50 pixels in each block to describe the lifetime estimation accuracy. Besides, we also calculated the standard deviation (St. Dev.) of the distribution of estimated lifetimes to describe the estimation precision. The graphs on the left panel in Fig. 4(a) show the comparison of lifetime estimation errors (MAE) among ADCG-FLIM, MLE-FLIM and LS-FLIM, while the graphs on the right panel in Fig. 4(a) show the comparisons of estimation precision (St. Dev.). In the case of 1000 photon count (the bottom rows in Fig. 4(a)), both the estimation errors and precisions obtained by the three methods are similar, which is consistent with what we observed directly from the lifetime images and histograms in Fig. 4(a) and (b). Specifically, the MAE values are 0.06 ns, 0.11 ns and 0.20 ns for ADCG-FLIM, 0.05 ns, 0.11 ns and 0.18 ns for MLE-FLIM, and 0.06 ns, 0.14 ns and 0.24 ns for LS-FLIM, respectively. Compared to the true lifetime value (1.0 ns, 2.0 ns and 3.0 ns), the relative estimation errors of the three methods range from 5% to 8%, which is an acceptable level. However, in the case of low photon count of 50 or 100 (top and middle rows in Fig. 4(a)), the MAE value for ADCG-FLIM analysis is significantly smaller than that for LS fitting under the same conditions. When compared with MLE-FLIM, the difference is not as significant, while it is bigger at longer lifetime (3.0 ns) than shorter lifetimes (2.0 ns and 1.0 ns). But it is worth noting that MLE-FLIM generated obvious anomalies which we eliminated before doing statistics. Specifically, when the photon count is 100, the MAE values are 0.16 ns, 0.34 ns and 0.53 ns for ADCG-FLIM, 0.16 ns, 0.37 ns and 0.67 ns for MLE-FLIM, and 0.30 ns, 0.53 ns and 0.89 ns for LS-FLIM, respectively. And there are 87 outliers for MLE-FLIM. When the photon count further decreases to 50, the MAE values increase to 0.23 ns, 0.48 ns and 0.74 ns for ADCG-FLIM, 0.24 ns, 0.58 ns and 1.20 ns for MLE-FLIM, and 0.52 ns, 0.85 ns and 1.88 ns for LS-FLIM, respectively. And there are 429 outliers for MLE-FLIM. Compared to the true lifetime value, the relative errors for LS-FLIM range from 43% to 63%, which means that LS-FLIM is actually difficult to estimate the lifetimes in this situation. In the same conditions, ADCG-FLIM performs much better, obtaining a relative error range from 23% to 25% without eliminating any outliers, which shows the robust estimation ability of ADCG-FLIM compared with MLE-FLIM. The graphs on the right panel in Fig. 4(a) show the standard deviations of the estimated lifetimes in each image block, representing the estimation precisions for different methods in each situation. It is shown that both ADCG and MLE analysis are able to maintain a relatively high level of estimation precision under conditions of different photon counts and lifetimes, with ADCG slightly outperforming MLE at low photon counts (top and middle rows). However, LS fitting shows a sharp decrease in estimation precision in the case of low photon count, and the longer the lifetime is, the lower the precision is. Overall, these results show that the ADCG-FLIM algorithm can be well applied to analyze lifetimes from TCSPC-FLIM data. The performance is similar to that of the traditional LS fitting and MLE analysis methods in the case of high photon count, but the estimation capability for low-photon-count data is significantly improved.
In real experiments, low-photon-count data usually also means low signal-to-noise ratio. To evaluate the performance of the ADCG lifetime analysis for TCSPC-FLIM data with different background noise levels, we analyzed the simulated dataset II. The comparison results in Fig. 3(c)-(d) and Fig. 4(b) show that for the 1000-photon-count data, all three algorithms are able to accurately estimate the lifetimes at different noise levels, and the estimation accuracy and precision are high, even though the photon count of the background noise is as high as 50% of the total photon count. But it is still worth noting that MLE analysis produced outliers, even at a low noise level (10%, shown by the three circles in Fig. 3(c)). However, for the 50- or 100-photon-count data, ADCG-FLIM is obviously shown to perform much better than LS-FLIM does, especially when the noise level is high. Both the estimated lifetime errors and standard deviations of LS fitting are significantly increased with the increase in noise level. For instance, in the case of 50 photon count, when the noise level is as high as 40% or 50%, the MAE values for LS-FLIM are 1.93 ns and 2.47 ns, respectively. Compared with the true lifetime value (2.0 ns), the relative estimation error is close to or even more than 100%. Meanwhile, the standard deviation is several times the true lifetime value. These results indicate that the analysis results of LS-FLIM in these situations are completely unreliable. In contrast, ADCG-FLIM is not sensitive to noise levels, and it is able to analyze and obtain the lifetime from low-photon-count data with a high noise level. This can be attributed to the truth that background noise has been taken into account in the detection model for compressed sensing analysis. When compared with MLE-FLIM, ADCG-FLIM only performs slightly better in terms of estimation error and precision (Fig. 4(b)). Specifically, when the photon number is 50, the MAE values range from 0.49 ns to 0.86 ns for ADCG-FLIM, and range from 0.59 to 0.97 ns for MLE-FLIM, while the standard deviations range from 0.65 ns to 1.18 ns for ADCG-FLIM, and range from 0.81 to 1.27 ns for MLE-FLIM, respectively. However, we should note that there are in total 152 anomalies for MLE analysis that were eliminated before carrying out statistics, further indicating that ADCG is more stable than MLE at low signal-to-noise ratios.
3.2 Evaluation of ADCG-FLIM algorithm on experimental data
We also evaluated the performance of the ADCG-FLIM algorithm on experimental data and again compared with the results of MLE-FLIM and LS-FLIM. The homogeneous solution of Rhodamine B was used as the sample, and the TCSPC-FLIM imaging was performed using the system described in the previous section. In order to compare the results with the simulation, we collected both high- and low-photon-count datasets. The maximum photon counts per pixel in the two datasets are around 1000 and 100, respectively, while the average photon counts are 896 and 93, respectively. Figure 5(a) and (f) show the intensity images for the two datasets. The image size is 128 × 128 pixels. It is worth noting that the total photon (including signal photons and background noise photons) count in each pixel in the experimental dataset is different, and the level of background noise is unknown, which is slightly different from the simulation. In the case of high photon count, ADCG-FLIM, MLE-FLIM and LS-FLIM are able to stably estimate the lifetime as shown in Fig. 5(b)-(d). The average estimated lifetimes obtained by ADCG-FLIM, MLE-FLIM and LS-FLIM are 1.58 ± 0.06 ns, 1.61 ± 0.08 ns and 1.58 ± 0.14 ns (Fig. 5(e)), respectively. However, as shown in Fig. 5(g)-(i), in the case of low-photon-count data, ADCG-FLIM estimates the lifetimes better than MLE-FLIM and LS-FLIM do. Specifically, the average estimated lifetimes calculated from the histograms in Fig. 5(j) are 1.58 ± 0.18 ns, 1.52 ± 0.24 ns and 1.63 ± 0.47 ns for ADCG-FLIM, MLE-FLIM and LS-FLIM, respectively. That is to say, the analysis result obtained by ADCG-FLIM with low-photon-count (around 100) data is similar to that achieved by LS-FLIM with high-photon-count (around 1000) data, which is usually regarded as a reliable measurement result. In contrast, carrying out LS fitting on low-photon-count data produced an obviously deviated result. The performance of MLE-FLIM is between the two, while we can also conclude from the comparison between ADCG-FLIM and MLE-FLIM that the former is more stable.
Fig. 5. Comparison results of lifetime analyses on experimental datasets of both high (a-e) and low (f-j) photon counts using ADCG-FLIM, MLE-FLIM and LS-FLIM algorithms, including fluorescence intensity images (a, f), reconstructed lifetime images by ADCG-FLIM (b, g), MLE-FLIM (c, h) and LS-FLIM (d, i), respectively, and the corresponding histograms of estimated lifetimes (e, j). Image size: 128 × 128 pixels.
The evaluation results on experimental datasets are consistent with those obtained from analyses on simulated datasets. And the finding that ADCG-FLIM is able to conduct lifetime estimation on low-photon-count data as reliably as traditional LS-FLIM does on high-photon-count data, and is more stable than MLE-FLIM makes us confident to combine this lifetime analysis algorithm into the above-mentioned FLIM system to realize lifetime tracking and imaging of moving particles. The combined technique is termed SPT-FLIM.
3.3 Moving particle tracking and lifetime imaging using SPT-FLIM
The proposed SPT-FLIM technique employs feedback-controlled addressing scanning, Mosaic FLIM mode imaging and ADCG-FLIM analysis algorithm to decrease the number of scanned pixels, the readout time and the photon count requirement in each pixel, respectively, to minimize the acquisition time for single frame lifetime image. To demonstrate that SPT-FLIM is able to be used for fluorescence lifetime tracking and imaging of moving particles, we used fluorescent beads dissolved in a 2:1 mixture of glycerol and water as a motion sample.
After the moving fluorescent bead of interest was found in the larger FOV intensity image captured by the EMCCD camera in real time, we selected a small scanning area of 16 × 16 pixels around the bead, and began the feedback-controlled addressing scanning and the Mosaic FLIM imaging procedure. 8 × 8 elements were set for a mosaic data acquisition, which means 64 consecutive frames of lifetime image were collected at different times (and subsequently at different positions) without data readout time between adjacent frames (Fig. S2, and Visualization 1). Using the same sample, preliminary experiments showed that, for each mosaic element, it required 20 repeat scans (with a corresponding scan time of 0.512 s) to obtain an average photon count of 1000 per pixel when the excitation power on the sample plane is 5 mW. However, when the average photon count requirement decreased to 100, just one tenth of that in the former situation, only two repeat scans (with a scan time of only 0.051 s accordingly) were needed, significantly reducing the data acquisition time for a single frame of lifetime image, and improving the imaging speed of TCPSC-FLIM. Considering the time required for centroid localization, feedback control and other communications (∼0.077 s) between two adjacent mosaic elements, the effective imaging frame rate in this experiment was 7.8 fps. Therefore, by using the ADCG-FLIM algorithm to reduce the photon count requirement from 1000 to 100, the scan time of a single frame of FLIM image is shorten to be 1/10, while the realistic imaging speed is improved by a factor of 4.6. As a control, collecting a normal 128 × 128 image with 1000 photon counts per pixel without feedback-control addressing scanning needs 32.8 s (an equivalent frame rate of only 0.03 fps).
Figure 6 shows the trajectory and lifetime analyses of a typical fluorescent bead captured by SPT-FLIM. Figure 6(a) is the fluorescence lifetime trajectory, in which the coordinates of bead come from the localization results obtained during the feedback-control addressing scanning procedure, and the pseudo-colored lifetimes come from the ADCG-FLIM analysis on the mosaic elements. This lifetime trajectory is definitely different from a normal trajectory obtained in simple SPT experiments, for that it also provides the real-time variations of the fluorescence lifetime along the motion path. In this demonstration experiment, since the fluorescence bead moves in a relatively uniform microenvironment, the fluorescence lifetime should be stable accordingly. Therefore, Fig. 6(b) analyzes the estimated lifetimes of bead in each frame of lifetime image, and an average lifetime is calculated to be 2.42 ± 0.15 ns. The measured lifetime does fluctuate within a relatively small range, but the result is close to that (2.42 ± 0.10 ns) obtained by LS fitting on a 1000-photon-count dataset for a static bead. This result shows that the ADCG-FLIM algorithm maintains its good performance for low-photon-count data analysis when analyzing the data of moving samples.
Fig. 6. Trajectory and lifetime analyses of a typical moving fluorescent bead captured by SPT-FLIM, including the reconstructed fluorescence lifetime trajectory (a), estimated lifetime in each frame (b) and MSD analysis of the trajectory (c).
Figure 6(c) calculated the mean square displacement (MSD) [43] of the trajectory and conducted a linear fitting with the time intervals. The result shows a good linear relationship between them, indicating that the captured motion of the fluorescence bead is a Brownian motion. According to the slope of the fitting curve, the two-dimensional diffusion coefficient D (1/4 of the slope) is around 0.027 µm2/s. It can be theoretically calculated as D = kBT/ (3πηa) to be 0.121 µm2/s, where kB is Boltzmann's constant, T is the sample temperature (estimated at 20°C/293 K), η is the viscosity coefficient of the solution (0.0177 Pa•s [44]), and a is the particle diameter (200 nm). The measured bead diffusion coefficient is close to but smaller than the theoretical value, which may due to the fact that the tracked bead was close to the interface between the glycerol/water mixture and the coverslip.
4. Conclusion
We have developed a SPT-FLIM technique for fluorescence lifetime tracking and imaging of moving objects, mainly based on feedback-controlled addressing scanning, Mosaic FLIM mode imaging and a compressed sensing analysis algorithm, ADCG-FLIM, for low-photon-count TCSPC-FLIM data. The method improves the imaging speed of TCSPC-FLIM in three aspects. Firstly, it speeds up the imaging by reducing the size of the scanning area (and consequently the number of scanned pixels), utilizing the real-time localization information of the moving target to update the AOD scanning region. Secondly, the data readout time is minimized by using the Mosaic FLIM mode. Thirdly, and most importantly, the ADCG-FLIM algorithm reduces the requirement of photon count per pixel to less than 100, which significantly shorten the acquisition time for single frame lifetime images.
The ADCG-FLIM algorithm implements analysis on low-photon-count data by global conditional gradient search and non-convex local search alternately, which enable it to estimate the accurate lifetime from low-photon-count data with high noise level. Evaluation results on both simulated and experimental datasets shows that compared with LS-FLIM and MLE-FLIM, the performance of ADCG-FLIM is similar in the case of high photon count data, but it is far better in the case of low photon count data, in terms of smaller estimation errors, higher estimation precisions (compared with LS-FLIM), and higher stability (compared with MLE-FLIM). However, since the algorithm is based on compressed sensing, the run time for an image of 150 × 150 pixels is around 2.3 min. Therefore, it is suitable for post-processing but not real time display. It is also worth stating that the current version of the ADCG-FLIM algorithm only works well for single-exponential decay data and dual-exponential data with known lifetimes, and the performance is not as good when either the lifetime of each component or the number of lifetime components is unknown. The main reason is that, for multi-exponential models, the increasing number of unknown variables (i.e., lifetimes and proportions) and the coupling between them make the estimation process of the algorithm even more time-consuming and may lead to inaccurate local optimal solutions. Since real samples may exhibit more than one fluorescence lifetime, or lifetime heterogeneity, it is important to continue improving the ADCG-FLIM algorithm.
The lifetime trajectory of moving fluorescent beads obtained with SPT-FLIM has validated the technique. Tracking Brownian motion with a measured two-dimensional diffusion coefficient of 0.027 µm2/s with speeds of up to 11.1 µm/s was demonstrated. This is much lower than the theoretical trackable limit, which for a scanning area of 16 × 16 pixels and an effective frame rate of 7.8 fps is more than 20 µm/s. However, the distribution sparsity and the fluorescence intensity of the moving objects, along with the required localization accuracy and lifetime accuracy, would affect the real tracking speed and should be taken into account in specific experiments. For macromolecules in biological samples, a lifetime trajectory may reflect real-time interactions between molecules along the motion path, which might provide direct information associating with some important mechanisms. We believe the proposed SPT-FLIM technique assisted by the ADCG-FLIM algorithm can benefit applications demanding real-time fluorescence lifetime imaging of moving particles in similar situations.
Funding
National Key Research and Development Program of China (2022YFF0706001, 2022YFF0712500, 2021YFF0502900); National Natural Science Foundation of China (62275165, 62235007, 61835009, 61975131, 62127819, 62175166,); Shenzhen Fundamental Research Program (JCYJ20200109105411133).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but maybe obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. H. Deschout, F. C. Zanacchi, M. Mlodzianoski, A. Diaspro, J. Bewersdorf, S. T. Hess, and K. Braeckmans, “Precisely and accurately localizing single emitters in fluorescence microscopy,” Nat. Methods 11(3), 253–266 (2014). [CrossRef]
2. L. Liang, J. Li, Q. Li, Q. Huang, J. Shi, H. Yan, and C. Fan, “Single-particle tracking and modulation of cell entry pathways of a tetrahedral DNA nanostructure in live cells,” Angew. Chem. Int. Ed. 53(30), 7745–7750 (2014). [CrossRef]
3. Y. Ma, E. Pandzic, P. R. Nicovich, Y. Yamamoto, J. Kwiatek, S. V. Pageon, A. Benda, J. Rossy, and K. Gaus, “An intermolecular FRET sensor detects the dynamics of T cell receptor clustering,” Nat. Commun. 8(1), 15100 (2017). [CrossRef]
4. K. O. Brien, K. Breyne, S. Ughetto, L. C. Laurent, and X. O. Breakefield, “RNA delivery by extracellular vesicles in mammalian cells and its applications,” Nat. Rev. Mol. Cell Biol. 21(10), 585–606 (2020). [CrossRef]
5. S. Hou, C. Johnson, and K. Welsher, “Real-time 3D single particle tracking: towards active feedback single molecule spectroscopy in live cells,” Molecules 24(15), 2826 (2019). [CrossRef]
6. S. Bálint, I. V. Vilanova, Á. S. Álvarez, and M. Lakadamyal, “Correlative live-cell and superresolution microscopy reveals cargo transport dynamics at microtubule intersections,” Proc. Natl. Acad. Sci. U. S. A. 110(9), 3375–3380 (2013). [CrossRef]
7. J. R. Lakowicz, Principles of Fluorescence Spectroscopy (Springer, 2006).
8. X. Liu, D. Lin, W. Becker, J. Niu, B. Yu, L. Liu, and J. Qu, “Fast fluorescence lifetime imaging techniques: a review on challenge and development,” J. Innov. Opt. Health Sci. 12(05), 783–790 (2019). [CrossRef]
9. A. J. Bower, J. Li, E. J. Chaney, M. Marjanovic, D. R. Spillma, and S. A. Boppart, “High-speed imaging of transient metabolic dynamics using two-photon fluorescence lifetime imaging microscopy,” Optica 5(10), 1290–1296 (2018). [CrossRef]
10. E. F. Fornasiero, S. Mandad, H. Wildhagen, et al., “Precisely measured protein lifetimes in the mouse brain reveal differences across tissues and subcellular fractions,” Nat. Commun. 9(1), 4230 (2018). [CrossRef]
11. C. Ma, W. Sun, L. Xu, Y. Qian, J. Dai, G. Zhong, Y. Hou, J. Liua, and B. Shen, “A minireview of viscosity-sensitive fluorescent probes: design and biological applications,” J. Mater. Chem. B 8(42), 9642–9651 (2020). [CrossRef]
12. W. Becker, “Fluorescence lifetime imaging - techniques and applications,” J. Microsc. 247(2), 119–136 (2012). [CrossRef]
13. K. Suhling, L. M. Hirvonena, J. A. Levitt, P. Chung, C. Tregidgoa, A. L. Marois, D. A. Rusakov, K. Zheng, S. Ameer-Beg, S. Poland, S. Coelhoc, R. Hendersone, and N. Krstajic, “Fluorescence lifetime imaging (FLIM): basic concepts and some recent developments,” Med. Photonics 27, 3–40 (2015). [CrossRef]
14. N. Hirmiz, A. Tsikouras, E. J. Osterlund, M. Richards, D. W. Andrews, and Q. Fang, “Highly multiplexed confocal fluorescence lifetime microscope designed for screening applications,” IEEE J. Sel. Top. Quantum Electron. 27(5), 1–9 (2021). [CrossRef]
15. C. García, A. Losada, M. A. Sacristán, J. F. Martínez-Leal, C. M. Galmarini, and M. P. Lillo, “Dynamic cellular maps of molecular species: Application to drug-target interactions,” Sci. Rep. 8(1), 1140 (2018). [CrossRef]
16. M. Jonsson, A. Ehn, M. Christensen, M. Aldén, and J. Bood, “Simultaneous one-dimensional fluorescence lifetime measurements of OH and CO in premixed flames,” Appl. Phys. B 115(1), 35–43 (2014). [CrossRef]
17. W. Becker, “Fluorescence lifetime imaging by multi-dimensional time correlated single photon counting,” Med. Photonics 27, 41–61 (2015). [CrossRef]
18. L. M. Hirvonen, W. Becker, J. Milnes, T. Conneely, S. Smietana, A. L. Marois, O. Jagutzki, and K. Suhling, “Picosecond wide-field time-correlated single photon counting fluorescence microscopy with a delay line anode detector,” Appl. Phys. Lett. 109(7), 071101 (2016). [CrossRef]
19. R. Datta, T. M. Heaster, J. T. Sharick, A. A. Gillette, and M. C. Skala, “Fluorescence lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis, and applications,” J. Biomed. Opt. 25(07), 1 (2020). [CrossRef]
20. S. Isbaner, N. Karedla, D. Ruhlandt, S. C. Stein, A. Chizhik, I. Gregor, and J. Enderlein, “Dead-time correction of fluorescence lifetime measurements and fluorescence lifetime imaging,” Opt. Express 24(9), 9429–9445 (2016). [CrossRef]
21. A. Cominelli, G. Acconcia, P. Peronio, M. Ghioni, and I. Rech, “High-speed and low-distortion solution for time-correlated single photon counting measurements: A theoretical analysis,” Rev. Sci. Instrum. 88(12), 123701 (2017). [CrossRef]
22. P. Peronio, G. Acconcia, I. Rech, and M. Ghioni, “Improving the counting efficiency in time-correlated single photon counting experiments by dead-time optimization,” Rev. Sci. Instrum. 86(11), 113101 (2015). [CrossRef]
23. J. Qi, Y. Shao, L. Liu, K. Wang, T. Chen, J. Qu, and H. Niu, “Fast flexible multiphoton fluorescence lifetime imaging using acousto-optic deflector,” Opt. Lett. 38(10), 1697–1699 (2013). [CrossRef]
24. L. M. Hirvonen and K. Suhling, “Wide-field TCSPC: methods and applications,” Meas. Sci. Technol. 28(1), 012003 (2017). [CrossRef]
25. M. Koenig, S. O. Mueller, R. Dowler, B. Kraemer, A. Tannert, O. Schulz, T. Roehlicke, M. Wahl, H. J. Rahn, M. Patting, F. Koberling, and R. Erdmann, “Rapid FLIM: The new and innovative method for ultra-fast imaging of biological processes,” Biophys. J. 112(3), 298a (2017). [CrossRef]
26. J. E. Sorrells, R. R. Iyer, L. Yang, E. J. Chaney, M. Marjanovic, H. Tu, and S. A. Boppart, “Single-photon peak event detection (SPEED): a computational method for fast photon counting in fluorescence lifetime imaging microscopy,” Opt. Express 29(23), 37759–37775 (2021). [CrossRef]
27. A. Margara, P. Peronio, G. Acconcia, G. Cugola, and I. Rech, “High-accuracy and video-rate lifetime extraction from time correlated single photon counting data on a graphical processing unit,” Rev. Sci. Instrum 90(10), 104709 (2019). [CrossRef]
28. Q. Zhao, B. Schelen, R. Schouten, R. Oever, R. Leenen, H. Kuijk, I. M. Peters, F. Polderdijk, J. T. Bosiers, M. Raspe, K. Jalink, J. Jong, B. Geest, K. Stoop, and I. T. Young, “Modulated electron-multiplied fluorescence lifetime imaging microscope: all-solid-state camera for fluorescence lifetime imaging,” J. Biomed. Opt 17(12), 126020 (2012). [CrossRef]
29. M. Raspe, K. M. Kedziora, B. V. D. Broek, Q. Zhao, S. D. Jong, J. Herz, M. Mastop, J. Goedhart, T. W. J. Gadella, I. T. Young, and K. Jalink, “siFLIM: single-image frequency-domain FLIM provides fast and photon-efficient lifetime data,” Nat. Methods 13(6), 501–504 (2016). [CrossRef]
30. Y. Zhang, I. H. Guldner, E. L. Nichols, D. Benirschke, C. J. Smith, S. Zhang, and S. S. Howard, “Instant FLIM enables 4D in vivo lifetime imaging of intact and injured zebrafish and mouse brains,” Optica 8(6), 885–897 (2021). [CrossRef]
31. Q. Wu, J. Qi, D. Lin, W. Yan, R. Hu, X. Peng, and J. Qu, “Simultaneous acquisition of trajectory and fluorescence lifetime of moving single particles,” Multiphoton Microscopy in the Biomedical Sciences XVII 10069, 1006922 (2017). [CrossRef]
32. S. Zeng, X. Lv, C. Zhan, W. R. Chen, W. Xiong, S. L. Jacques, and Q. Luo, “Simultaneous compensation for spatial and temporal dispersion of acousto-optical deflectors for two-dimensional scanning with a single prism,” Opt. Lett. 31(8), 1091 (2006). [CrossRef]
33. A. J. Walsh, A. Sedelnikova, G. P. Tolstykh, B. L. Ibey, and H. T. Beier, “Fluorescence lifetime imaging of calcium flux in neurons in response to pulsed infrared light,” Multiphoton Microscopy in the Biomedical Sciences XVII 10069, 100691B (2017). [CrossRef]
34. Y. Chen, Y. Chang, S. Liao, T. D. Nguyen, J. Yang, Y. Kuo, S. Hong, Y. Liu, H. G. Rylander III, S. R. Santacruz, T. E. Yankeelov, and H. Yeh, “Generative adversarial network enables rapid and robust fluorescence lifetime image analysis in live cells,” Commun. Biol 5(1), 18 (2022). [CrossRef]
35. M. Fazel, S. Jazani, L. Scipioni, A. Vallmitjana, E. Gratton, M. A. Digman, and S. Pressé, “High resolution fluorescence lifetime maps from minimal photon counts,” ACS photonics 9(3), 1015–1025 (2022). [CrossRef]
36. A. Taimori, D. Humphries, G. Williams, K. Dhaliwal, N. Finlayson, and J. Hopgood, “Fast and robust single-exponential decay recovery from noisy fluorescence lifetime imaging[J],” IEEE Trans. Biomed. Eng. 69(12), 3703–3716 (2022). [CrossRef]
37. X. Zhang, D. Lin, J. Niu, X. Liu, J. Zhang, B. Yu, W. Zhang, and J. Qu, “Low photon count fluorescence lifetime analysis based on alternating descent conditional gradient method,” Zhongguo Jiguang 47, 0207022 (2020).
38. S. Yang, J. Lee, Y. Lee, M. Lee, and B. Lee, “Estimation of multiexponential fluorescence decay parameters using compressive sensing,” J. Biomed. Opt 20(9), 096003 (2015). [CrossRef]
39. V. Mannam, Y. Zhang, X. Yuan, C. Ravasio, and S. S. Howard, “Machine learning for faster and smarter fluorescence lifetime imaging microscopy,” J. Phys. Photonics 2(4), 042005 (2020). [CrossRef]
40. N. Boyd, G. Schiebinger, and B. Recht, “The alternating t descent conditional gradient method for sparse inverse problems,” SIAM J. Optim. 27(2), 616–639 (2017). [CrossRef]
41. D. L. Donoho, “For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution,” Comm. Pure Appl. Math. 59(6), 797–829 (2006). [CrossRef]
42. R. F. Laine, “TCSPC-image-simulation,” Github, 2019, https://github.com/Romain-Laine/TCSPC-image-simulation.
43. D. Lin, W. Ma, X. Liu, Y. Sun, and Y. Wu, “Three-Dimensional imaging and tracking for constrained Brownian motion of single Nano-scaled Particles in solution,” Proceedings - 1st International Congress on Image and Signal Processing, CISP4, 244–248 (2008).
44. J. B. Segur and H. E. Oberstar, “Viscosity of glycerol and its aqueous solutions,” Ind. Eng. Chem. 43(9), 2117–2120 (1951). [CrossRef]
