1. Introduction

Fluorescence lifetimes are critical parameters for discriminating between two samples that have similar excitation and emission spectra. Information concerning molecular dynamics is also obtainable from lifetimes [1,2]. Thus, fluorescence-lifetime-imaging microscopy (FLIM) is widely used in biochemistry, biophysics, and analytical chemistry, and for observing living cells labeled by fluorescent probes [3,4].

FLIMs are categorized into two groups in terms of light detection. (i) A scanning FLIM uses a single-channel detector, such as a photomultiplier tube (PMT) or an avalanche photodiode; (ii) a wide-field FLIM uses a gated image intensifier and a two-dimensional (2-D) imaging sensor, such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor camera. In case (i), the sample stage or the focused excitation source is scanned mechanically [5,6], while the fluorescence is detected by the single-channel detector. Weak fluorescence can be detected with a high spatial resolution; and with a confocal microscope, three-dimensional depth resolution can be obtained [7]. However, mechanical scanning can be slow, making it difficult to image biological samples that rapidly photo-bleach. High-speed scanning with a galvanometer mirror device requires careful attention to mechanical vibrations. In case (ii), wide-field FLIM allows the whole imaging area to be acquired at the same time and the fluorescence can be time-resolved with the gated intensifier. However, the sensitivity of the imaging detector is usually lower than that of the single-channel detector.

In FLIM, there are two methods for sample excitation: pulsed or sinusoidally modulated. With pulsed light, the fluorescence decay can be monitored by delaying the gate with respect to the excitation. However, the time resolution is determined by the pulse width of the gate, which is at least 5.0 ns. In addition, it takes a long time to acquire the whole decay waveform, and the signal-gathering efficiency is extremely low. One way to overcome this problem is to use several time gates for each excitation [8]. However, this is difficult for multi-component samples because the fluorescence decays need to be single exponentials. Furthermore, the repetition rate of the excitation is limited by the frame rate of 2-D detectors. High-speed cameras are expensive [9] and are limited by their image size in pixels.

Phase modulation is an alternative FLIM technique that uses a sinusoidally modulated light source. Fluorescence lifetimes are estimated from the phase difference and/or the modulation ratio between reference and fluorescence waveforms. One efficient method is to prepare four time gates during one modulation period, each phase-shifted in turn by π/2 [10]. Another uses a mode-locked laser and a gated, intensified CCD camera [11] to acquire “lifetime-selective” fluorescence images. The phase-modulation method is less damaging to biological samples than pulsed excitation. However, for multi-component samples, several modulation frequencies must be used. (In fact, plural modulation frequencies should be used to verify single-component samples.) Finally, it is difficult to detect short-lived fluorescent molecules that have low quantum efficiencies.

To address the above problems, we have developed a Hadamard-transform fluorescence-lifetime imaging (HT-FLI) technique. It is a combination of a previously reported Fourier-transform phase-modulation fluorometer (FT-PMF) [12], and Hadamard-transform imaging (HTI) [13–15]. The use of HTI for fluorescence-lifetime measurements has been proposed [16]; however, the emphasis was on multiplexing advantages. Although the signal-to-noise ratio (SNR) and the signal-gathering efficiency are increased, the question remains whether the multiplex advantage is retained when a quantum photon detector such as a PMT is used. Here, we combined the two methods with the goal of solving the sensitivity and time resolution problems. An electronically driven cyclic HT excitation pattern solves the low frame-rate problem of 2-D detectors [13]. HT-FLI enables us to obtain fluorescence-lifetime images for every component in a sample. Although such an image could be obtained with a conventional FLIM, it is cumbersome for multi-component samples, and often results in averaged fluorescence-lifetime mapping [4–10].

The primary aim here is to discuss the HT-FLI concept, and demonstrate a proof-of-principle prototype. For convenience, the prototype has an 8 × 8-pixel, laboratory-made, light-emitting diode (LED) array as an excitation source, and an inline optical arrangement for fluorescence detection. Therefore, spatial resolution is determined by the 4.0-mm physical interval between neighboring LEDs. Because of the 50-MHz upper limit of the LED modulation frequency, the lower limit for fluorescence lifetimes is a few nanoseconds. Because those specifications are insufficient for a practical FLIM, we present guidelines for future FLIM construction.

2. FT-PMF and HTI

The HT-FLI system consists of the FT-PMF and the HTI units, which are described here. The former is used for the measurement of the fluorescence lifetime and the latter for the imaging. The combined HT-FLI system will be explained in the next section.

2.1 FT-PMF

FT-PMF has been discussed in detail elsewhere [12]. Briefly, it is a phase-modulation fluorometer, where the modulation frequency of the excitation source is swept (chirped) linearly from f_min to f_max during a fixed time T, as shown in Fig. 1. Then, the excitation waveform has the amplitude and phase distribution in the frequency domain as depicted in the figure. The amplitude and phase of the resulting fluorescence waveform varies with the modulation frequency. Therefore, from Fourier transforms of the excitation (reference) and fluorescence waveforms, the modulation ratio $m(f)=A_{\text{em}}(f)/A_{\text{ex}}(f)$, and the phase difference $\theta (f)={\theta }_{\text{em}}(f)-{\theta }_{\text{ex}}(f)$, of the two waveforms as a function of the modulation frequency f yield the impulse response function of the fluorescent sample. A is the amplitude and Δθ is the phase, and the subscripts em and ex denote emission and excitation, respectively. Data processing used in conventional phase modulation yields the fluorescence lifetimes. If necessary, we can obtain a fluorescent decay waveform by setting $f_{\text{min}}=0$ [12]. FT-PMF can also be used for multi-component samples. For a data-sampling frequency f_s, the number of data points L is given by $L=T{f}_{\text{s}}$, and the frequency resolution is given by $(f_{\max }-f_{\min })/f_{\text{s}}$.

 

Fig. 1 Schematic of the FT-PMF working principle.

Download Full Size | PDF

2.2 HTI unit

Figure 2(a) shows the working principle of the HTI system, which consists of a light source, N Hadamard masks, a sample object G, and a point detector. Light fluxes passed through the individual Hadamard masks and the sample object are gathered sequentially by the point detector, resulting in N time-series data y. In the HTI unit, we used a structured illumination light source made from an $n\times n$ LED array that is a simple extension of the one-dimensional HTI procedure described below. As shown in Fig. 2(b), we divide the observation (imaging) area into $n\times n$ pixels, from which we make an ($N\times 1$) column vector $g$ (where $N=n^2$) that represents the sample object G. By multiplying an $N\times N$ mask matrix $M$ with $g$, we obtain the $N\times 1$ observation vector $y=Mg$[Fig. 2(c)]. $M$ can be derived from an $N\times N$ Hadamard matrix $H$ as follows. All elements in the first row and the first column of $H$ are set to unity and the remainder is set to a $(N-1)\times (N-1)$ cyclic Hadamard matrix $H\text{'}$. If we create an $N\times N$ matrix $B$, where all of the elements are unity, we can make an $N\times N$ mask matrix via $M=(B-H)/2$. As shown in Fig. 2(d), elements of $M$ consist of 1 and 0, where we assign the value 1 to light transmission and 0 to light blockage. Then, the elements of the first row and the first column of $M$ become zero. The remaining elements are also replaced so that 1 and −1 in $H\text{'}$ become 0 and 1, respectively. Therefore, $M$ is still cyclic. The original cyclic pattern in $H\text{'}$ is generated by quadratic residue construction described in [13].

 

Fig. 2 (a) Schematic of the HTI system, (b) generation of an N × 1 object column vector g from an n × n object matrix (G), where N = n², (c) an N × 1 observation vector y is obtained by multiplying an N × N mask matrix (M) with g: y = Mg, and (d) generation of n illumination mask pattern n × n matrices from (M).

Download Full Size | PDF

If we set the first element of $g$ to $g_1=-(g_2+\cdots +g_N)$, then $g=(g_1,g_2,\cdots ,g_N)$. This is possible because all elements of the first row and the first column in $M$ are zero, and the value of $g_1$ does not affect the measurements. Then, because $Bg=0$, $y=Mg=(B-H)g/2=-Hg/2$ and $H^{\text{T}}y=-H^{\text{T}}Hg/2$, where $H^{\text{T}}$ is the transpose of $H$. Because $H$ is a normalized matrix, $H^{\text{T}}H=NI$, where $I$ is an $N\times N$ identity matrix. Thus, $g=-2H^{\text{T}}y/N=-2Hy/N$, because $H\text{'}$ is a cyclic matrix and $H^{\text{T}}=H$.

The illumination mask pattern for the $n\times n$ LED array is generated from the $N\times N$ mask matrix $M$. As shown in Fig. 2(d), we sequentially take $n$ sets of values from the first row of $M$, where each set has $n$ elements, and create an $n\times n$ square matrix. This is the first illumination mask pattern. The second $n\times n$ pattern is obtained from the second row of $M$ by a similar procedure. Henceforth, the $N$th $n\times n$ illumination mask pattern is obtained from the $N$th row of $M$. The $N$ illumination mask patterns can still work if the 0 and 1 entries in $M$ are reversed. Also, the first illumination mask pattern becomes a “black” pattern because entries in the first row of $M$ are zero. Therefore, the first element of the observation vector $y$ is zero: $y_1=0$. In addition, the (1,1)th element of the reconstructed object image is always zero because entries in the first column of $M$ are zero.

3. Experimental apparatus

3.1 Block diagram of the HT-FLI system

Figure 3 shows a schematic of the HT-FHI system. The excitation light is a laboratory-aligned $n\times n$, 525-nm LED array (OSTG1608C1A, OptoSupply), which is sequentially turned on and off by the illumination mask patterns. In our prototype, $n=8$. The interval between neighboring LEDs was 4.0 mm and the whole pattern is imaged 1:1 on the sample plane by lens L1. Fluorescence emitted from the sample is focused by lens L2 onto a photomultiplier tube (PMT; R7400U, Hamamatsu Photonics) after passing through a long-wavelength pass filter LPF in a collinear, afocal configuration. The afocal system prevents non-uniformity of the illumination pattern at the cost of fluorescence intensity on the PMT. We acquire $N(=n^2=64)$ sets of time-series data sequentially and synchronized with every exchange of the illumination mask pattern.

 

Fig. 3 Schematic of HT-FLI; (a) cyclic Hadamard mask patterns; (b) chirped excitation waveforms; (c) fluorescence waveforms; (d) N pairs of real and imaginary spectra; (e) k(<<N) pairs of real and imaginary images obtained from (d) by an inverse Hadamard transform; (f) amplitude ratio m and phase difference θ images calculated from (e); (g) shading image corresponding to each fluorescence lifetime.

Download Full Size | PDF

The timing diagram for acquisition of $N$ sets of time-series data is shown in Fig. 3(a–c). The modulation frequencies of the “on-state” LEDs are swept (chirped) linearly from the minimum frequency $f_{\min }(=0\text{Hz)}$ to the maximum frequency $f_{\max }(=50\text{MHz)}$ repeatedly within every period $T(=2.0\text{μs)}$ of the illumination (excitation) mask pattern, as shown in Fig. 3(b). The data-sampling interval is $\Delta t(=400\text{ps:}{f}_{\text{s}}=2.5\text{GHz})$, the number of data points for each period is $L(=5,000)$, and the number of accumulation for ensemble averaging is p( = 500). The time required for a single frequency sweep is $T_{\text{c}}(=4.0\text{μs)}$, which is in addition to $T(=2.0\text{μs})$ and the settling time $T_{\text{d}}(=2.0\text{μs})$ required for exchanging the illumination mask. The total time required to acquire the image is therefore $NT(=256\text{μs)}$, which is a higher frame rate than that of a normal 2-D detector. As shown in Fig. 3(c), we get $N(=64\text{)}$ sets of time-series fluorescence waveforms, with amplitudes and phases depending on the fluorescence lifetimes. Each fluorescence waveform is a sum of waveforms generated from $N/2(=32\text{)}$ pixels. The data are captured with a digital oscilloscope (Tektronix TDS5054B: 10⁶ data points; 400-ps sampling interval; 12-bit resolution) and processed by LabVIEW (National Instruments) software. As shown in Fig. 3(d), we get $N$ pairs of real and imaginary spectra by performing $L(=5,000)$-point discrete Fourier transforms (DFT) for individual time-series data. We select $k(<<N)$ frequencies that are suitable for estimation of fluorescence lifetimes. In practice, we pick up $k(=100)$ frequencies with an even frequency interval $\Delta f(=500\text{kHz)}$ from $f_{\min }(=0\text{Hz)}$ to $f_{\max }(=50\text{MHz)}$. We thus obtain a pair of $N\times k$real and imaginary spectral matrices. Then, by performing the inverse HT procedure, we get real- and imaginary-image column vectors for the $k$ frequencies, including zero frequency, from which we reconstruct $k$ pairs of $n\times n$ real and imaginary images, as shown in Fig. 3(e). If we obtain reference images in advance by a similar procedure, we can calculate the $n\times n$ modulation ratio $m$ and phase $\theta$ images for the $k$ frequencies, as shown in Fig. 3(f).

For a multi-component sample, the numerical fitting procedure for each pixel [e.g., point P in Fig. 3(f)] yields the modulation ratio and fluorescence lifetimes by the normal phase-modulation procedure. We can thereby derive “shading” images for the corresponding fluorescence lifetimes, where a shading image is essentially the same as the “lifetime-selective” fluorescence image [11]. For example, if there are two components, the fluorescence decay waveform is represented by $a_1\exp (-{\tau }_{1}t)+a_2\exp (-{\tau }_{2}t)$, where ${\tau }_1$ and ${\tau }_2$ are the two fluorescence lifetimes, and $a_1$ and $a_2$ are the corresponding initial amplitudes. Because we can obtain the dc-level fluorescence intensity of $a_1{\tau }_1+a_2{\tau }_2$by setting $f_1=0\text{Hz}$, we can calculate $a_1$ and $a_2$ for pixel P by knowing ${\tau }_1$, ${\tau }_2$, and $a_2/a_1$. Application of such procedures to all pixels yields $a_1$ and $a_2$ images. These are shading images that correspond to fluorescence lifetimes of ${\tau }_1$ and ${\tau }_2$, respectively. We set the error for the lifetimes at $\pm 5.0\%$: ${\tau }_1\pm {\tau }_1/20$ for the $a_1$ image and ${\tau }_2\pm {\tau }_2/20$for the $a_2$ image. If necessary, to enhance SNR, we acquire $p$ accumulations of the time-series data.

In the above data processing procedure, we picked up $k$ frequencies before performing inverse HTs, which saves calculation time. The linearity of DFT allows such a procedure. Also, it should be noted that the shading images of the fluorescence lifetimes cannot be obtained until HT-FLI is employed.

3.2 LED array driver

Figure 4 shows a schematic of an $n\times n$ LED array driver, which consists of a 64-bit cyclic shift register (CSR; Toshiba TC74HS194$\times$16, 70 MHz maximum frequency) for cyclic shifting of the Hadamard illumination mask pattern, a clock pulse function generator for the shift operation, an initial Hadamard pattern-setting circuit, a chirped sinusoidal waveform generator (arbitrary waveform generator; Tektronix AWG520, 500 MHz maximum frequency; $4.2\times {10}^6$ maximum data points), and an $8\times 8$-bit transistor array. As described above, the LED array is turned on/off according to the cyclic Hadamard illumination mask pattern with an interval $T_{\text{c}}=4.0\text{μs}$, and linearly frequency modulated from $f_{\min }=0$ to $f_{\max }=50\text{MHz}$ for every exchange of the pattern. The master clock for the pattern exchange is a function generator (WF1965, NF Corporation). The driver used for double modulation of each LED is a high-speed NAND gate (Toshiba Semiconductors TC74AC00FT, 25-mA/gate maximum sink current). Although the NAND gate changes the sinusoidal waveform to a two-valued rectangular waveform, it does not affect the entire operation. Each LED is driven by the synthesized wave at a 6.4-mA peak current. The timing diagram (A)–(E) in Fig. 4 displays the operation.

 

Fig. 4 Schematic of an n × n LED array driver (upper part). CSR; cyclic shift register used to generate Hadamard illumination mask patterns. AWG; arbitrary waveform generator for chirped sinusoidal waveform generation and its operating timing diagram (lower part); (A) master clock for pattern exchange. (B) settling time T_d for preparing Hadamard illumination mask patterns and dwell time T for frequency chirping, (C) start pulse for the chirped waveform, (D) frequency-chirped excitation waveform, and (E) resulting fluorescence waveform.

Download Full Size | PDF

4. Results and discussion

4.1 Preparation of fluorescent samples

To evaluate the HT-FLI system, we measured fluorescence lifetimes of two single-component samples: (i) $10\text{μM}$ rhodamine 6G in ethanol (R6G); and (ii) $10\text{μM}$ rhodamine B in ethanol (RB). The samples were contained in two separate 10 × 10 × 45-mm quartz cells aligned side-by-side, as shown in Fig. 5(a). This arrangement mimics a single-component sample with a fluorescence lifetime that varies spatially. We also prepared mixed solutions of R6G and RB with volume ratios of (iii) 3:1, (vi) 1:1, and (v) 1:3, respectively, that were aligned as shown in Fig. 5(b). Because of the 1.25-mm-thick quartz walls, there are two 2.5-mm-thick non-fluorescent barrier areas.

 

Fig. 5 (a) Schematic of two single-component samples with a fluorescence lifetime that varies spatially (R6G: 10-μM rhodamine 6G in ethanol; RB: 10-μM rhodamine B in ethanol) (b) Schematic of a two-component mixture of R6G and RB with volume ratios of (iii) 3:1, (vi) 1:1, and (v) 1:3, respectively. (c) Fluorescence excitation spectra of (i) R6G, (ii) RB, and (iii) 1:1 mixed solution. (d) Corresponding emission spectra.

Download Full Size | PDF

Fluorescence excitation spectra of R6G, RB, and the mixed solution are shown in Fig. 5(c); the peak wavelengths are ${\lambda }_{\text{ex}}$ = 540 nm, 550 nm, and 555 nm, respectively. Figure 5(d) shows their fluorescence emission spectra, with peak emission wavelengths of ${\lambda }_{\text{em}}$ = 560 nm, 569 nm, and 575 nm, respectively. In the fluorescence measurements, we inserted a 550 ± 10 nm band-pass excitation filter (VPF-25C10-50-55000, Sigmakoki Co., Ltd.) and a 580-nm low-pass emission filter (SCF-50S-58O, Sigmakoki Co., Ltd.).

4.2 Results

Figure 6 shows results for the two-cell sample illustrated in Fig. 5(a). Figure 6(a) is a dc (f = 0) fluorescence image obtained with the HT-FLI, which is the same as that obtained from the conventional HTI. Two columns in the center correspond to the air space and the cell walls. Also, as described above, the (1,1) image element has no information. Figure 6(b,c) show the modulation ratio m and the phase difference θ as a function of the modulation frequency f, calculated at positions X and Y, respectively, as marked in Fig. 6(a). Numerical fitting yields fluorescence lifetimes τ₁ = 4.2 ns and τ₂ = 2.5 ns for R6G and RB in ethanol, respectively. Figure 6(d) shows the initial amplitude ratio a₂/a₁ for cells (i) and (ii), calculated for each pixel assuming single-exponential fluorescence decays. From the image in Fig. 6(a), we can derive an averaged fluorescence shading image for a₁ and a₂, as shown in Fig. 6(e,f), respectively, corresponding to${\overline{\tau }}_1$ = 4.2$\pm$0.21 ns and ${\overline{\tau }}_2$ = 2.5$\pm$0.13 ns. Region (i) in Fig. 6(e) shows the fluorescence-lifetime image for R6G, while region (ii) has no signal. Region (ii) in Fig. 6(f) shows the fluorescence-lifetime image for RB, while region (i) has no signal. The degree of shading in the two images indicates the spatial fluctuation of the initial amplitude. The two averaged lifetimes were in good agreement with that reported in literatures [17,18].

 

Fig. 6 (a) A dc (f = 0) fluorescence image obtained with HT-FLI for the two-cell sample shown in Fig. 5(a). (b) Amplitude ratio m and phase difference θ as a function of the modulation frequency f derived at a position X marked in (a), and (c) at a position Y marked in (a). (d) Initial amplitude ratio image a₂/a₁ calculated for each pixel. (e) a₁ shading image for ${\overline{\tau }}_1$ = 4.2 ± 0.21 ns. (f) a₂ shading image for ${\overline{\tau }}_2$ = 2.5 ± 0.13 ns.

Download Full Size | PDF

Figure 7 shows the results for the multi-component cells shown in Fig. 5(b). Figure 7(a) shows the dc (f = 0) fluorescence intensity image, while Fig. 7(b-d) show modulation ratios and phase differences as a function of the modulation frequency at positions X, Y, and Z marked in Fig. 7(a), respectively. From the frequency response curves, we can derive τ₁ and τ₂; the initial a₂/a₁ is derived from numerical fitting. Figure 7(e) shows the a₂/a₁ image, while Fig. 7(f,g) show the a₁ and the a₂ shading images, respectively, corresponding to ${\overline{\tau }}_1$ = 4.2$\pm$0.21 ns and ${\overline{\tau }}_2$ = 2.5$\pm$0.13 ns. The intensity plots for the individual a images shown in Fig. 7(e,f) are relative in that we can compare the degree of the shading directly for each pixel in the same image as well as in the other image. The number of fluorescent components was determined by minimizing the residuals between the measured and the fitted curve at every pixel.

 

Fig. 7 (a) A dc (f = 0) fluorescence intensity image obtained from HT-FLI for the multi-component cells in Fig. 5(b). (b) Amplitude ratio m and phase difference θ as a function of the modulation frequency f derived at a position X, marked in (a), (c) for position Y, and (d) for position Z. (e) Initial amplitude ratio a₂/a₁ calculated for each pixel. (f) a₁ shading image for ${\overline{\tau }}_1$ = 4.2 ± 0.21 ns, and (g) a₂ shading image for ${\overline{\tau }}_2$ = 2.5 ± 0.13 ns.

Download Full Size | PDF

To construct the FLIM from the HT-FLI system and a fluorescence microscope, we replace the LED array with a combination of a digital light processing (DLP) projector and a laser diode (LD). For example, if we use the commercially available DLP Discovery 4100 (Texas Instruments), which has a 1024$\times$768 array of 13.6-μm-pitch mirrors, and a $\times$100 fluorescence microscope, diffraction-limited spatial resolution may be realized. High-speed measurements, one cycle measurement time T_c = 30.7 μs explained in Fig. 4(A), might be attainable as well, which might result in the total measurement time around 0.5 s for 128$\times$128 pixel, while T_c = 4.0 μs and the total measurement time 256 μs for 8$\times$8 pixel in the present system. If the LED is replaced by a LD, the maximum modulation frequency would exceed 1.0 GHz [19]. This would allow much faster fluorescence lifetimes (tens of picoseconds) to be measured. The present prototype system resulted in a few nanoseconds for the maximum modulation frequency of 50 MHz of the LED.

The maximum number of pixels resolved in the HT-FLI is determined by both the optical and detection systems. To acquire an $n\times n$ pixel image with l-bit ordinate resolution, the detection system should have $q(=l+{\log }_2(n^2/2))$bit resolution, with the corresponding dynamic range as a maximum. For example, when n = 64 and l = 6, q = 17. To increase n for a given q, one should conduct an accumulation procedure p times at the cost of the measurement time. This secures ${\log }_{2}p$ bits in a random noise situation. Other options include heterodyne light detection for the enlargement of the detection dynamic range [20] or photon counting for measurements of weak fluorescence [19].

5. Conclusions

We have discussed a Hadamard-transform fluorescence-lifetime imaging (HT-FLI) technique and demonstrated a prototype in proof-of-principle experiments. The HT-FLI technique combines HT imaging with a Fourier-transform phase-modulation fluorometer. It has advantages in detection sensitivity, resolution times, and measurement times over conventional FLIs that use 2-D gated image detectors. For mixed standard solutions of rhodamine 6G and rhodamine B in ethanol, the HT-PMF acquired shading images for every fluorescence lifetime. We also briefly discussed guidelines for the construction of a fluorescence-lifetime-imaging microscope.