1. Introduction

Fluorescence lifetime spectroscopy is a widely used tool for extracting functional information from biological assays [1–4]. The fluorescence lifetime, $\tau$, describes the time a fluorophore spends in the excited state. It is strongly dependent on the properties of the local microenvironment. Thus, fluorescence lifetime measurement is a common technique used in the elucidation of the underlying characteristics of a complex system. Amongst the most interesting insights such a tool can provide in biology is the extent of protein-protein interactions [5–7].

Applications of fluorescence lifetime imaging (FLIM) for spatio-temporal elucidation of physical, chemical or biological phenomena is extensive [6,8–10]. FLIM implementations are broadly divided into frequency- and time-domain implementations, under either scanning [11] or wide-field [12,13] imaging modalities. Wide-field imaging methodologies with time-domain lifetime spectroscopy capabilities can suffer from the disadvantage of out-of-focus contributions in the resulting FLIM data which negatively impact the accuracy of the lifetime measurements [13–15]. Therefore, a well-defined illumination plane is highly desirable. Strategies circumventing this problem have been demonstrated using total internal reflection fluorescence (TIRF) imaging [16,17], as well as confocal sectioning using a Nipkov spinning disk [18] or structured illumination known as programmable array microscopy (PAM) [19–24]. The key advantages of PAM compared to traditional laser scanning confocal methods include: the omission of moving parts in the system; fast data acquisition speeds; and system flexibility afforded by the incorporation of programmable spatial light modulators (SLM), particularly if compatible with random access scanning. In addition to confocal sectioning, other applications of PAM include multiplexed fluorescence recovery after photo-bleaching (FRAP) [23], controlled light exposure microscopy (CLEM) [25], spectral imaging [26,27], as well as structured illumination microscopy (SIM) [28,29]. Therefore, the application of a PAM system depends highly on the patterns displayed on the associated SLM display. Fluorescence lifetime imaging (FLIM) in a PAM context has been performed extensively using frequency domain measurements [13,23,24,30]. Despite lifetime spectroscopy in the frequency domain being intrinsically fast and relatively inexpensive, it suffers from complex analysis requirements for multi-exponential decays, is sensitive to poor imaging signal-to-noise ratio’s (SNR) and is prone to imaging artefacts [8,10,31,32].

The gold-standard methodology for lifetime spectroscopy is time-correlated single-photon counting (TCSPC), due to its single photon sensitivity [33], high SNR [34] and exploitation of well-defined Poissonian statistics describing photon arrival times [35]. Traditional laser scanning approaches benefit from the utilization of highly sensitive point detectors such as hybrid photomultiplier tubes (hPMT) which have superior temporal accuracy relative to traditional photomultiplier tubes (PMT) or avalanche photo diodes (APD), due to their large detector active area, low dark count contribution, high quantum efficiency, and measurement of fluorescent decays that are free of afterpulsing effects [36]. TCSPC works on the principle of the repeated measurement of photon arrival times relative to some reference. Typically, this reference is provided by a pulsed laser source. The difference in time between the detection of a reference pulse and the arrival time of a single photon of fluorescence is known as the microtime. Exponential fitting models can be applied to microtime histograms, representing the characteristic fluorescence decay of the fluorophore, to extract the measured lifetime at every pixel. This significantly simplifies the data analysis for more complex decay characteristics compared to frequency domain measurements [37]. TCSPC utilizing these superior point detection systems are often considered to be intrinsically incompatible with wide-field illumination methodologies, since the spatial information describing the structural detail of the assay will be lost in the absence of a suitable scanning mechanism. Previous attempts to overcome this limitation was demonstrated using a digital micromirror device (DMD) [38] to raster scan illumination across a fluorescent phantom [39]. Of note from this study, is that due to the architecture of the DMD display, it behaves in analogy to a blazed grating when the pixels have a tilt applied to them [40]. Therefore, a significant amount of the coherent illumination power will be distributed in higher order reflections that cannot be captured by low numerical aperture optics, thus significantly reducing the irradiance at the sample. The authors circumvented this by scanning “super-pixels” comprising 16 × 16 individual DMD pixels to make TCSPC measurements. However, the spatial resolution of their final FLIM data is a significantly degraded as a result. This would make the study of more structurally complex assays particularly challenging and quantitatively unreliable. It is also interesting to note that significant time is spent performing TCSPC measurements in areas of the sample where there is no information of value (i.e background). Importantly, this is also true for all point scanning systems studying particularly sparse assays on a pixel-by-pixel basis across the full field-of-view (FOV). This can make data acquisition of such samples particularly cumbersome and inefficient. More recent developments using multiplexed excitation with beamlet arrays coupled to single photon avalanche diode (SPAD) arrays have been successful at significantly reducing data acquisition times [41–43]. However, this technique does not fundamentally resolve the issue of performing TCSPC measurements where there is no data. Furthermore, one may consider using SPAD arrays as a TCSPC camera for assays imaged under a wide-field modality [44]. However, the improved temporal resolution obtained through the use of such technology in comparison to microchannel plates or time-gated implementations [45], is also at the significant expense of spatial resolution. Furthermore, the low fill factor of the pixels will significantly impact the sensitivity of the system under wide-field illumination.

As a further contribution to the PAM class of microscopy technology, we have designed a system that allows for fluorescence spectroscopy under wide-field illumination, attempting to address the previously described limitations. We term this technology Programmable Lifetime Imaging (PLI). We use DMD technology, whose display is located conjugate to the image plane, to segment fluorescence obtained under TIRF illumination in an object-oriented manner. Importantly, object-oriented segmentation in this way avoids sampling regions of the FOV in which there is no information of value, significantly improving the efficiency and speed of data collection. Each object that is addressed using this mechanism has its fluorescence selected at the DMD and projected onto a hPMT for accurate TCSPC measurements. This technology is demonstrated using both manual and automated segmentation of the fluorescence emission. Automated implementation of PLI includes retention of the localisation information, allowing the spatial information of the assay to be reconstructed for wide-field FLIM image representation. We envisage such technology to be particularly suited for imaging sparse data sets, such as single molecule assays, and FLIM specifically at the plasma membrane.

2. Methods

2.1 Optical Setup

Two different optical configurations of PLI were prototyped. The first comprised only off-the-shelf optics [Thorlabs Ltd., UK], as well as utilizing a pulsed supercontinuum laser source (SC400-4-20 [Fianium, UK]), providing 400 < ${\lambda }_{ex}$ < 2000 nm excitation, with a pulse duration of 6 ps at a repetition rate of 20 MHz (Fig. 1a). The excitation was passed through an interference filter to select 480 < ${\lambda }_{ex}$ < 490 nm, which was then free-space coupled into an optical fibre with ~30-40% efficiency. The output of the optical fibre was then mounted on a moveable stage allowing the transition between epi-fluorescent and TIRF illumination. The excitation then passed through a tube lens and a FITC-compatible dichroic filter and focussed onto the back aperture of a 100 × 1.47 N.A. objective [Leica Microsystems, UK] mounted on a microscope platform (DMi8 manual [Leica Microsystems, UK]). The emission wascollected and passed through a telescope performing a 1.5 × magnification on the image, before being projected onto the DMD (DLP Discovery 4100 development kit (D4100) [Texas Instruments, USA]) display. The DMD comprises a 1024 × 768 array of micro-mirrors on a complementary metal-oxide semiconductor (CMOS) substrate. Each pixel can be programmatically addressed to incur a ± 12° tilt across the diagonal, relative to the incident illumination, allowing the optical path to be split in two directions; one for the imaging side of the system, and another for the TCSPC acquisition. To keep the optical paths parallel with the optical table, the DMD was rotated by 45° along the axis normal to its surface to compensate for the tilt orientation of its pixels. The imaging side of the system comprised an additional set of relay lenses producing a 1 × magnification, with an aperture located in the Fourier plane to select the 0th order diffraction off the DMD display. This had the effect of sharpening the image at the camera (QuantEM: 512SC [Photometrics, USA]). On the TCSPC side of the system, the image was (de)magnified by 0.2 × via two consecutive telescopes, with a custom square aperture placed at the intermediate image plane to attenuate fluorescent emission reflecting from the periphery of the DMD display, before being projected into the hPMT.

 

Fig. 1 Schematic of the optical configuration of PLI. a) The focal lengths of the optical elements are: $L_1$ = 10 mm; $L_2$ = $L_9$ = 75 mm; $L_3$ = $L_4$ = $L_7$ = $M_4$ = 200 mm; $L_5$ = 100 mm; $L_6$ = $M_5$ = 150 mm; $L_8$ = $L_{10}$ = 50 mm. Intermediate image planes are depicted by $II{P}_x$ and the positions of circular apertures are depicted by $C{A}_x$. A short pass dichroic located at position $D_1$ reflects the infrared contribution of the supercontinuum into a beam block. The filter depicted at position $F_1$ selects the desired excitation wavelength. The dichroic depicted at position $D_2$ was FITC compatible [Leica Microsystems]. b) Modification to PLI to address the previous limitations. The focal lengths of the optical elements are: $L_1$ = 20 mm; $L_2$ = $L_4$ = $M_7$ = 200 mm; $L_3$ = 500 mm; $L_5$ = $L_6$ = 100 mm; $L_7$ = $M_9$ = 150 mm; $L_8$ = $L_9$ = 75 mm. Custom mirrors $M_4$ and $M_5$ are concave and convex, respectively. The radii of curvature are: $M_4$ = 500 mm and $M_5$ = 200 mm, providing an effective focal length of 166.07 mm. The filter wheels $F{W}_1$ and $F{W}_2$ were loaded with chromatic filters and neutral density filters, respectively. Intermediate image planes are depicted by $II{P}_x$. Dichroic $D_2$ is FITC compatible [Nikon Instruments].

Download Full Size | PDF

To address imaging limitations associated with the DMD display, a second configuration of the PLI system incorporated optics which allowed for the inclusion of a Schiefspiegler relay [Cairn Research Ltd., UK], in accordance with the specifications described elsewhere [46] (Fig. 1b). Other significant changes included, the removal of the optical fibre providing the excitation to prevent any patch cable associated losses in excitation intensity, the microscope platform was upgraded (Ti-E Eclipse [Nikon Instruments, UK]) to provide precise control of the $x,y$ and $z$ position of the sample, a 100 × 1.49 NA objective [Nikon Instruments, UK] was incorporated, and the consecutive telescopes on the spectroscopy side of the system provides a 0.375 × magnification of the emission before entering the hPMT.

TCSPC measurements were conducted using a SPC-830 module [Becker & Hickl GmbH, Germany] in first-in-first-out (FIFO) mode. The hPMT was controlled via a DCC-100 detector module [Becker & Hickl GmbH, Germany].

2.2 Microscope Control Software

All subsequently described software was written in 32bit LabVIEW 2011 [National Instruments, UK] and executed on a computer running Windows XP SP3 with an Intel Core 2 Duo E4600 CPU operating at 2.6 GHz containing 4 GB RAM.

The DMD controller board (D4100) via USB2.0, connected to a computer by USB2.0 interface, was controlled using an application program interface (API) [Vialux GmbH, Germany]. The DMD display is segmented into 16 blocks encompassing 48 rows of pixels. Data is typically loaded onto the CMOS substrate in a block-wise manner and the state of the pixels is changed following a “reset” clocking pulse. There are four different block-clocking modes which are typically implemented when using DMD technologies, including single block mode, dual block mode, quad block mode and global mode. However, a particularly useful feature of the D4100 in our case is its compatibility with random row addressing, in which only the rows containing the pixels that are required to have their state changed have data loaded onto them before being reset.

For simplicity, the DMD software was written such that a square region of pixels can be addressed at any time, which we refer to as a “super-pixel”. It should be noted that, in principle, the DMD can display any binary or 8bit greyscale image it is provided with. The centre of the n × n super-pixel corresponds to the pixel coordinate the DMD is supplied with. A 1D Boolean array, with elements being either “0” or “1”, is then generated describing the pixels that require their state changed across the top row of the super-pixel. This is then encoded as an unsigned 8bit numerical array, which is subsequently duplicated across all the required rows to describe the entire super-pixel data. This data is then sent to the DMD for display. This super-pixel is responsible for redirecting the fluorescence emission away from the camera, and towards the hPMT for TCSPC measurements.

The effect the size of the super-pixel has on the speed at which the DMD can display them was quantified by measuring the execution time of displaying super-pixels with various sizes using programmatic timing functionality. For each super-pixel size, 2000 timing measurements were made.

Custom software using an API [Becker & Hickl GmbH, Germany] to control the SPC-830 was written and incorporated into the microscope control software. Trigger signals were sent from the D4100 general purpose in-out (GPIO) socket upon application of every new super-pixel on the DMD directly to the SPC-830, routed via a custom cable consisting of a PicoBlade wire-to-board connector [Molex, USA] and pre-crimped cables [Molex, USA] for plugging into the GPIO slot on the D4100, in which three pins are allocated for sending trigger signals. The other end of the cables were soldered onto the pixel, line and frame clock pins on a D-Sub connector [Amphenol ICC, France], for direct interfacing with the SPC-830. To minimize latency between the sending of a trigger signal from the D4100 and its receipt at the SPC-830, synchronization tools in LabVIEW were utilized such that the receipt of a trigger at the SPC-830 and the application of a new super-pixel on the DMD (signifying the start of a new measurement) coincide. Of the 8 available unique trigger combinations that can be sent to the SPC-830, only [1 0 0], [0 1 0] and [0 0 1] can be recognised by the SPC-830 at any given time while operating in FIFO mode.

Due to the legacy operating system required for compatibility of the DMD drivers, it was acknowledged that perfect synchronicity is particularly challenging and that residual latency is likely to be present. To quantify this latency, three super-pixels of different sizes were sequentially displayed on the DMD under constant illumination. Each super-pixel was associated with a trigger signal across a known pin on the GPIO socket. The variation in size of the applied super-pixels also corresponded to measurable differences of photon flux measured at the photodetector, allowing for highly accurate determination of the time the super-pixel changed on the DMD display. The time a trigger is detected at the SPC-830 is also encoded in the photon data.

Efficient control of the DMD can be facilitated by using the camera as a feedback mechanism to indicate where in the imaged FOV objects of interest are located. For this, careful pixel-to-pixel correspondence between the DMD and camera image planes was required. Previous methods demonstrating this process include the phase shifting Moiré method [47], and using Affine transforms [28]. In our case several super-pixels were generated on the DMD display with known locations. The camera then takes an image of these displayed super-pixels and calculates each centre-of-mass (COM). This repeats until all DMD pixels seen by the camera have had super-pixels applied to them, resulting in a raster scanning effect. The COM information populated a 3D look-up-table (LUT) of dimensions 512 × 512 × 2, where the $x$ and $y$ dimension sizes correspond to the size of the camera detector, and the two elements along $z$ store the corresponding $x$ and $y$ location on the DMD. This allows for direct indexing of the corresponding DMD coordinates using the localisations found in the camera pixel space.

The microscope control software for automated data acquisition executes the following steps (Fig. 2): (1) an image of the sample is acquired by the camera. (2) The image is processed allowing the identification of the COM coordinates of all the objects located in the calibrated image field. (3) Each coordinate is sequentially used to generate the super-pixel data, which is then sent to the DMD and displayed. (4) TCSPC measurements from that object’s fluorescent emission are performed. Steps 3 and 4 repeats iteratively until all the identified objects are measured, after which the process repeats from step 1 for as many frames the user has specified to acquire. The COM for each object, single photon events, and camera image are streamed to disk throughout the data acquisition cycle to prevent RAM overflow on the host PC.

 

Fig. 2 Schematic illustration of PLI operating principle for automated data acquisition. (a) The camera acquires an image of the sample. (b) The image thresholded and the COM of each object in the FOV is identified. (c) The localisation information is populated into a list. (d) Each coordinate in the list is extracted and transformed to correspond to the location on the DMD display. The super-pixel data is then generated, loaded and displayed on the DMD to redirect the fluorescence away from the camera and towards the hPMT. This process repeats for all identified objects. (e) When all objects have been scanned, the next frame is acquired. This entire process repeats until the desired number of frames of the sample have been collected.

Download Full Size | PDF

2.3 Post-Processing Software

We developed further post-processing software to extract the photon arrival time-stamps acquired and represent them as histograms for elucidation of the fluorescence lifetime. Segmenting the individual measurements using trigger signals allows the photons recorded for a single object, over several frames, to be extracted and binned together to maximise the number of photons available for fitting against, improving the accuracy of the fluorescence lifetime measurement. This restructured raw data is then reformatted for compatibility with the pre-existing fluorescence lifetime analysis software, TRI2 [48]. The photon data is represented as an unsigned 32bit integer, in which the first 12bits encodes the microtime information associated with the measurement. For each measurement (in other words, for every super-pixel displayed on the DMD), this information is extracted and used to generate a microtime histogram which can subsequently be fit using exponential fitting models to extract the fluorescence lifetime. With knowledge of the location at which each measurement was made, a FLIM image akin to those extracted from traditional laser scanning techniques can be generated by embedding the associated microtime histogram in an image cytometry standard (ICS) file [49]. It is important to note that since the data is acquired autonomously over several frames, COM calculations are distributed over several pixels (dependent of SNR) for the same object when represented as an ICS format. As a result, the photon information associated with each localisation is then distributed across these pixels, despite them all being associated with the same object. To mitigate the extent of error arrising as a result of low photon counts when determining the lifetime of each object, we implemented an “object-binning” routine (Fig. 3), which identifies the cluster of localisations made during the acquisition cycle for each object. The COM for each cluster of localisations made for eachobject is then extracted. This allows all the photon data for each object to be binned into a single pixel, increasing the number of photons available for fitting and improving the accuracy of the lifetime extraction.

 

Fig. 3 Schematic illustration of the object-binning algorithm. (a) A scatter plot representing all the localisations made through an acquisition cycle is generated. (b) In most cases, the localisations associated with a single object are clustered around the true localisation of the object. (c) The object-binning algorithm identifies the extent of the localisation cluster (yellow box), and measure the COM (green cross) associated with that cluster. (d) It then assigns the location of that object with the COM measurement. Note that this does represent the true localisation of the object; it is in analogy to the average location, with single pixel accuracy, as seen by the camera over the entire acquisition. All the associated photon data acquired for that object is then binned into this single pixel.

Download Full Size | PDF

2.4 Cell Culture

Transfection media consisting of: 0.4 µg of DNA encoding Lifeact-eGFP, or FRET standards containing eGFP as a donor separated from a mRFP1 acceptor molecule by $X$ amino acids (aa) with an N-terminal CaaX motif (eGFP-$X$aa-mRFP1-CaaX), at a concentration of 1 µg µL⁻¹; 3.2 µL of enhancer [Qiagen, Germany]; and 50 µL of DNA-condensation buffer. The transfection media was vortexed for 10 s and incubated at room temperature for 5 min. After, 5 µL effectene [Qiagen, Germany] was added to the transfection media and vortexed for a further 30 s, before incubating at room temperature for 15 min, completing the transfection reagent.

HeLa cells were grown in DMEM growth media. The growth medium was discarded and the cells were washed with PBS. 600 µL of growth media at 37° was added to the transfection reagent and added directly to the cells. They then incubated for 4 – 8 hr at 37°. The cells were then washed twice with growth media at 37°, before being allowed to incubate overnight in fresh growth media at 37°. The media was then discarded and the cells were washed in PBS, before 250 µL of trypsin was added, which were allowed to incubate at room temperature for 5 min at 37°. 10 mL of fresh growth media at 37° was then added to the cells, before 500 µL were plated in an 8-well imaging chamber [Ibidi GmbH, Germany]. The media of the plated cells was removed before 4% PFA was added. The cells in the fixing media were incubated at room temperature for 15 min. The PFA was then removed and the cells were washed several times using PBS. The cells were then permeabilized for 10 min with 0.1% triton diluted in PBS at room temperature. The cells were then treated with 1 mg mL⁻¹ sodium borohydride diluted in PBS, before an extensive washing with PBS. The cells were then mounted in 200 µL of PBS.

2.5 Fluorescence Microsphere Preparation

Fluorescent microspheres of diameter 0.100 ± 0.001 µm [Thermo Fisher Scientific, UK] were diluted in 1:1600 v/v in PBS and pipetted onto a coverslip imaging chamber [Ibidi GmbH, Germany] and left to incubate at room temperature for at least 30 mins to allow them to settle on the sample surface.

2.6 Manual Imaging Using PLI

Samples were imaged under TIRF illumination and fluorescence measured at the camera with a 150 × multiplicative gain and a 500 ms integration time. Super-pixels comprising 10 × 10 individual DMD pixels were applied to perform TCSPC measurements. After manual application of the super-pixel in the desired location, TCSPC measurements were conducted in first-in-first-out (FIFO) mode with a 250 ms acquisition time using proprietary control software (SPCM [Becker & Hickl, Germany]).

2.7 Automated Imaging Using PLI

Samples were imaged under TIRF illumination and fluorescence measured at the camera with a 150 × multiplicative gain and a 100 ms integration time. The DMD was configured to apply super-pixels comprising 9 × 9 individual DMD pixels, with a 50 ms dwell time. Objects were probed across a 200 frame acquisition, and TCSPC measurements were conducted continuously in FIFO mode using our custom PLI control software for the entire duration of the experiment.

3. Results

3.1 Quantification of DMD Performance with Increasing Super-Pixel Size

We first quantified how the random row addressing feature of the D4100 effects the speed at which super-pixels can be displayed on the DMD, given the number of rows that they occupy. Measuring the time taken for the software to apply super-pixels on the DMD for a variety of super-pixel sizes showed that there is a linear decrease of approximately 0.53 ± 0.04 Hz per additional row which the super-pixel occupies (Fig. 4).

 

Fig. 4 Benchmarking data (mean ± standard deviation) showing the reduction in DMD super-pixel application rate with increasing number of pixel rows the super-pixel occupies.

Download Full Size | PDF

3.2 Synchronization of the DMD with the TCSPC Electronics

Next, we quantified the residual latency that exists between the identification of the start of a new measurement on both the DMD and SPC-830, represented respectively by the sending and receipt of a trigger signal associated with the display of a particular super-pixel (Figs. 5a-5c). It can be seen that the measured photon flux shows a step increase in magnitude as the super-pixel increases in size (Fig. 5d and Fig. 5e). Comparison between the time each rising edge of each step occurs with the time the associated trigger is received by the SCP-830 shows that there remains approximately 234.5 ± 30.5 µs latency between the detection of a trigger signal at the SPC-830. Observed outliers to the peak of the Gaussian fit can be ascribed to operating system interrupts and limitations associated with the USB2.0 bandwidth, as well as operation of other devices present on the same USB host controller.

 

Fig. 5 (a - c) Super-pixels of different sizes correspond to different magnitudes of photon flux onto the photodetector. (d) Cyclically displaying these super-pixels allows the extraction of a photon flux time trace. The display of a particular super-pixel corresponds to a trigger being sent though a unique pin on the GPIO socket of the D4100. The magnitude of photon flux follows a power law, corresponding to the area of the super-pixel itself, making the super-pixel responsible for a particular flux measurement easily identifiable from the trace alone. Coloured vertical lines correspond to the point in time the SPC-830 received the trigger. The inconsistency in the application rate is ascribed to the operating system prioritising other background processes. (e) A 200 ms section of the entire 5 s trace, highlighted by the yellow dashed box in (d). (f) A histogram to show the apparent latency between the triggers being sent from the DMD and their receipt by the SPC-830. Outliers from the Gaussian fit are ascribed to the operating system prioritizing other background processes.

Download Full Size | PDF

3.3 Lifetime Spectroscopy under Evanescent Field Illumination

We first demonstrate the TCPSC ability of PLI (system depicted in Fig. 1a) by performing lifetime spectroscopy on fixed HeLa cells transiently expressing Lifeact-eGFP (Fig. 6). Three super-pixel locations were chosen at visible stress fibres. A mono-exponential Levenberg-Marquardt fitting model was applied to the measured microtime histogram to extract the lifetime; the average lifetime was found to be $〈\tau 〉$ = 2.57 ± 0.002 ns.

 

Fig. 6 Programmable lifetime spectroscopy of fixed HeLa cells transiently expressing Lifeact-eGFP. (a - c) Super-pixels encompassing 10 × 10 individual DMD pixels were manually applied on three different stress fibres visible under TIRF illumination (scale bars are 10 µm). The position of the super-pixel’s are indicated with white arrows. (d - f) The microtime histograms extracted from the photon data measured at the super-pixel locations in panels a – c, respectively. A mono-exponential Levenberg-Marquardt fitting model was used to extract the lifetime. (g - i) The residuals associated with the fits in panels d – e, respectively. The random fluctuation of points around 0 within the extent of the fit demonstrates the suitability of the model for the data.

Download Full Size | PDF

We then assessed the ability of the PLI to distinguish two different lifetimes. Fixed HeLa cells transiently expressing FRET standards of the form eGFP-$X$aa-mRFP1-CaaX [50] (Fig. 7), whereby the donor molecule eGFP is separated from the acceptor mRFP1 by an amino acid linker of length $X$. Therefore, high FRET efficiency is expected with a linker of length 7aa, resulting in a decrease in the fluorescence lifetime of the eGFP compared to the low FRET construct with a linker length of 32aa. These constructs were previously shown to be distinguishable by the effect the proximity of the acceptor has on the measured lifetime of eGFP [50,51]. The N-terminal CaaX motif ensure expression primarily at the plasma membrane of the cell [52]. The average fluorescence lifetime extracted for each construct was $〈{\tau }_{7aa}〉$ = 2.01 ± 0.14 ns and $〈{\tau }_{32aa}〉$ = 2.26 ± 0.05 ns (N = 4). The increased standard deviation in the lifetime measurement for the 7aa linker construct compared to the 32aa linker construct can be ascribed to the error associated with the reduction in the number of photons measured over a fixed acquisition time. Given an eGFP lifetime of $\tau$ ≈2.6 ns (measured separately), this corresponds to FRET efficiencies of 23% and 13% for 7 and 32 amino acid FRET constructs respectively, which is in reasonable agreement with other studies [50,51].

 

Fig. 7 Representative images of fixed HeLa cells transiently expressing FRET rulers with a 32aa linker (a - c), and a 7aa linker (d - f), imaged under TIRF illumination (scale bars are 10 µm). White arrows indicate the positions of super-pixel manually applied to sample fluorescence at the cell membrane. (f) The measured fluorescence lifetime of eGFP (mean ± standard deviation) measured for both linker lengths at three positions on the membrane of each cell (N = 4).

Download Full Size | PDF

3.4 Object-Oriented FLIM under Evanescent Field Illumination

We demonstrate the automated implementation of PLI (system depicted in Fig. 1b) using diffraction limited fluorescent microspheres. Several microspheres were observed in the FOV with a sparse distribution (Fig. 8a). A binary scatter plot representation of all the localisations made throughout the acquisition cycle allows the object-binning algorithm to identify the centre of all localisation clusters for single pixel representation of the microspheres (Fig. 8b). The generated ICS file was then analysed in TRI2 using a mono-exponential Levenberg-Marquardt fitting model to extract the lifetime information. The average measured lifetime for these microspheres was measured to be $〈\tau 〉$ = 4.78 ± 0.79 ns (Fig. 8c) associated with an average goodness-of fit measure of $〈{\chi }_r^2〉$ = 1.11 ± 0.15 (Fig. 6d), indicating the suitability of the fitting model with this data set. It can be observed that throughout this particular acquisition, mechanical lateral drift manifests itself in the localisation scatter plot image (Fig. 8b). The benefit of the automated data acquisition is that the super-pixel applications follow such drift, making any impact of the acquired TCSPC data negligible. The single pixel representation of the imaged objects can also be considered a spatial average of each objects location, given the associated drift although, in principle full correction to the experimental origin could be performed. Particularly in the case where the assay has a sparse distribution of emitters, the data acquired over hundreds, or even thousands of frames, may be required for statistical significance in the lifetime measurements. As a result, it may also be useful to represent all the lifetime measurements from each object from several acquisitions as single image array (Fig. 9a-c). This allows the visualization of any global trends in the assay, or time-dependent variations that may inform on data quality or the presence of outliers. Across 10 independent acquisitions, the average number of measured photons from each bead was measured to be $〈I〉$ = 1164.3 ± 220.5 counts (Fig. 9a and Fig. 9d), the average measured lifetime was $〈\tau 〉$ = 4.41 ± 0.45 ns (Fig. 9b and Fig. 9e) and an average goodness-of-fit measure of $〈{\chi }_r^2〉$ = 1.02 ± 0.15 (Fig. 9c and Fig. 9f), consistent with measurements made from a single acquisition. To verify that there is no intensity-dependent trend to the data or the presence of errors used in the fitting model for lifetime extraction, an intensity vs lifetime histogram was generated (Fig. 10). It can be seen that the distribution of events shows no clear correlation with intensity, indicating that an adequate number of photons were acquired for lifetime extraction.

 

Fig. 8 Object-oriented FLIM on diffraction limited microspheres. (a) The first frame of a 200 frame acquisition of the sample, imaged under TIRF illumination (scale bar is 5 µm). (b) Localisation scatter plot of all objects in the FOV identified across the 200 frame acquisition, used for object-binning the associated photon data. (c) FLIM image representation of the data, analysed using a mono-exponential Levenberg-Marquardt fitting model. (d) the associated goodness-of-fit measure for each analysed object. The microspheres highlighted in red boxes are the same in panels a – d. (e and f) An enlarged view of the FLIM and goodness-of-fit data associated with the microspheres highlighted in red box in panels (c) and (d), respectively.

Download Full Size | PDF

 

Fig. 9 (a - c) Batched arrays of all measured diffraction limited microspheres (N = 333). Each row corresponds to a particular acquisition, and each column corresponds to the identified object index (0 being top-left-most in the acquisition FOV, $n$ being the bottom-right-most in the acquisition FOV). (a) Measured intensity array of all measured microspheres. (b) Measured FLIM array of all measured microspheres. (c) Measured goodness-of-fit array for all measured microspheres. Elements highlighted in the red box corresponds to the microspheres highlighted in Fig. 6. (d - f) Histograms of the measured intensity, lifetime, and goodness-of-fit from panels a – c, respectively.

Download Full Size | PDF

 

Fig. 10 Lifetime vs intensity histogram for the measured diffraction limited microspheres.

Download Full Size | PDF

4. Discussion

We first quantified the performance of the key operating aspects of our PLI imaging technique. It was demonstrated that the more rows the super-pixel occupies, the lower the achievable application rate of the system. We also quantified the residual latency that exists between the receipt of triggers at the SCP-830. The minimum required photon counts for accurate lifetime determination for a mono-exponential decay is 1000 measured counts. To prevent pile up at the photodetector, no more than 1 photon should be measured every 5 µs, which corresponds to 1% of the reference repetition rate. Therefore, a latency of 234.5 µs represents a 4.7% error in the origin of the measured photons associated with a particular object. This can be further improved with utilization of GPU optimization and/or real-time processing.

Next, we demonstrate the ability of our PLI imaging system to perform TCSPC lifetime spectroscopy under evanescent field illumination. For the first time, we demonstrate the advantages of using object-oriented lifetime detection rather than pixel-by-pixel scanning, as a method to improve the efficiency of the data acquisition and consequently data acquisition. By comparison, PLI operating in an automated scanning mode was able to acquire a data set with 26 objects (Fig. 8a) in 2.628 seconds per frame on average (including all the image processing and data transfer), whereas a point scanning system with a pixel dwell time matching our super-pixel dwell time would require 15.810 seconds per frame, regardless of the number of objects of interest that are present in the FOV. The acquisition speed of PLI is strongly dependent on the number of objects $N$, in the FOV that require measurement; imaging time will increase approximately linearly with $N$. For the experimental setup shown, this equates to PLI outperforming a laser scanning system with an assay consisting of up to 156 visible objects per frame. The maximum speed of PLI is limited by the USB2.0 control interface of the DMD. The total image processing time during runtime is approximately 25 ms, and it takes approximately 8 ms to generate, send and load the super-pixel data to the DMD. Therefore, the maximum super-pixel application rate is currently approximately 30 Hz. This could be reduced with improved processing power, real-time operating system capabilities, and more modern interface technologies such as USB3.0 which are now available.

Limitations associated with the system described in Fig. 1a primarily originate from the structure of the DMD display. The DMD forms a 2D grating causing a significant dispersive effect on the image, requiring an aperture at the Fourier plane to select the 0th order diffraction only. This improved the image quality significantly at the cost of achievable SNR. A number of different optical arrangements have been utilized that have been designed specifically to correct aberrations generated from the DMD display, for example: projection systems utilize high NA optics such as prisms [53]; customized condenser optics [54]; an Offner relay optical configuration [22]; and more recently a Schiefspiegler relay optical configuration [46,55]. The Schiefspiegler relay is a particularly attractive solution, since it projects the object to infinity, allowing for greater flexibility for subsequent optical transformations, if required.

An additional limitation associated with PLI more generally is the constant illumination of the sample while only a single object is being measured using TCSPC. This causes unnecessary photobleaching/photodamage to the assay. This could be circumvented by performing only the localisation step under wide-field illumination, followed by subsequent sequential targeted confocal excitation of the observed objects, mediated via illumination off the DMD. This would consequently allow for much longer measurement dwell times on each object while protecting the remainder of the assay. This may be particularly important for single molecule imaging, where SNR considerations become increasingly important, given their limited photon budget.

Imaging technologies such as PLI could be particularly useful of number applications requiring lifetime contrast where sparse fields of view are evident, such as imaging vesicles or single molecules. The advantage of a TIRF imaging modality is the sub-diffraction limited axial illumination depth and relatively low power densities at the sample in comparison with point scanning systems. It is also worth noting that TIRF is difficult to implement for a scanning system [56]. The ability to incorporate high performance photon detection systems also ensures optimum data quality. We anticipate that our PLI methodology will enhance the diversity of information that can be extracted from biological assays, providing more detailed insights into the underlying mechanisms governing cellular processes.