1. Introduction

Fluorescence microscopy has become firmly established as one of the chief tools available for the study of biological systems at the cellular level. Unfortunately, the resolution of optical microscopes in the lateral dimension is limited to 0.61 λ/NA (~250 nm). As many biological structures within cells are much smaller than this, increasing resolution is of prime importance. Recently, a great deal of progress has been made in this area, reviewed in [1].

Single molecule localization microscopy (SMLM) methods are one of the more common approaches for achieving superresolution [2–10]. They have quickly been adopted as they are relatively easy to implement, and offer multicolor, 3D imaging with resolution in the 20 nm range. SMLM methods work by isolating and imaging single molecules in an extended time series. The idea is to, by photochemical means, convert nearly all the fluorescent molecules in a sample to a temporary dark state. The molecules then return to the fluorescent state both infrequently and randomly, such that they can be imaged individually. The molecules are then localized by fitting the imaged Airy patterns to a suitable model. Typically, the accuracy achieved is about 20 nm, primarily depending on the number of photons collected from each molecule. In general, the sample must remain motionless during imaging, but SMLM is not incompatible with live cell imaging [11,12]. In this case, a compromise must be reached between the desired resolution and rate of movement of the cellular structures of interest.

The three basic requirements for imaging with SMLM are thus:

In the present study, we focused on the final point as the procedures involved are not yet standardized and researchers working in this field have a variety of choices to make when developing software to generate SMLM images. Firstly, we concentrated on accurate identification of candidate molecules. For this purpose we designed extensive Monte Carlo simulations which helped us to assess the performance of various aspects of the processing routine, and to ensure that we maximized information retrieval from the data, especially in data with low signal to noise ratios (SNR). Secondly, we briefly present a new SMLM data rendering algorithm based on adaptively jittered 2D histograms.

To develop our data analysis procedures, we chose Atto565-phalloidin labeled actin in LEP fibroblast cells as a reproducible, easy to generate model system. It is known that Atto565 and several other Atto dyes are suitable for SMLM [10]. The Atto family of dyes offer high quantum yields, high extinction coefficients, and good photostability. These features all make SMLM much easier to accomplish.

Equipped with our new algorithms, we turned to a more challenging case. We used single-laser SMLM to image fusions of the YFP mCitrine to the receptor tyrosine kinase erbB3 [13] as expressed in A431 cells, an epidermoid carcinoma cell line. Using mCitrine as a probe for SMLM proved to be more difficult, as the SNR was much lower. The data processing methods presented here were instrumental in allowing us to work with these samples.

2. Materials and methods

2.1 Cell lines and reagents

A431 cells expressing mCitrine-erbB3 [14] were a kind gift from Dr. Donna Arndt-Jovin and Dr. Tom Jovin of the Max Planck Institute for Biophysical Chemistry (Göttingen, Germany). Both the A431 cells and the LEP fibroblast cells were maintained in phenol red-free DMEM supplemented with 10 % FCS, 100 U/ml penicillin, 100 U/ml streptomycin, and L-glutamate (all from Invitrogen, Carlsbad, CA, USA) at 37 °C and 100 % humidity. Mowiol containing 1,4-diazabicyclo[2.2.2]octane (DABCO) was from Fluka (St. Louis, MO, USA). DTT and MEA were from Sigma (St. Louis, MO, USA). Point speck and tetraspeck beads for measuring the microscope's PSF and lateral drift characteristics were from Invitrogen. Atto565-phalloidin was from Atto-Tec (Siegen, Germany).

2.2 Sample preparation

Cells were grown on high performance #1.5 coverslips (Zeiss, Jena, Germany) which had been cleaned in glacial acetic acid followed by distilled water and 100 % ethanol. Cells were first washed with PBS, then fixed with fresh 4 % paraformaldehyde for 15 min at 4 °C. For F-actin imaging, we permeabilized fixed cells with 0.1 % Triton X-100 for 15 minutes at 4 °C, then labeled the cells with 2 nM Atto565-phalloidin for 30 min at room temperature. For both samples, we then mounted the coverslips in moviol containing DABCO and 50 mM DTT (or 100 mM MEA), and sealed them on acid-water-ethanol cleaned slides with clear nail polish.

2.3 Microscopy

We used an Olympus IX71 microscope equipped with an Olympus planapochromatic 100 × / 1.35 NA objective and a front-illuminated Ixon DU885 EMCCD camera (Andor, Belfast, Northern Ireland). The back-projected CCD pixel size in the sample was 80 nm. For mCitrine-erbB3 imaging, laser illumination from a 473 nm, 400 mW DPSS laser (Dragon laser, ChangChun, China) was filtered using a 470/40 nm filter (Chroma, Bellows Falls, VT, USA), then coupled into the microscope using a 0.39 NA multimode optical fiber (M29L05, Thor Labs, Dachau, Germany). The fiber output was collimated with a 2 inch, 60 mm FL lens (LA1401-A, Thor Labs) and introduced into the microscope using an Olympus L-shape fluorescence illuminator (IX2-RFAL). The L-shape illuminator incorporates field and aperture stops located outside the microscope body. This configuration resulted in a roughly evenly illuminated field. We isolated mCitrine-erbB3 fluorescence using an Olympus U-MNIBA3 filter set (excitation 470 – 495 nm, dichroic 505 nm, emission 510 – 550 nm). For Actin imaging we used a 1000 mW, 532 nm laser (Dragon laser). Fluorescence was isolated using an Olympus U-MWG-2 filter set (excitation 510 – 550 nm, dichroic 570 nm, emission 590 LP) and an additional HQ622/36 filter (Chroma). We closed the field stop so that only a small area of the sample (~30 μm diameter) was illuminated by the full power laser. This had the effect of reducing the background intensity.

Image sequences were acquired using Andor IQ software. We typically recorded 20,000 to 40 000 frames with an exposure time of 50 – 100 ms, an EM gain of 100, and preamplifier gain of 3.7 using frame transfer mode. With these settings, a saturated pixel in our 14-bit camera corresponds to ~150 photons.

2.4 Data processing

We identified candidate molecules in noise-reduced images by finding local intensity maxima in 4-connected pixel neighborhoods. To reduce the number of candidate molecules, we thresholded the local maxima with a user defined sensitivity, and only locations which are not closer than 7 pixels (~560 nm) were taken into account.

Noise reduction is important for detection of candidate molecules [15,16]. Our implementation applies convolution of the raw images with a Gaussian kernel of size 19 × 19 pixels with σ = 2.6 pixels (about double the size of the PSF of our microscope). In our hands, this approach achieves the best performance in image sequences with lower SNRs. We explored this aspect using Monte Carlo simulations, see Section 3.2 below.

Next, every detected position along with a square region containing the neighboring raw image area was isolated for further data analysis. Assuming σ_PSF = 1.3 pixels, we set the region size to 7 × 7 pixels, covering almost 99 % of the PSF area. The precise positions of the centers of the detected molecules, along with other parameters, were estimated by fitting the data in each of the stored images of single molecules to a PSF model [7–9,17]. Generally, a Gaussian function is a good approximation to the more complex PSF, especially in the case of noisy data [18]. We used a cylindrically symmetric, two-dimensional Gaussian function

We then used the results from nonlinear curve fitting to calculate the localization accuracy of the detected molecules as

Construction of the final SMLM image is based on the determined positions and localization accuracies of the detected molecules. The traditional method for eliminating outliers is to apply several constraints to the fitted parameters [7]. The final superresolution image is then formed by displaying only a subset of the localized molecules. When we applied traditional thresholding, we used the following constraints: 0.7 σ_PSF < σ₀ < 1.3 σ_PSF, N ₀ > α b, where α defines sensitivity and is typically in a range 20 – 40 depending on the sample, and 5 < Δx < 100 nm, cf. Eqs. (1) and (2).

A further important consideration is the choice of methods by which to render the superresolution image [19]. We developed a new visualization technique based on averaging a series of 2D histograms. Here, in every histogram, the positions of all molecules are randomly jittered around their localized positions. The chosen jitter is normally distributed with an amplitude (radius of the jitter) proportional to the localization accuracy, cf. Eq. (2), of corresponding molecules. In this way, we adaptively blur each localized molecule based on the calculated localization accuracy. This method is superior to conventional 2D histogram rendering as the localized molecules are not forced, on average, onto a regular grid. Moreover, the final superresolution images preserve the gain in resolution while also providing information both about the density of molecules and their localization accuracy.

We developed our data processing procedures using Matlab (The MathWorks, Natick, Massachusetts, USA) with a combination of C/C + + MEX code to accelerate the processing.

2.5 Simulated data

An artificial sequence of images was created to test the performance and accuracy of the algorithm for detection of candidate molecules. The molecules are simulated as PSFs with known, but random position, size and intensity. Additional background noise is created using a random number generator with a probability distribution based on a numerical model determined from the real data (taken from an area of the sample free from cells).

In our simulation, we generated 2000 artificial images of size 512 × 512 pixels each containing 500 molecules (i.e., 10⁶ molecules in total) on a noisy background. To avoid the possibility of overlapping locations, the molecules are positioned sparsely on a grid of size 16 × 16 pixels. We thus ensure there is a one-to-one correspondence between generated and detected points. To create some uncertainty in the positions, the centers of the molecules are placed randomly around grid nodes (random jitter), in our case up to 4 pixels away in any direction. Molecules are simulated as cylindrically symmetric Gaussian functions with sizes in the interval σ = [0.3, 2.2] pixels, and intensity N such that the SNR is in the interval SNR = [0, 100]. Here SNR = N / σ_noise; σ_noise = 1.7 photons. This was the measured standard deviation of the background recorded in the mCitrine-erbB3 sample in a region free from cells.

3. Results

3.1 Background

An algorithm for the detection of candidate molecules in SMLM data must find locations in the camera image where the PSF model will be fitted. The algorithm should avoid false positive detections (localizations of non-existing molecules – type I errors) and false negative detections (missed molecules – type II errors). Although we wish to minimize the number of missed molecules, the problem of false positive detections is more serious as these false molecules are propagated further in the processing and can distort the final SMLM image reconstruction.

There are many approaches for detection of single molecules in SMLM data proposed in the literature [7,8]. However, the properties of these algorithms with respect to false positive and false negative detections have not been thoroughly analyzed. It is therefore difficult to determine what results can be expected. Recently, a study [15] stressed use of noise reduction algorithms as a preprocessing step for single molecule detection. The authors analyzed some properties of several standard noise reduction methods followed by a non-maxima suppression algorithm used together for detection of single molecules. However, the authors of this study considered only two cases: large spots with high noise, and small spots with low noise.

Our experiments extend the study [15] by providing more comprehensive results about properties of the discussed noise reduction methods. For this purpose, we designed the following Monte Carlo simulation using the simulated data described in Section 2.5. The aim was to find the best match between the noise reduction method and the algorithm for detection of candidate molecules with respect to the SNR and the imaged size of the detected objects. Our results also suggest a nonlinear, sample-dependent threshold boundary, ensuring a minimum number of detection errors.

Using the detection methods described in Section 2.4, we counted the number of correctly identified molecules and the number of false positive and false negative detections in the simulated data in Section 2.5. This is depicted in Fig. 1 .

 

Fig. 1 Counting correctly detected molecules and type I and II errors (there are 3 correct, 4 false positive, and 2 false negative localizations) in Monte-Carlo simulations; red dot – generated molecule within a jittered grid, blue cross – detected candidate molecule, dashed circle – localization radius.

Download Full Size | PDF

Statistical measures related to correctly detected molecules and to type I and type II errors are the recall rate (also called sensitivity) and the precision rate (also called the positive predictive value) [20]. Their definitions are as follows:

For purposes of comparison between multiple algorithms, it is convenient to combine precision and recall into a single measure of performance with some trade-off between both values. A traditional method for this applies the balanced F-measure (also called the F ₁ score) [20] defined by

3.2 Comparing noise reduction methods

Similarly as in [15], the algorithm for detection of single molecules was examined with several noise reduction methods: (i) no noise reduction, (ii) an average mask, (iii) a median filter, (iv) convolution with a Gaussian kernel, and (v) morphological erosion [15,21]. All algorithms were evaluated on the same simulated data as described in Section 2.5. The sensitivity threshold of the detection algorithm was set such that the number of detected molecules was 500 per image, i.e., the same as the number of generated molecules.

For every detected point in the noise reduced image, we found the closest generated PSF. If the detected molecule was within a radius of two pixels from the generated PSF then we assigned this point to be a PSF and counted it as a true positive. If there was no PSF within the localization radius, we counted it as a false positive. Generated PSFs which were not detected were counted as false negatives. The localization radius was chosen to fully cover the largest objects in the image. For all three counted variables, we created two-dimensional histograms with respect to the SNR = N ₀ / b and the imaged sizes of the detected objects. Here the fitted parameters (imaged size and intensity of detected points) were used in the case of false positive detections and generated parameters otherwise. The performance was determined in each bin of the histogram using the F-measure according to Eq. (5). The results are visualized in Fig. 2 for each of the tested noise reduction methods. In this way, we evaluated every possible combination of SNR and imaged object size in the chosen intervals.

 

Fig. 2 Monte Carlo simulation using the F-measure to compare the performance and accuracy of the algorithm for detection of candidate molecules with respect to SNR and imaged size of the detected objects. Noise reduction methods used: a) no noise reduction b) median filter of size 3 × 3 pixels, c) morphological erosion with a diamond structuring element of size 3 × 3 pixels, d–f) average filter of size 2 × 2 / 3 × 3 / 4 × 4 pixels, g–i) convolution with Gaussian kernel with σ = 0.7 / 2.6 / 3.5 pixels. The number in the top right corner of each graph reports the percentage of the displayed area where F = 1 and the solid curve indicates an approximate boundary of this area. Dashed lines in h) indicate possible bounds for traditional thresholding.

Download Full Size | PDF

The results of the Monte-Carlo simulation clearly visualize the fraction of correctly detected objects (F → 1.0) and the fraction of false positive and false negative detections (F → 0.0). Our results indicate that choosing an appropriate noise reduction method can significantly enlarge the interval of correctly detected molecules while avoiding errors, especially for data with low SNR.

To be able to compare the performance of different algorithms using a single number, we estimated the relative area where the F-measure is above a chosen threshold. As we do not wish to allow any errors, we set the threshold to F = 1. The results in Fig. 2 indicate that noise reduction applying convolution with a Gaussian kernel of about double the size of the PSF achieves the best performance, covering 66.9 % of the area in the displayed interval. This is followed by a 3 × 3 average filter covering 59.5 % of the area. We can also see that too little blurring (Figs. 2d and 2g) degrades detection of bigger objects, while too much blurring (Figs. 2b, 2c, and 2f) negatively affects detection of objects with smaller imaged sizes.

3.3 Optimal nonlinear threshold

Another important result can be derived from the Monte Carlo simulations. The thresholded F-measure profile suggests an optimal, nonlinear threshold boundary on the imaged size and SNR of the detected molecules so that a minimum number of false positive detections are accepted for further data processing and a minimum number of acceptable molecules are missed. In the case of the chosen noise reduction method (a Gaussian kernel about double the size of the PSF, Fig. 2h) we modeled the decision boundary using a second order polynomial function. We therefore propose that only molecules satisfying the inequality

The effect of the proposed constraint, cf. Eq. (6), on the real SMLM data can be seen in Figs. 3 – 6 both for Atto565 and for mCitrine. The histograms in Fig. 3 compare the distributions of the imaged sizes of the detected molecules. The comparison is done for all detected points, when constrained by the optimal threshold boundary in Eq. (6), and when constrained by traditional single value thresholding as described in Section 2.4. A remarkable peak corresponding to false positive detections can be observed for localized objects with an imaged size of σ₀ ~50 nm. The situation for mCitrine-erbB3 is more severe, as the level of noise in the data was higher.

 

Fig. 3 Histograms comparing distributions of the imaged sizes of localized molecules for all detected points, when constrained by Eq. (6), and when traditional thresholding is used for a) Atto565-phalloidin labeled Actin and b) mCitrine-erbB3.

Download Full Size | PDF

An even more comprehensive view of the data is given in Fig. 4 , where 2D histograms depict the measured SNR versus the imaged sizes for all localized molecules. The decision boundaries for the proposed multiparametric threshold, cf. Eq. (6), and for the traditional thresholding are also indicated. It can be seen that both data sets suffer from very low SNRs as the center of mass of the histogram sits very close to the edge of the optimal lower bound. In our data, we estimated the overall SNR by taking the peak value of the 2D histograms, which was about 36 for Atto565-phalloidin, while it was only 28 for mCitrine-erbB3.

 

Fig. 4 Histograms of the measured SNR versus the imaged sizes of all localized molecules for a) Atto565-phalloidin labeled Actin and b) mCitrine-erbB3. The solid curves indicate the proposed threshold boundary according to Eq. (6), dashed lines show bounds for traditional thresholding, and color codes percentage of localized molecules in each bin of the histograms. Here SNR = N ₀ / b; N ₀ is the fitted amplitude corresponding to the number of photons detected for each molecule, b is the standard deviation of the pixel intensities in the fitting region after subtraction of the fitted PSF, cf. Eqs. (1), (2).

Download Full Size | PDF

The images in Figs. 5 and 6 show our results for two LEP fibroblast cells in close proximity forming a network of filopodia, and for an A431 cell expressing mCitrine-erbB3. Conventional widefield images are displayed in Figs. 5a and 6a. The other sub-images demonstrate the rendered SMLM data (red) overlaid on the conventional images (grey) in the indicated yellow regions. The images in Figs. 5b and 6b show SMLM images (red) with all detected molecules. Molecules selected by traditional thresholding are displayed in Figs. 5c and 6c. False positive localizations (red) introduced by conventional thresholding, and those molecules (green) which would be missed compared to the proposed approach, are visualized in Figs. 5d and 6d (contrast stretched for visualization purposes). Finally, the results of the proposed optimal nonlinear threshold, cf. Eq. (6), are indicated in Figs. 5e and 6e. The SMLM images were rendered using an adaptively jittered 2D histogram as described in Section 2.4. The numbers of visualized molecules in the selected regions of interest are given in Table 1 .

 

Fig. 5 Atto565-Phalloidin labeled Actin; a) widefield image with SMLM imaging area indicated, and an overlay with SMLM image indicating b) all detected molecules, c) traditional thresholding, d) false positive molecules (red) introduced by traditional thresholding, and molecules missed (green) by traditional thresholding (compared to panel e), and e) the proposed nonlinear threshold, cf. Eq. (6). The resolution achieved was ~15 nm, cf. Eq. (2).

Download Full Size | PDF

 

Fig. 6 mCitrine-erbB3; a) widefield image with SMLM imaging area indicated, and an overlay with SMLM image indicating b) all detected molecules, c) traditional thresholding, d) false positive molecules (red) introduced by traditional thresholding, and molecules missed (green) by traditional thresholding (compared to panel e), and e) the proposed nonlinear threshold, cf. Eq. (6). The resolution achieved was ~15 nm, cf. Eq. (2).

Download Full Size | PDF

Table 1. The number of molecules visualized in the regions indicated in Figs. 5 and 6.

View Table

5. Discussion

Low SNRs in SMLM data can result in significant numbers of missed molecules, and worse, an unacceptably high rate of false positive detections. This can be especially deleterious in the case of weakly fluorescent samples, such as we encountered when imaging mCitrine-erbB3. Systematic analysis of the algorithms used for the detection of candidate molecules can significantly reduce the problem. Using Monte Carlo simulations we have shown that, of the tested methods, the best noise reduction in the input data is achieved by convolution with a Gaussian kernel of about twice the size of the PSF. Our results also suggest the use of a nonlinear threshold boundary on the measured data, combining both the signal to noise ratio and the imaged sizes of the detected objects. This approach minimizes detection errors in an unbiased way. As for the computational complexity of the proposed nonlinear threshold, it is asymptotically the same as for traditional single valued cut offs, in particular O(n), where n is the number of molecules to be processed. Other aspects of the time required for processing have been previously discussed [15], and depend on the speed of the computer used and the level of code optimization.

We imaged both Atto565-phalloidin labeled actin in LEP fibroblast cells, and A431 cells expressing mCitrine-erbB3. In our data we found that, compared to the proposed nonlinear threshold, traditional thresholding can introduce about 1 – 5 % false positive detections into the final SMLM reconstruction, and may discard up to 30 % of the useful information (measured in the entire SMLM imaging area, ~30 μm diameter), depending on where the SNR threshold is placed. These values are only approximate, and will vary depending on the chosen fluorophore and labeling density. Taken together, our approach allows us to work, in a controlled way, with probes having lower single molecule photon efficiencies.