Abstract
Dynamic speckle illumination (DSI) microscopy is a widefield fluorescence imaging technique that provides depth discrimination. The technique relies on the illumination of a sample with a sequence of speckle patterns. We consider an image processing algorithm based on a differential intensity variance between consecutive images, and demonstrate that DSI sectioning strength depends on the dynamics of the speckle pattern. Translated speckle patterns confer greater sectioning strength than randomized speckle patterns because they retain out-of-focus correlations that lead to better background rejection. We present a theory valid for arbitrary point-spread-functions, which we corroborate with experimental results.
©2006 Optical Society of America
1. Introduction
Confocal fluorescence microscopy [1] is a well established technique in the bioimaging community, mainly because it provides optical sectioning. Implementations with laser illumination usually involve scanning of a single laser focal spot, whereas implementations without laser illumination typically make use of multi-spot array scanning [2, 3]. More light-efficient alternatives have been proposed such as aperture correlation microscopy [4, 5], or programmable array microscopy [6] that involve illuminating the sample with patterns of light defined by illumination masks, and collecting the fluorescence through these same masks. Alternative strategies that allow direct collection of the fluorescence without a return passage through the illumination mask include the simple and well-established technique of incoherent structured illumination microscopy [7, 8].
We recently proposed a technique similar to structured illumination microscopy wherein the incoherent grid illumination pattern is replaced with a dynamic speckle illumination (DSI) pattern from a laser, and we demonstrated that this new technique confers optical sectioning in biological tissue [9]. In our initial implementation, the sample was illuminated with a sequence of statistically independent (random) speckle patterns. The root-mean-square (rms) of the corresponding sequence of fluorescent images comprised the optically sectioned image. A drawback of this technique is that its sectioning strength is weaker than that of a confocal microscope. In this paper, we show that the sectioning strength can be significantly improved by using the same image processing algorithm but with a better control of how the DSI pattern is rendered dynamic. While in our initial implementation, the speckle pattern was completely randomized between image acquisitions, here we apply small lateral translations to the speckle pattern, on the order of a speckle grain size, and demonstrate both theoretically and experimentally that the resulting sectioning strength is near-confocal.
2. Principle of dynamic speckle illumination microscopy
Figure 1 depicts speckle illumination inside a sample and the detection point-spread-function (PSFdet) for an arbitrary pixel in the imaging CCD camera. The photocounts recorded by this pixel are plotted as a function of time, or, more precisely, image frame k. These photocounts can be thought of as arising from two contributions: signal fluorescence arising from the in-focus plane (denoted by S) and background fluorescence arising from out-of-focus planes (denoted by B). Our goal with DSI microscopy is to estimate the fluorophore concentration at the in-focus plane based only on these photocounts. Our strategy for achieving this is to render the speckle illumination dynamic and to exploit an a-priori knowledge on how the temporal statistics of S and B differ.
In particular, there is a fundamental difference between how the photocounts from zones S and B are acquired. Because the in-focus PSFdet is narrow and the out-of-focus PSFdet is wider, we may roughly think of PSFdet as locally sampling the in-focus fluorescence while spatially averaging the out-of-focus fluorescence. Correspondingly, when the speckle pattern is rendered dynamic, different realizations of the speckle pattern are locally sampled in zone S whereas they are spatially averaged in zone B.
The temporal average of the detected photocounts is Ī d = S̄+B̄. Assuming that the different realizations of the speckle pattern are statistically independent for S (though not necessarily for B) then S̄ provides a direct estimate of the fluorophore concentration at the in-focus plane. Our goal, therefore, is to distinguish S̄ from B̄. The temporal average of the photocounts does not, by itself, provide enough information to do this. However, in the case of DSI the additional knowledge garnered from the temporal variance of the photocounts does provide enough information. In particular, it is known that detected photocounts from locally sampled independent speckle patterns obey Bose-Einstein statistics [10]. Accordingly, the temporal variance of S is given by Var(S) = S̄2+S, which can be interpreted as arising from both “classical” local intensity fluctuations of speckle (which obey negative-exponential statistics - first term) and “quantum” shot noise associated with photodetection (which obeys Poissonian statistics - second term). In contrast, the local intensity fluctuations in B are spatially averaged and hence contribute little to its temporal variance, giving Var(B) ≈ B̄. The total variance is then Var(Id ) ≈ S̄2 + S̄+B̄, and an estimate of S̄ becomes straightforward: . This estimate is our intended goal.
A caution should be made regarding this simple idea since the separation between zones S and B in the sample is nowhere near as clear-cut as suggested in Fig. 1. In particular, we have assumed that PSFdet in zone B is so wide as to completely spatially average out the “classical” intensity fluctuations of speckle in B and extinguish their contribution to Var(B). This is an approximation. In fact, a simple variance algorithm cannot completely extinguish residual intensity fluctuations, and better algorithms for their reduction constitute the main motivation for this paper.
3. DSI sectioning with translated versus randomized speckle patterns
The algorithm we consider in this paper is based not on an intensity variance, as suggested above, but rather on a structure function D defined as half the differential intensity variance:
where I d,k is the intensity recorded by the CCD camera on pixel d in frame k (we consider only “classical” intensities for the moment), and N + 1 is the total number of acquired images. This algorithm was introduced in ref. [9] and has the advantage of being insensitive to long-term power variations in the illumination beam. We note that if consecutive I d,k’s are statistically independent, then D is formally equivalent to the intensity variance Var(Id ). Such statistical independence is achieved, for example, if the speckle pattern is completely randomized between camera frames, as was done in ref. [9]. However, if correlations are induced between consecutive I d,k’s, then D can be much smaller than Var(Id ). In particular, if correlations are induced only in the out-of-focus background and not in the in-focus signal, they can lead to significantly enhanced background rejection. A simple technique for introducing such background correlations is with DSI based on speckle translation rather than speckle randomization. In this section we present an intuitive argument for why background rejection is improved, and defer a more rigorous argument to section 4.
To quantify DSI sectioning strength, we consider as we did in ref. [9], a sample that is a thin uniform fluorescent plane and calculate the expected structure function D as a function of the sample’s axial position zc (see Fig. 2). To gain an intuitive understanding, we adopt the following simplifications:
- We approximate the lateral profile of PSFdet( ,z) to be a circular top-hat with a diameter that expands linearly with distance z away from the focal plane. PSFdet( ,z) is normalized such that PSFdet(0,0) = 1 and ∫PSFdet( ,z)d 2 = Ad , independently of z. That is, Ad corresponds to the area of PSFdet at the focal plane.
- We further approximate the speckle pattern to consist of a mosaic of speckle “grains” of equal lateral areas As (the speckle correlation area) and of uniform intensities within these areas. The intensity of each speckle grain is statistically independent of its neighbors’, as illustrated in Fig. 2(c). The speckle grain intensities are denoted I s,n and obey the negative-exponential statistics of a fully developed speckle pattern [10]. Hence, the spatial variance of these grain intensities sampled over many grain areas is given by the average speckle intensity squared (again, we neglect shot-noise for the moment). We note that for widefield speckle illumination, as is the case in DSI microscopy, the speckle correlation area (or grain area) does not significantly depend on z [9].
To calculate the intensity detected by a CCD pixel, we define Ns to be the number of speckle grain areas “seen” by PSFdet. That is, when the sample plane is in focus then Ns = 1 (we assume Ad ≤ As ); when it is out of focus then Ns is the ratio of the PSFdet area to the speckle grain area, which increases as . The intensity detected by a CCD pixel is then given by the spatial summation
where C is the fluorophore concentration and I s,n is the intensity of speckle grain n. We can now readily calculate the structure function D for the cases when the speckle pattern is randomized (DR ) versus when it is translated (DT ). Our final DSI image in both cases is given by √D.
We consider the random case first. As mentioned above, DR can be directly interpreted as the variance of the detected intensities, and therefore reads:
When the sample is at the focal plane, the DSI image is simply given by the product of the averaged illumination intensity Ī s , the concentration C of fluorophores, and the area of PSFdet at the focal plane, as we would obtain with a confocal microscope. When the sample is out of focus, then Ns is proportional to and the final DSI image (√DR ) decays as 1/|zc |. Spatial averaging caused by PSFdet therefore reduces the out-of-focus speckle fluctuations.
For the translation case, we first define ΔI d,k as the difference between two consecutive intensities (ΔI d,k = I d,k-I d,k-1). For a uniform fluorescent plane (as considered here), a translation of the speckle pattern relative to a fixed PSFdet is equivalent to a translation of PSFdet relative to a fixed speckle pattern, allowing us to write:
where ΔNs is the (zc dependent) number of speckle grain areas encompassed in either of the differential areas between PSFdet and its shifted replica. These differential areas are colored in green and red in Figs. 2(c) and 2(f), green corresponding to the gain in new speckle grain areas (left term in Eq. (4)) and red corresponding to the loss in old speckle grain areas (right term in Eq. (4)). The corresponding structure function DT is given by:
If the lateral shift Δρs of the speckle pattern is larger than the width of PSFdet (as is the case, for example, in Fig 2(f)), then ΔNs = Ns and DT = DR . Two consecutive raw images I d,k are then completely uncorrelated and the final DSI image is the same regardless of whether the speckle pattern is translated or randomized.
In contrast, if the lateral shift Δρs is smaller than the width of PSFdet, then ΔNs < Ns and hence DT < DR . In this case, two consecutive raw images are highly correlated because of the overlap in the areas covered by PSFdet and its shifted replica, meaning consecutive raw images comprise many of the same speckle grains. Assuming that Δρs is small (in practice it is of order the speckle grain size) then ΔNs scales linearly with zc for large |zc | (i.e. out of focus) and the final DSI image √DT decays as 1/|zc |3/2. We therefore expect better DSI sectioning strength with translating speckle patterns than with randomized speckle patterns.
We recall that the signal from a fluorescent plane scales as 1/ for a confocal microscope [1]. The DSI sectioning strength for randomized speckle patterns (1/|zc |) was previously referred to as “quasi-confocal” [9]. For lack of a better term, we refer to the DSI sectioning strength for translated speckle patterns (1/|zc |3/2) as “near-confocal”.
4. DSI sectioning for arbitrary PSF
In this section, we present a rigorous calculation of the structure function DT that is valid for arbitrary illumination and detection PSF’s (denoted PSFdet and PSFill respectively). This calculation assumes a translating speckle pattern, however we note that in the limit where the translation step size is infinite, then DT reduces to DR . Hence, this calculation applies to both DSI techniques.
In general, the fluorescence intensity at a position d on the CCD detector plane is given by
where Is ( ,z) is the speckle intensity in sample and C( ,z) the fluorophore concentration. We normalize our PSF’s such that PSFill(0,0) = PSFdet(0,0) = 1, ∫PSFill( ,z)d 2 =As and ∫PSFdet( ,z)d 2 =Ad .
Again we consider a uniform fluorescent plane located at axial position zc . The fluorophore concentration is then C( ,z) = C δ(z - zc ), and the intensity detected at the CCD camera can be simplified to
Denoting as ΔId the difference between two consecutive images, we find,
where is the speckle pattern translation vector. Equivalently,
For ease of notation we recast the structure function DT (Eq. (1)) as , where the overline indicates an average over many successive speckle patterns. Expanding, we obtain
At this point, we invoke a-priori knowledge on the speckle statistics. Assuming that the speckle pattern is fully developed and that the speckle size (correlation length) is roughly constant over a large depth of field [9], we have [10]:
where Δρ = | - ′|. Defining R det as the lateral autocorrelation of PSFdet:
and noting that , we obtain finally:
This expression is valid for arbitrary PSFill and PSFdet and we can progress no further without specifying these. In the case where both PSF’s exhibit Gaussian-Lorentzian profiles, then DT ( d ) can be solved analytically (see below). However, in practice, PSFdet is usually defined by a circular pupil. To consider this latter more realistic case, we assume that is smooth on the scale of a speckle grain size, defined by PSFill(Δρ,0) (we are mostly concerned here with out-of-focus background rejection, and for large enough |zc | this assumption is reasonably valid). We then make the approximation PSFill , and Eq. (12) readily simplies to
Equation (13) is equally valid for translated or randomized speckle patterns. In the case of randomized patterns (Δρs → ∞), then only the first term in Eq. (13) is non-zero, and we recover our previous result (Eq. (3)). Note: to recover this result, we invoke the general definition for the lateral area of PSFdet at depth zc , given by
which is valid for arbitrary PSFdet.
Further insight into the behavior of DT ( d ) is gained by reformulating Eq. (13) in terms of the 2D Optical Transfer Function (OTF):
where ⊥ is a lateral spatial frequency. We note that
from which we readily obtain:
Once again, this equation is equally valid for translated or randomized speckle patterns. In the case of randomized patterns (Δρs → ∞), then the term can be replaced by its average value, namely 0. The random case expression for DT thus simplifies to
The advantage of expressing DT and DR in terms of OTFdet is that a well-established analytical approximation for OTFdet is available for circular pupils, given by [11]
where J 1 is Bessel function of the first kind of order 1, 2Δk ⊥ = 2k sinα is the cutoff frequency for incoherent imaging (sinα corresponds to the detection numerical aperture and k= 2π/λ where λ is the optical wavelength), and
Figure 3(a) shows plots of |OTFdet(⊥,zc )|2 for different values of zc . This function is always equal to 1 at zero spatial frequency, but then rapidly decays at higher frequencies. The greater the defocus, the faster the decay.
Numerical integration of Eq. (18) indicates that DR decays as 1/ (see Fig. 3(b)). The random DSI out-of-focus background rejection (√DR ) therefore scales as 1/|zc |, as expected from our simple model in section 3.
In contrast, numerical integration of Eq. (17) indicates that DT decays as 1/ for and as 1/|zc |3 for (see elbow in Fig. 3(b)). This transition defocus can be interpreted as the defocus where the width of PSFdet is roughly equal to Δρs . When |zc | is less than this transition defocus, the speckle translation is effectively seen as a randomization. However when |zc | is greater than this transition defocus, the speckle translation preserves correlations in consecutive speckle patterns. To keep this transition defocus as tightly confined to the focal plane as possible, Δρs must be chosen to be small (though no smaller than the speckle grain size!). In practice, Δρs is chosen to be of order the speckle grain size. The translation DSI out-of-focus background rejection (√DT ) therefore scales as 1/|zc |3/2, also as expected from our simple model in section 3. The reason for this improved background rejection is clear from Fig. 3(a), and stems from the extra factor in the integrand of Eq. (17). This term effectively quenches the low-frequencies contributions to |OTFdet(k⃗⊥,zc )|2, which decay only weakly (if at all) with |zc |.
Having developed a model that is valid for arbitrary PSF, we now examine how the DSI sectioning strength is influenced by the PSF shape. In the case of speckle randomization, DSI sectioning strength seems to depend little on the exact profile of PSFdet since both circular (see above) and Gaussian (see ref [9]) pupil functions lead to quasi-confocal sectioning.
The situation is different, however, in the case of speckle translation. For example, if PSFill and PSFdet are both defined by a same Gaussian-Lorentzian profile: (where and W 0 is the PSF waist), and we calculate DT directly from Eq. (12), we find
where A = /2. (this expression reduces to that found in ref. [9] when Δρs → ∞). In the case of small speckle translations then and we find that the DSI signal √DT exhibits truly confocal sectioning (as opposed to near-confocal sectioning for a circular pupil function). This suggests that we could improve out-of-focus background rejection by using Gaussian apodization in the detection pupil.
5. Experimental results
Our experimental setup, shown in Fig. 4, is based on a standard widefield microscope with an argon laser as a light source. A spatial light modulator (SLM - Holoeye LC-R-768) is used to apply a random binary phase mask to the laser beam, by randomly assigning to each pixel a phase-shift of 0 or π. The SLM is imaged onto the back focal plane of the microscope objective, thereby producing widefield speckle illumination. The fluorescence emitted at the focal plane of the objective is then imaged onto a CCD camera (Retiga 2000R, Q-Imaging).
Various strategies can be used to translate the speckle pattern, the simplest of which is to apply a linear phase-gradient across the SLM (a phase slope in the objective back focal plane leads to an amplitude translation in the front focal plane). Unfortunately, the dynamic range of our SLM was limited to about π, and hence we had to resort to an alternative strategy: It is well known that a lateral translation of the random phase mask applied at the SLM plane will result in a translation of the speckle pattern everywhere except at the corresponding SLM Fourier plane [12, 13]. We therefore decided to displace this Fourier plane away from the objective focal plane by illuminating the SLM with a diverging beam instead of a collimated beam. The SLM Fourier plane is then located at the geometric image of the effective illumination point source (see Fig. 4), and with this configuration a simple translation of the SLM phase mask induces a translation of the speckle pattern in the region of the objective focal plane.
A more detailed schematic of this illumination configuration is shown in the inset of Fig. 4 where, for simplicity, we removed the 1×1 telescope and depicted the SLM directly at the objective back focal plane. The dotted lines illustrate the conjugation between the effective point source and the SLM Fourier plane. Moreover they provide a simple geometric picture relating a phase-mask translation ΔρM at the SLM plane to the corresponding speckle pattern translations inside (or outside) the sample. At the SLM Fourier plane, this corresponding translation is equal to zero, and the speckle pattern is said to be “boiling”[12]. Away from the SLM Fourier plane, the speckle is almost purely translated. The amount of translation Δρs is linearly proportional to the distance d between the observation plane and the SLM Fourier plane: Δρs = ΔρMd/fo (where fo is the objective focal length)[12]. The SLM Fourier plane can thus be regarded as a pivot plane about which the 3D speckle pattern becomes sheared. In our experimental setup fo = 4.5mm, d FP ≈ 100 μm, and we choose ΔρM such that the speckle translation Δρs at the objective focal plane is on the order of a speckle grain diameter (< 1 μm). We note that as long as |Δρs /d FP|≪ sinα (the case here), then the speckle translation always remains smaller than the width of the out-of-focus PSFdet, as required to ensure out-of-focus correlations between consecutive speckle patterns.
We note that the theory presented in sections 3 and 4 is equally valid for translations comprising directed walks (fixed translation steps and directions) or random walks (fixed translation steps but random directions). The former was found to cause residual streaking in the final DSI image, and we generally opted for the latter in practice (we verified experimentally that both walks lead to the same depth sectioning strength).
For demonstration purposes, we directly compare randomization versus translation DSI imaging by applying a speckle update protocol that alternates between randomization and translation. In this way, I d,k+1 and I d,k are statistically independent when k is odd (by randomization) and statistically dependent when k is even (by translation), and the corresponding DR and DT are evaluated from:
Such an alternating speckle update protocol would not normally be used in practice, however it has the advantage here of allowing us to compare the sectioning strength of DR and DT with rigorously identical data sets.
We emphasize that we have only considered “classical” speckle fluctuations throughout sections 3 and 4, and have neglected the contributions of shot-noise to DR and DT . In practice, to remove these shot-noise contributions we must subtract them from DR and DT before applying the square-root operation to retrieve our final DSI images, as prescribed in section 2 (and being careful to take into account CCD-camera gain and offset).
To experimentally quantify the DSI sectioning strength we measure the signal from a thin fluorescent plane as a function of its axial position zc . Figure 5 displays the measured signal √DT for two different translation step sizes Δρs , and for randomization. The green trace corresponds to a step size Δρs that is larger than the maximun out-of-focus PSFdet width. As expected, such a large step size is equivalent to randomization, and we observe the same quasi-confocal sectioning strength for translation and randomization (red trace), where √DT decays as 1/ |zc |. In contrast, when Δρs is of order a speckle grain size (blue trace), we confirm that DSI sectioning is improved to near-confocal and that √DT decays as 1/|zc |3/2, as expected from theory.
Finally, we present images of a fluorescent pollen grain (Fig. 6). From the same set of raw images, we compute three different images: the translation DSI image (a), the random DSI image (b) and the widefield image (c) which is obtained simply by averaging all the raw images. The translation Δρs is set to approximatively 1 speckle grain size. Again, we observe that translation DSI confers better sectioning than random DSI: the center of the pollen grain is darker, the details appear finer, and the overall image quality is significantly improved. Figure 6(d) illustrates a 3D image of the pollen grain reconstructed from a z-stack of 75 2D images.
6. Conclusion
In conclusion, we have studied the sectioning capacity of DSI microscopy based on a (half) differential intensity variance algorithm, and have shown that this sectioning capacity can be improved by controlling the dynamics of the speckle. We have demonstrated both theoretically and experimentally that out-of-focus background rejection scales as 1/|zc |3/2 when we slightly translate the speckle pattern and as 1/|zc | when we randomize it, where zc is the defocus distance. The improved sectioning strength with translation DSI is near-confocal. In addition, we displayed images of a fluorescent pollen grain and demonstrated a significant improvement in image quality with translation DSI as compared to random DSI. This work is a step toward further improvements in DSI microscopy algorithms designed to increase sectioning strength while reducing the number of raw images required for DSI image processing.
Acknowledgment
We thank Rainer Heintzmann for helpful discussions on shot noise and its subtraction. This work was funded by the Whitaker Foundation.
References and links
1. T. Wilson and C. Sheppard, Theory and practice of scanning optical microscopy, Academic Press, London (1984).
2. G. Q. Xiao, T. R. Corle, and G. S. Kino, “Real-time confocal scanning optical microscope,” Appl. Phys. Lett. 53, 716–718 (2000). [CrossRef]
3. E. M. McCabe, D. T. Fewer, A. C. Ottewill, S. J. Hewlett, and J. Hegarty, “Direct-view microscopy: Optical sectioning strength for finite-sized, multiple-pinhole arrays,” J. Microscopy 184, 95–105 (1996) [CrossRef]
4. T. Wilson, R. Juskaitis, M. A. A. Neil, and M. Kozubek, “Confocal microscopy by aperture correlation,” Opt. Lett. 21, 1879–1981 (1996). [CrossRef] [PubMed]
5. R. Juskaitis, T. Wilson, M. A. A. Neil, and M. Kozubek, “Efficient real-time confocal microscopy with white light sources,” Nature 383, 804–806 (1996). [CrossRef] [PubMed]
6. P. J. Verveer, G. S. Hanley, P. W. Verbeek, L.J. Van Vliet, and T. M. Jovin, “Theory of confocal fluorescence imaging in the programmable array microscope (PAM),” J. Microscopy 189, 192–198 (1998). [CrossRef]
07. M. A. A. Neil and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22, 1905–1907 (1997). [CrossRef]
8. M. A. A. Neil, A. Squire, R. Juskaitis, P. I. H. Bastiaens, and T. Wilson, “Wide-field optically sectioning fluorescence microscopy with laser illumination,” J. Microscopy 197, 1 (2000). [CrossRef]
9. C. Ventalon and J. Mertz, “Quasi-confocal fluorescence sectioning with dynamic speckle illumination,” Opt. Lett. 30, 3350–3352 (2005). [CrossRef]
10. J. W. Goodman, Statistical Optics, Wiley, New-York (1985).
11. P. A. Stokseth, “Properties of a defocused optical system,” J. Opt. Soc. Am. 59, 1314–1321 (1969). [CrossRef]
12. T. Yoshimura, “Statistical properties of dynamic speckles,” J. Opt. Soc. Am. A 3, 1032–1054 (1986) [CrossRef]
13. H. T. Yura, S. G. Hanson, R. S. Hansen, and B. Rose, “Three-dimensional speckle dynamics in paraxial optical systems,” J. Opt. Soc. Am. A 16, 1402–1414 (1999) [CrossRef]