Interpretation of the optical transfer function: Significance for image scanning microscopy

Colin J. R. Sheppard; Stephan Roth; Rainer Heintzmann; Marco Castello; Giuseppe Vicidomini; Rui Chen; Xudong Chen; Alberto Diaspro

doi:10.1364/OE.24.027280

1. Introduction

The optical transfer function (OTF) is a central concept in Fourier optics. For each component of spatial frequency in the object intensity, it determines the strength and phase of the corresponding component in the image. The OTF is a property of the optical system alone, and once calculated, can be used to model the image formation process for different objects. It can also be used to compare different optical systems, because in order to achieve high resolution the cut-off spatial frequency must be as high as possible, and to achieve good contrast the OTF should have a large magnitude.

The concept of the OTF can be applied to three-dimensional (3D) imaging, but here we restrict our attention only to the 2D case. The OFT is applicable to incoherent imaging systems. The OTF is the Fourier transform of the (intensity) point-spread function (PSF). As the PSF is real, the OTF must be Hermitian. For a symmetrical system, the OTF must therefore be real, but can be negative, thus resulting in artifacts in the form of contrast reversal. Negative values of OTF occur with defocus [1] (or in the presence of other aberrations), with confocal systems with finite confocal pinhole size [2], in vectorial (polarized) systems [3], and also with nonlinear effects such as fluorescence saturation. Often the magnitude of the OTF is plotted, which can be confusing as a negative value is shown as positive. The negative sign is important when calculating an image from the object spectrum.

For coherent imaging systems, many of the properties of the OTF apply to the analogous coherent transfer function (CTF), which determines the strength and phase of the amplitude in the image for a component of spatial frequency in the amplitude of the object. There are stricter conditions (as compared with the case of the OTF) that need be satisfied for the CTF to be space-invariant [4]. For partially coherent optical systems, the transfer function approach can be generalized to a bilinear transmission cross-coefficient (TCC) [5].

The OTF is conventionally normalized to unity at zero spatial frequency. This assumption is natural for the investigation of a conventional optical system that does not absorb energy, because then a featureless object is transmitted perfectly. As power is conserved, the value of the OTF for zero frequency is invariant either under defocus or in the presence of aberrations.

However, there are cases when normalization of the transfer function is not appropriate. One case is in a dark field system, when the transfer function is zero at the origin, so that normalization is not possible. A second case is for the comparison of different systems, such as how an increase in the aperture of a system results in both increased cut-off frequency and better collection efficiency. Also, in a confocal system, the signal strength increases as the pinhole size is increased. In a confocal microscope, defocus reduces the signal even for a featureless object, such as a thin fluorescent sheet. Thus for the comparison of the relative imaging performance of different optical systems, it is important to compare the absolute values of the OTF. The magnitude of an unnormalized OTF determines the strength of a spatial frequency component in the image. Spatial frequencies are imaged efficiently only if the value of their strength in the image is above the noise floor of the optical system. Unnormalized OTFs are also useful when considering hybrid systems, which combine optical acquisition and digital processing, because the OTF then gives the true strength of the spatial frequency component.

This paper considers the general properties of OTFs, and unnormalized OTFs in particular, for general imaging applications. We consider two simple examples, firstly the effect of a finite value of the Fresnel number on focal shift, and secondly the effect of varying the aperture of a lens. We then discuss as practical examples the cases of confocal microscopy and image scanning microscopy (ISM), where signal strength is important. ISM is basically confocal microscopy with a detector array instead of a confocal pinhole [6–11], which allows the signal strength in a confocal microscope to be increased, while at the same time retaining, or even improving upon, the resolution of a true (point-like detector) confocal microscope.

In a confocal microscope, the object is illuminated with a scanned focused spot of light. In order to speed up the scanning process, the single illumination spot is sometimes replaced by an array of spots, as in a spinning disc microscope. A detector array can then be used to record a full-field image at each scan position. The similarity of ISM with structured illumination microscopy (SIM), which is a conventional microscope with a fringe pattern projected on to the object, is then apparent. We can consider confocal microscopes and SIMs as particular cases of a general patterned illumination microscope (PIM).

2. Some properties of the OTF

If light is focused through a circular aperture stop with radius $a$ , on to a point a distance $z$ away, with a Fresnel number $N = a^{2} / (λ z)$ not large compared with unity, it is found that the axial maximum in intensity is displaced towards the aperture. This is called the focal shift effect [12]. It can be explained by the properties of the OTF [13]. For points closer to the aperture than the geometrical focus, the effective numerical aperture of the system is increased, and thus the spatial frequency cutoff is also increased. There is also defocus present, but because of the increased numerical aperture (NA) the defocused OTF is rescaled. A property of the Fourier transform is that the 2D integral under the OTF gives the on-axis intensity of the PSF. The on-axis intensity reaches its maximum value when the integral under the OTF is a maximum, thus determining the focal shift.

Because of the reciprocal nature of the Fourier transform, the 2D integral under the PSF gives the magnitude of the OTF for zero spatial frequency. As a result of conservation of energy, the value of the OTF at the origin is independent of defocus, and is conventionally set to unity.

The relationship between an integral and the corresponding central value, in a domain and its Fourier domain, can be considered as a special case of the projection/slice theorem of tomography. This result can also be generalized to higher order moments, the integral being just the zero order moment. The cusp (a discontinuity in the first derivative) in the OTF at zero spatial frequency (i.e. the second derivative is infinite) is related to the $1 / ρ^{3}$ decay in PSF intensity (where $ρ$ is cylindrical radius) [14]. This results in the fact that the second moment of the PSF is infinite for a hard-edged aperture, and cannot therefore be used as a measure of resolution in this case.

3. The unnormalized OTF

Next we consider some general properties of unnormalized OTFs. A simple example to illustrate the behavior is that of changing the diameter $2 a$ of an ideal lens of fixed focal length $f$ , obeying the sine condition. Taking $f$ to be the radius of the Gaussian reference sphere, the NA is just $N A = n a / f$ , where $n$ is the refractive index of the immersion medium. The intensity at the focal point is proportional to $a^{4}$ . The power of four comes from a power of 2 from the stronger focusing with change in NA, and another power of 2 from the increase in area, and therefore increased focused power.

An effect of increasing the NA by a factor of $\sqrt{2}$ (corresponding to a change of aperture of one stop), on the unnormalized OTF, is that the value of the OTF at zero spatial frequency, which is proportional to the power in the focused light, is doubled. The cut-off frequency is also increased by a factor of $\sqrt{2}$ , so that the intensity at the focal point is increased in total by a factor of 4 $(= 2 \times {(\sqrt{2})}^{2})$ . If we consider the noise floor to be at a constant value of say 0.1 relative to the peak in the original OTF, the effective cut-off frequency is increased by a factor of 1.54. This example stresses the importance of not normalizing the OTF when considering the performance in the presence of noise.

4. The confocal fluorescence microscope

Figure 1 shows the normalized in-focus OTF for a confocal fluorescence microscope for different pinhole sizes, calculated using a scalar paraxial theory [2]. The pinhole radius is measured in Airy units. It is seen that the strength of the high spatial frequency components is increased, relative to the zero frequency, as the pinhole size is decreased. The normalized cut-off frequency is 4, where the normalized frequency is $l = l_{t} (NA/ λ)$ , and $l_{t}$ is the true frequency. As the pinhole size tends to infinity, the response for normalized frequencies greater than 2 tends to zero, so the cut-off frequency becomes 2, and the OTF becomes identical to that in a conventional fluorescence microscope.

Fig. 1 The normalized OTF for a confocal microscope with different pinhole sizes in Airy units. The OTF for ISM with a large array is also shown.

Download Full Size | PDF

However, this does not tell the whole story, because as the pinhole size becomes smaller the signal measured from a thin, featureless, planar fluorescent object decreases. The unnormalized OTF has already been presented [15], and is replotted in Airy units in Fig. 2(a). The OTF was calculated from the two dimensional convolution:

C (l) = v_{d} \iint_{A} C_{1} C_{2} \frac{J_{1} (l_{2} v_{d})}{l_{2}} l^{'} d l^{'} d ϕ,

where

C_{1, 2}

are the OTFs for the two lenses,

\begin{array}{l} C_{1, 2} = Re {\frac{2}{π} [arc \cos (\frac{l_{1, 2}}{2}) - \frac{l_{1, 2}}{2} \sqrt{1 - {(\frac{l_{1, 2}}{2})}^{2}}]}, \\ l_{1, 2} = \sqrt{{l^{'}}^{2} + \frac{1}{4} l^{2} \mp l l^{'} \cos ϕ}, \end{array}

and

v_{d}

is the radius of the pinhole in optical units, so that

v_{d} = 3.83 \times A U

with

A U

the pinhole size in Airy units. The integral is evaluated over the region of overlap of

C_{1} C_{2}

. The normalized OTF can then be calculated using the signal level from a uniform fluorescent plane [2], given by putting

l = 0

in Eq. (2).

Fig. 2 The unnormalized OTF for (a) confocal microscopes and (b) ISM with different pinhole/array sizes.

Download Full Size | PDF

It is seen that the magnitude of the OTF for $l > 2$ is weak if the pinhole size is larger than 0.75 AU. This behavior is shown in more detail in Fig. 3(a). The OTF exhibits negative values for pinhole sizes greater than 0.5 AU, which degrade the imaging performance. The frequency at which the OTF first becomes zero drops from 4 to 2.2 as the pinhole size increases from 0.5 to 1 AU, as shown in Fig. 4. Interestingly, the negative values of the OTF are strongest for a pinhole size of around 1AU, which is a commonly used size in experimental confocal microscopy. There is therefore some advantage in limiting the size of the pinhole to a smaller value. Actually, a pinhole size of 0.68 AU has been shown to optimize the ratio of the signal to noise from the background [16]. Alternatively, if a larger pinhole size is used, digital filtering should be employed to suppress or invert the negative parts of the OTF. If the image is low-pass filtered, a modest resolution improvement is attainable for a pinhole size in the range 0.75-1 AU. Note that simple high-frequency enhanced filtering is a poor strategy, as it enhances the negative parts of the OTF. It should be combined with apodization or sign reversal of the high spatial frequencies.

Fig. 3 A close up of the unnormalized OTF at high spatial frequency for a confocal microscope with different pinhole sizes (dashed lines), and for ISM with different array sizes (solid lines). The behavior of ISM with a large array is also shown (black line) for comparison. Note that the confocal case results in negative values of OTF, but ISM does not.

Download Full Size | PDF

Fig. 4 The normalized spatial frequency for the first zero of the OTF, for confocal imaging with pinhole size in Airy units. The cut-off frequency is 4 for pinhole sizes less than 0.5 AU, and tends to 2 for large pinhole sizes.

Download Full Size | PDF

5. Subtractive imaging

A popular imaging technique at present is the general class of methods based on subtractive imaging, where a lower resolution image is subtracted from a high resolution image [17–22]. If a multi-element detector array is used, then images from two different pinhole sizes can be acquired simultaneously. Usually, subtraction reduces the signal and amplifies the noise. But if the subtracted image is recorded with a system with negative components in the OTF, subtraction actually reinforces these components in the final image [22]. Figure 2(a) suggests that pinhole radii of about 0.5 AU and 1 AU would be suitable. The multiplication factor of the subtracted signal can be chosen to give a resolution improvement while avoiding strong negative intensities in the final image: subtracting a quarter of the second image from the first gives empirically a good compromise between frequency response and signal level, as shown in the logarithmic plot of Fig. 5, where only the first positive lobe of the confocal OTF is shown. Of course, combining these two images using multi-image deconvolution (e.g. using Richardson-Lucy) or weighted averaging [19] would be even better, but requires more computational effort and the need to know the original OTFs. Figure 5 also shows the result of using a simple matched filter, which combines the OTFs from a 0.5 AU pinhole and a 0.5-1 AU ring. The processed OTF is taken as $C = {(C_{1}^{2} + C_{2}^{2})}^{1 / 2}$ . The performance is quite good, but to implement this approach the signs of the OTFs must be known a priori.

Fig. 5 A logarithmic plot of the unnormalized OTFs for a confocal microscope (dashed lines) and ISM (solid lines) with different pinhole/array sizes. The first positive lobe only of the confocal OTF is shown. The behavior for subtracting images from two pinhole sizes $(I_{0.5 A U} - \frac{1}{4} I_{1 A U})$ , or using a matched filter with two ring detectors is also shown.

Download Full Size | PDF

6. Image scanning microscopy

In ISM, a 4D data set is acquired by scanning a laser spot in 2D as in confocal microscopy, and detecting an image from a detector array at each scan position. The image can be processed using the concept of pixel reassignment, where it is recognized that a combination of the illumination point and the detection point effectively produces an image of the object point midway between them [6]. These signals can then be integrated over the detector array, which then gives an optical sectioning effect very similar to that in a confocal system with pinhole size equal to the array size. The resulting OTF is shown in Fig. 2(b), which when compared with Fig. 2(a) sums up the comparison in the imaging behavior for ISM and confocal microscopy. The OTF was calculated from the Hankel transform

C (l) = \frac{1}{2 π} \int_{0}^{\infty} \int_{0}^{v_{d}} \int_{0}^{π / 2} h_{1} h_{2} J_{0} (l v) v v^{'} d θ d v^{'} d v,

where the offset intensity point spread functions for illumination and detection after reassignment are

h_{1, 2} = \frac{4 J_{1}^{2} (\sqrt{v^{2} + \frac{1}{4} {v^{'}}^{2} \pm v v^{'} \cos θ})}{v^{2} + \frac{1}{4} {v^{'}}^{2} \pm v v^{'} \cos θ} .

Again the signal level can be calculated, by putting

l = 0

in Eq. (3). For a small array (e.g. 0.25 AU), the OTF is almost identical to that in confocal microscopy, but for larger arrays the high spatial frequency response is much increased [Fig. 2(b)]. The value of the OTF never exhibits negative values (Fig. 3), as are observed with confocal microscopy with a finite pinhole size. The comparison between the OTFs for confocal and ISM is illustrated dramatically in the logarithmic plot in Fig. 5. Note that the OTF for a finite array sometimes has a value slightly greater than that for an infinite array, labelled ISM in Figs. 2, 3 and 5. The performance of ISM with an array bigger than 0.5 AU is better than applying the matched filter strategy without pixel reassignment.

The peak in the PSF is given by the (2D) integral under the OTF. As it is apparent that the OTF is greater for ISM than for confocal microscopy, the peak PSF intensity is higher. For large arrays and simple integration with reassignment, the peak intensity is 1.84 times that of conventional fluorescence [8]. The mechanism is that most of the light is detected, but squeezed into a smaller PSF. This effect has been called superconcentration [23].

ISM with simple integration over a large array exhibits a cusp in the OTF. The PSF is sharper (by a factor 1.53) than in conventional fluorescence, but decays as $1 / ρ^{3}$ as does conventional fluorescence. For low spatial frequencies, the OTF for ISM and confocal microscopy for the same size of array/pinhole are very similar. For a large array, ISM thus behaves similarly to conventional fluorescence for low frequencies. A large, featureless object contains only low spatial frequencies, so that the superconcentration effect is not observed in this case, but is only observed for small objects.

7. Discussion

We have discussed several properties of the general OTF. We have stressed the merits in calculating unnormalized OTFs, which can be used to compare the strengths of the image signal in different optical systems, and to determine its strength relative to the noise level. We have also stressed the importance in retaining the sign rather than only the magnitude of the OTF. Phase is also important, including the case of a phase of $180^{\circ}$ . In addition we have discussed the significance of the integral under the OTF, the value of the OTF for zero spatial frequency, and the low frequency behavior, including the presence of a cusp. These properties are applicable to general optical systems.

These principles were applied to the performance, and comparison, of confocal microscopy and image scanning microscopy. They are also relevant for other types of microscopy, including superresolution microscopy.

The OTF of image scanning microscopy for different array sizes was presented. The combination of improvement in signal strength and resolution of ISM will likely result in this general approach eventually replacing, confocal microscopy with a pinhole, for most applications.

References and links

1. H. H. Hopkins, “The frequency response of a defocused optical system,” Proc. R. Soc. Lond. A Math. Phys. Sci. 231(1184), 91–103 (1955). [CrossRef]

2. M. Gu and C. J. R. Sheppard, “Confocal fluorescent microscopy with a finite-sized circular detector,” J. Opt. Soc. Am. A 9(1), 151–153 (1992). [CrossRef]

3. C. J. R. Sheppard and K. G. Larkin, “Vectorial pupil functions and vectorial transfer functions,” Optik (Stuttg.) 107, 79–87 (1997).

4. D. A. Tichenor and J. W. Goodman, “Coherent transfer function,” J. Opt. Soc. Am. 62(2), 293–295 (1972). [CrossRef]

5. H. H. Hopkins, “On the diffraction theory of optical images,” Proc. R. Soc. Lond. A Math. Phys. Sci. 217(1130), 408–432 (1953). [CrossRef]

6. C. J. R. Sheppard, “Super-resolution in confocal imaging,” Optik (Stuttg.) 80, 53–54 (1988).

7. C. B. Müller and J. Enderlein, “Image scanning microscopy,” Phys. Rev. Lett. 104(19), 198101 (2010). [CrossRef] [PubMed]

8. A. G. York, S. H. Parekh, D. D. Nogare, R. S. Fischer, K. Temprine, M. Mione, A. B. Chitnis, C. A. Combs, and H. Shroff, “Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy,” Nat. Methods 9(7), 749–754 (2012). [CrossRef] [PubMed]

9. C. J. R. Sheppard, S. B. Mehta, and R. Heintzmann, “Superresolution by image scanning microscopy using pixel reassignment,” Opt. Lett. 38(15), 2889–2892 (2013). [CrossRef] [PubMed]

10. G. M. R. De Luca, R. M. P. Breedijk, R. A. J. Brandt, C. H. C. Zeelenberg, B. E. de Jong, W. Timmermans, L. N. Azar, R. A. Hoebe, S. Stallinga, and E. M. Manders, “Re-scan confocal microscopy: scanning twice for better resolution,” Biomed. Opt. Express 4(11), 2644–2656 (2013). [CrossRef] [PubMed]

11. S. Roth, C. J. R. Sheppard, K. Wicker, and R. Heintzmann, “Optical photon reassignment microscopy (OPRA),” Opt. Nanoscopy 2(1), 5 (2013). [CrossRef]

12. Y. Li and E. Wolf, “Three-dimensional intensity distribution near the focus in systems of different Fresnel numbers,” J. Opt. Soc. Am. A 1(8), 801–808 (1984). [CrossRef]

13. C. J. R. Sheppard, “Imaging in optical systems of finite Fresnel number,” J. Opt. Soc. Am. A 3(9), 1428–1432 (1986). [CrossRef]

14. R. N. Bracewell, The Fourier Transform and its Applications (McGraw Hill, 1978).

15. C. J. R. Sheppard and M. Gu, “The significance of 3-D transfer functions in confocal scanning microscopy,” J. Microsc. 165(3), 377–390 (1992). [CrossRef]

16. X. S. Gan and C. J. R. Sheppard, “Detectability: A new criterion for evaluation of the confocal microscope,” Scanning 15(4), 187–192 (1993). [CrossRef]

17. T. Wilson and D. K. Hamilton, “Difference confocal scanning microscopy,” Opt. Acta (Lond.) 31(4), 452–465 (1984). [CrossRef]

18. V. Sarafis, C. Johnson, and G. Boyer, “Confocal microscopy with pinhole super-resolution,” Cell Vis. 4, 264 (1997).

19. R. Heintzmann, V. Sarafis, P. Munroe, J. Nailon, Q. S. Hanley, and T. M. Jovin, “Resolution enhancement by subtraction of confocal signals taken at different pinhole sizes,” Micron 34(6-7), 293–300 (2003). [CrossRef] [PubMed]

20. H. Dehez, M. Piché, and Y. De Koninck, “Resolution and contrast enhancement in laser scanning microscopy using dark beam imaging,” Opt. Express 21(13), 15912–15925 (2013). [CrossRef] [PubMed]

21. S. You, C. Kuang, Z. Rong, and X. Liu, “Eliminating deformations in fluorescence emission difference microscopy,” Opt. Express 22(21), 26375–26385 (2014). [CrossRef] [PubMed]

22. R. Gauderon and C. J. R. Sheppard, “Improvement in imaging in confocal fluorescent microscopes using detector arrays,” Bioimaging 6(3), 126–129 (1998). [CrossRef]

23. S. Roth, C. J. R. Sheppard, and R. Heintzmann, “Superconcentration of light: circumventing the classical limit to achievable irradiance,” Opt. Lett. 41(9), 2109–2112 (2016). [CrossRef] [PubMed]

Interpretation of the optical transfer function: Significance for image scanning microscopy

Abstract

1. Introduction

2. Some properties of the OTF

3. The unnormalized OTF

4. The confocal fluorescence microscope

5. Subtractive imaging

6. Image scanning microscopy

7. Discussion

References and links

Cited By

Figures (5)

Equations (4)

Optics Express