1. INTRODUCTION

Optical sectioning has enabled imaging of large volumes by a fluorescence microscope (FM), as realized in confocal, two-photon microscopy [1], structured illumination microscopy (SIM) [2], and light sheet microscopy [3]. For optimal imaging, aberrations need to be minimized by avoiding a RIM between the detection microscope objective immersion medium ${n_1}$ and specimen ${n_2}$ [1]. Failing to do so results in the blurring of the point spread function (PSF) of the microscope and therefore a loss in resolving power, as well as in a deformation of the recorded volume along the optical axis.

This axial deformation arises from the refraction of the peripheral rays on the RIM interface, causing an axial shift of their focal point with respect to the focal point of the paraxial rays [4]. This effect can be characterized using focal shift $\Delta f = {\rm AFP} - {\rm NFP}$, where AFP is the actual focal position (the real position of the object) and NFP is the apparent or nominal focal position (the microscope ${z}$-position where the object is found in focus) [1]. Just as the true lateral distances are recalculated back from an image using the magnification of the objective $M$, so ought the AFPs (i.e., the true axial distances) be re-scaled using the re-scaling factor $\zeta = {\rm AFP}/{\rm NFP}$. The accurate knowledge of the re-scaling factor enables reliable quantitative volumetric microscopy.

While it is generally unfavorable to have a refractive index mismatch for, e.g., illumination intensities in confocal microscopy, there are common imaging scenarios in which a mismatch is still present in the most optimal configuration [1,4]. For instance, as the resolving power of the microscope depends on the NA of the detection objective, high-NA oil immersion objectives (${n_1} = 1.52$) are used to image water-like specimens (${n_2} \approx 1.33$). In addition, embedding and fixation media rarely match exactly, in terms of refractive index, the immersion media of air, water, silicone, glycerol, or oil objectives, leading to the axial deformation of the imaged volumes.

Refractive index mismatches are often more pronounced in integrated cryogenic fluorescence and/or correlative microscopy [5–8]. In such systems, light and electron microscopies are combined in a single setup, and air objectives are often used for imaging specimens with a higher refractive index. The use of air objectives is a straight-forward choice, as the specimen resides in a vacuum chamber for electron microscopy. In cryogenic fluorescence microscopy the specimen is cooled to temperatures below 120 K, which makes the use of a non-touching air objective favorable, as any other (touching) immersion objectives are challenging in terms of engineering [9–12].

The development of the confocal microscope, along with its optical sectioning capability, allowed for 3D imaging, and hence the need for axial scaling theories arose. In the paraxial approximation, the axial distances are simply re-scaled using the ratio of the two refractive indices [13], which works well for low-NA objectives. Visser et al. presented a scaling theory that is based on the contribution of the high-angle (or marginal) rays to the axial scaling [14]. More recently, several scaling theories have been presented that result in re-scaling factors somewhere in between the paraxial and high-angle approximations [4,15,16]. The most accurate method to determine axial scaling is full wave-optics calculations of the microscope’s point spread function under RIM [17–19]. As these calculations are computationally expensive and complex, they are hardly used by microscopists to calculate the axial re-scaling factor.

The re-scaling factor can also be measured experimentally by observing the NFP of a fluorescent bead or interface through a medium with refractive index ${n_2}$, while knowing the AFP of said bead or interface. This can be done by constructing a sample cell where the fluorescence is present far from the coverslip, which is later filled with a liquid with ${n_2}$ [4,20,21]. The AFP can be obtained through imaging with an objective where ${n_1} = {n_2}$ [4,20,21] or by measuring Fabry–Pérot fringes in the transmission spectrum of the cell [21]. Alternatively, the apparent axial deformation of a spherical object, larger than the PSF of the microscope, can be used to measure the axial scaling [14,22]. Recently, a different approach was presented where a coverslip was step-wise coated with a low-index polymer (${n_2} \approx 1.33$) and the AFP was measured using stylus profilometry [23]. With this method, the axial scaling can be measured in the range of a few microns from the coverslip, while the former methods are used to measure tens of microns away from the coverslip.

While the explicit axial scaling theories in literature are all depth independent, there exist some reports in literature that this factor is actually depth dependent. In 1993, Hell et al. wrote “…it can be expected that the regions close to the cover glass are slightly more scaled than those in deeper regions of the specimen.” [17]. Later, wave-optics calculations showed a non-linear dependence of the focal shift $\Delta f$ on the imaging depth for high-NA objectives and large RIMs [18,19]. For instance, Sheppard and Török reported a non-linear dependence of the focal shift $\Delta f$ at a depth ${\lt}{30}\;{\unicode{x00B5}{\rm m}}$ from the coverslip using wave-optics calculations (${\rm NA} = {1.3}$, ${n_1} = 1.52$, ${n_2} = 1.33$) [18], where the re-scaling factor was 5% larger close to the coverslip than at large distances. More recently, the measurements by Petrov et al. showed significant non-linear axial scaling for imaging depths ${\lt}4\;\unicode{x00B5}{\rm m}$ [23].

There is, up to now, no straightforward equation that can be used to explicitly calculate the re-scaling factor as a function of depth. Moreover, a depth dependence of the re-scaling factor has not been measured experimentally for large depths, and for several NAs and multiple RIM conditions.

Here, we present an analytical theory that calculates the depth-dependent re-scaling factor as a function of the NA, the refractive indices ${n_1}$ and ${n_2}$, and the wavelength $\lambda$. The gist of the theory is in the determination of the leading constructive interference band in the objective lens’ pupil under RIM. We compare the theory to both full wave-optics calculations and experiments. In the experiments, we have imaged the gap between two substrates that were brought closer to each other step by step. By filling the space between the substrates with a liquid with index ${n_2}$ we were able to measure the NFP, while the AFP was determined independently from the microscope by the piezo-stage holding the top substrate. We have measured the re-scaling factor $\zeta$ for several objectives with various NAs, immersion refractive index ${n_1}$, and sample refractive index ${n_2}$, with both RIMs where ${n_1} \lt {n_2}$ and ${n_1} \gt {n_2}$ in a wide range of depths and compared them to the analytical theory and the wave-optics calculations. We demonstrate that the axial re-scaling of 3D microscopy data, recorded with a refractive index mismatch, using the depth-dependent re-scaling factor outperforms the re-scaling using existing linear re-scaling theories from literature. Finally, we provide the reader with an online web applet where one can visualize the depth-dependent axial re-scaling factor for their specific optical setup and Python software to re-scale data acquired under RIM.

2. SCALING OF AXIAL DISTANCES DUE TO REFRACTIVE INDEX MISMATCH

A. Geometrical Optics

Given a fluorescent object emitting light in an ideal, spherical fashion, a flat interface with a refractive index mismatch will cause disturbance to the fluorescent light propagation–in the form of refraction. When an objective lens is collecting light under RIM, this effect will be more pronounced with increasing NA of the objective lens (OL) (and therefore collection angle); see Fig. 1(a) for the overall geometry.

 

Fig. 1. Effects of RIM demonstrated with geometrical optics. (a) OL imaging a fluorescent object (green star) without and with refractive index mismatch, left and right, respectively. The dotted boxes are detailed out in (b)–(d). (b) A fluorescent object (green star) sits ${z_A} = 10\;\unicode{x00B5}{\rm m}$ deep from the refractive index interface (dotted line) between ${n_1}$ and ${n_2}$. With ${n_1} = {n_2} = 1.0$, rays (black) and wave-fronts (blue) are depicted. (c) In the case where ${n_1} = 1.0$, ${n_2} = 1.33$ the ideal spherical wave-fronts (dashed blue) are deformed (solid blue) after crossing the refractive index (RI) interface. Rays are depicted in red to indicate RIM and the caustical surface (purple) shows where geometrical optics breaks down. (d) In the opposite RIM case, where ${n_1} = 1.52$, ${n_2} = 1.33$, again the wave-fronts are deformed under RIM and total internal reflection occurs. The NA is (b), (c) 0.95 and (d) 1.4.

Download Full Size | PDF

This is further illustrated using geometrical optics in Fig. 1(b) through Fig. 1(d), where a light source is located 10 µm away from the interface between ${n_1}$ and ${n_2}$. In Fig. 1(b) no RIM is present (${n_1} = {n_2} = 1.0$), where the undisturbed wave-fronts and rays are shown in solid blue and black, respectively. The dotted horizontal line depicts the interface with the RIM. Figures 1(c) and 1(d) show two cases where RIMs are present, respectively, for ${n_1} \lt {n_2}$ and ${n_1} \gt {n_2}$. The black rays follow Snell’s law when crossing the interface, resulting in the wave-fronts (solid blue) deviating from the ideal spherical shape (dashed blue). The caustic surface (purple) indicates where geometrical optics breaks down. From this simple illustration, we see that the effect of RIM becomes more pronounced with increasing collection angles (numerical aperture) and increasing RIM contrast $\Delta n = |{n_2} - {n_1}|$.

When imaging with a microscope, there are two (independent) axial coordinates to be considered: the actual depth of the fluorescent emitter ${z_A}$, as measured from the RIM interface (${z_A} = 10\;\unicode{x00B5}{\rm m}$ in Fig. 1) and the nominal depth of the focal plane of the microscope into the sample ${z_N}$. A 3D image stack recorded with a microscope under RIM has ${z_N}$ as an axial coordinate. Similar to using the magnification $M$ to re-calculate the true lateral dimensions, the axial coordinates should be re-scaled using the re-scaling factor $\zeta \equiv {z_A}/{z_N}$ to obtain a 3D stack with the actual depth ${z_A}$ as an axial coordinate.

Geometrical optics can produce several estimates for the re-scaling factor $\zeta$. The paraxial rays are nicely focused, even under RIM, producing the estimate [13]

Another geometrical optics estimate comes from Visser et al., obtained from the marginal rays still fitting into the NA [14]:

The two re-scaling factor values ${\zeta _{{\rm paraxial}}}$ and ${\zeta _{{\rm marginal}}}$ are irreconcilable in modern high-performance objective lenses (with ${\rm NA} \to 0.95 \times {n_1}$). For the example shown in Fig. 1(c), ${\zeta _{{\rm paraxial}}} = 1.33$ and ${\zeta _{{\rm marginal}}} = 2.98$, while for Fig. 1(d) ${\zeta _{{\rm paraxial}}} = 0.88$ and ${\zeta _{{\rm marginal}}} \to 0$. Although these estimates are derived from geometrical optics, ${\zeta _{{\rm paraxial}}}$ and ${\zeta _{{\rm marginal}}}$ do provide lower and upper bounds when ${n_1} \lt {n_2}$ (reverse when ${n_1} \gt {n_2}$) and are still useful checks in the wave-optics treatment.

B. Analytical Expression Describing Depth-Dependent Axial Scaling

When considering the PSF of a wide-field microscope under RIM two intrinsic length scales need to be considered: (i) several strictly geometrical parameters ${n_1}$, ${n_2}$, ${z_A}$, ${z_N}$, and NA, and (ii) the physical parameter of the (vacuum) wavelength of light used $\lambda = 2\pi /k$, which is independent of the exact RIM geometry, where $k$ is the wave number. With changing imaging depth, the geometrical parameters ${z_A}$, ${z_N}$ do change, while the wavelength of light $\lambda$ does not. Following the derivation outlined below, we find it is exactly the interplay between the two length scales that yields a depth-dependent re-scaling factor:

 

Fig. 2. Structure of $\Phi (\theta)$ for ${n_1} \lt {n_2}$, where the diffraction integral in k-space is illustrated with the arcs of Ewalds’ spheres. $k_z^\prime $ and ${k_z}$ are the axial wave numbers with and without RIM. (a) After transition to a new medium the NA is preserved (due to the Snell’s law) while a different Ewalds’ sphere is used, going from black (no RIM) to red (under RIM). (b) The RIM phase-function $\Phi (\theta)$ re-scales (multiplies) the spheres in the axial directions in ${z_A}$ (red) and ${z_N}$ (black), and the phase derivative can be set to zero only at one point on the arc depending on the exact values of ${z_N}$ and ${z_A}$ (for non-zero $\theta$). The Ewalds’ sphere curvature under RIM changes with changing depth (red translucent, $\Delta {z_N}$) and with it the stationary point ${\rm d}\Phi /{\rm d}\theta = 0$.

Download Full Size | PDF

C. Derivation of Analytical Theory through Wave-Optics

With geometrical optics ambiguous in defining the re-scaling factor $\zeta$ we have to turn to wave-optics. The most complete wave-optics theory describing aberrations occurring under RIM was developed by Hell et al. [17]. Their calculation scheme takes into account the vectorial nature of light and is, in essence, equivalent to the three-fold application of the scalar model described in [24]. The scalar model in turn is based on the general diffraction integral applied to the ray fans in RIM [25–27]. A scalar (radially symmetric) PSF of a wide-field microscope under RIM is an integral over Bessel beams:

Compared to the ambiguous behavior of geometrical optics, Eq. (4) provides a clear prediction of the behavior of a microscope imaging under RIM. For a given depth ${z_A}$ of a fluorescent object there is a focal plane depth ${z_N}({z_A})$ where the PSF is maximal (i.e., the object is in focus), and $\zeta ({z_A}) \equiv {z_A}/{z_N}({z_A})$. Under RIM, two intrinsic length scales need to be considered: (i) several strictly geometrical parameters ${n_1}$, ${n_2}$, ${z_A}$, ${z_N}$, and NA, and (ii) the (vacuum) wavelength $\lambda = 2\pi /k$, which is totally independent of the RIM geometry. These two length scales are fused together in Eq. (4) via the phase function $\Phi (\theta)$. With changing imaging depth, the geometrical parameters also change (e.g., the path length difference between the paraxial and the marginal rays). Reversely, the wavelength of light does not depend on the depth. So it is the interplay between geometrical and physical length scales that results in a depth-dependent re-scaling factor $\zeta ({z_A}) \ne {\rm const}$.

To find $\zeta$ without calculating the integrals like Eq. (4) we estimate the maximum of such an integral by finding the leading terms of a simpler integral:

To estimate the maximum positions of ${I_{{\rm approx}}}$ and assuming that ${e^{{i\Phi}}}$ is oscillating fast (for example, at large depth), we try to find the stationary points ${\theta ^*}$ where ${\rm d}\Phi /{\rm d}\theta = 0$ [28]. The structure of $\Phi (\theta)$ in the case of ${n_1} \lt {n_2}$ is shown in Fig. 2. The axial wave numbers under RIM and without RIM, $k_z^\prime $ and ${k_z}$, belong to the Ewalds’ spheres [Fig. 2(a), red and black], with $k_r^\prime (\theta) \equiv {k_r}(\theta) = {k_1} \times \sin \theta$ conserved under RIM due to Snell’s law. Radial wave number conservation allows the use of $\sin \theta {\rm d}\theta = {\rm d}{k_r}/{k_2}$, making Eq. (4) more similar to ${I_{{\rm approx}}}$. The two integrals [Eqs. (4) and (6)] are taken over spherical segments having the same width, but different radii due to RIM, where the larger sphere is flattened compared to the smaller one. The RIM phase-function $\Phi (\theta)$ re-scales these spheres in the axial directions in ${z_A}$ and ${z_N}$; see Fig. 2(b). The stationary points ${\theta ^*}$ are found where the two re-scaled spheres are running in parallel to each other (dashed lines, ${\rm d}\Phi /{\rm d}\theta = 0$). There is always a stationary point at $\theta = 0$, and a stationary point might appear for a single non-zero $\theta$. For a given ${z_A}$ and ${z_N}$, there is an interference between the contributions from the two critical points (${e^{i\Phi (0)}}$ and ${e^{i\Phi ({\theta ^*})}}$), which controls the shape of the PSF. And when the interference is the most constructive, then the object appears in focus.

To find the non-zero stationary point ${\theta ^*}$ we rewrite $\Phi$ using Ewalds’ spheres and Snell’s law:

The stationary phase points of the integral in Eq. (6) correspond to the rays of geometrical optics [26–28]. Hence, the re-scaling factor must fall within the bounds set by ${\zeta _{{\rm paraxial}}}$ and ${\zeta _{{\rm marginal}}}$. Contrary to the purely geometrical optics approach, we have now two contributions to balance: the paraxial one ${e^{i\Phi (0)}}$ and the intermediate one ${e^{i\Phi ({\theta ^*})}}$.

We estimate for which ${z_A}/{z_N}$ the two stationary point contributions ${e^{i\Phi (0)}}$ and ${e^{i\Phi ({\theta ^*})}}$ build the largest constructive interference. The relative strengths of the two leading contributions in the integral ${I_{{\rm approx}}}$ [Eq. (6)] depend on the width of the stationary points of $\Phi$, and hence on the second derivative of $\Phi$. The contributions of stationary-points in Eq. (4) are also weighted by pupil-amplitude factor $A(\theta)$. Instead of finding the exact weight of each contribution, we study how the phase function $\Phi (\theta)$ behaves in several focal positions ${z_N}({z_A})$, which can be determined by full vectorial PSF calculations.

This is shown in Fig. 3(a) for ${\rm NA} = 0.95$, ${n_1} \lt {n_2}$. The phase function at the second stationary point lies between ${-}\frac{\pi}{3}$ and ${-}\frac{{2\pi}}{3}$ from the phase function at the first stationary point. Therefore, we select the following condition when deriving an explicit formula for $\zeta$:

Using Ewalds’ spheres, Snell’s law, and Eq. (8) yields

Going back to the left hand-side of Eq. (10),

When imaging under the RIM condition ${n_1} \gt {n_2}$, the derivation is similar, with some slight differences. The red and black colors in Fig. 2 should be switched, which changes the second stationary point from a minimum to a maximum:

 

Fig. 3. RIM phase function difference $\Phi - \Phi (0)$ taken in the focal position ${z_N}({z_A})$, determined from the wave-optics calculations for fluorophore depths ${z_A}$ set to 10 µm (red), 20 µm (green), 50 µm (blue), and 100 µm (black). (a) Objective NA of 0.95, ${n_1} = 1.0$, ${n_2} = 1.33$ (solid lines) and ${n_2} = 1.52$ (dashed lines). (b) Objective NA of 1.4, ${n_1} = 1.52$, ${n_2} = 1.33$ (solid lines) and ${n_2} = 1.0$ (dashed lines). For both cases, the second stationary point is somewhere between $\pi /3$ and $2\pi /3$ from the paraxial stationary point at ${k_r}/{k_0} = 0$.

Download Full Size | PDF

D. Critical Value at Shallow Depths

At very large depths, the re-scaling factor $\zeta$ approaches the paraxial regime [Eq. (1)], as $\epsilon \to 0$. At very shallow depths (i.e., large $\epsilon$) Eq. (13) becomes problematic: $\zeta ({z_A} = 0) = \infty$ when ${n_2} \gt {n_1}$ and $\zeta ({z_A} = 0) = 0$ for ${n_2} \lt {n_1}$, going beyond the bounds of the geometrical optics [${\zeta _{{\rm marginal}}}$ in Eq. (2)]. This is related to the fact that the actual value of the NA of the objective lens was omitted in the derivation so far. At ever decreasing depths, the second critical point ${\theta ^*}$ moves to higher wave numbers ${k_r}$ and higher angles $\theta$ (peak of the phase difference in Fig. 3) until it falls outside the NA of the optical system [Fig. 2(a)]. Hence, there is some (NA-dependent) depth below which the assumptions we used to derive Eq. (13) break down. We will hence derive a NA-dependent critical value of re-scaling ${\zeta _{{\rm crit}}}$ to be used at small depths.

With the whole range of phase function values $\Phi (\theta)$ becoming smaller than $\pi /2$ at small depths, the earlier condition on the separation of the critical points [Eq. (9)] cannot be satisfied anymore within the confines of the NA. At such small depth, the entire pupil participates in constructive interference and the breakdown condition can be written as

From the definition of $\Phi (\theta)$ and ${k_r}({\theta _{{\max}}}) = {k_0} \times {\rm NA}$,

Equation (15), together with the universal NA-independent expression for $\zeta$ Eq. (13), forms Eq. (3) and describes the depth-dependent axial scaling when imaging under RIM.

E. Re-scaling Microscopy Data

When re-scaling microscopy data from the native ${z_N}$ coordinate into the true depth coordinate ${z_A}$, it is convenient to use the re-scaling factor as a function of the nominal depth ${z_N}$ instead of the actual depth ${z_A}$ utilized in Eq. (13). In this case, a different dimensionless parameter $\delta$ can be used for the derivation:

Table 1. Overview of Microscope Objectives and Refractive Indices in the Measurements

View Table

3. METHODS

A. Wave-Optics Calculations

For the wave-optics calculations of the PSFs we adapted the most complete description from Hell et al. [17]. The radially symmetric fields (of both excitation and emission light) are represented as integrals over component Bessel beams. The wave-optics is linked with geometrical optics through an elegant mathematical correspondence between Bessel beams and light-cone sections of the ray manifold [25,26,29]. The effects of RIM are taken into account as (RIM-dependent) phase shifts of the component Bessel beams, resulting in a new interference pattern of the field and thus in a new PSF shape. The (dimensionless) intensity of a focal spot can be found by computing three integrals ${I_0}$, ${I_1}$, and ${I_2}$:

Integrals ${I_i}$ each have a Bessel function of the first kind ${J_i}(r{k_r}(\theta))$ weighted by some coefficients of geometric origin. The common factors are $\sin \theta$ (from the Jacobian), $\sqrt {\cos \theta}$ (from Abbe’s sine condition), and ${e^{i\Phi (\theta)}}$ (the RIM induced phase shifts). The argument of the Bessel functions is $r{k_r}(\theta) = r \times {n_2}{k_0}\sin \theta$. Taking into account the light polarization, the following factors appear: ${I_0}$ contains a factor of $(1 + \cos \theta)$, ${I_1}$ a factor of $\sin \theta$, and ${I_2}$ a factor of $1 - \cos \theta$. For each polarization orientation $s$ and $p$, the partial transmission is taken into account as ${t_s}(\theta)$ and ${t_p}(\theta)$ coefficients. Thus, the polarization factors become ${t_p}(\theta) + {t_s}(\theta)\cos \theta$ in ${I_0}$, ${t_s}(\theta)\sin \theta$ in ${I_1}$, and ${t_p}(\theta) - {t_s}(\theta)\cos \theta$ in ${I_2}$. The integrals then become

The computations of the integrals were performed through a MatLab script producing 2D PSF cross-sections ($r \times {z_N}$, $10 \times 20\;\unicode{x00B5}{\rm m}$) on an ${N^2} = 2048 \times 2048$ pixel grid from $M = 2000$ Bessel beam components. To minimize the number of Bessel function invocations, each of the integrals ${I_i}$ was represented as a double matrix product:

For every RIM condition, a series of 2D PSF cross-sections (${\rm lateral }\times {\rm axial}$, $XZ$ planes) was generated with the actual depth ${z_A}$ changing from 0.2 to 5 µm with a step of 0.2 µm. Step sizes of 0.5 and 1.0 µm were used respectively from 5.5 to 25 µm and from 26 to 150 µm. The axial center of the calculated PSF frame was shifted with the actual depth ${z_A}$ using the paraxial scaling factor $z_N^0 = {z_A}({n_1}/{n_2})$. In the lateral center of each frame, the $Z$ profile was extracted and passed through a Butterworth low-pass filter (fifth order, sampling and critical frequencies ${f_s} = 0.1\;{{\rm nm}^{- 1}}$, ${f_c} = 0.003\;{{\rm nm}^{- 1}}$) to remove high-frequency ringing oscillations. The axial focal position $z_N^*$ was determined by taking the maximum of the filtered profile and the re-scaling factor was computed as $\zeta = {z_A}/z_N^*$.

B. Axial Scaling Measurements

We measured the depth-dependent axial scaling using a fluorescence microscope setup described earlier [9], for different refractive index mismatches and numerical apertures, as listed in Table 1. A schematic of the measurement scheme is shown in Fig. 4(a), where a sapphire ball (BA, Ceratec, 2 mm diameter, grade 10) is separated from a coverslip (CS, Thorlabs #CG15CH2, $22 \times 22\;{\rm mm}$, #1.5H thickness). The sapphire ball was glued to a glass strip (GS) with a small droplet of UV curable glue (Norland Optical Adhesive 63). Fluorescent beads (FB, diameter 190 nm, excitation 480 nm, emission 520 nm, Bangs Labs #FSDG002) were dropcasted on the coverslip using a micropipette. Both surfaces were cleaned with acetone and isopropyl alcohol and dried with nitrogen. The glued sapphire ball was dipped in the solution and dried three times to apply the beads to the surface. Beads on both surfaces were imaged by acquiring a $z$-stack with the fluorescence microscope using an excitation wavelength of 485 nm (Lumencor SPECTRA X), dichroic mirror (Semrock #FF410/504/582/669-Di01-25x36), and an emission filter (Semrock #FF01-525/30-25). The objective lens was mounted upright and was moved by stick-slip piezo positioners (SmarAct GmbH) in $X,\;Y$, and $Z$. The correction collar setting of the OL was optimized to minimize spherical aberrations when imaging beads on the coverslip. The glass strip with a sapphire ball was mounted to the sample shuttle holder (SSH), Fig. 4(b). This holder was moved by piezo positioners, and hence the distance between the coverslip and the ball was controlled. Details on the positioning systems can be found in [9].

 

Fig. 4. Experimental setup, and data analysis procedure, used to measure depth-dependent axial scaling. (a) Schematic showing the movable optical objective lens (OL) and sapphire ball (BA) at opposite sides of the static coverslip (CS), with fluorescent beads (FB) on their surfaces. (b) Photograph showing the objective lens (OL), coverslip (CS), and sapphire ball (BA). The ball is glued to a glass strip (GS), which is mounted to the sample shuttle holder (SSH). Scale bar 3 mm. (c) Maximum intensity projection along the $Y$ axis of a recorded ${ z}$-stack where ${n_1} = {n_2} = 1.0$. The bottom fluorescent signal originates from beads residing on the coverslip, and the signal at the top from beads on the sapphire ball, shown by the visible curvature. (d) Schematic showing three different distances between ball and coverslip, going from clear separation (I) to contact (II) and deformation of the coverslip (III). The dashed lines indicate the position of both ball (red) and coverslip (black). (e) Position of the coverslip and ball versus the set ball-coverslip distance ${z_S}$ (top) and the same data near the coverslip (bottom). (f) Landing curve (in ${z_N}$ coordinates), plotted against the set depth ${z_S}$ (top). The fit values from a piecewise linear fit are used to determine the contact point between ball and coverslip, and this yields the observed focal depth ${z_N}$ versus the actual depth ${z_A}$ from which the re-scaling factor $\zeta = {z_A}/{z_N}$ was calculated (bottom).

Download Full Size | PDF

We used different objective lenses, but all were compatible with the Nikon CFI60 optical system, and were used together with a 200 mm tube lens (Nikon #MXA20696). Care was taken to center the pupil of the objective lens to the center of the tube lens, after which the objective lens was solely moved along the optical axis. With the distance between ball and coverslip set, a ${z}$-stack was acquired by moving the objective lens. In Fig. 4(c) the maximum intensity projection along $Y$ is shown for such a ${z}$-stack, where the fluorescent beads were imaged using a $100 \times$, 0.85 NA objective (Nikon #MUE35900) in the absence of a refractive index mismatch (${n_1} = {n_2} = 1.0$). The bottom fluorescent signal originates from fluorescent beads residing on the coverslip. The signal at the top comes from the beads on the sapphire ball, as recognized by the visible curvature.

The AFP was varied by changing the distance between ball and coverslip, and the contact point between them was first approximately set by having both the fluorescent beads on the coverslip and bottom side of the ball in focus of the microscope. Next, the starting AFP (${z_A}$, typically around 100 µm) was set by moving the ball away from the coverslip and the AFP was reduced step by step [see Fig. 4(d), I through III]. At each set depth ${z_A}$, a ${z}$-stack is recorded around the coverslip and ball separately (by moving the objective lens, ${-}4$ to 8 µm and ${-}8$ to 8 µm, respectively, both with a step size of 0.25 µm). To ensure the second $z$-stack was acquired around the beads on the surface of the ball, we used ${\zeta _{{\rm marginal}}}$ and ${\zeta _{{\rm paraxial}}}$ estimates for ${\lt}20\;\unicode{x00B5}{\rm m}$ and ${\gt}20\;\unicode{x00B5}{\rm m}$, respectively. At the end of the measurement sequence the ball was forced into contact with the coverslip, slightly deforming it by setting negative values of AFP on purpose (to about –2 µm); see Fig. 4(d), III.

The 3D positions of all fluorescent beads in the recorded ${z}$-stack were acquired using the PSF Extractor software [9,30] and the $X,\;Y$, and $Z$ locations of all fluorescent beads were extracted. From these, the position of the coverslip ${z_{\textit{CS}}}$ was determined by taking the median of the $z$ positions of the beads on the coverslip, and a least squares paraboloid fit on the coordinates of the beads residing on the ball surface was used to determine the lowest point of the ball ${z_B}$. These $z$ positions are plotted against the set depth ${z_S}$ in Fig. 4(e) for the ball (black) and coverslip (red). Around 5.5 µm the ball and coverslip make contact, effectively deforming the coverslip at lower set depths. The coverslip position was subtracted from the lowest point of the ball (${z_N} = {z_B} - {z_{\textit{CS}}}$) and plotted against (${z_S}$); see Fig. 4(f) (top). A piecewise linear function of the form

 

Fig. 5. Axial re-scaling factor $\zeta$ versus AFP for different NAs and RIM contrasts, obtained through analytical expression Eq. (3) (colored solid) and as calculated using wave-optics (colored dashed). For each NA the re-scaling factor as calculated by Lyakin et al. is included as a black solid line [16]. (a) $\zeta$ as a function of AFP for NAs of 0.95, 0.85, and 0.70 at ${n_1} = 1.0$ and ${n_2} = 1.33$. (b) $\zeta$ as a function of AFP for a NA of 1.4 at ${n_1} = 1.52$ and ${n_2} = 1.33$.

Download Full Size | PDF

C. Re-scaling of 3D Microscopy Data

To demonstrate and test the re-scaling of 3D microscopy data, we recorded z-stacks of fluorescent beads embedded in an agarose hydrogel for ${n_1} \lt {n_2}$, ${n_1} = {n_2}$, and ${n_1} \gt {n_2}$, where the ${n_1} = {n_2}$ case serves as a ground truth for the re-scaling of the mismatched cases. To prepare the sample, we first coated a coverslip with fluorescent beads (190 nm, excitation 480 nm, emission 520 nm, Bangs Labs #FSDG002) by dropcasting from isopropyl alcohol (IPA). Next, an agarose gel was prepared (2w%) by mixing Milli-${ Q}$ water and agarose (Low EEO, Fisher Scientific) and heating to boil in a microwave. 25 µL of the same beads in IPA were added to 225 mL agarose gel while warm. The mixture was pipetted into a glass sample cell composed of a microscope slide to which two small pieces of microscope slide as spacers were glued (using Norland 81 optical adhesive) roughly 0.5 cm apart, on top of which a #1.5 coverslip (180 µm thickness) was glued. After filling the cell with the agarose gel containing the beads, the cell was sealed with optical adhesive. After each gluing step, the sample was cured by exposure to UV light (${\sim}350\;{\rm nm}$) for 90 s, and during the last curing step the beads were protected using a piece of aluminum foil.

Confocal ${ z}$-stacks of the same volume were repeatedly recorded with different objectives using a Nikon C2-SHS C2si confocal on a Nikon Eclipse Ti inverted microscope with a 488 nm excitation laser. If the microscope objective had a correction collar, we optimized its position to minimize spherical aberrations while inspecting beads on the coverslip before the recording of the ${ z}$-stack. We used the following procedure to record the ${ z}$-stacks (with a 500 nm ${ z}$ step size). On the back of the sample (i.e., the side of the microscope slide) a spot was drawn using a felt marker. We then imaged this spot in bright field using a $10 \times /0.4\;{\rm NA}$ air objective and chose a reference feature in this spot to find back the same position in the sample. Next, we switched to the $100 \times /1.4\;{\rm NA}$ oil objective, and recorded a confocal ${ z}$-stack including the coverslip interface. We then removed the immersion liquid from the sample, and found the same position again using the $10 \times /0.4$ air objective. At this position, we again recorded a ${ z}$-stack, but now with the $100 \times /0.85\;{\rm NA}$ air objective. Finally, we recorded at the same position a (ground truth) $z$-stack using a $40 \times /1.25\;{\rm NA}$ water objective. For single bead comparison, we cropped volumes in the three stacks comprising the same beads in the ground truth (water immersion) stack and the axially deformed stacks (air, oil immersion). We determined the coverslip-to-sample interface as the intensity peak of the beads deposited on the coverslip and cropped the ${ z}$-stack above the coverslip.

Re-scaling of the data was done either using the linear theories [4,13,15,16] where the voxel size in $z$ was simply corrected, or using the depth-dependent re-scaling. For the latter, the AFP of each slide in the stack was calculated using Eq. (17), with the Lyakin scaling factor as the critical value instead of Eq. (15). Next, a linear ${z}$-stack was generated with a $z$ step size corresponding to the original stack and a range corresponding to the final AFP. The intensities in this ${ z}$-stack were interpolated (using inverse distance weighting) from the two nearest slices in the (non-linear) AFP ${z}$-stack. A Jupyter Notebook to perform the linear and non-linear re-scaling is available at [32].

D. Refractive Index Measurements

The refractive indices of the immersion oils used in this study were measured using an Abbe refractometer (Atago 3T). The indices were measured at $T = {25^ \circ}{\rm C}$ and at a wavelength ${\lambda _D} = 589.3\;{\rm nm} $. Using a measured dispersion value, the indices were converted using Cauchy’s relation to $\lambda = 520\;{\rm nm} $ [33]. This resulted in the following refractive indices: $n_{520}^{25} = 1.5197$ (type DF immersion oil, Cargille) and $n_{520}^{25} = 1.3421$ (Immersol W 2010 immersion oil, Zeiss).

4. RESULTS

A. Validation through Wave-Optics Calculations

We first validate the derived analytical expression by comparing it to wave-optics calculations, performed as described in Section 3.A. This is shown in Figs. 5(a) and 5(b), respectively, for RIM contrasts of ${n_1} = 1.0 \to {n_2} = 1.33$ and ${n_1} = 1.52 \to {n_2} = 1.33$. The axial re-scaling factor $\zeta$ is plotted versus depth, as obtained through analytical expression Eq. (3) (colored solid) and as calculated using wave-optics (colored dashed). For ${n_1} \lt {n_2}$ [Fig. 5(a)] the re-scaling factor increases at shallow depths, where the re-scaling (constant) maximum is determined by the numerical aperture of the optical system (i.e., the ${z_A}$-independent value of ${\zeta _{{\rm crit}}}$). At increasing depths it levels towards the ${n_2}/{n_1}$ limit, following the universal (i.e., NA-independent) curve ${\zeta _{{\rm univ}}}$. The wave-optics calculations deviate from the analytical expression due to some oscillations remaining after data processing of the computed 2D PSFs. Despite this, they follow the general trend in the NA-independent regime [${\zeta _{{\rm univ}}}$ in Eq. (3)] for the different NAs plotted. At shallow depths, the plateau from the analytical expression [${\zeta _{{\rm crit}}}$ in Eq. (3)] is reproduced by the wave-optics calculations, though the exact re-scaling factor value fits between the analytical expression and the re-scaling factor as calculated by Lyakin et al. [16].

 

Fig. 6. Axial re-scaling factor $\zeta$ versus AFP for a NA of 0.85, imaging under a RIM contrast of ${n_1} = 1.0 \to {n_2} = 1.34$. Measurement data (solid blue dots) are plotted alongside the wave-optics calculations (dashed blue lines), analytical solution [solid black lines, Eq. (3)], and depth-independent theories (solid lines). Two sets of measurement data are plotted individually and from these data the mean (solid black dots) is computed by binning along ${z_A}$, plotted in the center of each bin (bin sizes of 1 and 10 µm, respectively, for 0 to 10 µm and 10 to 100 µm). The measurement error (shaded blue area) is the sum of the measurement standard deviation and an estimated upper error limit.

Download Full Size | PDF

In the case of ${n_1} \gt {n_2}$ [Fig. 5(b)] the re-scaling factor as given by Eq. (3) decreases at shallow depths, also reaching a plateau value determined by the numerical aperture. For this RIM contrast the plateau is very small, yielding a fast changing re-scaling factor in the first 10 µm after the refractive index interface. Imaging deeper, the re-scaling factor again levels towards the ${n_2}/{n_1}$ limit. The wave-optics calculations show less extreme axial scaling at shallow depths, while we find good agreement for depths ${\gt}15\;\unicode{x00B5}{\rm m}$.

B. Validation through Experiments

Five different objective lenses are used to measure depth-dependent axial re-scaling factors as explained in Section 3.B, varying in numerical aperture, immersion refractive index ${n_1}$, and magnification. Immersol W 2010 immersion oil (Zeiss, $n_{520}^{25} =\def\LDeqbreak{} 1.3421$) and type DF immersion oil (Cargille, $n_{520}^{25} = 1.5197$) were used in experiments. We will first discuss two cases in detail below, ${n_1} = 1.0 \to {n_2} = 1.34$ for a NA of 0.85 and ${n_1} = 1.52 \to {n_2} = 1.34$ for a NA of 1.4. For the other cases listed in Table 1, figures are presented in Supplement 1 and will be discussed afterwards.

1. NA = 0.85, n₁ = 1.0, n₂ = 1.34

In Fig. 6, the axial re-scaling factor $\zeta$ is plotted against the actual depth ${z_A}$. The measurements (solid blue dots) are plotted alongside the wave-optics calculations (dashed blue lines), analytical solution (solid black lines), and literature theories (solid colored lines). From our individual sets of measurement data, the mean (solid black dots) is computed by binning along ${z_A}$, plotted in the center of each bin (bin sizes of 1 and 10 µm, respectively, for 0 to 10 µm and 10 to 100 µm). We estimate an upper error limit of $({100\;{\rm nm}/{z_N} + 100\;{\rm nm}/{z_A}})\zeta$, in determining the various $z$ positions, composed of errors from: (i) positioning of the piezo stages, (ii) determining the position of ball and coverslip by fluorescent bead localization, and (iii) fitting uncertainties in the landing curve. The total measured error for the re-scaling factor (shaded blue area) is the sum of the rolling standard deviation of the measurements and the upper error limit. It increases significantly at shallow depths, which is confirmed by acquiring data without RIM present, and applying the same data analysis as shown in Figs. S4 and S10.

Results are shown for a RIM contrast of ${n_1} = 1.0 \to {n_2} = 1.34$ in Fig. 6, where the uncertainty in the measurement data increases at shallow depths, while at larger depths the measurements capture the decay of the re-scaling factor but are slightly higher than the wave-optics calculations and analytical theory. Although we have taken care to align the optical system, we do expect some misalignment and/or residual optical aberrations to be the cause of this deviation. Especially, residual spherical aberration induced by a non-perfect correction collar setting could influence the measurements.

Looking at the exact value of the plateau at shallow depths, the analytical expression overestimates the re-scaling factor compared with the measurements and wave-optics calculations. In fact, both re-scaling factors from Lyakin and Stallinga give better agreement with the measured data [15,16] and in practical terms seem to provide a better value for ${\zeta _{{\rm crit}}}$. At extreme depths of ${z_A} = 10\;{\rm mm}$ the depth-dependent re-scaling factor approaches the ${n_2}/{n_1} = 1.34$ limit set by Carlsson et al. (not shown in Fig. 6) [13]. At more realistic imaging depths of 100 µm, the median from Diel coincides with the depth-dependent re-scaling factor and their mean gives a good agreement at ${z_A} = 40\;\unicode{x00B5}{\rm m}$.

2. NA = 1.4, n₁ = 1.52, n₂ = 1.34

In Fig. 7, the axial re-scaling factor $\zeta$ is plotted against the actual depth ${z_A}$. We find good agreement between measurement data and the analytical theory when imaging under a RIM contrast of ${n_1} = 1.52 \to {n_2} = 1.34$, as shown in Fig. 7. The measurements presented agree with the analytical theory and only start to drop more at shallow depths (${\lt}4\;\unicode{x00B5}{\rm m}$). The relative error when approaching ${z_A} = 0$ increases for ${z_A}$ and ${z_N}$ as the error in both is affected by determining the point at which the sapphire ball touches the coverslip. We have also included re-scaling factors as measured by Petrov et al. where they use a method with much smaller measurement errors (see also inset) [23]. Combining the measurement data from Petrov at shallow depths, and our own measurements data at larger depths, we find very good agreement with the analytical theory presented. Although the analytical expression provides a plateau for the re-scaling factor ${\zeta _{{\rm crit}}} = 0.64$, this is not reproduced in the measurement data from Petrov.

 

Fig. 7. Axial re-scaling factor $\zeta$ versus AFP for a NA of 1.4, imaging under a RIM contrast of ${n_1} = 1.52 \to {n_2} = 1.34$. Measurement data (solid blue dots) are plotted alongside the wave-optics calculations (dashed blue lines), analytical solution [solid black lines, Eq. (3)], and depth-independent theories (solid lines). Three sets of measurement data are plotted individually and from these data the mean (solid black dots) is computed by binning along ${z_A}$, plotted in the center of each bin (bin sizes of 1 and 10 µm, respectively, for 0 to 10 µm and 10 to 100 µm). The measurement error (shaded blue area) is the sum of the measurement standard deviation and an estimated upper error limit. Measured data and theory as presented by Petrov et al. and wave-optics calculations from Egner and Hell are included [19,23].

Download Full Size | PDF

Our wave-optics calculations also match the analytical theory at larger depths, but deviate below ${z_A} = 4\;\unicode{x00B5}{\rm m}$. At the same time, they do overlap with wave-optics calculations from Egner and Hell [19]. At these shallow depths, an offset of a few tens of nanometers in the NFP has a dramatic effect on the re-scaling factor and hence super-critical angle fluorescence (SAF) effects are significant [34]. Although SAF is not included in the analytical derivation, it is considered in the wave-optics calculations through the Fresnel transmission coefficients when using the vector PSF model. The axial deformation is, however, enhanced by SAF, as the NFP shifts $\lt 40\; {\rm nm}$ away from the objective [34]. Therefore, the difference in the re-scaling factor between the wave-optics calculations and the analytical theory has already been reduced by the incorporation of SAF in the calculations.

Up to depths of ${z_A} = 1\;\unicode{x00B5}{\rm m}$, there is a ${\sim}250\;{\rm nm}$ difference in the NFP found between $\zeta = 0.64$ (analytical theory) and $\zeta = 0.78$ (wave-optics calculations). This is too large to be caused by the chosen method of determining the focal position (i.e., maximum axial intensity [19] versus minimal lateral width [23]). As such, we do not have a fitting explanation for the discrepancy between the wave-optics calculations and the analytical theory along with measurements from Petrov et al.

3. Other Imaging Scenarios

The measurement data for the remaining cases listed in Table 1 are shown in Figs. S1 through S11, where the re-scaling factor $\zeta$ is plotted against ${z_A}$, alongside the wave-optics calculations, analytical solution, and depth-independent theories. For each objective lens used we acquired data without a refractive index mismatch present and applied the same data analysis procedures as for axially scaled measurements (Figs. S1, S4, S6, S9, S10). In all cases, the spread in the measurement data increases below ${z_A} = 10\;\unicode{x00B5}{\rm m}$ as uncertainties in determining both ${z_A}$ and ${z_N}$ increase. At higher depths, no reproducible deviations away from $\zeta = 1.0$ are found in these data, and we note a maximum error of about 2% in the measurements. The data shown in Fig. S9 are of special interest where we have used a water immersion objective to verify our measurement method and analysis when having immersion oil ($n = 1.34$) present between the coverslip and sapphire ball. We find a mean re-scaling factor of $0.99 \pm 0.05$ (standard deviation) over the entire depth range, and apart from the increased scatter near ${z_A} = 0$, the re-scaling factor remains approximately constant ($\zeta = 1.0$) as a function of depth for each individual dataset.

The objective lens with the lowest numerical aperture that we used in our measurements had a NA of 0.7, of which the results are presented in Figs. S2 and S3 (${n_1} = 1.0 \to {n_2} = 1.34$ and ${n_1} = 1.0 \to {n_2} = 1.52$). In both cases, the wave-optics calculations reproduce the analytical expression, while the measurements only reproduce the general trend for higher RIM contrast of ${n_1} = 1.0 \to {n_2} = 1.52$. In the case of ${n_1} = 1.0 \to {n_2} = 1.34$, the measurement uncertainties obscure a clear trend in the measurement data.

The RIM of ${n_1} = 1.0 \to {n_2} = 1.52$ for the 0.85 NA objective lens is shown in Fig. S5. Both the wave-optics calculations and measurement data reproduce the behavior of the re-scaling factor as predicted by the analytical expression. Comparing against the measurement data, Lyakin and Stallinga again provide the best critical value ${\zeta _{{\rm crit}}}$.

In the case of the 0.95 NA air objective (Figs. S7 and S8), again the wave-optics calculations reproduce the analytical expression over the entire depth range. For the measured data, however, the trend of the NA-independent regime ${\zeta _{{\rm universal}}}$ is reproduced, but the measurements fail to replicate the strong increase to the plateau given by ${\zeta _{{\rm crit}}}$ below ${z_A} = 20\;\unicode{x00B5}{\rm m}$. Along with depth-dependent re-scaling being more prominently present with increasing NA, so is the sensitivity to any remaining (i.e., spherical or tilt) aberrations in the optical setup. We have taken care to minimize these, but we estimate that residual aberrations lower the effective NA, obscuring the rise to the plateau in the measurements.

For the 1.4 NA oil immersion objective, we present the RIM case of ${n_1} = 1.52 \to {n_2} = 1.0$ in Fig. S11. Again the general depth-dependent behavior is reproduced in the measurements, but the re-scaling factor is underestimated at the coverslip while it is overestimated when imaging deeper into the sample.

C. Axial Re-scaling of 3D Microscopy Data

To test axial re-scaling of 3D microscopy data using our depth-dependent theory, we recorded confocal ${z}$-stacks of beads suspended in an agarose hydrogel, from the coverslip to 100 µm depth (${z_N}$). These ${ z}$-stacks were recorded using the $100 \times /0.85\;{\rm NA}$ air, $40 \times /1.25\;{\rm NA}$ water, and $100 \times /1.4$ oil objectives (see Table 1). As the refractive index of the agarose gel (${n_2} = 1.3356$ [35]) is very close to the refractive index of the immersion medium of the water objective, we can use these ${ z}$-stacks as a ground truth for the axial re-scaling of the data. We recorded the ${ z}$-stacks at the exact same location in the sample to allow for direct comparison of the same beads in different imaging scenarios.

1. n₁ < n₂

The case of ${n_1} \lt {n_2}$ is particularly relevant for cryo-fluorescence microscopy where air immersion is used to observe a frozen or vitrified sample. Thus, we evaluate a typical situation with ${\rm NA}= 0.85$, ${n_1} = 1.0$, and ${n_2} = 1.336$ [9]. Before re-scaling the data, we quantitatively compare the re-scaling factors of the depth-dependent and linear theories, as plotted in Fig. S12. As we have found that the theories of Lyakin and Stallinga result in approximately equal re-scaling factors, we choose to plot only the Lyakin re-scaling factor. As the critical value of the depth-dependent re-scaling factor, we use the Lyakin re-scaling factor. Therefore, the depth-dependent re-scaling factor is 1.532 close to the coverslip and falls off to ${\sim}1.4$ deeper into the sample (see Fig. S12a). When plotting the focal shift $\Delta f$ as a function of depth (AFP) we see that depth-dependent focal shift equals the Lyakin focal shift close to the coverslip and gradually transitions to the Diel (median) focal shift at larger depth (see Fig. S12b).

 

Fig. 8. Axial re-scaling of 3D microscopy data with ${\rm NA}= 0.85$, ${n_1} = 1.0$, ${n_2} = 1.336$ using the depth-dependent and linear theories. Maximum intensity projections along $Y$ of the (re-scaled) confocal ${ z}$-stack of beads embedded in an agarose hydrogel. The stacks have been re-scaled using the depth-dependent (left), Diel (mean) (middle), and Diel (median) (right) re-scaling factors. Overlays are plotted of the re-scaled ${ z}$-stacks (grays) and the ground truth ${ z}$-stack recorded with the water immersion objective (reds). The two bottom rows are cut-outs of the upper row, where the cut-outs at larger depths appear noisier due to the intensity re-scaling due to fluorescence intensity loss at large depths. The beads are slightly displaced in $X$ due to imperfections in the manual overlay of the ${ z}$-stacks.

Download Full Size | PDF

To see how significant the difference between the theories is for this imaging scenario, we plot the absolute and relative difference between the depth-dependent and linear theories as a function of depth (AFP). Figure S12c shows that for Lyakin the difference is obviously zero near the coverslip, but quickly increases to a large difference (${\gt}{1}\;{\unicode{x00B5}{\rm m}}$) after a depth of ${\gt}{25}\;{\unicode{x00B5}{\rm m}}$. On the other hand, Diel (median) differs significantly (${\sim}1\;\unicode{x00B5}{\rm m}$) at a depth of 20 µm, but is the linear theory with the smallest relative difference at ${\gt}{50}\;{\unicode{x00B5}{\rm m}}$ depth. Finally, Diel (mean) is in between the other two linear theories, having a smaller difference close to the coverslip than Diel (median), and a smaller difference at large depth than Lyakin. Where the relative difference between the depth-dependent theory and the theories of Lyakin and Diel median can exceed 5%, the relative difference for the Diel mean theory is below 3% (see Fig. S12d).

Figure 8 shows the re-scaling of the 3D microscopy data of beads in an agarose hydrogel with ${\rm NA}= 0.85$, ${n_1} = 1.0$, and ${n_2} = 1.336$. Overlays are plotted of the re-scaled ${z}$-stacks (grays) and the ground truth ${ z}$-stack recorded with the water immersion objective (reds). As ${n_1} \lt {n_2}$, the recorded ${ z}$-stack was stretched in the axial direction to return to the ground truth axial distances.

Deep into the sample (${z_A} \approx 90\;\unicode{x00B5}{\rm m}$), Diel (mean) shows the largest re-scaling error where the ${ z}$-stack is overstretched, whereas the depth-dependent and Diel (median) re-scaling collapses onto the ground truth data, although the PSF is elongated in the axial direction due to spherical aberrations induced by the refractive index mismatch. Closer to the coverslip (${z_A} \approx 20\;\unicode{x00B5}{\rm m}$), both the depth-dependent and Diel (mean) re-scaling overlap with the ground truth ${ z}$-stack, whereas Diel (median) has a slight (absolute) error as the axial distances have not been stretched enough. This agrees with the re-scaling factors plotted in Fig. S12.

2. n₁ > n₂

The case of ${n_1} \gt {n_2}$ is relevant for oil immersion observation of samples in water. Thus, we here evaluate a typical situation with ${\rm NA}= 1.4$, ${n_1} = 1.52$, and ${n_2} = 1.336$. We quantitatively compare the re-scaling factors of the depth-dependent and linear theories in Fig. S13. As ${\rm NA}{\gt}{n_2}$, we cannot use Diel (mean). The depth-dependent re-scaling factor equals the Lyakin re-scaling factor close to the coverslip, but surpasses the Diel (median) at ${z_A} \approx 30\;\unicode{x00B5}{\rm m}$. The same trend is seen in the focal shift $\Delta f$. In this imaging scenario, the axial positional difference between Lyakin and the depth-dependent re-scaling theory increases after a few micrometers in depth, resulting in a significant difference ${\gt}{6}\%$. While the absolute difference between Diel (median) and the depth-dependent theory is small close to the coverslip (${\lt}1\;\unicode{x00B5}{\rm m}$) and ${\sim}2\;\unicode{x00B5}{\rm m}$ um at large depths, the relative error is large (${\sim}20\%$) close to the coverslip and small at large depths (${\sim}2\%$).

 

Fig. 9. Axial re-scaling of 3D microscopy data with ${\rm NA}= 1.4$, ${n_1} = 1.52$, ${n_2} = 1.336$. Maximum intensity projections along $Y$ of the (re-scaled) confocal ${ z}$-stack of beads embedded in an agarose hydrogel. The stacks have been re-scaled using the depth-dependent (left) and Diel (median) (right) re-scaling factors. Overlays are plotted of the re-scaled ${ z}$-stacks (grays) and the ground truth ${ z}$-stack recorded with the water immersion objective (reds). The two bottom rows are cut-outs of the upper row. The beads are slightly displaced in $X$ due to imperfections in the manual overlay of the ${ z}$-stacks.

Download Full Size | PDF

 

Fig. 10. Relative error of the re-scaled $z$ position compared to the ground truth ${z_A}$ as recorded using a water immersion objective. The relative error is binned along ${z_A}$ in bins of 10 µm and the mean of each bin is plotted in the center of each bin. The standard deviation is plotted as an error bar. The dashed gray line indicates zero and acts as a guide to the eye. (a) Results for ${\rm NA}= 0.85$, ${n_1} = 1.0$, ${n_2} = 1.336$, where we measure an error in the axial re-scaling using the depth-dependent theory below 2%, whereas the linear theories result in errors close to 5%. (b) Results for ${\rm NA}= 1.4$, ${n_1} = 1.52$, ${n_2} = 1.336$, where we find that the maximum error in the axial re-scaling using the depth-dependent and Diel (median) linear theory is 5%.

Download Full Size | PDF

In Fig. 9 the re-scaling of the 3D microscopy data is plotted. As ${n_1} \gt {n_2}$, the recorded ${z}$-stack was compressed in the axial direction to return to the ground truth axial distances. At larger depth (${z_A} \approx 65\;\unicode{x00B5}{\rm m}$), the depth-dependent re-scaling nicely coincides with the ground truth, whereas the axial distances in the Diel (median) re-scaling have been compressed too much (see second row). Closer to the coverslip (${z_A} \approx 15\;\unicode{x00B5}{\rm m}$) the depth-dependent re-scaling again nicely coincides with the ground truth, but the Diel (median) re-scaling has been compressed too little.

3. Quantitative Comparison

To quantify how well the acquired data can be corrected using the depth-dependent re-scaling factor, we localize all individual fluorescent beads present in the data recorded using the PSF-Extractor software [30]. We compare the $z$ position found in the re-scaled data to the ground truth value ${z_A}$ and divide this difference over ${z_A}$. This is shown in Fig. 10 for both imaging scenarios, where we bin the data (bin size 10 µm), plot the mean of each bin along with the standard deviation as the error bar, and plot the data centered with respect to the bin. In Fig. 10(a) (${\rm NA}= 0.85$, ${n_1} = 1.0$, ${n_2} = 1.336$), we omit the first bin (0 to 10 µm) as it only contains three data points. The depth-dependent theory outperforms both Diel theories in the axial re-scaling of this data, where we note a maximum error of 2% when using depth-dependent axial re-scaling, whereas the error for the linear theories approaches 5%.

The same analysis and data processing are done for ${\rm NA}= 1.4$, ${n_1} = 1.52$, ${n_2} = 1.336$, to quantify the error in re-scaling using the Diel (median) and depth-dependent theory; see Fig. 10(b). There is not one theory that outperforms the other over the depths measured here. Still, the depth-dependent theory results in a smaller error closer to the coverslip, as is to be expected following Fig. 7. In addition, this measurement did not include beads near the cover glass (${z_A} \lt 20\;\unicode{x00B5}{\rm m}$) where a relative difference up to 20% between the two theories is expected (see Fig. S13).

We should note that the objective lens used in this experiment did not have a correction collar. This means that we could not completely get rid of spherical aberration (SA) when imaging close to the coverslip. The presence of SA near the coverslip contradicts the optical conditions in the derivation of the analytical theory and therefore affects the legitimacy of the depth-dependent re-scaling factor, resulting in a higher re-scaling error. This shows that proper re-scaling also requires an optimization of the correction collar before the acquisition of the 3D microscopy data.

D. Software for Depth-Dependent Re-scaling

1. Re-scaling Acquired $z$-stacks Using Python

To re-scale the ${ z}$-stack acquired under RIM, we have written Python software, which is available at https://github.com/hoogenboom-group/SF [32]. Jupyter Notebooks are used to read image data and, based on the specific imaging conditions, use inverse distance weighting to rescale the intensities along the axial coordinate correctly. The same axial pixel size from the original data is used, but depending on the RIM mismatch, the re-scaled ${ z}$-stack will map a shorter or larger axial range.

2. Interactive Online Tool for Plotting the Re-scaling Factor versus Imaging Depth

We have made an online interactive tool where the depth-dependent re-scaling factor can be plotted and compared to existing depth-independent scaling theories using Plotly Dash [36], the source code of which is available at [32]. The code is accessible via the URL https://axialscaling.pythonanywhere.com/. The refractive indices ${n_1}$ and ${n_2}$, the NA, and the wavelength $\lambda$ can be varied. In addition, the focal shift $\Delta {f}$ can be plotted for the depth-dependent re-scaling factor, as well as for two depth-independent scaling theories. Both the re-scaling factors and the focal shift resulting from the depth-dependent theory can be exported to file. All the measurement data and wave optics calculation results found in this manuscript and the accompanying Supplement 1 are also included in the interactive plot.

5. DISCUSSION

The various re-scaling theories found in the literature can be understood as a result of different (overt or covert) assumptions on which maximum constructive interference contribution is leading. For example, Lyakin et al. used an analysis similar to ours, but explicitly sets the critical point ${\theta ^*}$ such that $k_z^\prime ({\theta ^*}) =\def\LDeqbreak{} \frac{1}{2} \times (k_z^\prime (0) + k_z^\prime ({\theta _{{\max}}}))$, and then from Eq. (8) follows [16]

The same value was proposed earlier by Stallinga while posing different criteria of minimal variation of the phase function: ${\theta ^*}:\min \langle {{{(\Phi - \Phi ({\theta ^*}))}^2}} \rangle$ [15]. For Diel et al. the ${\zeta _{{\rm median}}}$ corresponds to setting ${\theta ^*}:{k_r}({\theta ^*}) = 1/2{k_r}({\theta _{{\max}}}) = \frac{1}{2}{k_0} \times {\rm NA}$ [4], such that

These estimates for $\zeta$ are geometrical in nature, where the actual wavelength is not taken into account–with the radii of the Ewalds’ spheres being cancelled out of the fraction in Eq. (8). This results in an implicit omission of the interference nature of the PSF under RIM. Somewhat less obviously, the criterion of Stallinga is also geometrical [15]. While it does demand minimal variation in the value of $\Phi$ in order to achieve the highest possible constructive interference, it does not take into account that the well minimized phase function can become several $2\pi$ in range (when both ${z_A}$ and ${z_N}$ are large enough) and thus lead to destructive interference anyway.

In order to be able to correct for depth-dependent axial scaling in experiments where imaging under RIM is unavoidable, it is crucial that (i) the position of the coverslip (or more general, the RIM interface) is determined precisely and (ii) spherical aberration at the interface is reduced to an absolute minimum by selecting the cover glass the objective is optimized for, or by optimizing the position of the correction collar. In practical terms, this would require that the objective lens used is fitted with a correction collar, which is often the case for high-NA objective lenses.

One such case would be cryo-fluorescence microscopy used in the cryo-electron tomography workflow, where a RIM is generally present due to the large temperature difference between the optical objective and the sample. With fluorescence microscopy, targets can be identified in frozen hydrated cells and consequently sufficiently thin sections can be prepared by ablating the excess cellular material surrounding the target with a focused ion beam [37–39]. The aimed thickness of this frozen section is approximately 100 to 200 nm, which makes it crucial to precisely determine the target position with respect to the RIM interface (outer cell surface) [40]. If axially scaled distances measured with the light microscope are not corrected, targeting errors in the range of 300 to 1200 nm will occur with cell thicknesses ranging up to micrometers. Moreover, the depth dependence of these errors will be important when fabricating sections out of thicker ice while using fluorescence microscopy to find targets in, for instance, organoids, as can be done in high-pressure frozen samples [41,42].

With the provided software [32], one can easily plot the depth-dependent re-scaling factor for the relevant imaging scenario, but also re-scale their 3D microscopy data, without having to judge which linear theory will hold in this scenario or perform full wave-optics calculations. We should note, however, that although the data will be axially re-scaled, our software does not provide any correction of the shape of the point spread function (PSF) due to spherical aberrations. To correct for this, one would require deconvolution with a depth-dependent PSF as demonstrated in [43].

Our measurements are limited by the relatively high uncertainty in determining the re-scaling factor near ${z_A} = 0$. It would therefore be interesting, for future research, to use the method from Petrov et al. to measure the re-scaling factor for the scenario with a 0.85 NA air objective when imaging ${n_1} = 1.0 \to {n_2} = 1.33$.

6. CONCLUSION

We have presented an analytical theory to correct for the depth-dependent axial deformation when imaging with light microscopy in the presence of a refractive index mismatch between the sample and the microscope objective immersion medium. Using the resulting equation, a re-scaling factor as a function of depth can be calculated from imaging parameters’ numerical aperture (NA), the refractive indices of the objective (${n_1}$) and sample (${n_2}$), and the wavelength ($\lambda$), for a RIM with both ${n_1} \lt {n_2}$ and ${n_1} \gt {n_2}$. We performed wave-optics calculations to verify the theory and find a very good agreement between the two. In addition, we performed experiments to measure the axial scaling, where we find a good agreement at larger depths, whereas closer to the coverslip, the measurements suffer from large uncertainties. We do find good agreement with the accurate measurements done by Petrov et al. [23] in the case when imaging with a high-NA oil immersion objective into a water sample.

Next, we tested the depth-dependent theory versus existing linear theories in the literature on 3D microscopy data with a known ground truth. We find that for ${\rm NA} = 0.85$, ${n_1} = 1$, ${n_2} = 1.33$ the depth-dependent theory outperforms existing linear theories for depths up to 80 µm with a maximum relative error of 2%. For ${\rm NA} = 1.4$, ${n_1} = 1.52$, ${n_2} = 1.33$, the depth-dependent theory performs as good as the best linear theory in literature. However, we think its performance was compromised by the presence of a small spherical aberration near the coverslip, which could not be corrected for as the objective did not have a correction collar. Moreover, we could not compare close to the cover glass (${\lt}10\;\unicode{x00B5}{\rm m}$), where the linear theory used in the comparison was expected to break down.

Finally, we have presented a web applet to be used by microscope users to calculate the re-scaling factor for their imaging parameters. In addition, we have shared software to re-scale 3D data sets using the depth-dependent scaling factor.

Our re-scaling theory is the first to include the depth dependence of axial scaling due to a refractive index mismatch. It will be of use in imaging scenarios where the refractive indices of the sample and objective cannot be matched, such as in integrated, cryogenic, and correlative light and electron microscopy setups, or in the imaging of water-like samples using high-NA oil immersion objectives.