1. INTRODUCTION: PRINCIPLES OF POLARIZATION MICROSCOPY

Optical microscopy is an area of research whose steady growth has accelerated over the last few decades due to several factors: technological advances in optics, optoelectronics, laser sources, and detectors; new labeling strategies in biology; nanoscience fabrication tools in physics; and remarkable advances in computational imaging. Today optical microscopy has matured into a field in which algorithms, computing technology, and data science are of an importance comparable to that of optical design and sample optimization.

A key element in the search for improvement in optical imaging is the polarization of light. Polarization not only provides a way to obtain information on how matter is structured and organized at the molecular level but is also a degree of freedom that, along with amplitude and phase, plays a central role in structuring light. Importantly, far-field microscopy offers situations in which unconventional properties of light can be manipulated: high numerical aperture conditions and optical interfaces involve non-paraxial geometries in which longitudinal field components play an important role. It is, therefore, necessary not to rely only on the standard paraxial formalism for polarization but to consider instead a complete 3D description [1,2]. Polarized optical microscopy is a field in which fundamental and applied physics, computational sciences, and their applications in nanoscience, material science, biology, and medical sciences are still evolving and are in need of new concepts and technological advances.

An important application of polarized optics is the imaging of organization in molecular environments at the nanometric scale. Molecular interactions, which are at the core of biological phenomena and pathologies, lead to complex and heterogenous organization that is challenging to decipher, particularly in living samples. Accessing orientational organization at the molecular level is crucial for understanding key processes such as protein clustering in immunology responses, cytoskeleton filament reorganization under cell and tissue mechanical perturbations, protein aggregation in neurodegenerative diseases, and DNA organization in repair processes within the nucleus. Orientation is also a key element in biological functions at the single-molecule level. Indeed, conformational changes in proteins involve a strong reorganization of their domains at the scale of a single protein.

 

Fig. 1. (a) Principle of polarization microscopy. A dipole aligned with the unit vector $\vec\mu$ is generated at the coordinates $(x,y,{z_0})$ from the excitation of a molecule or particle by an incoming excitation field. The longitudinal distance ${z_0}$ is measured from the flat interface between the medium with index ${n_0}$ that embeds the dipole and another medium with index ${n_1}$. The radiation field emitted by the dipole is captured by a high numerical aperture objective lens that collimates it. A diaphragm is placed at the back focal plane (or pupil plane) to determine the numerical aperture of the system. A lens (or combination of lenses) is then used to image the dipole. The image is the result of a coherent (e.g., nonlinear optical process) or incoherent (e.g., fluorescence) summation of fields coming from all orientations explored by one dipole (e.g., single molecule imaging) or many dipoles (e.g., ensemble imaging) present within the spatial resolution of the objective and averaged in time. The dotted line in the inset at the top right represents the plane at the object space that is conjugate to the image plane, and that is at a distance ${z_1}$ from the interface. (b) First row, dipole directions aligned with the axes. Second row, the corresponding intensity and polarization distributions at the pupil plane, where the gray circle corresponds to $u = {n_0}$, which separates the homogeneous and evanescent components emitted by the dipole. The polarization inside the circle is always linear, and its direction is indicated by the green lines. Outside of the circle (SAF), the polarization is typically elliptical (with handedness represented in red or blue). Third row, resulting PSFs for the case ${\rm NA} = {n_0}$ for the corresponding three dipole directions. The scale bar represents the distance that corresponds to a wavelength in the object space.

Download Full Size | PDF

In this review, we aim to describe several ways in which light polarization, in particular in the non-paraxial regime, has led to important progress in imaging molecular organization. In optical microscopy, the signal depends on optical polarization via a photo-selection vectorial process (excitation) and a radiation process (emission). We begin by providing an overview of the fundamentals of polarized light–matter interaction leading to such processes, both at the single-molecule and ensemble levels (Section 2). We then review advances on the two main categories of molecular organization imaging approaches: the first, mostly developed at the ensemble level, uses control of the excitation polarization of light in both fluorescence and nonlinear optical interactions (Section 3); the second, dedicated to the single-molecule level, uses control of the polarization of the fluorescence light radiated by emitters (Section 4). We focus particularly on advances in single-molecule orientation and localization microscopy (SMOLM), which is a rapidly growing field (Section 5). Section 3 and Sections 4 and 5 can be read independently. Section 6 then summarizes the properties of both excitation and emission polarized approaches, emphasizing their advantages and limitations. Finally, we discuss future challenges in the field, highlighting the possible benefits of manipulating light polarization in 3D (Section 7). Note that, while the methods addressed in this review concentrate on inverse microscopy imaging in which the molecular radiation is collected by the same objective as the illumination, transmission modalities have also been developed, some of which are mentioned in Section 3.

2. THEORETICAL BACKGROUND

The sensitivity to polarization of light–matter interaction in molecular media is due to the orientation of molecular transition dipole moments between the ground and excited states. In an incoherent process such as fluorescence, the involved molecular dipoles originate first from one-photon or multiphoton transitions in the absorption process, followed by the radiation of a de-excitation dipole in the emission process. Coherent optical processes occur instead through the radiation of induced molecular dipoles, which depend linearly (in linear scattering) or nonlinearly (in nonlinear scattering) on the incident field. In this section, we begin by giving a simplified description of the radiation of dipoles originating from molecular emitters in a microscope. We then specify the mechanisms used to excite them in both incoherent fluorescence and coherent nonlinear optical regimes.

A. Mathematical Model for the Radiation of a Dipole through a Microscope

Consider a radiating dipole with transverse coordinates ${\textbf x} = (x,y)$, whose direction is aligned with the unit vector $\vec\mu$. This dipole is embedded in a medium with refractive index ${n_0}$ at a distance ${z_0}$ from the interface of this medium with a second medium with refractive index ${n_1}$, e.g., the coverslip supporting the sample, as shown in Fig. 1(a). (Note that in this article we use boldface characters for vectors with components only in the $x$ and $y$ directions, and characters with arrows on top for higher dimensional vectors.) As discussed later, the dipole is linked either to the direction of a fluorescent molecule or induced directly by the excitation/illumination (through a linear or nonlinear process). In both cases, the excitation of the dipole is through epi-illumination, as shown in the figure. Part of the dipole’s radiation is captured by the microscope objective.

 

Fig. 2. (a) Schematic representation of the dipole of a fluorophore attached to a protein and definition of the angles used in this work. A single molecule or an ensemble of molecules, averaged in time and space, explore a distribution $f(\Omega)$ with $\Omega = (\theta ,\phi)$ being the local orientation of the instantaneous molecular dipole in the distribution frame. The distribution, represented here as a cone, is centered at the mean direction $(\eta ,\xi)$, and its aperture is quantified by the cone angle $\delta$. (b) Illustration of polarization microscopy results, showing the fluorescence intensity from emitters (here phalloidin conjugates AlexaFluor488) attached to F-actin filaments forming stress fibers in a fixed cell. (c) Ensemble molecular organization imaging, the color code representing the degree of orientation of many dipoles within the focal volume at each pixel of the image. The observed order is the result of the alignment of actin filaments, convolved with single molecule orientational fluctuations that remain despite the rigidity of the phalloidin linker to actin. (d) Single-molecule orientation and localization microscopy (SMOLM) allows the discrimination of single molecules wobbling (represented here by the color code of sticks) from their orientation (the sticks’ orientation). Each stick represents a single molecule whose orientation information is added on top of its localization (adapted from [4]). (e) Schematic representation of the decomposition of the 2D projection of a molecular angular distribution into increasing symmetry orders that can be detected by different optical contrasts as described in the text.

Download Full Size | PDF

It is convenient to make the microscope telecentric, which means that the back focal plane of the objective is chosen as the pupil plane of the system, namely the location for placing an aperture stop that determines the numerical aperture (NA), because this plane is Fourier-conjugate to the object and, hence, the image. At the back focal plane, the field is essentially paraxial and is denoted as $\tilde {\textbf E}({\textbf u})$, where ${\textbf u} = (u\cos \varphi ,u\sin \varphi)$ is a dimensionless pupil coordinate normalized so that its maximum value equals the NA of the system. If the microscope objective satisfies the Abbe sine condition and has negligible amounts of aberrations, the field at the back focal plane can be calculated through the Richards–Wolf model [3],

An image is formed by performing an optical Fourier transform with a lens whose focal distance $f$ contributes to the magnification $m$. The field at a detector coordinate ${\boldsymbol \rho}$ is then given by the Fourier transform,

In polarized microscopy, a single dipole $\vec\mu$ can be either the emission dipole from a fluorescence process (one-photon fluorescence 1PF, two photon fluorescence 2PF) or an induced nonlinear dipole from a coherent nonlinear process (e.g., second harmonic generation). In both cases, this dipole radiates a field expressed at the pupil plane by Eq. (1). Optical contrasts are based on the measurement of the light radiated by molecular dipoles averaged in space (over the focal volume of the microscope objective) and in time (over the integration time of the detector). Let us parametrize the molecule’s dipole direction as $\vec\mu(\Omega) = (\sin \theta \cos \phi ,\sin \theta \sin \phi ,\cos \theta)$, with $\Omega = (\theta ,\phi)$ representing the orientation of the dipole in the frame of its angular distribution $f(\Omega)$ [Fig. 2(a)]. This distribution characterizes the explored orientations of either a single dipole or many of them, averaged in time and space, with a mean orientation lying along the direction angles denoted as $(\xi ,\eta)$ [Fig. 2(a)], and an angular width denoted as $\delta$ in what follows. The goal of polarized microscopy is to access both the mean direction and width/shape of this distribution.

B. Polarized Microscopy in Fluorescence

In this section, we describe the case of fluorescence, for which polarization-resolved modalities have led to a large volume of work both at the ensemble and single-molecule levels. In fluorescence imaging, fluorescent molecular labels are used as reporters of the proteins or molecules they are attached to. Figures 2(b)–2(d) illustrate polarized fluorescence imaging results obtained on a fixed cell whose F-actin filaments are labeled with a phalloidin conjugate fluorescent molecule, which is known to lie along the filaments. Polarized ensemble imaging gives an image of the molecular order, i.e., the width of $f(\Omega)$ from many molecules at each pixel location [Fig. 2(c)]. In contrast, polarized single-molecule imaging provides information on the orientation and extent of the wobbling of each molecule over time [Fig. 2(d)]. In most of the cases considered, the distribution $f(\Omega)$ is assumed to be cone-shaped with cone aperture angle $\delta$ [5,6]. Other distributions have been considered depending on the molecular labeling being used, including Gaussian-shaped [7] or “disk-shaped” distributions [8,9]. It has been shown, however, that only a limited amount of information [e.g., symmetry orders as represented in Fig. 2(e)] is accessible in polarized microscopy due to the intrinsic order of light–matter polarized interactions [10]. More details will be provided in Section 3.

Fluorescence is the result of a two-step process: absorption, which occurs with a probability density ${{\cal P}_{{\rm abs}}}({\Omega _{\rm a}})$ when the molecule is at an orientation ${\Omega _{\rm a}}$, and emission, which corresponds to the radiation efficiency of the emission dipole when the molecule is at an orientation $\Omega$ (which can be different from ${\Omega _{\rm a}}$, e.g., due to fast rotation) [10]. The emission efficiency depends on the fluorescence quantum yield of the molecule and the collection efficiency of the field radiated by its dipole. Since fluorescence is an optically incoherent process, the total intensity from a collection of molecules illuminated in a focal volume is the sum of the intensities radiated by their individual dipoles due to the absence of phase correlations between their respective emitted fields.

To express the fluorescence intensity as a sequential process between absorption and emission, one must account for possible orientation decorrelation between these two steps. At the scale of a fixed single molecule, these steps occur at the same orientation ${\Omega _{\rm a}}$, and the radiated fluorescence intensity distribution at the pupil plane is expressed as

For molecules that possibly rotate in time, the fluorescence emission intensity is the probability of emitting a photon from the dipole orientation $\Omega$ at time $t$ while the orientation was ${\Omega _{\rm a}}$ at time $t = 0$, averaged over time during the long integration time of the detector. This change of orientation can occur within a time window of duration ${\tau _f}$, the fluorescence lifetime, which is typically of several nanoseconds. This process depends on $g(\Omega ,{\Omega _{\rm a}},t)$, the conditional probability density function to find a dipole along $\Omega$ after rotation from its initial position ${\Omega _{\rm a}}$. The distribution $g(\Omega ,{\Omega _{\rm a}},t)$ is a solution to the rotational diffusion equation that has been described in detail within the context of constrained fluorophores [6,11]. The resulting time-averaged intensity is

These two situations, therefore, accept a similar description, keeping in mind that the result from an ensemble measurement is a combination of single-molecule wobbling and time-averaged organization of their ensemble. In both the ensemble and single-molecule situations, it appears that different approaches can be developed in order to decipher the way in which the molecules are organized:

The retrieval of information about $f(\Omega)$ from Eq. (6) can be better visualized when considering two extreme cases: slow rotation (e.g., compared to the fluorescence lifetime), where the molecules do not change orientation between the absorption and emission steps, and fast rotation, where their rotational diffusion time is much shorter than the fluorescence lifetime. Any intermediate scenario can be considered as a linear combination of these two limits [12].

In the slow rotation regime, the direction of the emission dipole of each single molecule is the same as that for the absorption dipole since the rotational diffusion of the molecule (with the characteristic time ${\tau _r}$) is much longer than its fluorescence lifetime ${\tau _f}$ (${\tau _r} \gg {\tau _f}$). The fluorescence intensity averaged over the integration time of the detector ($\tau \gg {\tau _r} \gg {\tau _f}$) can then be written as

In the fast rotation regime, on the other hand, the rotational diffusion is much faster than the fluorescence process (${\tau _r} \ll {\tau _f}$), so the emission and absorption dipole directions are totally decorrelated in time, within the limits of the angular constraint of the molecular diffusion (e.g., their orientational distribution in the considered sample). This leads to a form of the fluorescence intensity in which the absorption probability and radiated fields are averaged separately. The fluorescence intensity can be written as

C. Polarized Microscopy in Coherent Nonlinear Optics

Second harmonic generation (SHG) and higher-order nonlinear processes such as third harmonic generation (THG), four wave mixing (FWM), and coherent anti-Stokes Raman scattering (CARS) are coherent processes in which $\vec\mu$ is a nonlinearly induced dipole. The excitation process of these optical contrasts, therefore, implies a very different readout of the orientational properties of the probed molecules: not only is the emission coherent but also there is no dependence on the rotation timescale of the molecules since the emission occurs instantaneously after excitation.

Nonlinear coherent processes have been used extensively to study biological molecules in a label-free manner, benefiting from their intrinsic nonlinear optical responses. SHG is a second-order structural contrast sensitive to non-centro-symmetric molecular organizations and has been applied to the measurement of membrane potentials thanks to SHG-active molecular lipid labels [17] or to the imaging of endogenous structural proteins such as collagen type I [18], acto-myosin, and tubulin in cells and tissues. THG is a third-order interface-sensitive contrast applied to dense structure imaging such as in lipid bodies [19]. CARS results from a third-order nonlinear interaction in resonance with molecular vibrations. It has been applied as a chemically sensitive contrast specific to biomolecular structures such as aliphatic C–H stretching vibrations in lipids [20,21]. Finally, FWM is a non-chemically specific contrast that can be applied to the visualization of molecular density, similarly as in non-resonant CARS microscopy [22].

In what follows, we illustrate the influence of light polarization on nonlinear imaging by addressing SHG, the lowest-order contrast. Similar results for higher orders can be easily derived by using the same analysis. SHG signals are generated by a linear incident field ${\vec E^{{\rm in}}}$ at frequency $\omega$. The resulting SHG intensity is the result of the coherent sum of all fields radiated at frequency $2\omega$ by neighboring molecules within a focal volume, which are assumed to be in phase (here in the pupil plane):

Note that, in the paraxial approximation, the SHG intensity can be written as the modulus squared of the macroscopic SHG dipole. [A complete expression can be obtained by using Eq. (1).] The approximate SHG intensity along the polarization component $i$ is, therefore,

In the following sections, we describe separately two techniques: polarized excitation microscopy, which consists of tuning the excitation (i.e., incident) field ${\vec E^{{\rm in}}}$ to provide information on molecule organization (Section 3), and polarized detection microscopy, which consists of analyzing the polarization of the radiated light (Sections 4 and 5). While the former is generally limited to the exploration of organization within a 2D projection onto the sample plane and is mostly focused on ensemble microscopy, the latter has been used for the exploration of 3D orientation organization and has been recently studied extensively in single-molecule orientation imaging.

3. POLARIZED EXCITATION MICROSCOPY: QUANTIFYING MOLECULAR ORGANIZATION IN 2D

A. Probed Organization in Ensemble Microscopy

The previous section showed that polarized excitation or detection allows us to decipher molecular organization through information about the distribution $f(\Omega)$. However, an investigation of the integrals involved in the light–matter interaction process shows that not every detail of $f(\Omega)$ is accessible [10]. In summary, the integrals involved in polarized microscopy are all in the form of $n$ th-order moments made of the orientational average of dipoles to the $n$th power,

The integral in Eq. (14) and its related selection rules are at the origin of the concept of field tensor formalism, a useful picture that allows us to derive simple selection rules from light–matter interaction in high-order tensorial interactions [30,31]. Selection rules have been exploited in particular by the use of rotationally invariant field tensors, generated through the use of circular polarizations [32,33].

Both polarized fluorescence and nonlinear microscopy are based on the tuning of the incident field ${\vec E^{{\rm in}}}$ to access information on the distribution $f(\Omega)$ with unpolarized detection. This is typically implemented by rotating a linear incident polarization in the plane of the sample, e.g., by using a rotating motor holding a half-wave plate, or through electro-optic modulators. An important aspect of these methods is that they restrict the accessible components to projections in the plane of the sample since this field is essentially manipulated in the sample plane. Very few examples in the literature have addressed the possibility of measuring an ensemble dipole distribution in 3D, even though methods exist that indirectly retrieve this information, exploiting both excitation and emission [34] or measuring far field scattering patterns [35]. When the incident fields are strongly focused, such as in scanning microscopy, a longitudinal polarization contribution is expected at both the excitation [3] and collection steps. This contribution can be significant when the nonlinear coupling involves axially oriented crystals [36], but it is negligible in most biological samples studied, which are generally tilted toward the sample plane [37]. For the vast majority of polarized microscopy techniques based on in-plane polarization tuning, the measured distribution is, therefore, a reduced version of $f(\Omega)$, limited to an in-plane distribution $p(\phi)$, with $\phi$ being the azimuthal dipole orientation in the sample plane [28,38]. A circular decomposition of this distribution allows us to identify its measurable symmetry orders [10,28,38] [Fig. 2(e)],

 

Fig. 3. 1PF/2PF polarization microscopy in 2D: linear dichroism in fluorescence. (a) Color-coded molecular order measured over a fluorescence image (gray background) in cell membranes labeled with fluorescent lipid probes. The molecular order (e.g., cone aperture of an ensemble of molecules in the focal volume) is shown to decrease when the membrane is stretched in the absence of the underlying actin cortex. (Reprinted from Biophys. J. 105, A. Kress et al., “Mapping the local organization of cell membranes using excitation-polarization-resolved confocal fluorescence microscopy,” 127–136, Copyright 2013 with permission from Elsevier.) (b) Molecular order (stick colors) and orientation (stick orientation) shown in actin labeled with phalloidin AlexaFluor488 conjugates, in fixed drosophila embryos undergoing cellularization. At the apical surface of the cells, actin is disorganized, while at the basal membrane the presence of septin organizes actin (adapted from [39]). (c) Top, live cell imaging of microtubules in cells (labeled tubulin-GFP) using polarized standard wide field fluorescence imaging (left) and polarized structure illumination (pSIM, right). The color code corresponds to the GFP mean orientation measured at each pixel, and the pSIM image exhibits a gain in resolution. Bottom, representation of pSIM in the wave-vector space, showing the diversity of high frequency regions explored by rotating a polarized fringe pattern on the sample plane (adapted from [40]). (d) Top, examples of angular distributions probed by 2PF polarization microscopy in 2D. Bottom, illustration of the measurement of second and fourth order in distributions made of lipid probes inserted in cell membranes. (Reprinted from Biophys. J. 106, A. Ferrand et al., “Ultimate use of two-photon fluorescence microscopy to map orientational behavior of fluorophores,” 2330–2339, Copyright 2014 with permission from Elsevier.)

Download Full Size | PDF

B. Linear Dichroism in Fluorescence in 2D

Linear incident light polarization interacts with fluorescent labels in a way that maximizes absorption (and thus fluorescence) when its direction lies along the fluorophore direction, defined by its absorption transition dipole moment, which is along the main conjugation direction of the label. In polarized fluorescence (called more rigorously linear dichroism), the incident field ${\vec E^{{\rm in}}}$ is a linear polarization that is rotated within the sample plane at an angle $\alpha$ with respect to the horizontal $x$ axis. The incident field is supposed to have a negligible longitudinal contribution and can be written as ${{\textbf E}^{{\rm in}}}(\alpha) \propto (\cos \alpha ,\sin \alpha)$. Varying $\alpha$ leads to a modulation of the absorption efficiency in a molecular ensemble, ${{\cal P}_{{\rm abs}}}(\alpha ,\Omega) = |{{\textbf E}^{{\rm in}}}(\alpha) \cdot {{\boldsymbol\mu}^{{2{\rm D}}}}(\phi {)|^2}\mathop {\sin}\nolimits^2 \theta$, with ${{\boldsymbol\mu}^{{2{\rm D}}}}(\phi) = (\cos \phi ,\sin \phi)$ being the normalized projection of the dipole onto the sample plane. In ensemble molecular imaging, the molecules are supposed to be fixed within the distribution function $f(\Omega)$ (slow rotation regime), and the detection is unpolarized. The contribution of the radiation field $|\tilde {\textbf E}({\textbf u},\Omega {)|^2}$ in Eq. (7) depends only on the off-plane molecular tilt angle $\theta$. Therefore, the total fluorescence intensity, averaged over the full back focal plane, can be expressed as an $\alpha$-modulated function,

The goal of polarized fluorescence imaging is to deduce the accessible orders ${c_n}$ and ${\phi _n}$ of this reduced distribution function $p(\phi)$ from the ${I^{{1{\rm PF}}}}(\alpha)$ dependence. Since the dipole appears at a power $2$ in this expression, only the parameters $({c_2}/{c_0},{\phi _2})$ can be retrieved. A simple way to analyze this dependence is to decompose Eq. (17) onto circular functions as

Note that the order and orientation parameters $({c_2}/{c_0},{\phi _2})$ are directly related to the characteristics of the distribution function. Assuming that $f(\Omega)$ is a cone of aperture $\delta$ oriented along an azimuthal angle $\xi$ with respect to the $x$ axis [Fig. 2(a)], in the paraxial approximation where $J(\theta)$ can be ignored, these angular parameters $(\xi ,\delta)$ can be easily related with ${({\rm sinc} \delta)^2} = ({c_2}/{c_0}{)^2} ; \def\LDeqbreak{}\tan \xi = \tan {\phi _2}$. Other distribution shapes would lead to different dependencies. Polarized fluorescence microscopy consists of retrieving, at each pixel of an image, the order parameters $({c_2}/{c_0})$ (or the distribution aperture $\delta$) [Fig. 2(c)] and the mean orientation $\xi$ of the distribution of molecules within the focal volume of the objective. This approach has been successfully implemented in lipids using fluorescent probes embedded in membranes, providing sensitivity to lipid phases [41] and membrane nanoscale morphological changes [8] [Fig. 3(a)].

In contrast to lipids, fluorophores attached to proteins are not necessarily attached rigidly enough to be able to report on the orientation of the protein of interest. Using polarization-sensitive fluorescence imaging to study protein organization then requires labeling protein domains in a rigid way that minimizes orientational flexibility. This has been possible in vitro, e.g., in protein aggregates made of amyloids fibrils [42], but also in fixed cells and tissues with phalloidin conjugates labeling of actin [39,43,44] [Fig. 3(b)]. Live imaging requires engineering smaller or more rigid sequences in the linker between fluorescent proteins and the protein of interest. Imaging the organization of proteins in live cells has been the object of an increasing number of strategies, such as the investigation of membrane receptor proteins [45–47], septin in yeast [48], nuclear pore complexes [49], integrin [50], and actin [51]. This approach was later applied to the determination of complex geometries in protein organization at the adhesive interface between cells, targeting the desmosomal cadherins (DCs) in cell–cell adhesion regions [52].

Recently, polarized fluorescence/linear dichroism imaging has been extended to super-resolution microscopy, using polarized structured illumination microscopy (pSIM) [40] [Fig. 3(c)] and polarized minimal photon flux (MINFLUX) microscopy [53]. The use of light polarization to also enhance spatial resolution in fluorescence imaging has been explored using sparse deconvolution of polarization-modulated fluorescent images [54], an approach that was further improved by the use of a joint reconstruction of both spatial and polarization modulation information without invoking sparsity constraints [55,56]. Finally, while polarized fluorescence imaging has mainly been developed under epi-illumination, it can also be adapted to transmission geometries [48], allowing access to thickness-averaged quantities such as birefringence and diattenuation [57], which are important readouts of molecular organization, as demonstrated in several works [58,59]. Interestingly, this geometry is applicable to other fluorescence modalities such as multi-focus microscopy (MFM) [60] and can potentially be extended to the exploration of other illumination angles such as in light-sheet microscopy.

 

Fig. 4. Coherent nonlinear polarization microscopy. (a) General scheme of a nonlinear microscope in which polarization control of the incident beam provides a polarized contrast for SHG, 2PF, THG, and CARS (adapted from [61]). Bottom, typical response from a nonlinear polarized SHG (pSHG) signal, exhibiting the presence of several harmonics in the polarization response (adapted from [62]). (b) pSHG applied to collagen in a sagittal section of bovine articular cartilage, depicting a graduation of molecular order toward the cartilage surface in an arcade structure (adapted from [62]). (c) pTHG in human skin biopsies, evidencing lipid orientation and ordering within interfaces. A strong modulation is observed from ordered phases (adapted from [61]). (d) pCARS in lipids from the myelin sheath of fixed spinal cord mouse tissue (intensity). The zoomed region shows molecular order (color) and mean orientation (stick direction) of the lipids. (Reprinted from Biophys. J. 113, A. Gasecka et al., “Lipid order degradation in autoimmune demyelination probed by polarized coherent Raman microscopy,” 1520–1530, Copyright 2017 with permission from Elsevier.) (e) pSHG imaging of the structural alteration of collagen in cell lung carcinoma ex vivo tissues, depicting different contrasts such as the degree of linear polarization (e.g., molecular order), orientation, and chiral and achiral components of the nonlinear SHG tensor (adapted from [64]).

Download Full Size | PDF

C. Polarized Nonlinear Microscopy

The capacity of higher-order matter-optical field interactions to study fine molecular organization in materials and biological samples has been largely exploited. Nonlinear microscopy is particularly attractive for its capacity to detect molecular species without any fluorescence labeling and to access depths of several hundreds of microns in tissue by use of near-infrared illumination. A polarized nonlinear microscope is generally based on a scanning modality, in which the scan is combined with a rotation of the incident polarization, the detection being performed either in the forward or epi-direction depending on the nonlinear contrast being investigated [Fig. 4(a)]. The recorded response generally exhibits more than one harmonic in the polarization dependence due to the high order of the input fields involved [Fig. 4(a)]. Below we summarize their main properties, illustrated by important findings using these approaches.

Since they depend on the fourth power of the incident fields, both polarized 2PF (un-polarized detection) and polarized SHG give rise to a fourth-order dependence of the incident polarization angle $\alpha$,

Different directions have recently emerged in nonlinear polarized microscopy that show the fast pace of progress in this field. First, technological advances have led to high-speed polarization imaging, using in particular fast rotating electro-optical devices that can be synchronized with high-speed scanning [82–84]. Second, the application range of nonlinear polarized microscopy is now reaching biomedical optics interests, in particular for disease diagnosis [85–87] [Fig. 4(e)]. To make nonlinear polarized microscopy largely applicable to biological tissue research, an important element to consider is light scattering and its effect on depolarization. In this direction, formalisms including Mueller and Jones tensor descriptions are currently being developed [88], which provide robust estimates for depolarizing effects in a nonlinear Mueller formalism [89].

4. POLARIZED EMISSION MICROSCOPY: THE SECOND MOMENT MATRIX

In this section and the next, we address imaging modalities based on the way the polarization of light emitted by molecular dipoles evolves while traveling through a microscope, and how it can be manipulated in order to gain information about the molecules’ orientational behavior. Since these modalities are for the most part dedicated to the study of single molecules, we start by deriving the theoretical principles used in the field of single-molecule orientation measurements. Such results can nevertheless also be used to express responses from molecule ensembles.

The fluorescence intensity from a wobbling single molecule can be expressed in a simple way following Eq. (2), assuming an isotropic excitation that eliminates the dependence on the absorption cross-section. More complex situations involving non-isotropic excitation and slow rotational motion have been taken into account indirectly [90] or introduced in more complex expressions [12,91], using similar principles as in what follows. The measured PSF is given by the directional average of the intensity at the detector over the wobbling distribution $f(\Omega)$,

Given its linear dependence on the second moment matrix, it is useful to write the PSF in Eq. (23) in terms of a basis of PSF components ${{\cal I}_{\textit{ij}}}({\boldsymbol \rho}) = \{{\mathbb{K}^\dagger}({\textbf x},{z_0},{z_1};{\boldsymbol \rho})\mathbb{K}({\textbf x},{z_0},{z_1};{\boldsymbol \rho}{)\} _{\textit{ij}}}$ as [92–94]

In what follows, we take advantage of analogies between the analysis of wobbling/disordered dipoles and that of 3D polarized fields that appear in the context of non-paraxial optics [1,2]. The relation between fluorescence and 3D polarization was recognized almost a century ago by Soleillet [95]. Theoretical formalisms that have been used for 3D polarization can prove useful in the study and development of polarized microscopy, one example being the decomposition of the matrix into an appropriate basis of Hermitian matrices. The elements of the $3 \times 3$ second-order matrix can then be slightly rearranged by using the Stokes–Gell–Mann parameters [2,94,96], which are the coefficients of the expansion of the matrix in terms of Gell–Mann matrices (much like the standard Stokes parameters for $2 \times 2$ polarization matrices are the coefficients of an expansion in terms of Pauli matrices). The Stokes–Gell–Mann parameters can be defined as

We can now write an alternative version of Eq. (25), in which the measured PSF is expressed as a linear combination of PSF elements whose coefficients (except for the first) are the normalized Stokes–Gell–Mann parameters,

Here, the first PSF element ${{\cal I}_0}({\boldsymbol \rho}) = ||\mathbb{K}({\textbf x},{z_0},{z_1};{\boldsymbol \rho}{)||^2}/3$ is the only one that is guaranteed to be non-negative at all pixel positions ${\boldsymbol \rho}$; the remaining PSF elements ${{\cal I}_{\textit{mn}}}({\boldsymbol \rho})$ contain both positive and negative regions, but in a way that guarantees that $I({\boldsymbol \rho}) \ge 0$ for all pixels if the parameters ${s_{\!\textit{mn}}}$ are within their valid ranges, explained later in this section. Several of the experimental methods described in the next section were proposed with the aim of measuring fluorophore direction and wobble. Essentially, these methods provide estimates of the second moment matrix for individual fluorophores, in addition to their spatial position. Clearly, the more different the elements ${{\cal I}_{\textit{mn}}}({\boldsymbol \rho})$ are, the better the estimation of ${s_{\!\textit{mn}}}$ (and hence $\mathbb{M}$) from the measured PSFs, leading to a better characterization of the directional behavior of the molecule [96]. Interestingly, it can be shown that the PSF basis formed from the Stokes–Gell–Mann parameters ${s_{\!\textit{mn}}}$ is, in this respect, more appropriate than that based on the direct second-order matrix elements ${M_{\textit{ij}}}$, as it naturally separates the redundancy due to normalization.

The five-component vector $\vec s$ fully characterizes the second-order statistics of the fluorophore’s direction and wobble. It is, therefore, important to understand the range of values that its components can take, that is, the shape of the region in the abstract 5D space that this vector inhabits. As mentioned earlier, this vector satisfies $|\vec s| \le 1$, with the equality being possible only for fixed dipoles. This serves as the basis for a measure of wobble, inspired by a measure of 3D polarization [2,97], given by

 

Fig. 5. Restrictions for the normalized Stokes–Gell–Mann parameters. (a) The sub-vector $({s_{11}},{s_{12}})$ is constrained to the interior and edges of an equilateral triangle centered at the origin and whose corners are at a unit distance from the origin. (b) Definition of the quantities ${H_n}$ for $n = 1$ (orange), 2 (green), and 3 (blue), given by the intersections of a line emerging from $({s_{11}},{s_{12}})$ normal to this plane with three half-cones (in the colors mentioned earlier) that fit with the equilateral triangle. (c) The sub-vector $({s_{21}},{s_{22}},{s_{23}})$ is constrained to the interior and surface of a rounded tetrahedron contained in a box defined by ${s_{2n}} \in [- {H_n},{H_n}]$ for $n = 1,2,3$. The shape of this rounded tetrahedron is such that any cross-section for a fixed value of a parameter ${s_{2n}}$ is an ellipse.

Download Full Size | PDF

Another way to describe the second moment matrix [93] is in terms of its eigenvalues ${\lambda _n}$ and eigenvectors ${\vec v_n}$ as

5. SINGLE-MOLECULE ORIENTATION AND LOCALIZATION MICROSCOPY

While ensemble polarized microscopy is very useful for obtaining insight into the way molecules are organized, the associated methods are typically limited to a 2D projection, and most importantly they report on averaged information. In fluorescence microscopy, this means that the reported quantity is the addition of two effects: the effective structural disorder and the angular fluctuations of each molecule in the ensemble. This second effect can be addressed by measuring molecules individually. We now give a description of the rapidly growing field of single-molecule localization and orientation imaging, where different strategies have been developed to exploit super-resolution single-molecule-based methods while adding information about polarization (and hence molecule orientation). Deciphering fluorescent single-molecule orientation and localization with degree- and nanometer-level precision opens the door to new research possibilities beyond the ensemble observations described in Section 3. Downscaling orientational measurements to the single-molecule level offers the possibility of accessing new information such as slow rotational diffusion behavior and conformational changes in proteins under work [100]. If used in combination with single-molecule localization microscopy (SMLM), orientational imaging leads to a new modality known as single-molecule orientation and localization microscopy (SMOLM), where structural spatial information permits visualization of the orientational organization of proteins independently from the rotational flexibility (or wobble) of their fluorescent labels since this quantity can be estimated separately for each single molecule. In this section, we describe several of the conceptual and experimental advances that have led to the development of SMOLM.

 

Fig. 6. (a) Bias introduced by a fixed dipole orientation on its spatial localization estimation [adapted from [101]; licensed under a Creative Commons Attribution (CC BY) license]. (b) SMOLM imaging of intercalants of DNA using excitation modulation (adapted from [102]). (c) SMOLM in isotropic media, using polarization splitting for anisotropy imaging (adapted from [16]) with permission. (d) SMOLM in organized media using a four-polarization split to retrieve both orientation and molecular order projected onto 2D. This application is illustrated in the structural imaging of actin in the dense meshwork of a cell lamellipodia (adapted from [4]).

Download Full Size | PDF

A. Single-Molecule Localization Microscopy: Spatial Localization

Given that they are telecentric and have low levels of field-dependent aberrations, optical microscopes are able to produce PSFs whose shape is approximately independent of the transverse position ${\textbf x}$ of the molecule being imaged. The transverse localization of a molecule beyond the diffraction limit is then possible in fluorescence microscopy through the techniques [103] collectively known as SMLM, such as photoactivated localization microscopy (PALM) [104,105], stochastic optical reconstruction microscopy (STORM) [106,107], and point accumulation in nanoscale topography (PAINT) [108]. Single fluorescing molecules have dimensions that are much smaller than the fluorescent wavelength $\lambda$, so in standard imaging techniques they can only be resolved to within scales of the order of $\lambda /2$. SMLM techniques rely on the stochastic nature of fluorescence: the final image is composed from a stack of sequential images, where for each image only a subset of the molecules fluoresce, producing PSFs that are typically sufficiently spaced as to not overlap. The $(x,y)$ coordinates of each molecule can then be estimated with an accuracy well below the diffraction limit. Many techniques rely on the assumption that the PSF follows an Airy pattern that is symmetric and not too different from a Gaussian. The molecule’s transverse position can then be estimated as the centroid of the PSF or by fitting a Gaussian through an approach known as fluorescence imaging within 1 nm accuracy (FIONA) [109]. In addition to their lateral localization, the longitudinal position ($z$) of single molecules is an important parameter that is challenging to measure with high precision [103] due to the relatively poor sensitivity of their PSF to defocusing. Different strategies have been developed based on interferometric pattern illumination [110], multiple plane imaging [111–113], and PSF engineering [114,115], the latter being described in more detail below.

As discussed in Section 2, the shape of a PSF depends not only on the $z$ coordinate of the fluorescing molecule but also on its 3D orientation, and even when in focus, it can differ considerably from the standard Airy pattern. Not long after the first SMLM approaches were proposed, it was noticed that the fluorophore’s lateral localization accuracy and precision can be significantly affected by its orientation, particularly when this orientation is fixed [101,116–119] [Fig. 6(a)], but also for low wobbling conditions [120–122]. Strategies of PSF manipulation such as the use of azimuthal polarization filtering have been proposed to minimize localization biases [123]. However, the dependence of the PSF’s shape on the fluorophore’s orientation and longitudinal position can be viewed as a feature rather than a source of error, as it might provide a means of estimating these parameters jointly with the transverse coordinate. In fact, many authors [119–126] have envisioned ways to achieve higher spatial accuracy once a single molecule’s orientation is estimated.

In what follows, we describe techniques that have been proposed to enhance the dependence of the PSF on orientation and/or longitudinal position in order to better estimate these parameters for single molecules. Under the assumption of uniform wobbling within a cone, deciphering both the 3D localization and 3D orientation of single molecules requires the estimation of six parameters: three spatial ($x,y,z$) and three orientational ($\xi ,\eta$ azimuthal and elevation tilt angles, and $\delta$ wobbling extent). For anisotropic wobble, the number of parameters grows to eight.

B. Polarized Excitation (Single-Molecule Linear Dichroism)

The first measurements of single-molecule orientation were performed not by using the shape of the PSF but by using the linear dichroism inherent in the excitation of the molecules through variations in the polarization of the excitation light. This approach was not coupled at the time with single-molecule localization, but it did shed some light on the level of new information that results from considering isolated molecules as individual radiating dipoles. These dipoles were assumed to have fixed directions during the integration time of the measurements, and only the projection onto the 2D sample plane was considered [127,128]. Note that, as explained in Section 3, linear dichroism allows only the determination of two independent parameters so that 3D orientation information is not accessible except if an additional assumption is made, such as the absence of wobble. Attempts to estimate 3D orientation were performed by adding extra information, such as using multiple angles of incidence [129–133] or azimuthal illumination [134]. Although spatial localization was not used at that time, this approach led to new biological insights on DNA bending [127] and rotational mobility in lipid membranes [128] and polymers [135], as well as biological motor dynamics in vitro [136–140]. Single-molecule linear dichroism was later combined with 2D spatial localization by exploiting the long emission times of single emitters attached to DNA strands lying on a glass surface [102] [Fig. 6(b)].

C. Polarization Splitting

A convenient way to acquire extra information on molecular orientation without sacrificing significant numbers of photons is the one known as polarization splitting. The light emitted by a fluorophore travels through the imaging system and arrives at the detector array where it is measured. At each pixel of the detector, this light has a polarization that contains information about the fluorophore’s orientation, but these detectors are typically insensitive to polarization. To capture this missing information, the light is separated into different polarization components, each of which is imaged onto a different detector or a different section of the same detector. Separation into two orthogonal linear polarizations can be easily implemented by using a polarization beamsplitter or a Nomarskii or Wollaston prism. While sufficient for many purposes, this separation into two polarization components does not allow a full polarimetric characterization; if this was required, the image can be projected onto more paths to allow more polarization projections. Similarly, a rotating analyzer can provide sequential access to polarization information. If the polarization distribution is assumed to be fairly uniform across the PSF, the separation into different polarizations allows measurement of the ratio between their intensities, which gives information about the orientation parameters, similarly to what is done in standard polarimetric techniques.

Polarized intensity estimation with no spatial localization. The first type of polarization-splitting single-molecule measurements, prior to the advent of localization microscopy, was aimed at estimating the 3D orientation of dipoles assumed to be fixed in time by using a projection onto two orthogonal linear polarizations for fluorophores specifically anchored at two ends to motor proteins [141], attached to biotin-streptavidin linkers [142], or immobilized into DNA strands [143]. This strategy was later made more precise by using total internal reflection illumination to favor longitudinal excitation and using four polarization detection channels (along the 0°, 90°, 45°, 135° directions relative to the horizontal axis of the sample) [144]. This approach was demonstrated in motors labeled with quantum rods [145] or bifunctional rhodamine molecules [146]. Polarized detection can also be achieved by a rotating analyzer [147–149], a strategy that is appropriate for non-bleaching nano-emitters [150] due to its sequential nature. The strategy of detecting multiple polarization projections was especially successful for deciphering polarization emission properties of quantum dots, in particular degenerate levels from CdSe nanocrystals [148,151,152]. Accessing information about molecule wobbling requires adding further information to polarized microscopy. If localization is ignored, this can be performed by combining both excitation and detection polarization tuning [130,139].

Polarized intensity estimation including spatial localization. Most of the early work on SMOLM consisted of combining SMLM with polarization splitting. A direct application of this approach was for fluorescence anisotropy imaging, particularly the estimation of single molecules’ rotational diffusion in isotropic environments, similarly to ensemble measurements [14]. Gould et al. [16] obtained two-polarization split images of single-molecule anisotropy in cells, showing contrasts in membranes in particular [Fig. 6(c)]. Ohmachi et al. [153] used four-polarization imaging and high tilt illumination to provide both orientation and 2D spatial motion of single quantum rod-tagged myosin V proteins, assuming fixed orientations. This approach was further extended to the 3D tracking of freely moving nanoprobes [154]. Valades Cruz et al. [155] used a two-polarization split SMLM estimation of phalloidin conjugate single molecules attached to actin in fixed cells, providing information on their 2D structural organization under the hypothesis of homogeneous wobbling. A similar approach was used to image a single Thioflavine T molecule reporter of amyloids local structures [156]. Later, Rimoli et al. [4] used a four-polarization split SMLM approach to quantify the 2D projection of wobbling molecules attached to actin, providing indirect evidence of their 3D orientations [Fig. 6(d)]. Other indirect methods for the estimation of 3D orientation using a four-polarization split were applied to live actin [157] and integrin [50,158] in focal adhesion sites of live cells, using knowledge of the structure of the fluorescent construct location in the protein. Finally, a combination of polarization splitting with pupil splitting was applied recently for a complete determination of 3D orientation including wobbling, in an approach based on single-molecule intensity estimations [159,160]. The advantage of intensity estimation is that there is no need for PSF-shape fitting and that it can potentially work for compact PSFs that are more appropriate for high density SMLM imaging.

In all the work mentioned in the previous paragraph, localization is limited to 2D lateral $x$, $y$ information, and since a PSF fit is not required, this estimation is based on the hypothesis of symmetric PSFs, which is valid under relatively high wobbling conditions [120]. The combination of four-polarization imaging with PSF shape fit estimation was nevertheless studied recently [161] to achieve higher precision on the estimation of the spatial and orientation parameters. This comes, however, at the cost of heavier computational work. Note also that under such geometry the estimation of the 3D orientation information and wobbling from single molecules is incomplete, as some of the PSF basis elements (${{\cal I}_{21}}$ and ${{\cal I}_{22}}$) vanish [96].

Polarized PSF imaging. Separation into two polarization components can be particularly useful in cases where the polarization varies appreciably within the PSF, hence encoding more information. An appropriate choice of separation basis can enhance considerably the ability to estimate the 3D orientation of the fluorophore. The first approaches in this direction were designed to determine the 3D positions and 3D orientations of dipoles, without accounting for wobbling. A direct method used maximum likelihood estimation (MLE) of the shape of enlarged polarized PSFs [162], allowing separation of intra-molecular fluorescent molecules on DNA using different colors in addition to orientational information [163]. A decomposition of the PSF model onto a Hermite–Gauss basis was also implemented on four polarization projections [164]. By taking into account the proximity of the fluorophore to a dielectric interface, wobble information was incorporated in a precise 3D orientation estimation in order to image single Nile Red molecules transiently bound to amyloid aggregates [165]. However, this method requires knowledge of the distance between the dipole and the substrate interface. As described in Section 5.E, polarization splitting is particularly effective when used in combination with PSF engineering.

D. Pupil Plane Imaging and Defocused Imaging

In parallel to polarization-splitting strategies that address mostly the image plane, a direct observation of the Fourier pupil plane can provide information on the 3D orientation of fixed dipoles [122,166], as can be appreciated from the second row in Fig. 1(b). Despite more challenging detection conditions in terms of signal-to-noise level, direct pupil plane imaging takes advantage of modifications of the emission pattern of fluorophores due to their 3D orientation [166,167] or near dielectric or metal interfaces [168,169], as described in Section 2.

An effective way to decipher a fixed single molecule’s 3D orientation was later proposed based on a defocused image due to the significant difference between the longitudinal and lateral dipole radiation patterns [166,170,171]. This approach has also been widely used to analyze nanoparticles [172–175]. The analysis of defocused images in the presence of wobbling was later performed by accounting for orientational mobility in the formalism describing dipolar emission and propagation in a high numerical aperture microscope [93]. The determination of localization in addition to fixed orientations was also explored based on defocused images [92,176].

E. Point-Spread Function Engineering

The dependence of the PSF on parameters such as the orientation and longitudinal coordinate can be enhanced (or reduced) by applying a suitable filter or mask in the imaging system. In order to maintain the approximate shift invariance of the PSF to the fluorophore’s transverse position, such masks must be placed at the pupil (i.e., back focal) plane. These pupil masks can modify different properties of the field distribution at the pupil, such as its amplitude, phase, or polarization. The modeling of such modified PSF uses the expressions given in Eqs. (1), (3), and (2), in which the Green matrix $\mathbb{G}({\textbf u})$ is multiplied by a amplitude/phase/polarization mask matrix $\mathbb{J}({\textbf u})$ [94]. We now describe these three different types of filtering, which of course can be combined. These methods, known as PSF engineering, were first introduced for 3D localization and were later extended for the measurement of orientation and wobble.

Amplitude filters. The amplitude distribution at the pupil plane can be modified either in a piecewise or smooth (apodized) way. The latter is less common, so we focus on the former. Amplitude filtering techniques have been proposed to measure dipole orientation based on the different distributions of optical power at the pupil plane in terms of the orientation of the fluorophore: for a dipole oriented longitudinally, a larger fraction of the power is located at the edges in comparison to a dipole contained in the transverse plane. These techniques then typically rely on splitting the wavefront before forming an image and using different sections of the pupil on each. Prior to their use for spatial localization, amplitude masks at the pupil plane were used to estimate fixed 3D dipole orientations [177]. Hohlbein [178] then used an annular mirror to separate different parts of the numerical aperture onto different paths, demonstrating high sensitivity to 3D orientations [179,180]. Pupil plane segmentation and splitting were later combined with phase masks and polarized detection to achieve complete 3D spatial localization and orientational measurements in SMOLM, as described below.

 

Fig. 7. Phase masks used for suppressing or enhancing PSF dependence on $z$. (a) Cubic phase mask (where the phase range must span many cycles, 10 in this simulation). The top row of PSFs shows for comparison the PSFs at five equally spaced distances about the focal length without the mask, while the bottom row shows those with the cubic phase masks at steps 5 times larger than for the first row. (b) Astigmatic phase resulting from using a cylindrical lens, and PSFs with an without the lens. (Reprinted from Biophys. J.67, H. P. Kao and A. S. Verkman, “Tracking of single fluorescent particles in three dimensions: use of cylindrical optics to encode particle position,” 1291–1300, Copyright 1994 with permission from Elsevier [181].) (c) Mask for the “double helix” PSF and corresponding PSFs at several distances (adapted from [182]). (d) Mask for the “tetrapod” PSF and corresponding PSFs at several distances. (Reprinted with permission from Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, Phys. Rev. Lett.113, 133902 (2014) [183]. Copyright 2014 by the American Physical Society. https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.113.133902.)

Download Full Size | PDF

Phase filters. The most common type of mask is that of phase filters since they allow strong modifications of the PSF shape while sacrificing hardly any photons. Much work in this direction focused on either suppressing or enhancing the dependence of the PSF on the longitudinal position $z$ of the emitter, but they have also been used to retrieve directional information, as explained in what follows:

The complete retrieval of 3D spatial localization with 3D orientation (including wobbling), which is the core of SMOLM, requires careful design of strategies that are able to detect all parameters with high precision and with the least correlation between them. The goal of PSF engineering is to capitalize on the PSF dependence to orientation [Fig. 8(a)], emphasizing its sensitivity to angles and axial position parameters [197]. An implementation of a polarized double helix PSF has led to 3D localization and orientation estimations supposing fixed dipoles [115,120], which, however, lead to poorer estimation performances at high off-plane orientation angles and low SBRs due to a large spread of the PSF. A refinement of the quadrated pupil (the bisected pupil) [195] was proposed in which the PSF is separated into two spots with phase ramps, but in different directions for each polarization channel [198]. This approach indicated for the first time the possibility to estimate wobbling from PSF engineering [120] [Fig. 8(b)]. Zhang et al. proposed a version called the tri-spot phase mask, in which the pupil is separated into three sectors (with more irregular shapes), each with a linear phase ramp in a different direction to create three spots [90] [Fig. 8(c)]. The implementation of this phase mask requires the use of a spatial light modulator (SLM), and the retrieval of the parameters uses an appropriate algorithm such as maximum likelihood estimation. This work demonstrated high orientational estimation performances able to minimize 3D localization bias, which was applied to the measurement of single Atto 647 N molecules embedded in a thin layer of polymethyl methacrylate (PMMA). The tri-spot phase mask was used later to analyze the orientation behavior of Nile Red lipophilic probes transiently bound to artificial lipid membranes [199]. Finally, a phase vortex (i.e., a phase ramp in the pupil’s azimuthal angle) was also employed, which gives a relatively compact PSF in comparison to multiple-spot phase masks [200] [Fig. 8(d)]. This strategy of direct phase imaging was modified by incorporating two orthogonally polarized detection channels [201] to enhance the sensitivity of estimation in combination with lateral $x$, $y$ position estimation. This polarized vortex PSF, applied to single molecules in supported lipid layers, permitted to reach, with typically 500 photons detected, precisions of 12 nm in lateral localization and 4–8° in orientation, together with 12° wobbling angle precision. It also allowed us to estimate possible anisotropy of the directional distribution function (that is, estimating $\mathbb{M}$ even when its two lowest eigenvalues are different).

 

Fig. 8. SMOLM based on PSF fitting or PSF engineering. (a) Enlarged PSF from a fixed dipole (adapted from [162] with permission). (b) Bisector phase plate and PSF (adapted from [120]). (c) Tri-spot phase plate and PSF. (Reprinted with permission from O. Zhang, J. Lu, T. Ding, and M. D. Lew, Appl. Phys. Lett.113, 031103 (2018) [90]. Copyright 2018 AIP Publishing LLC.) (d) Vortex phase plate and corresponding PSF (adapted from [200]). (e) Optimized phase plate and PSF (adapted from [202]). (f) CHIDO birefringent phase plate and PSF (adapted from [94]). (g) Radially and azimuthally polarized multi-view reflector and PSF (adapted from [203] with permission), with the corresponding microscope setup depicted below.

Download Full Size | PDF

Filters with spatially varying birefringence. Another property of a nominally transparent filter that can be used for PSF engineering is birefringence. Recently, methods were proposed based on masks whose birefringence changes as a function of the pupil coordinate ${\textbf u}$. These techniques are used in combination with polarization splitting in order to better encode both fluorophore orientation, wobble, and longitudinal position. Curcio et al. [94] proposed a method called coordinate and height super-resolution imaging with dithering and orientation (CHIDO), whose filter is a glass window subjected to stress with trigonal symmetry [Fig. 8(f)]. The birefringent Fourier filter, referred to as a stress-engineered optic (SEO), naturally displays all the range of linear birefringence over the pupil, since the fast-axis orientation varies as ${-}\varphi /2$, and the retardance is proportional to $u$. Because it uses circular (rather than linear) polarization splitting, the variation in retardance has an effect similar to that of a smooth pupil splitting. CHIDO was applied to AF488 phalloidin conjugates attached to single F-actin filaments, demonstrating a tilt angle relative to the filament. In theory, this approach is able to estimate anisotropy in the wobbling. A second method [204] uses a birefringent mask corresponding to what is known as an S-plate (or a q-plate of charge 1/2), whose retardance is fixed to a half-wave, and the fast axis rotates as $\varphi /2$. This technique, called radially and azimuthally polarized (raPol) microscopy, uses linear polarization splitting and was shown to be particularly sensitive for axially oriented molecules. raPol SMOLM, which achieves 2.9 nm localization precision, 1.5° orientation precision, and 0.17 sr precision in wobbling, was applied to spherical supported lipid layers in which Nile Red fluorescent probes were inserted. Finally, Zhang et al. [203] developed a radially and azimuthally polarized multi-view reflector (raMVR) microscope that splits the pupil plane in addition to radial and azimuthal filtering, using a square pyramidal mirror. This approach, which provides improvement in precision at the price of increased implementation complexity [Fig. 8(g)], is based on the analysis of eight measured images.

Generation of masks. Many of the pupil masks described here have been generated with SLMs, and in theory all of them can be implemented this way. This, however, comes at the cost of photon losses by diffraction and extra complexity of the system. Photon loss can be reduced by using instead deformable mirrors [205], although the number of degrees of freedom that can be controlled is smaller. The amplitude masks are implemented with diaphragms or mirrors. Some of the phase masks can be implemented by holography or lithography [182]. Birefringent elements can now be created through several technologies such as liquid crystals, which allow for some active control. Modern nanofabrication techniques allow the fabrication of metasurfaces whose birefringence distribution can be tailored [206]. Elements like the S-plate are now commercially available. The birefringence pattern for the SEO arises naturally around an equilibrium point of a glass window under stressed-induced birefringence.

F. Optimization Criteria

Several factors must be taken into account when implementing techniques for PSF engineering. Perhaps the main one is the maximization of the information encoded. Metrics such as the Fisher information with its associated Cramér–Rao bounds [207] provide criteria for optimization, even though this is not trivial when one seeks to optimize several parameters simultaneously over a range of situations. A few recent articles reported the search for optimal phase plates in order to estimate efficiently the orientation, wobbling, and 3D positions from single molecules. One optimization strategy used the tuning of all pixels from polarized phase masks [Fig. 8(e)] [202], and deep learning approaches have also been developed [208]. While the resulting PSFs are quite large, these works show that optimization is an interesting approach through the search for efficient estimation methods.

In what follows, we summarize the important features of the use of Fisher information theory for the optimization of the efficiency of SMOLM approaches. Suppose that we have a model for the PSF intensity distribution $I({\boldsymbol \rho};p)$ in terms of some source parameter $p$, where ${\boldsymbol \rho}$ represents the coordinates of the detector pixels. (Note that this intensity model can include some expected amount of background, not emitted by the fluorophore.) The probability that a photon will fall on a given pixel ${\boldsymbol \rho}$ is found by simply normalizing the intensity,

The same concepts apply to the estimation of a collection of parameters $\vec p = ({p_1},{p_2},...)$. The Fisher information is now a matrix, $\mathbb{J}$, and it provides a Cramér–Rao bound for the covariance matrix $\mathbb{C}$,

Another criterion for optimization is the size of the engineered PSFs. There are two reasons for trying to achieve PSFs that are as small as possible while efficiently encoding the relevant parameters: a higher concentration of photons that improves the signal-to-noise ratio and a reduction in the occurrence of overlap of PSFs for different molecules in dense samples. The first of these criteria, however, is naturally integrated into the Fisher information if the model used to calculate it includes background [96]. Different measures for the width of a PSF exist. Some of them lend themselves to analytical minimization [211], while others must be minimized through numerical techniques.

G. Retrieving the Parameters

Several strategies can be used to retrieve the 3D positions and orientations of the fluorophores from the measured PSFs. All these approaches require a forward model able to predict the PSF position and shape for a given value of the parameters. This model can be based on theory, e.g., the one described in Section 2. However, aberrations, residual birefringence (e.g., originating from mirrors used to fold the light path) and other practical issues create discrepancies between the model and the real PSFs. In standard polarimetry techniques, these issues can be addressed by creating a model from a set of calibration measurements. This is challenging in SMOLM, however, given the nontrivial combination of the spatial (particularly the longitudinal) and directional degrees of freedom, and due to the difficulty of having controllable sources for calibration. One can use, for example, fluorescent beads [54,212–216], which combined with polarizers can mimic dipoles with given orientations, although there are inherent limitations to this approach [217]. The best option is then perhaps to use a combination of calibration measurements with theoretical calculations to arrive at a model that is complete and as close to reality as possible. Many software solutions for extracting a model from calibration measurements and for subsequently extracting 3D localization data have been proposed [205,218,219]. Some of the approaches used to retrieve the parameters from measured data are now listed.

Basis decomposition. This technique is based on the linear decomposition of the PSF in terms of a basis of elementary PSF elements, each associated with an element of the second moment matrix as in Eq. (25), or equivalently with a Stokes–Gell-Mann parameter as in Eq. (27). This approach is particularly useful for a rapid estimation of the orientation parameters for known 3D position. As mentioned earlier, the more different the basis elements are, the more accurately the parameters can be retrieved, so the goal of approaches such as polarization splitting and pupil masks is precisely to make these basis elements as large (in intensity but not in spatial extension) and as different as possible. While this approach is very simple and intuitive for the directional degrees of freedom, the mixing of the molecule’s spatial coordinates makes it a bit more challenging. A method for estimating the molecule’s transverse position was described in the supplementary document of [94], which also presented a way to address longitudinal localization by enlarging the basis through an expansion in $z$. For more accurate estimations, the results of this approach can be used as starting points for the techniques described next.

Maximum likelihood estimation. MLE [207,220] is a standard parameter retrieval technique that is optimal when the dominant noise is Poissonian and, hence, has been used in several SMOLM approaches [94,118,221,222]. The probability density ${\cal P}({\boldsymbol \rho};\vec p)$ can be interpreted as a measure of the likelihood of a given value of the parameters $\vec p$ given that a photon hit pixel ${\textbf x}$. If an experiment yields a measured PSF $\hat I({\boldsymbol \rho})$ so that ${\langle \hat I({\boldsymbol \rho})\rangle _{\textbf x}} = N$, the log-likelihood distribution can be calculated as

Deep learning. Like for many other areas in science and technology, machine learning techniques such as deep learning are rapidly becoming popular. One application in SMOLM is for the retrieval of molecule parameters from large stacks of images. The two main objectives of applying this type of technique are the rapid acquisition of data and the ability to handle superposed PSFs by training the network using either simulated or experimental data. To our knowledge, the first publication in this direction appeared in 2018 for the retrieval of 2D localization [223], and by 2020, Möckl, Roy, and Moerner had already written a review of deep learning and its use by many authors in single-molecule microscopy, particularly for the retrieval of 2D and 3D localization [224]. Deep learning was subsequently applied to orientation-dependent complex PSF shapes [225] and to the joint estimation of 3D orientation and 2D localization [208,226].

6. SUMMARY

In this section, we provide a summarized comparative view of both the polarized excitation and emission microscopy approaches, addressing their main advantages and limitations as well as their applicability to a more extended range of contexts. Different factors contribute to the challenges in the development of a polarized microscopy technique, which we list in what follows.

Finally, as mentioned in Section 2.B, exploiting both the excitation and emission mechanisms can open the path to new information, such as high orders of symmetry of the molecular angular distribution, or the rotational diffusion dynamics of molecules.

7. POSSIBLE FUTURE DIRECTIONS

We would like to conclude this review by listing some of the many future directions that can be explored in molecular orientation imaging. The list is by no means exhaustive, but it aims to point out some of the challenges that remain in this growing area of research and to suggest some of the strategies that could be used to tackle them, whether they be theoretical, computational, and/or experimental. Most of these challenges relate to situations in which the dimensionality of the problem is increased, either by adding degrees of freedom such as time or frequency, by considering higher-order moments of the fluorophore direction, or by extending the treatment to cases when the radiation pattern is more complex.

A. Measurement Optimization

Many of the techniques described in this review have the aim of estimating one or several parameters of one or a group of fluorescing molecules. Some of the SMOLM methods, for example, do so by using pupil masks that are either pre-existing devices [94,200,204] or tailor-made masks designed through optimization [183,188–190,202,208]. When the goal is to measure a considerable number of degrees of freedom (e.g., three spatial coordinates and five directional parameters related to the second moment matrix for a single molecule), optimization requires defining a merit function that involves all parameters (such as the determinant of the Fisher information matrix), each within a range of interest. Even for a given combination of parameters of interest, the definition of a merit function is typically not unique: there are, for example, choices to be made on how to average this function over the range of values of interests or whether PSF compactness should be included as part of this function. Notice also that not only can the dimensionality of the measurement increase but also the number of degrees of freedom to be optimized in a measurement technique. Modern fabrication techniques allow the creation of increasingly more complex distributions of amplitude, phase, and birefringence in a mask. In cases where the optimization becomes overly complex due to the large number of parameters, deep learning techniques can be used to propose novel solutions that can at least serve as starting points for down-hill optimization routines.

B. Probing Orientational Dynamics

While polarized microscopy addresses mostly steady-state situations in which the information is averaged over time, there is a great deal of insight to be gained from time-resolved measurements. Time-resolved fluorescence anisotropy is a well-established tool for ensembles, and it is based on the measurement of the arrival time of photons that are polarized in both perpendicular and parallel directions with respect to an incoming known polarization. Using such extra information, rotational motion times shorter than the fluorescence lifetime can be accessed, providing new information on fast rotational trajectories. Time-resolved anisotropy is, however, performed mostly in isotropic environments and solutions, and it has not yet been used to address 3D rotational motion in constrained environments. It has also been developed at the single-molecule scale in fluorescence correlation spectroscopy (FCS), in which molecules diffuse freely in solution through a focus [227]. Accessing such configuration both at the ensemble and single-molecule levels is an interesting challenge, given that the current technology of time-sensitive cameras may even allow the combination of PSF engineering with time-resolved measurements. Time-resolved information could also lead to the simultaneous quantification of the rotational diffusion times of probes attached to proteins and their nanometer-scale localization, as well as to the measurement of the local viscosity or the amount of energy transfer between two nearby molecules, occurring at sub-nanometer distances.

C. Measurement of the Fourth Moment Tensor

As mentioned in Section 2.B, in the slow rotation regime in which the fluorescence lifetime is shorter than the characteristic time of molecule wobble, the excitation and detection stages are not decoupled in polarization. The measurements are, therefore, not linked to the second moment matrix $\mathbb{M}$, but to a fourth moment tensor of the form in Eq. (14) with $n = 4$. This fourth rank tensor provides more detailed information than $\mathbb{M}$ about the fluorophore’s orientational dynamics and the shape of its orientational distribution, but measuring it poses interesting challenges. Unlike in the fast rotation regime where either the excitation or emission processes can be used to access the second-order moment, here both excitation and detection must work together to probe all components of the tensor. The measurement scheme will then be qualitatively similar to Mueller polarimetry, where different polarizations of the excitation field are applied, and for each the output is analyzed. This is also valid for the ensemble measurement configurations.

We can again consider techniques that are either restricted to measuring a 2D projection of the molecule orientation or that probe orientation in all three dimensions. For the 2D projection case, for which both excitation and collection can stay within the paraxial regime, the fourth moment tensor has ${2^4} = 16$ elements, but by symmetry only five of them are independent, and the normalization of the direction vector $\vec\mu$ removes one more degree of freedom, leaving only four quantities to be measured. This number is considerably smaller than the number of elements of a Mueller matrix due to the symmetry of the tensor (and because the elements related to circular polarization are not required), so appropriate choices of polarization for the excitation and analysis can simplify the scheme. For the 3D case, on the other hand, the tensor has ${3^4} = 27$ elements, which given symmetry and normalization constraints result in nine independent quantities to be measured. As discussed earlier in this article, up to five parameters related to the rotational dynamics can be extracted for a given excitation field. However, a complete measurement of the tensor will probably require more than three different excitation fields that engage in different ways all three Cartesian components of the dipole orientation. Effective techniques will then involve a cleverly chosen sequence of measurements, which will pose technological challenges associated with applying the required excitation fields, especially if this method is to be used in a super-resolution context in which both the position and directional dynamics of several fluorophores are measured simultaneously over a broad field of view.

D. Polarized Microscopy and Nanooptics: Probing Plasmonic Vector Properties, 3D Polarization States, and Complex Vector Field Structures

An interesting extension of some of the methods described in this work is to use them for probing fields rather than molecules. That is, so far we have considered that we know the excitation field and we make measurements of the radiation emitted by a molecule or collection of molecules excited by the field so that these measurements give information about these molecules. However, one can consider the situation in which we use particles whose response is well characterized to measure excitation fields. For example, the particles (e.g., metallic) can be chosen to be sufficiently small as to give a linear response in the Rayleigh scattering regime in which the electric dipole component dominates but also to be sufficiently large as for this response to provide a sufficiently strong signal. If the 3D position of these particles is known with accuracy, they can serve as probes for measuring the unknown incident field at the particles’ locations [228]. This type of approach has been used in recent years to measure a variety of different fields presenting interesting polarization features such as polarization Möbius strips [229] and transverse spin [230]. Most of these approaches have relied on using a single particle that is scanned spatially and where the measurement is performed by analyzing the field at the pupil plane. It is possible, however, to parallelize the procedure by using multiple particles while working in the image plane, using PSF engineering to best encode the information.

The measurement of local fields beyond the paraxial regime is not limited to fully deterministic fields but can also be applied to partially polarized ones. Partial polarization results from an averaging of fluctuations that is analogous to fluorophore wobble. However, unlike for the case of fluorescent dipoles, the scattered dipole emission from a nanoparticle (whether the probed field is fully or partially polarized) can include spin components, meaning that the $3 \times 3$ polarization matrix that is analogous to the second moment matrix can be complex [96]. As a result, three more Stokes–Gell–Mann parameters must be used, resulting in a polarization space composed of eight parameters [1,2]. These extra parameters should also be taken into consideration in the optimization routines used to develop appropriate measurement techniques.

E. Fluorescence from Chiral Molecules

The chirality of a molecule can result in its fluorescence being composed not only of electric dipole radiation but to also include a magnetic dipole component (and in the case of plasmonic particles, higher-order multipolar components). These different components of the radiation pattern can in theory also be resolved by a high NA microscope. In the case in which only dipolar radiation is considered, the second moment matrix must now be replaced by a $6 \times 6$ matrix that encodes both the electric and magnetic dipole components. If the electric and magnetic dipoles are assumed to be linked at all times, the number of parameters will not grow significantly, but if the directions of both types of contributions are not necessarily linked and their relative amplitudes and phases are not predetermined as a result of a known molecular structure, the space of parameters to be determined can be considerably larger. In either case, by appropriately designing a pupil mask, the PSF could in principle be engineered to encode all these parameters while still allowing localization for multiple molecules simultaneously.