1. INTRODUCTION

Optical multimode fiber (MMF) is a high-throughput, narrow diameter, flexible waveguide that opens visual access to otherwise difficult to reach tissue sites [1]. Imaging through MMF for endoscopic applications offers many potential advantages over other optical probes: flexible geometry and lower cost compared to wide-field rod-lens endoscopes, minimized probe size, and variable sampling rate and working distance compared to micro–electromechanical-system (MEMS)-based endoscopes, and denser mode population over the core area compared to fiber bundles and multi-core fibers. Several studies have highlighted the promising biomedical significance of MMF imaging in neuroscience [2–4] and its potential for endoscopic imaging [5]. Current approaches to imaging through MMF, however, may involve complex hardware and have a small tolerance to bending. Simplified and label-free imaging paradigms, as well as continued progress to reduce tolerance to bending, are needed to enable clinical MMF endoscopy [6–9].

Several strategies have been proposed to compensate for the chaos caused by mode mixing and dispersion in MMF transmission, including wavefront shaping (WFS) [2,4,10–15], speckle imaging [5,16], compressive imaging [17,18], and deep learning [19,20]. Extensive research on WFS has yielded unheralded advances with imaging through MMF. It is the primary strategy for endoscopic imaging of complex three-dimensional (3D) biological samples, by generating a sharp distal focus through a calibrated MMF and scanning the distal focus through the sample for image formation. An alternative approach that avoids WFS may reduce system complexity and circumvent hardware limitations in imaging speed, focus accuracy, and power efficiency. Furthermore, while physical scanning through MMF combined with fluorescent labeling has great potential in preclinical imaging with high sensitivity and specificity, there is a need for exogenous-label-free imaging for clinical diagnostic applications. Toward this end, Loterie et al. have demonstrated focusing through MMF using WFS and digital/optical phase conjugation on the detection pathway to enhance the optical image contrast of non-fluorescent samples and resolve axial structure [12,21]. Choi et al. reported on wide-field label-free imaging through MMF that avoids WFS and offers high imaging speed by employing galvanometer scanning mirrors and a turbid lens imaging algorithm [5]. We focus here on extending this prior work to include optical sectioning and phase information, leading to higher contrast images.

We present work that builds on the simplicity of WFS-free MMF imaging and incorporates 3D optical sectioning by introducing a new computational strategy that enables synthetic confocally gated imaging through MMF. After initial calibration of a MMF—a process that is common to all current MMF imaging strategies—to obtain its full forward transmission matrix (TM) by scanning a focal spot over the proximal side of the MMF, we measure the round-trip reflection matrix of the fiber including the sample at its distal end, using the same proximal spot basis. We then digitally compensate for the effect of forward and backward transmission through the MMF on the reflection matrix. Using numerical refocusing, we demonstrate how a single measurement of the reflection matrix may yield volumetric images of complex samples with versatile imaging contrast, including dark-field and birefringence imaging, achieved through TM manipulations.

 

Fig. 1. Overview of computational confocal imaging through MMF. (a) We calibrate the MMF by measuring ${\textbf{T}}$. In the imaging phase, we measure the coherent round-trip ${\textbf{M}}$ through the same set of proximal channels as used for calibrating ${\textbf{T}}$, but now in the presence of a distal object. The ${\textbf{H}}$ matrix accounts for free-space propagation from the calibration plane to an OP. The sample is illuminated by different speckle realizations due to the proximal 2D scan, and ${\textbf{R}}$ denotes light–sample interaction. The measured ${\textbf{M}}$ and ${\textbf{T}}$ are used for computational reconstruction. (b) By modeling the round-trip light transmission with matrix multiplications, we can compensate for MMF scrambling using the measured ${\textbf{T}}$ and modeled ${\textbf{H}}$ to isolate the reflection matrix ${\tilde{\textbf{R}}}$ at a certain depth. The image of the sample is reconstructed from the diagonal of $|{\tilde{\textbf{R}}}{|^2}$, corresponding to confocal illumination (red) and detection (gray) through all available channels. Note that the illumination and detection spots are overlapping on the sample. With numerical refocusing, we can generate a complete 3D scan from a single measured ${\textbf{M}}$. The color map encodes complex values. amp., amplitude. (c) $\gamma$ curves show the intensity reflection signal summed in the lateral directions as a function of physical distance $d$ from the fiber facet for the schematically depicted sample. Only the full fiber ${\rm NA} = 0.22$ can correctly resolve the three reflective interfaces. The en face images show confocal images of buccal epithelial cells deposited on the front and back surfaces of the coverslip, reconstructed with the full fiber NA. The scale bar is 50 µm.

Download Full Size | PDF

2. OVERVIEW

The monochromatic optical transmission through a general medium from an input surface to an output surface can be expressed by a coherent matrix, specifying the amplitude and phase evolution of the transmitted field between pairs of input and output spatial channels. As illustrated in Fig. 1(a), we express forward light transmission through a MMF as matrix ${\textbf{T}}$. The measurement of ${\textbf{T}}$, by determining the amplitude and phase of the output speckle pattern arising from focal illumination at each independent transverse location on the input fiber facet, calibrates the transmission from the proximal end (P) to a distal calibration plane (at $z = 0$). The MMF output speckle pattern on the calibration plane per each proximal input realization constitutes a column vector of ${\textbf{T}}$. In imaging mode with a sample at the fiber distal end, we illuminate and detect light from the fiber proximal end. This corresponds to a bi-directional light transport consisting of forward transmission through the MMF, free-space propagation to an observation plane (OP) modeled by Fresnel diffraction (${\textbf{H}}$), speckle illumination on and backscattering from the sample, coupling back into the same MMF, and backward transmission to the proximal facet. The TM representing backward transmission ${{\textbf{T}}^{\rm T}}$ is the transpose transformation of the forward TM ${\textbf{T}}$ due to underlying reciprocity [22], and the overall round-trip reflection matrix ${\textbf{M}}$, describing optical transmission from and to the proximal side can be expressed as

The computational reconstruction is conceptually illustrated in Fig. 1(b). With a previously measured ${\textbf{T}}$, we can digitally compensate for light scrambling during round-trip MMF propagation and isolate the scattering signal of the various sample layers, including propagation from and to the fiber facet. This is similar to modeling conventional confocal microscopy, with the fiber facet taking the place of the principal output plane of the objective lens. The ${\textbf{H}}$ matrix is independent of fiber shape and can be numerically generated for any desired OP position ${z_i}$ (see Supplement 1 S2). Right- and left-multiplying ${\textbf{M}}$ with the inverse and transpose inverse of ${\textbf{H}}({z_i}){\textbf{T}}$, respectively, extracts the in-focus signal from the layer at ${z_i}$. The signal from adjacent layers ${z_j} \ne {z_i}$ is suppressed with increasing layer separation owing to the confocal gating effect, which arises from the defocusing ${\textbf{H}}({z_j}){{\textbf{H}}^{- 1}}({z_i})$ acting independently on both the illumination and detection sides. For simplicity, we drop the subscript of ${z_i}$ and use $z$ to specify the OP position from here on.

In practice, the measured ${\textbf{T}}$ is generally non-square, ill posed, and noisy, so we used Tikhonov regularization to approximate the inversion and transpose inversion of the fiber transmission, ${{\textbf{T}}^{- {{1(\rm tik)}}}}$ and ${{\textbf{T}}^{- {{\rm T(\rm tik)}}}}$, with the regularization parameter set to 10% of the largest singular value as justified by the L-curve method [23]. The approximated sample signal ${\textbf{R}}(z)$, ${\tilde{\textbf{R}}}(z)$ thus can be derived as

3. METHODS

A. MMF Calibration and Sample Reflectance Measurement

All experiments used a 1-m-long step-index MMF with 105 µm core diameter and a NA of 0.22 (FG105LCA, Thorlabs) that theoretically supports $\sim550$ guided modes per polarization at $\lambda = 1550\; {\rm nm}$. The fiber was coiled with a minimum radius of curvature of $\sim50\; {\rm mm}$. The monochromatic calibration matrix T was measured by sequentially probing the MMF input channels with a focal spot, while holographic detection was used for all distal output channels concurrently. Each input and output channel included two orthogonal polarization states: horizontal (H) and vertical (V). The focal spot position on the proximal input side was indexed by $u$, and the speckle pattern exiting on the distal side was imaged with an off-axis holographic imaging system, whose object plane determined the calibration plane, approximately 100 µm away from the MMF distal facet. Output images were flattened into column vectors of ${\textbf{T}}$ directly in the Fourier domain. The input and output spatial channels of ${\textbf{T}}$ were ordered first by spatial coordinates, then by polarization.

In imaging experiments, we again sequentially coupled light into the MMF through the same set of proximal input states. We then recorded the round-trip light transmission on the proximal side by the same off-axis holography setup. To preserve the symmetry between the illumination and detection configurations and to obtain a square matrix ${\textbf{M}}$, we sampled the recorded complex output fields at the ordered positions identical to the set of input states. Numerical corrections to compensate for the physical misalignment, including field curvature, were then applied to the output channels of the measured ${\textbf{M}}$ to accurately match the input channels and recover the previously described transpose symmetry. The detailed experimental setup and data processing are explained in Supplement 1 S1.

B. Confocal Image Reconstruction

Using the sample measurement ${\textbf{M}}$, the pre-measured ${\textbf{T}}$, and H($z$), we then computed ${\tilde{\textbf{R}}}$ following Eq. (2). The free-space propagation matrix ${\textbf{H}}(z)$ from the calibration plane to a selected OP corresponds to a diagonal matrix in the Fourier domain, parameterized by optical path length [illustration in Fig. S1 (b-1), details in Supplement 1 S2). We converted from optical to physical distance by considering the sample medium’s refractive index, including for layered samples, without computing the accurate impulse response of a stratified medium. The input and output bases of ${\tilde{\textbf{R}}}$ were converted from the Fourier domain to real-space indexed distal channel $\nu$ by multiplication with a pre-computed inverse discrete Fourier transform matrix. A 2D confocal intensity image ${\textbf{I}}$ of sample reflectance at the OP was then reconstructed by reshaping the diagonal of ${\tilde{\textbf{R}}}$ as

C. Wide-Field Image Reconstruction

From the same measured ${\textbf{M}}$, we can also obtain wide-field imaging that is equivalent to the turbid lens imaging algorithm [24]. We compensated for the reverse MMF transmission of reflectance from the sample under the variety of speckle illuminations, and then incoherently averaged the reflectance to statistically compose a uniform illumination. In terms of matrix operations, we left-multiply Eq. (1) with ${{\textbf{H}}^{- {\rm T}}}{{\textbf{T}}^{- {{\rm T(\rm tik)}}}}$:

D. Quantitative Phase Imaging

In confocal imaging, due to the complex nature of ${\tilde{\textbf{R}}}$, quantitative phase imaging can be accomplished by taking the complex values of the diagonal elements of a computed ${\tilde{\textbf{R}}}$ to form a complex 2D image (${\textbf{X}}$):

E. Dark-Field Imaging

Off-diagonal elements of ${\tilde{\textbf{R}}}$ also contain abundant information of sample optical properties, which can be extracted through manipulations on ${\tilde{\textbf{R}}}$. For instance, each column of ${\tilde{\textbf{R}}}$ represents the scattering at OP in response to an illumination focused on a single channel $q$. Instead of collecting the intensity at the corresponding location on the matrix diagonal, the intensity of surrounding output channels $\nu$ was summed with weights $L(\nu, q)$ given by their Euclidean distance from the input channel on the ${xy}$ plane up to an empirical cutoff of $\sim2.83$ Airy disk radius (12 µm on the fiber facet). The on-diagonal confocal signal thus has a zero weight and minimal contribution. This sum was furthermore normalized by the overall intensity:

F. Polarization Contrast

So far, computation of images from ${\tilde{\textbf{R}}}$ have considered only co-polarized illumination and detection, where each distal spatial channel $\nu$ degenerates into ${\nu _H}$ and ${\nu _V}$, and entries corresponding to input and output channels were used with the same polarization state. For birefringent samples such as collagen, illumination through a channel in a certain polarization state may induce cross-polarized backscattering. While ${\tilde{\textbf{R}}}$ is symmetric and has inputs and outputs ordered first by coordinates and then by polarization, the diagonals of the two off-diagonal matrix quadrants represent cross-polarized detection, and the sample birefringence at individual image positions $(x,y)$ can be resolved and characterized by assembling 2-by-2 Jones matrices,

4. RESULTS

A. Depth-Gated Imaging through MMF without WFS

To demonstrate the depth gating effect of our computational reconstruction, we imaged a 3D sample through the MMF, as shown in Fig. 1(c). The sample is a coverslip in air with buccal epithelial cells deposited on both surfaces. We computed the confocal image for each OP at varying distance $d$ from the MMF distal facet ($d = 0$) and calculated the corresponding integrated reflectivity ($\gamma$) by summing the intensity over the entire en face image. The $\gamma$ versus depth profile reveals three separated peaks (blue curve), which inform on the position of the reflective MMF facet and coverslip surfaces, with their axial positions at $d = 0$, 120, and 320 µm considering the medium’s refractive index. To evidence the confocal gating mechanism at play, we computed the $\gamma$ profiles corresponding to a reduced fiber NA (0.14, 0.1, and 0.05) by truncating the spatial frequencies of ${\textbf{T}}$. The axial signal peaks blur with reduced fiber NA and fail to resolve the individual sample interfaces, confirming the expected strong dependence of axial rejection power on the NA. The green and brown insets show high-contrast images of cells on the front ($d = 120\,\unicode{x00B5}{\rm m}$) and back ($d = 320\,\unicode{x00B5}{\rm m}$) surfaces of the coverslip, respectively, using the full fiber NA. Our matrix approach, which achieves confocal gating with numerical refocusing, thus enables 3D imaging from a single measured ${\textbf{M}}$ without WFS. For more details and additional results of this experiment, please see Supplement 1 S6.

To further evaluate the confocal gating effect, we imaged a USAF resolution chart (R1D21P, Thorlabs) in air or intralipid and distances $d$ through the MMF, as sketched in Fig. 2(a). In each medium, a sample reflectance matrix ${\tilde{\textbf{R}}}$ was computed from a single measured ${\textbf{M}}$ for each OP at varying depths, and processed to reconstruct confocal and wide-field images for direct comparison, as shown in Fig. 2(b). In each imaging condition and modality, we calculated the corresponding $\gamma$ profile, which is normalized by the highest value along the axial OP positions. The $\gamma$ profile allows us to find the in-focus position. Since the chart has a binary reflectance pattern across its surface, we can quantify the intensity image contrast as

 

Fig. 2. Depth gating of computational confocal imaging. (a) Imaging geometry: a resolution chart was imaged a distance $d$ in front of the distal MMF facet. (b) From the computed sample reflectivity matrix ${\tilde{\textbf{R}}}$, pixels of a confocal image were obtained by taking the intensity of diagonal elements, and of a wide-field image by taking the energy of row vectors for all available distal channels, visualized by the yellow circle and the brown rectangular box, respectively, for a single channel. Images at multiple depths were computed from the same measured ${\textbf{M}}$ with numerical refocusing. The image diagrams show the fundamental difference in illumination (red) and detection (gray) patterns of computational image formation. c, confocal; wf, wide-field. (c), (d) Confocal images show high, axially localized contrast compared to wide-field images, in both (c) air (at $d = 120\,\unicode{x00B5}{\rm m}$) and (d) intralipid media (at $d = 400\,\unicode{x00B5}{\rm m}$). Images show logarithm of normalized intensity. The solid line (dashed line) boxes indicate chrome pattern (glass substrate) areas for image contrast quantification. (e), (f) Normalized integrated reflectivity plots of confocal (blue) and wide-field (orange) images in (e) air and (f) intralipid media. The scale bars are 50 µm.

Download Full Size | PDF

We first demonstrated optical sectioning by imaging the chart placed at $d = 120\,\unicode{x00B5}{\rm m}$ in air. In Fig. 2(c), the confocal method renders the chart patterns with a high contrast of 0.9 due to the rejection of background signals from reflection at the MMF facet. The value is close to the expectation $\sim0.92$, assuming full reflection from the chrome pattern and 4% reflection at the air–glass interface. In Fig. 2(e), the profile reveals two prominent and separated peaks at $d = 0$ and $d = 120\,\unicode{x00B5}{\rm m}$, corresponding to the MMF facet and the resolution chart, respectively.

To test the capacity of computational confocal gating in the presence of additional sample scattering, we imaged the chart placed at $d = 400\,\unicode{x00B5}{\rm m}$ in agarose gel mixed with ${2}\,{\rm wt}.\%$ intralipid, which corresponds to $\sim0.36$ mean free paths. In Fig. 2(d), the reduced confocal image quality may be due to: intralipid scattering that distorted the wavefront and the reconstructed images, and degraded spatial resolution upon beam divergence at large $d$ (elaborated in the following section). Despite the scattering medium and lower signal to background ratio when the chart is far from the facet, the confocal image maintained a high contrast of 0.96, compared to a theoretical value of $\sim0.99$ (assuming 0.4% reflection at the gel–glass interface). In Fig. 2(f), the peak in the $\gamma$ profile at $d = 400\,\unicode{x00B5}{\rm m}$ corresponds to the chart, and, although weak, precisely informs on its physical location when assuming the medium’s refractive index to be 1.4. These results evidence the effective suppression of out-of-focus scattering and reflection signal without active WFS.

To theoretically compare the confocal method to wide-field processing, one can juxtapose Eqs. (4) and (2) to find that the matrix multiplication also on the right side of ${\textbf{M}}$ pre-compensates for the light scrambling effect of the MMF forward transmission, and synthesizes sharp foci through the MMF on a selected OP. In contrast, the wide-field processing of ${\tilde{\textbf{R}}}$ corresponds to speckle illumination, as illustrated in Fig. 2(b). When imaging in air as in Fig. 2(c), while the pattern with wide-field imaging stands out from the background on the OP at $d = 120\,\unicode{x00B5}{\rm m}$, the strong background reflection from the facet results in a low contrast of 0.48, and the corresponding $\gamma$ profile in Fig. 2(e) remains constant throughout the entire observation range. When imaging through intralipid, the wide-field image in Fig. 2(d) can barely distinguish the pattern from the background, resulting in poor contrast of 0.11. The uniform $\gamma$ plot of wide-field imaging in Fig. 2(f) again exposes the lack of optical sectioning.

B. Efficient Imaging with Flexible Reconstruction, Field of View, and Spatial Resolution

Imaging through a MMF using WFS typically scans a focus along a pre-defined scanning trace and records a single image point from each focus location. In contrast, our method illuminates the sample with a sequence of MMF-induced speckle patterns and utilizes the camera for parallel sampling of all addressable locations in the imaging volume. This allows an arbitrary definition of the sampling grid and working distance in post processing of a single measurement of ${\textbf{M}}$ (elaborated in Supplement 1 S3). As light diverges upon exiting the MMF distal end governed by the fiber NA, computational reconstruction can adapt to a growing field of view (FOV) with increasing OP distance from the fiber facet. Here, we demonstrate this flexible reconstruction in MMF reflectance imaging and evaluate the resulting FOV and 3D spatial resolution as a function of distance from the tip of the fiber.

To mimic endoscopic imaging with a variable working distance, a resolution chart was mounted on a translation stage and positioned at different distances $d = 10$, 600, or 1200 µm. For each $d$, a round-trip ${\textbf{M}}$ was measured, and computational reconstruction with numerical refocusing was utilized to locate the axial position of the resolution chart. Figures 3(a)–3(c) show the in-focus confocal images of different chart areas [color boxes in (d)] with physical dimensions of 123, 184, and 247 µm, when $d = 10,600$, and 1200 µm, respectively. The illumination power on the chart was kept at $\sim0.5\; {\rm mW}$. Due to the beam divergence and limited laser power, camera exposure time was increased from 200 µs up to 1 ms for larger $d$ to compensate for the declining photon collection. We filled the space between the fiber and the chart with index-matching gel (G608N3, Thorlabs) to mitigate the specular reflection from the MMF distal facet. In Figs. 3(a)–3(c), imaging from farther away captures a more complete picture, as the FOV expands with increasing $d$. However, this comes at the expense of spatial resolution and collected reflectance power, as the patterns are severely blurred at $d = 1200\,\unicode{x00B5}{\rm m}$, and background speckle becomes apparent. The finest detail of the chart, element 6 in group 7, can be resolved when the MMF is in close proximity of the facet, $d = 10\,\unicode{x00B5}{\rm m}$, where the FOV is determined on a lower bound by the fiber core size.

 

Fig. 3. Characterization of FOV and resolution of computational MMF imaging. (a)–(c) In-focus confocal intensity images of a USAF target at a distance (a) $d = 10$, (b) $d = 600$, and (c) $d = 1200\,\unicode{x00B5}{\rm m}$ away from the fiber facet. The green, brown, and purple boxes correspond to the ones in (d) that highlight the areas of the target. At $d = 1200\,\unicode{x00B5}{\rm m}$, the speckle background reduces the image quality, and the dynamic range of the image was increased to reveal weak signals. (e) Analysis of imaging performance at $d = 10\,\unicode{x00B5}{\rm m}$ distance. The blue plot shows the normalized intensity profile in linear scale along the dashed line in the image in (a); the red plot indicates $\gamma$ at varying $d$, and its FWHM estimates the axial resolution, with the shadowed areas indicating the standard deviations around the mean value of several independent realizations; the black curve shows the radial mean intensity in logarithmic scale, with 1% cutoff at dashed line roughly equal to the fiber radius. (f) Experimental 3D resolution and FOV at various $d$ and corresponding theoretical values. The scale bars are 100 µm.

Download Full Size | PDF

To quantify the spatial resolution at varying $d$, we inferred the lateral resolution, $\delta x$, from the smallest resolvable pattern on the chart, as shown by example by the dashed blue line and its linear intensity profile plot in Fig. 3(e). Also, since the chart serves as a sharp edge in the axial direction, we utilized the FWHM of the $\gamma$ profile around the reflectance peak to measure the axial resolution $\delta z$. The FOV of each computed image was characterized by its diameter $\emptyset$, set as twice the radius where the radially averaged image intensity dropped below 1% of the center. We tested several imaging realizations and computed corresponding axial profiles and radial mean intensities for statistical analysis, as plotted in red and black curves in 3(e), respectively, for $d = 10\,\unicode{x00B5}{\rm m}$. For comparison, the theoretically expected spatial resolution was derived considering the effective on-axis NA defined by the minimum between the fiber NA and the solid angle subtended by the fiber core at the corresponding distance from the facet. The theoretical FOV was estimated using ${\textbf{T}}$ by measuring the radial extent of averaged synthetic illumination patterns away from the facet (further details on theoretical resolution and FOV are presented in Supplement 1 S4). The overall quantification results are shown in Fig. 3(f). While the experimental spatial resolution is consistent with diffraction-limited theoretical values, the experimentally determined FOVs are up to 50% smaller than expected at increasing distance. This may be due to the low light collection efficiency of reflectance from distant planar objects. Nevertheless, the agreement in scaling properties of experimental and theoretical values corroborates the flexibility in addressable spatial dimensions given by the degrees of freedom guided through the MMF. These results demonstrate the convenience of reconstructing the entire sample volume without a pre-defined scan pattern in a practical setting where the sample distance is unknown. Moreover, this flexibility of the matrix approach allows confocal image reconstruction from partial measurements of ${\textbf{M}}$ with illumination through only a subset of proximal spatial channels (as shown in Supplement 1 S5). While reconstruction from partial measurements compromises background suppression, it accelerates the volume rate, which may be critical for real-time applications.

C. Multimodal Computational Confocal Imaging of Label-Free Complex Samples

To improve specificity in confocal MMF imaging without WFS for visualizing unlabeled biological specimens in reflection mode, we leveraged the matrix approach to generate diverse contrasts from a measured round-trip M by applying different post-processing, and synthesized multiple imaging modalities to create signal specificity. Different strategies for image formation are described in Section 3 (Methods) and illustrated in the following figures. The multi-contrast images in addition to the intensity images of the 3D imaging example in Fig. 1(c) can be found in Supplement 1 S6.

Figure 4(a) shows a typical reflection matrix ${\tilde{\textbf{R}}}$. The sample arrangement for these experiments, shown in Fig. 4(b), allowed imaging of a sample on a microscope glass slide in reflection mode through the MMF, and also in transmission mode (t) with the distal imaging system and bright-field illumination through the MMF as ground truth images. Figure 4(c) shows the images of a monolayer of 3 µm polystyrene beads spread on the surface of a microscope slide and imaged in air at $d = 100\,\unicode{x00B5}{\rm m}$. Since the reflectivity of the beads is orders of magnitude lower than that of the air–glass interface, the obtained round-trip ${\tilde{\textbf{R}}}$ at in-focus OP has diagonal elements dominated by the specular reflection from the glass slide, resulting in beads silhouetted against the glass signal in the confocal intensity image (${\textbf{I}}$) and featuring negative contrast, similar to other reports of reflectance imaging through optical fibers [12,24,26]. With the full knowledge of ${\tilde{\textbf{R}}}$ and following Eq. (7), we are able to extract scattering signal specifically from the beads and create a dark-field-like image (${\textbf{S}}$) through numerical engineering of the system point spread function (PSF). Physically, forward scattering by the beads followed by reflection at the glass interface created a multiply scattered signal that partially cross-coupled to neighboring spatial channels. Intriguingly, the cross-coupling signals that delineate the beads provided a slightly higher resolving power than the confocal intensity image, as verified by comparing the line profiles of clustered particles. This exemplifies the benefit of computational reconstruction, whereas physical implementation of dark-field imaging would traditionally require an annular filter, axicon lens, or customized pinhole, and increase the system complexity [11,25,27].

 

Fig. 4. Multimodal MMF imaging of unlabeled samples including confocal intensity, quantitative phase, and dark-field scattering imaging. (a) Diagonal elements of ${\tilde{\textbf{R}}}$, corresponding to pixels in confocal intensity (${\textbf{I}}$) and complex (${\textbf{X}}$) images. PSF engineering by an appropriate weighting function applied using Eq. (7) generates scattering contrast (${\textbf{S}}$). (b) Sample arrangement at the MMF distal end. Transmission images (t) serve as ground truth for verification. (c) Confocal intensity and scattering contrast images of 3 µm polystyrene beads. The ring-shaped detection PSF is shown in the inset. Imaging based on scattering contrast features slightly better resolving power, judging from the plotted line profiles across a cluster of beads. (d) Stitched images of buccal epithelial cells with phase and dark-field contrasts. (left) Phase contrast depicts nuclei and intracellular morphology. (right) Dark-field scattering contrast reveals positive signals at cell boundaries and membrane roughness. The scale bars are 50 µm.

Download Full Size | PDF

To demonstrate multimodal MMF imaging including phase and dark-field imaging from the same measurement of an unlabeled biological specimen, human buccal epithelial cells were smeared on a microscope glass slide and placed at $d = 120\,\unicode{x00B5}{\rm m}$ in air. The sample was laterally translated to image several overlapping areas, and at each lateral location, a round-trip ${\textbf{M}}$ was measured to reconstruct the corresponding image. Multiple images were then stitched together to make a composite image with a wider FOV. Figure 4(d) shows phase contrast (left, ${\textbf{X}}$), revealing the contour of cellular membranes and nuclei in its amplitude (coded in brightness), likely because they deflect the focused illumination, which attenuates the reflected signals, thereby resulting in negative contrast. Furthermore, as shown by its color-coded phase, the variation in sample thickness or refractive index inhomogeneity provides an intrinsic phase contrast of the unlabeled sample likely caused by sub-cellular structures, revealing information not contained in the intensity image alone. The scattering contrast image (right, ${\textbf{S}}$) delivers complementary information, likely arising from multiple scattering by the sample and coverslip, and positively outlines the cellular membrane morphology along with some cytoplasmic organelles that can be roughly correlated with the transmission image.

To demonstrate polarization sensitive computational imaging through MMF based on our matrix approach and Eq. (8), as illustrated in Fig. 5(a), we obtained reflection matrices of anisotropic materials including a quarter-wave plate (QWP) and cholesterol crystals through the MMF. To validate quantitative retardation and OA measurements, a QWP (WPQ501, Thorlabs) was placed on a microscope slide, of which the proximal reflection was measured either through the wave plate or adjacent to it. As shown in Fig. 5(b), the edge of the wave plate was imaged in different orientations to verify the OA orientation retrieved from polarization analysis. One ${\textbf{M}}$ was measured for each orientation. Due to the round-trip light propagation, the QWP has an effective half-wave retardance, which leads to full attenuation in the co-polarized detection for confocal intensity images when the slow axis is 45° to the H or V polarizations, and partial attenuation in between. Consistently, the corresponding retardance images reveal a constant $\pi\; {\rm rad}$ retardance of the wave plate regardless of the orientation. On the other hand, the color-coded OA images (combined with brightness-coded retardance images) show a rotating OA of the QWP with an orientation matching the set slow axis angle. Note that the OA colormap has a periodicity of $\pi$ instead of $2\pi$ used in phase colormaps. Figure 5(c) shows another example with a homemade plate-like cholesterol crystal (S25677, Fisher Science Education) on a microscope slide, which has a much weaker retardance due to its small thickness (tens of µm) yet uniform OA orientation. Judging from the values of retardation, the crystal may be thicker towards the bottom of the image. Since the crystal thickness is smaller than the confocal gate, interference between the front and back surfaces results in en face fringes. A tighter confocal gate may be achieved by switching to MMF with higher NA or choosing a shorter operating wavelength.

 

Fig. 5. Birefringence imaging of anisotropic samples through the MMF by utilizing reflection matrices. (a) Illustration of processing for computational polarization sensitivity. The full reflection matrix ${\tilde{\textbf{R}}}$ at an OP provides both co- and cross-polarized entries for assembling a Jones matrix at each spatial channel, which informs on retardation and OA orientation. (b) Accuracy evaluation of anisotropy reconstruction using a QWP with slow axis (white arrows) oriented at 45°, 90°, and 135° with respect to the $ x $ axis. While the confocal intensity images show only the sharp edge of the plate, the retardation and OA images unveil its intrinsic optical properties. (c) Retardance imaging through the MMF of a cholesterol crystal precipitated on a microscope slide. The visible fringes are attributed to thin film interference between the crystal and glass surfaces. The scale bars are 50 µm.

Download Full Size | PDF

5. DISCUSSION

Computational confocal imaging through MMF is a novel matrix-based method to obtain depth-gated images using a proximal scanning spot basis for reflectance measurement without WFS, yielding multimodal 3D reflectance of unlabeled samples including confocal intensity, quantitative phase, dark-field, retardance, and OA orientation contrast modalities. Pushing the frontier of MMF imaging techniques, this is the first report of numerical PSF engineering, phase, and polarization-resolved imaging through MMF in a reflection geometry.

High-contrast imaging through MMF frequently relies on fluorescent labeling or is operated in a transmission regime [2,13,15,17,28], which may be incompatible with practical endoscopic applications. Fluorescence scanning microendoscopy furthermore has potential photobleaching issues [29,30]. Our computational imaging approach instead efficiently extracts weak elastic scattering signals from unstained samples and operates in a reflection regime, making it favorable for practical endoscopic applications. The matrix approach moreover offers an elegant way of achieving full polarization management and leverages polarization as an additional contrast mechanism. In comparison, WFS for physical focusing through MMF typically addresses only a single polarization state [2,12], to avoid a complicated hardware setup required for full polarization-control [13]. While inspired by the arbitrary PSF engineering through complex media in a transmission regime [31], our method here does not need a spatial light modulator, digital micro-mirror device or optimization. More broadly speaking, illumination and detection with any respective PSF and in any polarization state can be readily engineered by weighting the entries of ${\tilde{\textbf{R}}}$ accordingly. Access to the reflection matrix may offer the ability to optimize image contrast for a given sample through post-processing, to synthetically adjust the confocal pinhole size, or to correct for sample-induced aberrations.

The matrix approach employs a simple proximal spot basis for illumination, which relaxes hardware requirements by accepting any 2D scanning module without using WFS. The limiting factor in imaging speed of this work is the InGaAs-camera frame rate of 120 Hz, which may be directly improved by an order of magnitude by replacing it with a faster one or by shifting the operation wavelength towards visible wavelengths with more and even faster camera options. Recently, MMF calibration covering 256 degrees of freedom within only 34 ms has been demonstrated by using a field programmable gate array (FPGA) to address the general latency issue in hardware interfacing and communication [32]. This work exemplifies the potential of software system optimization and is readily applicable to speed up our implementation. Our approach also allows image formation from a partial round-trip measurement with as few as 200 input realizations, offering an attractive trade-off between image quality and measurement time. Because the reconstruction of individual spatial channels is independent of other channels, this offers a high potential for parallelization of the processing using GPU acceleration. With careful engineering of the data acquisition and processing pipeline, fast video-rate imaging should be achievable. Fundamentally, the imaging speed of our MMF imaging method is limited by computational complexity, and no longer by hardware as for WFS methods.

MMF imaging has a notorious intolerance to small fiber perturbations such as bending or looping. In Eq. (1), while ${\textbf{H}}$ and ${\textbf{R}}$ are independent of fiber shape, the round-trip matrix ${\textbf{M}}$ changes with fiber transmission ${\textbf{T}}$. Even small fiber alterations result in distinctive ${\textbf{T}}$ and typically require MMF re-calibration, which is exceedingly difficult in an endoscopic setting [22]. This admittedly remains the most fundamental limitation towards practical use of flexible MMF endoscopy. In our experiments, the calibrated 1-m-long MMF was looped and fixed on the optical table and remained stable for several hours without a perceivable TM change such that the same measured TM could be used for imaging different samples through the MMF. In a practical setting, a segment of MMF could be mechanically shielded inside a rigid needle or hypodermic tubing to enable high-quality imaging through MMF without re-calibration. Furthermore, several promising strategies are being pursued to address the need for TM calibration without physical access to the distal fiber end: the installation of carefully designed passive optics or a guide star at the MMF distal tip [6,8,33], compressive sampling of TM with a sparsity constraint [34], or the use of graded-index MMF, which has increased robustness of light transport to bending deformations [35].

The disclosed method offers 3D confocal imaging through MMF with high signal specificity, yet is less hardware-demanding than common WFS methods. Moving forward, we may use MMF with higher NA to achieve better optical sectioning and resolve finer axial structure. Imaging weakly scattering samples may also require suppression of specular reflection from both the proximal and distal fiber facets, e.g., through the use of anti-reflection coating or other engineering solutions, to favor detection of the weak sample signal. Even with the present limitation that the MMF must be mechanically stable, our approach may expedite or create applications of minimally invasive MMF-based endoscopy in biomedicine, where probe size and cost are critical factors. For instance, deep brain imaging in neurosurgery, in situ inspection in needle biopsy, collagen imaging in arthroscopy, and tympanic cavity imaging in middle and inner ear surgery are potential uses of MMF endoscopy. The same methodology may also be extended to optical imaging through other complex or turbid media, or other imaging technologies such as ultrasound tomography.

6. CONCLUSION

Accurate knowledge of light propagation through MMF can transform this low-cost optical component into a high-throughput and ultra-thin optical conduit for measuring elastic optical scattering by remote samples in a reflection regime. The demonstrated computational imaging through MMF based on round-trip measurements in a proximal spot-basis may prove to be broadly applicable and clinically viable since it avoids the requirement for WFS and the use of fluorescent labeling. Our approach may streamline the system design and, in combination with future progress in calibration stability of MMF, stimulate the advent of hair-thin imaging probes that improve diagnostic performance, enhance guidance of existing interventions, and enable novel image-guided therapeutic procedures in clinical medicine.