1. Introduction

Optical microscopy is an essential tool in various fields of science and technology. For a long time, far-field optical imaging was held back by the Abbe limit: the resolution is confined by $\lambda /(2$NA$)$, where $\lambda$ is the wavelength of light and NA is the numerical aperture of the optical system [1]. The diffraction limit has been bypassed by the help of fluorescent molecules [2]. Many biological discoveries would not have been possible without super-resolution techniques, such as stimulated emission-depletion microscopy (STED) [3], photoactivation localization microscopy (PALM) [4] and stochastic optical reconstruction microscopy (STORM) [5].

Nowadays, optical nanoscopy is widely used worldwide. However, to significantly (more than twice) overcome the diffraction limit, a specific fluorescent labeling is required [6]. Moreover, the acquisition speed of super-resolution microscopy is rather slow. For a 40 $\times$ 40 $\mu$m$^{2}$ image, the acquisition time varies from several seconds per frame for STED and confocal microscopies to several minutes per frame for single-molecule localization methods [7,8]. Fluorescent labelling is not always possible, because it can change the sample properties or even have a destructive influence. The important application area where additional fluorescent markers are not desirable includes wafer defect inspection. Microcracks reduce the fracture strength of Si wafers and lead to wafer breakage during manufacturing, which reduces manufacturing yield. The microcracks can be detected by near-infrared-bright-field imaging [9]. However near-infrared cameras are costly and require temperature stabilization [10]. There is still a high demand for a fast super-resolution technique that does not rely on fluorescent labeling.

The endoscopic probe is often put inside the region of interest to overcome the limitations posed by scattering of light in the tissues [11]. To make the imaging process minimally invasive, the probe should be as compact as possible. Multimode fibers (MMFs) – miniature flexible optical waveguides that can carry thousands of modes – have a massive potential in endoscopic imaging [12]. Recently, compressive imaging through a MMF has been introduced [13]. Multiple speckle patterns generated in a MMF are projected on the sample and a bucket detector records the fluorescence intensity. The sparsity constraint allows the sample to be computationally reconstructed from a small set of speckle patterns combined with the fluorescent signal [14]. The robustness of the approach to fiber bending was demonstrated [15]. The method can be merged with acoustic imaging to successfully produce a hybrid technique that permits imaging with two contrast mechanisms of fluorescence and nonradiative absorption [16].

Here we demonstrate the new method of label-free MMF-based compressive imaging (MMFCI). The advantages of MMFCI over ghost imaging and conventional raster-scan microscopy has been demonstrated by imaging with a spatial resolution 2.5 times better than the diffraction limit at a speed of 5 fps (for $400 \times 400$-pixel frame) without any fluorescent labeling. Imaging speed up to 250 fps can be easily achieved by implementing fast beam scanning system, such as acousto-optic deflectors. The MMFCI approach is different from traditional methods of MMF imaging, where fields on the fiber output facet are pre-optimized to take the form of diffraction-limited foci and the number of single-pixel measurements is equal to the number of pixels in the final image. In contrast, in MMFCI, the computationally reconstructed image of a sample contains much more pixels than the number of single-pixel measurements.

Fiber-format and demonstrated long-term stability of our approach preface rapid, sub-wavelength endo-microscopy in unlabelled live specimens.

2. Materials and methods

2.1 MMF-based compressive imaging

In MMFCI, the process of projecting different speckle patterns and measuring the transmitted intensity can be formulated as the following linear problem

 

Fig. 1. (a) Schematic image of the experimental setup. MMF, multimode fiber; BS, beamsplitter; APD, avalanche photodiode, HWP, half-waveplate, PBS, polarizing beamsplitter. The sample was illuminated with various speckle patterns and the corresponding transmitted intensity was recorded by the avalanche photodiode. The camera records the illumination speckle patterns. (b) Schematic representation of multimode-fiber-based compressive imaging. Measured speckle patterns form measurement matrix A, recorded transmitted intensity forms measurement vector y. The number of measurement is equal to the number of projected speckle patterns.The sample image is computationally reconstructed by solving the problem Ax=y+n, where x is a sample image flattened into 1D vector,n is a noise in a measurement system. (c) Geometrical parameters of the samples normalized by the $54\times$ magnified diffraction limit of the MMF d$_{\textit {diffr}}$. All sizes are in normalized units

Download Full Size | PDF

The problem is ill-posed: in the case of $M\ll N^{2}$, there are multiple $\textbf {x}$ that satisfy Eq. (1). However a prior information about $\textbf {x}$ can be used to solve Eq. (1) exactly [17]. Compressive sensing uses a sparsity constraint – the knowledge that in some domain $\textbf {x}$ contains a lot of zeros. Sparsity helps to solve the problem by:

A gradient projection for sparse reconstruction (GPSR) algorithm solves Eq. (3) by the quadratic reformulation [18]. The advantage is that the $l_{1}$ norm is replaced by a linear function (functional), making the objective function smooth. Therefore the gradient method of minimization can be applied. For multimode-fiber-based imaging, we used the GPSR algorithm MATLAB code developed for compressed sensing and other inverse problems in signal processing and statistics [19]. Reconstructed vector $\textbf {x}$ ($N^{2}\times 1$) was transformed into $N\times N$ sample image $O(x,y)$.

2.2 Ghost imaging

Ghost imaging uses the same measurement procedure as described by Eq. (1): different speckle patterns are projected on the sample, and the transmitted intensity is recorded. Sample image $O(x,y)$ is obtained by correlating the transmitted intensity with the corresponding illumination pattern intensity distribution [20]:

Signal-to-noise ratio (SNR) of ghost imaging scales as the square root of the number of realizations, and in practise $M\,\gg \,N^{2}$ patterns are required to achieve SNR$\,\gg \,$1 [20]. The spatial resolution of the system is bound by the diffraction limit [21]. For reconstruction by ghost imaging, we used the same experimental data as for MMFCI.

2.3 Diffraction-limited raster-scan imaging

Conventional raster-scan imaging was simulated by taking into account the theoretical diffraction limit of the MMF and the known geometrical dimensions of the samples. To obtain $O(x,y)$, ground truth image $S(x,y)$ was filtered in Fourier domain with the 2D incoherent optical transfer function [1]:

2.4 Experimental setup

The experimental setup is presented in Fig. 1(a). The laser beam (532$\,$nm, Cobolt Samba, continuous wave) was expanded and then reflected by the galvo-mirrors. The system of relay lenses projected two galvo-mirrors (GVS411/M, Thorlabs) on the entrance aperture of the objective (NA$\,$=$\,$0.65, Olympus). The objective focused the beam on the input facet of the MMF (NA$\,$=$\,$0.22, diameter, $d\,$=$\,$50 $\mu$m, FG050UGA, Thorlabs). By tilting the mirrors, the beam was moved in the focal plane of the objective and different sets of modes have been excited producing different intensity patterns on the MMF output facet. The magnified image of the output facet was projected on the sample and the camera (pixel size$\,$=$\,$4.8 $\mu$m, Basler acA 800-510 um) by the objective (NA$\,$=$\,$0.75, Olympus), the tube lens (ACT508-250-A-ML, $f$ = 250 mm, Thorlabs) and the beamsplitter (BS025, Thorlabs). We measured the magnification of the system, MG = 54 by imaging of $1951$ USAF resolution test chart (R3L3S1N, Thorlabs). The cut-off frequency of the magnified speckle pattern was calculated as $\nu _{0} = 2\text {NA}/(\lambda \cdot \text {MG})$. The total intensity transmitted through the sample was measured by the avalanche photodiode (Thorlabs APD410A2). The incident laser power was adjusted by the half-waveplate and the polarizing beamsplitter cube (CCM1-PBS251/M, Thorlabs) to be 2$\,$-$\,$200 $\mu$W. The photodiode and the camera were synchronized.

In general terms, the resolution can be defined as the minimum separation distance at which two objects can be sufficiently distinguished. As a sample we used round defects with a known size at a known distance from each other (as shown in Fig. 1(c)). The samples were specifically designed for these experiments and made in the cleanroom environment using CMOS-compatible technologies and standard photolithography procedures without any fluorescent labelling. Maskless ultraviolet photolithography, followed by lift-off of sputtered aluminum patterns has been used to fabricate a sample that represents a simple model of the defects on a wafer surface or a microcrack in a silicon solar cell. Samples were accurately characterized by a high-NA optical microscope. The results are summarized in the table in Fig. 1(c). The geometrical parameters of the samples are normalized by the $54\times$ magnified diffraction limit of the MMF. Each sample was placed between the beam splitter and avalanche photodiode in the image plane of the tube lens. Samples were illuminated with the magnified image of the output facet of the MMF. The same image was recorded by the camera.

3. Results

3.1 Speckle-based imaging system

The quality of the reconstruction in compressive sensing largely depends on the properties of matrix $\textbf {A}$: it is desirable to have a linearly independent set of speckle patterns. Otherwise, some measurements do not add any new information. The absence of correlation between vectors guarantees their linear independence [22]. Here we analyse how the correlation between speckle patterns and their linear independence can be controlled by the scanning strategy (number of points and the distance between points) for a square scanning grid.

Optimal compressive imaging in a diffraction limited system requires the number of measurements to be less or equal to the number of modes, $M\leq M_{modes}$ [23]. We estimated the number of modes by the formula for an ideal cylindrical optical waveguide [24]:

An uncorrelated set of speckle patterns requires an optimal scanning grid. We consider a coupling coefficient between the $i$-th linearly polarized input field $E^{\text {in}}_i(x,y)$ and the $j$-th mode $E_{j}(x,y)$ as an overlap integral $\alpha _{ij}=\iint E^{\text {in}}_i(x,y)E^{*}_{j}(x,y) dx dy$. Since the NA of the coupling objective is greater than the NA of the fiber, we can approximate the input field as a delta function: $E^{\text {in}}_i$ $=\delta (x-x_{0i})\delta (y-y_{0i})$ and therefore $\alpha _{ij}=E^{*}_{j}(x_{0i},y_{0i})$, where $x_{0i}$ and $y_{0i}$ are the center coordinates of the $i$-th input beam. If the two input beams are closer than the diffraction limit of the fiber, their coupling coefficients are approximately the same. As a result, they excite two similar sets of fiber modes and the produced output speckle patterns are linearly dependent and correlated. The distance between input scanning points should be not smaller than the diffraction limit of the MMF. However, we can also excite another set of linearly independent speckle patterns by using orthogonal polarization on the input [25].

In our experiments, the distance between adjacent points was set to 1.14 $\mu$m that is equal to the diffraction limit of the MMF. The illumination patterns were formed by scanning the focused beam over the input facet on a 31 $\times$ 31 points equidistant square grid with a size of 35 $\times$ 35 $\mu$m (Fig. 2(a)). The total number of incident speckle patterns was $M$ = $961$. The typical speckle pattern produced by the MMF is presented in Fig. 2(c).

 

Fig. 2. (a) Schematic image of the input MMF facet with the scanning grid. (b) Experimentally measured PS (orange dots), background PS (blue line), and theoretical PS (green line) of the speckle patterns along the dashed line shown in (d). The cut-off frequency is marked by the black dotted line. The speckle patterns are diffraction-limited. (c),(d) Typical intensity distribution (c) and mean power spectrum density PS$_{\text {exp}}$ (d) of $54\times$ magnified speckle patterns. (e) Probability density function of speckle correlation coefficients r$_{\text {ij}}$ (blue dots) and the Gaussian fit (orange line). The low mean value of r$_{\text {ij}}$ guarantees that the speckle patterns are uncorrelated. (f) Distribution of singular values for measurement matrix A

Download Full Size | PDF

In the first set of experiments, we characterized the measured speckle patterns, $I_{i}$($x$, $y$). We calculated the mean power spectrum density (PS), as PS$_{\text {exp}}$($\nu _{x}',\nu _{y}')=<|\mathscr {F}(I_{i}(x,y)|^{2}>$, where $\nu _{x}'=\nu _{x}/\nu _{0},\nu _{y}'=\nu _{y}/\nu _{0}$ are the normalized spatial frequencies. The theoretical PS is [26,27]:

Complicated effects of light propagation in MMF lead to the existence of a correlation between incident and transmitted fields or optical memory effects [28]. For example, in MMF angular memory effect, the rotation of the input field around the fiber axis results in the rotation of the output field by the same angle. These effects were not taken into account in our simple analysis of scanning strategy optimization and can lead to unexpected correlations of speckle patterns. To ensure that experimentally measured patterns are both linear independent and uncorrelated we calculated the probability distribution of correlation coefficients and singular values. To get rid of high-frequency noise in measured speckle patterns, we filtered the recorded images in the Fourier domain: $I'(x,y) = |\mathscr {F}^{-1}\left (\mathscr {F}\left (I_i(x,y)\right )\cdot H(\nu _{x},\nu _{y})\right )|$, where $H(\nu _{x},\nu _{y})~=~1$, if $\sqrt {\nu _{x}^{2}+\nu _{y}^{2}}~\leq ~\nu _{0}$ and $H(\nu _{x},\nu _{y})~=~0$, if $\sqrt {\nu _{x}^{2}+\nu _{y}^{2}}~>~\nu _{0}$. We characterized the correlation between different speckle filtered patterns by the Pearsson correlation coefficient:

We studied the linear independence of the measured speckle patterns. We reshaped each $I'_i(x,y)$ into 1D vectors $(1\times N^{2})$ and formed matrix $\textbf {A}$ $(M\times N^{2})$. To calculate the rank of matrix $\textbf {A}$, we performed singular value decomposition. All the singular values are far from zero ranging from $1.5\times 10^{3}$ to $2.5\times 10^{5}$ with a well-pronounced peak around 3000 (Fig. 2(e)). It indicates that there are no values that can be neglected. The rank of matrix $\textbf {A}$ coincides with the number of measured speckle patterns meaning that they are linearly independent. Therefore, we ensured that the diffraction-limited speckle patterns are optimal for MMFCI.

3.2 Fiber-based label-free super-resolution imaging

In the second set of measurements, we experimentally investigated the resolution limits for fiber-based label-free MMFCI. The samples were reconstructed using the GPSR algorithm with $\tau _{\text {opt}}$ (see Appendix). Figure 3(b) represents the experimental results. Columns first till last corresponds to samples 1-4. The high-resolution bright-field images of the samples are presented in Fig. 3(a). We defined the resolution improvement factor as the theoretical diffraction limit divided by the minimum feature size of the sample resolved by MMFCI. The smallest features of samples 1-4 were 1.4, 1.8, 2.5, and 4 times smaller than the diffraction limit, respectively. Our experiments demonstrated that samples 1-3 are nicely resolved by the proposed approach, meaning that the resolution of MMFCI is at least 2.5 times better than the diffraction limit.

 

Fig. 3. (a) Bright-field high-resolution ground truth images. (b) Experimental results for MMFCI. (c) Theoretical results for raster-scan diffraction-limited imaging. (d) Experimental results for Ghost imaging. Columns from left to right correspond of samples 1-4. Scale bars are $100\mu$m. The corresponding intensity of the signal across two dots is presented by solid lines for ground truth images (green), MMFCI (orange), raster-scan diffraction-limited imaging (blue) and ghost imaging (white)

Download Full Size | PDF

However, we were not able to experimentally resolve features that are four times smaller than the diffraction limit. Whereas classical compressive sensing theory does not put any restrictions on the resolution, it has been shown that the successful reconstruction cannot always be achieved for a diffraction limited system [23]. The theoretical limit of the spatial resolution improvement factor calculated for our samples was more than an order of magnitude larger than the experimentally achieved values. Unsuccessful reconstruction of sample 4 is not a result of the fundamental limit of MMFCI and can be explained by the presence of noise in matrix $\textbf {A}$ and measurement vector $\textbf {y}$ [23].

We compared the performance of the proposed MMFCI with conventional approaches. Figures 3(c) and 3(d) present the results for raster-scan microscopy and ghost imaging, respectively. Columns first to last correspond to samples 1-4. These methods can resolve only sample 1 because its feature size is comparable to the diffraction limit.

To analyze imaging resolution in more detail, we plotted the cross-sections along the $x$ axis passed through the centers of the dots for the MMFCI (orange line), conventional microscopy (blue line), ghost imaging (white line), and ground truth (green line) in Fig. 3. Ghost imaging and raster-scan approaches can resolve the sample only if the feature size is close to the diffraction limit. Two distinctive peaks in the image of Sample 1 can be easily identified by all the methods (Fig. 3, first row). However, the contrast (the difference between the peak and the gap intensities) is close to one for MMFCI and only about 0.5 for conventional microscopy and ghost imaging techniques. MMFCI also provides an almost perfect contrast for sample 2, whereas it is barely resolvable by conventional imaging and not resolvable by ghost imaging (Fig. 3, second row). While two peaks in sample 3 are not distinguishable neither by ghost imaging nor by conventional microscopy, MMFCI clearly resolves them (Fig. 3, third row). Our results demonstrate the supremacy of MMFCI over classical microscopy techniques in terms of spatial resolution and imaging contrast.

In the next set of measurements, we studied the stability of the MMFCI. We repeated the measurements of sample 3 using the same signal, $\textbf {y}$, but different matrices $\textbf {A}$ constructed from speckle patterns measured in 2 hours and 2 days. The results are presented in Fig. 4. Even though the resolution slightly degrades in comparison with the measurements performed synchronously, the two dots are well resolved even if 2 days between the actual sample measurements and the speckle pattern pre-calibration have passed. The experimental setup and the proposed approach of MMFCI are stable enough to perform long-term super-resolution measurements with a single pre-calibration episode.

 

Fig. 4. Stability of MMFCI measured for sample 3. The time gap between the calibration and the measurements was: no gap (left), 2 hours (middle), 2 days (right)

Download Full Size | PDF

3.3 Image compression

We are working in a highly compressive regime by reconstructing a $400$ $\times$ $400$-pixel image from only 961 single-point measurements. Moreover, our computational approach allows for forming final images of almost any size. To characterize the real compression ratio, we take into account the amount of spatial information that we reconstruct. We imaged $1.9$ $\times$ $1.9$ mm sample resolving two objects placed at a distance of 26 $\mu$m from each other. To be able to image this sample with a conventional approach of similar quality, we would need to measure at least $150$ $\times$ $150$ pixel. This results in a real compression ratio of $150^{2}/961 = 23$.

In the final set of experiments, we investigated the minimal number of measurements required to resolve the sub-diffraction features with the proposed approach. We performed the MMFCI reconstruction for the different number of speckles $M$. The examples of Sample 3 images measured with $M = 48$, $192$, $432$ and $961$ are presented in Fig. 5(a), (b), (c), (d), respectively.

 

Fig. 5. (a), (b), (c), (d) compressive sensing images corresponding to different number of measurements: (a) minimal number of measurements, (b) saturated regime transition point, (c) optimized number of measurements and (d) maximum number of measurements $M_{\text {max}}$ (sample 3). (e) Correlation function between the compressive sensing images $O(x,y)$ and the ground truth $S(x,y)$ R$_{\text {O,S}}$ as a function of number of measurements $M$ for sample 1 (yellow curve), sample 2 (red curve), sample 3 (blue curve). The saturated regime and optimal reconstruction transition points are marked by dotted blue and green lines correspondingly

Download Full Size | PDF

The correlation between the reconstruction results and the ground truth as a function of $M$ for samples 1-3 is presented in Fig. 5(e). For $M \geq 48$, correlation $\text {R}_{\text {O,S}}>0.3$, meaning that the reconstructed image and the ground truth are correlated even for such a very low number of measurements. The correlation increases with the number of measurements followed by the saturation, which is marked by a dotted blue line in the Fig. 5(e) and corresponds to $M_{\text {sat}}=192$. At this moment, two dots can be resolved, as shown in Fig. 5(b). To ensure the good reconstruction quality, we chose a twice higher number of measurements despite the fact that the correlation does not show rapid growth with the number of measurements for $M\geq M_{\text {sat}}$. We show that $M=432$ provides to $\text {R}_{\text {O,S}}>0.8$ for samples 1-3. Thus, about 432 measurements are enough to get a high-quality image with MMFCI approach. As a result, the compression ratio can be increased up to $150^{2}/432 \approx 52$.

High compression allows to improve the imaging speed significantly. Because pre-calibration can be done several hours in advance (as we showed in the previous section), the measurement speed is only limited by a beam scanner. Imaging speed of $200~$ms per frame ($5$ fps) is available with the standard galvo mirror system.

4. Conclusion

Here we report on the successful reconstruction of sub-diffraction features (super-resolution) by fast fiber-based label-free compressive imaging. The prior knowledge that the sample is sparse allows achieving superior image quality and speed compared to the conventional techniques. We believe that sparsity-based imaging can be applied to any sample because all natural objects have a sparse representation in an appropriate basis, which is supported by many popular image compression techniques, such as JPEG.

We have experimentally demonstrated the imaging speed of $5$ fps and the spatial resolution 2.5 times better than the diffraction limit without using any fluorescent labeling. In our experiments, the imaging speed was limited by the time that galvo mirrors switch between adjacent points of the scanning grid. Acousto- or electro-optical deflectors, which do not contain any mechanical moving parts, can improve scanning speed to approximately $10~\mu$s per point [29]. As a result, high-speed super-resolution imaging with $4~$ms per frame or 250 fps for 1.6-megapixel images can be achieved by using non-mechanical beam scanning system. It is three orders of magnitude faster than state-of-the-art imaging through the same multimode fiber with a diffraction limited resolution [8].

Currently, the results are largely influenced by choice of the reconstruction algorithm. As we show, the best performance can be achieved with the proper selection of regularisation parameter. The future direction would be the development of algorithm optimization methods that do not rely on the prior information about the sample.

To summarize, the proposed MMFCI approach merges high-speed super-resolution optical label-free imaging with a potentially compact optical probe. We have shown that MMFCI is stable enough to be used for long-term (more than a day-long) ultra-high-speed super-resolution imaging. We experimentally demonstrated the resolution improvement better than could be achieved by state-of-the-art label-free techniques, such as structured illumination microscopy. Super-resolution fluorescence methods such as STED, PALM and STORM can theoretically achieve infinitely small resolution. However this comes at a cost of long acquisition time of about several minutes per frame [7]. Therefore, the demonstrated imaging speed of MMFCI is significantly higher than the imaging speed of fluorescence super-resolution microscopy. The fiber nature of the proposed approach paves the way toward high-speed and high-quality endoscopy.

Appendix: Algorithm optimization for a MMF-based super-resolution imaging

We experimentally investigated the optimal parameters of the GPSR reconstruction algorithm for MMFCI. We illuminated the sample with $M = 961$ speckle patterns produced by the MMF and recorded the total transmitted intensity for each illumination pattern. The intensities formed measurement vector $\textbf {y}$ and the speckle patterns filtered in the Fourier domain formed measurement matrix $\textbf {A}$.

In Eq. (3), regularisation parameter $\tau$ defines the balance between the $l_{1}$ norm and noise. The reconstruction results are greatly influenced by $\tau$: too large values of $\tau$ leads to overestimation of sample sparsity. For example, for $\tau >\tau _{\text {max}} = |\textbf {A}^{T}\textbf {y}|_{\infty }$, $\textbf {x}=0$ and the reconstruction fails [30]. To study the influence of the regularisation parameter on the image quality, we performed the reconstruction with different values of $\tau$. The reconstructed images for Sample 1 in comparison with ground truth are presented in Fig. 6(a). For $\tau < \tau _{\text {max}}$, the algorithm always reproduced the number and the positions of dots. To quantify the image quality, we calculated the Pearson correlation coefficient R$_{\text {O,S}}$ between the reconstructed image and the ground truth. The correlation coefficient as a function of normalized regularization parameter $\tau /\tau _{\text {max}}$ for all the samples is presented in Fig. 6(c). We defined $\tau _{\text {opt}}$ as the regularization parameter for which R$_{\text {O,S}}$ is maximal. We see that for $\tau _{\text {opt}}<\tau <\tau _{\text {max}}$, the feature size is smaller and for $\tau <\tau _{\text {opt}}$ is bigger than the ground truth (Fig. 6(a)).

 

Fig. 6. (a), (b) Reconstructed images of sample 1 for different values of regularization parameter $\tau$ in comparison with the ground truth in experiment (a) and in simulations (b). (c) Correlation between the measured image and the ground truth as a function of $\tau /\tau _{\text {max}}$ for sample 1 (violet), sample 2 (yellow), sample 3 (orange) and sample 4 (blue). (d) Optimal regularisation parameter $\tau _{\text {opt}}/\tau _{\text {max}}$ as a function of noise ratio in simulations for sample 1 (solid violet line), sample 2 (dashed yellow line), sample 3 (solid orange line), sample 4 (solid blue line). (e) Correlation between reconstruction result and ground truth as a function of regularization parameter $\tau /\tau _{\text {max}}$ for the noise-free case (blue), 5 $\%$ noise ratio (red), 10 $\%$ noise ratio (yellow), 50 $\%$ noise ratio (violet). From top to bottom: sample 1, sample 2, sample 3, sample 4

Download Full Size | PDF

To explain this behavior, we performed numerical experiments with the measured set of speckle patterns and corresponding $\textbf {A}$. We simulated the photodiode signal as $\textbf {y}_{\text {sim}}=\textbf {A}\textbf {x}_{\text {sim}}+\textbf {n}$, where $\textbf {x}_{\text {sim}}$ is the 1D vector $(N^{2} \times 1)$ reshaped from the ground truth image and $\textbf {n}=\text {max}(\textbf {A}\textbf {x}_{\text {sim}})\cdot \textbf {r}\cdot \phi /100\%$, where $\text {max}(\textbf {A}\textbf {x}_{\text {sim}})$ is the maximum value of noise-free photodiode signal, $\phi$ is a noise ratio in percent, and $\textbf {r}$ is a vector $(M\times 1)$, which entities are randomly and uniformly distributed numbers in the range (0,1).

We simulated the imaging procedure of samples 1-4 with different noise ratio $\phi$ and different values of $\tau$. The examples of Sample 1 reconstructions for 5 $\%$ noise level are presented in Fig. 6(b). The correlation coefficients as functions of $\tau /\tau _{\text {max}}$ are presented in Fig. 6(e). Different columns correspond to different samples, and violet, yellow, red and blue colors correspond to $\phi$ = 50%, 10%, 5% and 0%, respectively. In the noise-free case, the correlation function for all the samples has a plateau – a range of $\tau$ values where the correlation is equally high (Fig. 6(e), blue areas). However in the presence of noise, the R$_{\text {O,S}}$ curves have peaks that define $\tau _{\text {opt}}$. For $\tau < \tau _{\text {opt}}$, the algorithm overestimates the feature size, for $\tau _{\text {opt}}<\tau <\tau _{max}$, the size is underestimated (Fig. 6(b)) similar to the experiment (Fig. 6(a)). Higher noise level corresponds to the larger value of $\tau _{\text {opt}}$. It is in agreement with Eq. (3), where the larger value of $\tau$ makes the sparsity term more significant compared to the difference between measurement result $\textbf {y}$ and $\textbf {A}\textbf {x}$. As the noise in the measurement increases, the value of the term $||\textbf {y}-\textbf {A}\textbf {x}||_{2}$ in problem Eq. (3) increases. As a result, the sparsity term $||\textbf {x}||_{1}$ is overestimated and the size of the sample features is underestimated. To reconstruct sample vector $\textbf {x}$ correctly the significance of the sparsity term $||\textbf {x}||_{1}$ should be increased by increasing $\tau$. The higher noise level corresponds to the smaller value of R$_{\text {O,S}}$ or the degradation of reconstruction (Fig. 6(e)).

The numerical result qualitatively reproduces experimental behavior. To achieve the highest quality of the MMFCI image, GPSR algorithm requires the optimized value of regularisation parameter $\tau _{\text {opt}}$. In case of $\tau \neq \tau _{\text {opt}}$, the feature size is reconstructed incorrectly. For $\tau <\tau _{\text {opt}}$ the resolution of the image degrades, for $\tau _{\text {opt}}<\tau <\tau _{\text {max}}$ the feature size is underestimated and for $\tau \geq \tau _{\text {max}}$ the reconstruction fails. The value of optimal regularisation parameter $\tau _{\text {opt}}$ is specific to both the sample and the noise level. To overcome this problem either an empiric adaptive choice or an automated selection of regularisation parameter can be used [31,32]. In further experiments, we use the experimentally determined optimal values of $\tau$.

Funding

Nederlandse Organisatie voor Wetenschappelijk Onderzoek (WISE).

Acknowledgments

This work has been carried out within ARCNL, a public-private partnership between UvA, VU, NWO and ASML, and was financed by ‘Toeslag voor Topconsortia voor Kennis en Innovatie (TKI)’ from the Dutch Ministry of Economic Affairs and Climate Policy. We thank Sergey Amitonov (TU Delft) for custom-made samples, Gerwin Osnabrugge (ARCNL) for fruitful discussions and Marco Seynen (AMOLF) for his help in programming the software.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. J. W. Goodman, “Introduction to fourier optics, second edition,” Opt. Eng. 35(5), 1513 (1996). [CrossRef]  

2. L. Möckl, D. C. Lamb, and C. Bräuchle, “Super-resolved fluorescence microscopy: Nobel prize in chemistry 2014 for eric betzig, stefan hell, and william e. moerner,” Angew. Chem. Int. Ed. 53(51), 13972–13977 (2014). [CrossRef]  

3. H. Blom and J. Widengren, “Stimulated emission depletion microscopy,” Chem. Rev. 117(11), 7377–7427 (2017). [CrossRef]  

4. E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006). [CrossRef]  

5. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (storm),” Nat. Methods 3(10), 793–796 (2006). [CrossRef]  

6. M. Yamanaka, N. I. Smith, and K. Fujita, “Introduction to super-resolution microscopy,” Microscopy 63(3), 177–192 (2014). [CrossRef]  

7. L. Schermelleh, A. Ferrand, T. Huser, C. Eggeling, M. Sauer, O. Biehlmaier, and G. P. Drummen, “Super-resolution microscopy demystified,” Nat. Cell Biol. 21(1), 72–84 (2019). [CrossRef]  

8. S. A. Vasquez-Lopez, R. Turcotte, V. Koren, M. Plöschner, Z. Padamsey, M. J. Booth, T. Cižmár, and N. J. Emptage, “Subcellular spatial resolution achieved for deep-brain imaging in vivo using a minimally invasive multimode fiber,” Light: Sci. Appl. 7(1), 110 (2018). [CrossRef]  

9. Z. Liu, S. Wieghold, L. T. Meyer, L. K. Cavill, T. Buonassisi, and E. M. Sachs, “Design of a submillimeter crack-detection tool for si photovoltaic wafers using vicinal illumination and dark-field scattering,” IEEE J. Photovoltaics 8(6), 1449–1456 (2018). [CrossRef]  

10. O. Denk, A. Musiienko, and K. Žídek, “Differential single-pixel camera enabling low-cost microscopy in near-infrared spectral region,” Opt. Express 27(4), 4562–4571 (2019). [CrossRef]  

11. G. Zada, C. Liu, and M. L. Apuzzo, “Through the looking glass: optical physics, issues, and the evolution of neuroendoscopy,” World Neurosurg. 79(2), S3–S13 (2013). [CrossRef]  

12. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109(20), 203901 (2012). [CrossRef]  

13. L. V. Amitonova and J. F. de Boer, “Compressive imaging through a multimode fiber,” Opt. Lett. 43(21), 5427–5430 (2018). [CrossRef]  

14. L. V. Amitonova and J. F. de Boer, “Endo-microscopy beyond the abbe and nyquist limits,” Light: Sci. Appl. 9(1), 81 (2020). [CrossRef]  

15. M. Lan, D. Guan, L. Gao, J. Li, S. Yu, and G. Wu, “Robust compressive multimode fiber imaging against bending with enhanced depth of field,” Opt. Express 27(9), 12957 (2019). [CrossRef]  

16. A. M. Caravaca-Aguirre, S. Singh, S. Labouesse, M. V. Baratta, R. Piestun, and E. Bossy, “Hybrid photoacoustic-fluorescence microendoscopy through a multimode fiber using speckle illumination,” APL Photonics 4(9), 096103 (2019). [CrossRef]  

17. S. Foucart and H. Rauhut, A mathematical introduction to compressive sensing (Springer, 2013).

18. M. A. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems,” IEEE J. Sel. Top. Signal Process. 1(4), 586–597 (2007). [CrossRef]  

19. http://www.lx.it.pt/mtf/GPSR/.

20. O. Katz, Y. Bromberg, and Y. Silberberg, “Compressive ghost imaging,” Appl. Phys. Lett. 95(13), 131110 (2009). [CrossRef]  

21. X.-H. Chen, F.-H. Kong, Q. Fu, S.-Y. Meng, and L.-A. Wu, “Sub-rayleigh resolution ghost imaging by spatial low-pass filtering,” Opt. Lett. 42(24), 5290–5293 (2017). [CrossRef]  

22. R. G. Brereton, “Orthogonality, uncorrelatedness, and linear independence of vectors,” J. Chemom. 30(10), 564–566 (2016). [CrossRef]  

23. B. Lochocki, K. Abrashitova, J. F. de Boer, and L. V. Amitonova, “Ultimate resolution limits of speckle-based compressive imaging,” Opt. Express 29(3), 3943–3955 (2021). [CrossRef]  

24. B. E. A. Saleh and M. C. Teich, Fundamentals of photonics, 2nd edition (Wiley, 2007).

25. J. Pauwels, G. V. der Sande, and G. Verschaffelt, “Space division multiplexing in standard multi-mode optical fibers based on speckle pattern classification,” Sci. Rep. 9(1), 17597 (2019). [CrossRef]  

26. J. W. Goodman, Speckle Phenomena in Optics: Theory and Applications, Second Edition (Roberts and Company Publishers, 2020).

27. M. Imai, “Statistical properties of optical fiber speckles,” Bull. Fac. Eng. 130(8), 89–104 (1986).

28. S. Li, S. A. Horsley, T. Tyc, T. čižmár, and D. B. Phillips, “Memory effect assisted imaging through multimode optical fibres,” Nat. Commun. 12(1), 3751 (2021). [CrossRef]  

29. G. R. Römer and P. Bechtold, “Electro-optic and acousto-optic laser beam scanners,” Phys. Procedia 56, 29–39 (2014). [CrossRef]  

30. K. Koh, S. J. Kim, and S. Boyd, “An interior-point method for large-scale l₁-regularized logistic regression,” Journal of Machine Learning Research 8, 1519–1555 (2007).

31. C. Fang, T. Yu, T. Chu, W. Feng, F. Zhao, X. Wang, Y. Huang, Y. Li, P. Wan, W. Mei, D. Zhu, and P. Fei, “Minutes-timescale 3d isotropic imaging of entire organs at subcellular resolution by content-aware compressed-sensing light-sheet microscopy,” Nat. Commun. 12(1), 107 (2021). [CrossRef]  

32. Batu Özge and M. Çetin, “Parameter selection in sparsity-driven sar imaging,” IEEE Trans. Aerosp. Electron. Syst. 47(4), 3040–3050 (2011). [CrossRef]