1. Introduction

In a conventional imaging experiment, a lens maps every input pixel of an object to its conjugated output pixel at the sensor. Input and output measurement modes correspond to highly specific and localized information and all modes are simultaneously measured with an optical array detector, being transfer of information physically limited, in the ideal case, by diffraction. However, in a scattering medium the relationship between input and output pixels suffers the effects of light propagation by multiple scatterers. As a result, the spatial information of an input mode is scrambled and coupled through all output modes. Alternatively, an output pixel retains a tiny fraction of the optical field coming from every input mode and the interference between the different light fields generates an image that looks like a speckle field. Indeed, the ability of a lens to provide a clear image of an object is critically limited, even when a very thin layer of a scattering material is placed in the light path.

Scattering is the dominant extinction process that limits the imaging depth range inside biological tissue [1]. Although optical clearing techniques based on tissue manipulation has recently resulted in a see-through tissue [2], manipulation of photons provides a parallel avenue to control light-matter interactions without alteration of the biological material. Fine control of wave fields with current megapixel programmable spatial light modulators is by far the most employed approach. Examples of such technology are adaptive optics [3] and, more recently, control of disorder [4]. Transmission matrix characterization or feedback control based on iterative methods has allowed to undo the modifications that a scattering medium performs on the incoming wavefront by wavefront shaping [5–9] and seeing through static scattering media has also been demonstrated with this approach [10, 11]. Additionally, the angular correlation exhibited by speckle patterns within a small range of angles has been exploited for non-invasive imaging of a fluorescent object completely embedded in an opaque scattering medium [12]. Some recent attempts to extend the above techniques to steady focusing light through a slowly evolving dynamic medium are based on the use of faster spatial light modulators or by all-optical feedback [13, 14]. In this direction, recent advances in both DMD technology and GPU processing allow to compute the transmission matrix in tens of milliseconds [15]. However the dynamic nature of scattering in living tissue still remains an open question. Micrometer-scale translations of the scatter centres generate a decorrelated speckle field on the millisecond timescale, which makes dificult to follow the changing state of the medium for current feedback control techniques.

Image transmission through dynamic scattering media requires a paradigm shift to remove the use of feedback algorithms. Here we address this challenge. To do this, we use computational imaging of projected patterns with measurements being captured sequentially by a single-pixel sensor. The programmed patterns are used as generalized measurement modes where the object information is expressed. The same principle enables retrieval of the spatial information of an object with the use of a single-pixel detector in ghost [16, 17] and compressive imaging [18, 19]. In the latter case, the development of data collection strategies based on compressive sampling allows image compression to be performed at the sensing stage [20]. Applications that benefit from the advantages of single-pixel cameras are, among others, 3D imaging [21], entanglement imaging [22], fluorescence microscopy [23] or imaging at regions of the electromagnetic spectrum where current pixelated sensors are unavailable [24]. In this work we demonstrate that the presence of a scattering medium between the object and the light detector, even in the dynamically varying case, does not invalidate the operation principle of the proposed single-pixel scheme. Notably, scrambling of light due to disorder mixes information from all the regions of the sample but does not destroy the object information that can be retrieved from the generalized modes. As we will show later, we emphasize that this statement is accurate even if the medium is dynamic. Image transmission through disordered media has an immediate impact in the case of multimode fibers, which are subject to mode coupling. This crosstalk effect behaves in a similar way to a disordered medium. In this sense, an increment of the imaging capabilities of multimode fibers has been achieved by exploiting the properties of highly scattering media with a ‘single-pass’ architecture [25]. This approach, conceptually similar to that presented in Ref. [10], implies a precise calibration of the scattering effects and assumes a static medium. Both limitations are overcome by our single-pixel-sensor method.

2. Operation principle

To demonstrate our technique for image transmission through a scattering medium with a single-pixel camera, we refer to the diagram illustrated in Fig. 1(a). First, we consider the situation of static scattering. The sample (in this example a binary version of the famous Cheshire Cat in Alice‘s Adventures in Wonderland) is sequentially illuminated with a set of microstructured projecting patterns, which are codified onto a spatial light modulator. The light transmitted through the sample undergoes strong scattering so that any object is completely hidden, as is shown in Fig. 1(b). For each pattern, an optical sensor without spatial resolution averages the signal generated by the noise-like light distribution that covers its active surface. Each photodetected signal corresponds to a fraction of the light arising from the inner product between the sample and the projecting pattern and contains information of the whole sample thanks to light scrambling imposed by the scattering medium. Given that the sensor averages a sufficient number of speckle grains and that the number of input patterns is suitable, the information arising from the different modes is decorrelated and equally weighted. This fact guarantees that their contributions become independent for incoherent imaging and that the image-retrieval process can be formulated as a sequential measurement of the coefficients of the object intensity in the input basis (see Section 5). Notably, most of the coefficients of the object expressed in the function basis do not contribute in a significant way to image retrieval (see Fig. 1(c)). This means that the signal is sparse and compressive sensing can be used at the sampling stage by selecting in a random way the projecting patterns. In Section 3 we include a detailed study about the applicability of compressive sensing to our technique.

 

Fig. 1 Schematic diagram of the experimental set-up. (a) Set of microstructured patterns is projected onto the sample, which is placed in front of the scattering layer. The transmitted light undergoes strong scattering and a fraction is recorded by a photodiode. Within single-pixel imaging schemes, we need to project several patterns to form the image with measurements captured sequentially. (b) Photograph of the scattering layer covering partially a text located at a distance 10mm from the diffuser. In this way we show that the portion of the text under the diffuser is completely hidden. (c) Plot of two thousand projections derived from the photodetected signal in terms of the input modes.

Download Full Size | PDF

In our experiments, the Walsh-Hadamard basis is used to express the object xⁱⁿ as a linear combination, ${\mathbf{x}}^{\mathit{in}}={\sum }_i{E}_i^{\mathit{in}}{\mathbf{H}}_i$ (i = 1,...,N). A Walsh-Hadamard matrix of order n, denoted by H_i(n), is an N = n × n matrix with entries that satisfies ${\mathbf{H}}_i^T(n){\mathbf{H}}_i(n)=n\mathbf{I}(n)$, where I(n) is the identity matrix and ${\mathbf{H}}_i^T(n)$ stands for the transposed matrix. Hadamard matrices form an orthonormal basis of matrices that was first proposed by researchers in statistics and is considered to be the optimum weighting design for extracting information from random noise. A shifted and rescaled version of H_i(n) generates a binary pattern taking on the values 0 or 1, which can be simply encoded onto a spatial light modulator as an intensity pattern.

The scattering medium placed after the object is characterized at the macroscopic scale through the transmission matrix K with complex coefficients k_mn. The individual element k_mn connects the nth input Hadamard mode with the mth output measurement mode expressed in the canonical basis [27]. We assume that the matrix-coefficient statistics follows from the classical random walk phenomenon, as each k_mn results from the sum of contributions from many elementary pathways inside the medium that connect incoming and outgoing modes. As a result, the complex phasor k_mn is said to be a circular complex Gaussian variate and the transmission matrix amounts to a random matrix of independent identically distributed entries of Gaussian statistics [28]. The outgoing optical field corresponding to the mth output mode is given by $E_m^{\mathit{out}}={\sum }_n{k}_{mn}{E}_n^{\mathit{in}}$ and the field intensity from the mth mode is $I_m^{\mathit{out}}={\left|{\sum }_n{k}_{mn}{E}_n^{\mathit{in}}\right|}^2$. The input modes add together on an amplitude basis and the signal is affected by strong correlations between the signals provided by the different input Hadamard modes. In addition, the measurement in our scheme of single-pixel detection is given by the summation of the signals provided by the output canonical modes that cover the active surface of the sensor, i.e., $I^{\mathit{out}}={\sum }_m{I}_m^{\mathit{out}}$ (m = 1,...,S). However, the averaging over the photodetector surface forces the cross terms that couple different input modes to vanish and their contributions become uncorrelated and equally weighted so that the modes now add together on an intensity basis, i.e., $I^{\mathit{out}}={\sum }_m{I}_m^{\mathit{out}}\propto {\sum }_n{\left|E_n^{\mathit{in}}\right|}^2$. Consequently, the sequential projection of the different Hadamard modes onto the input object allow us at the detection level to measure separately any image expansion coefficient ${\left|E_n^{\mathit{in}}\right|}^2$. This result, derived in the Section 5, is extremely crucial for the validity of our proposed disorder-assisted single-pixel image-retrieval method and states that the incoherent nature of single-pixel imaging is preserved even through disordered media.

Applying the above ideas to our test object, we obtained the results shown in Fig. 2. The Cheshire Cat was directly codified onto the spatial light modulator with an image size of N = 128 × 128 pixels. The number of Walsh-Hadamard patterns was M = 0.2N. We used a diode-pumped laser at 532 nm (Oxxius slim-532) coupled to a single mode optical fiber. The beam was expanded and modulated by a liquid crystal reflective spatial light modulator (Holoeye LC-R 2500, with XGA pixel pitch of 19μm) inserted between an appropriate combination of polarizers. The scattering layer was a commercial diffuser (Edmund Optics T54-497). After passing through the diffuser, the light was partially guided by a lens to the photodiode (Thorlabs DET36A with active area size of 3.6 × 3.6mm²) to measure the fraction of the transmitted light intensity and an analog-to-digital converter digitized the photodetected signal. We found that the scattering layer completely hides the presence of the object so that the image recorded by a charge-coupled device camera is simply a speckle pattern (see Fig. 2(a)). On the contrary, the reconstructed image from computational imaging of projected patterns with single-pixel detection showed an excellent resemblance with the original object (see Fig. 2(b)).

 

Fig. 2 Seeing through scattering media with a single-pixel camera. (a) The Cheshire Cat is placed in front of a scattering layer that completely hides it. We show the image of the object as seen by a charge-coupled-device camera (Stingray-F-145 with square pixel size of 6.45 × 6.45μm²). The image contains not information at all on the shape of the object. (b) Image of the retrieved object from the photodetected values recorded by the photodiode. The good quality of the reconstructed image confirms that disorder-assisted imaging can indeed recover fine details of the object even though a sensor without spatial resolution is used.

Download Full Size | PDF

3. Compressive imaging through a static scattering layer

3.1. Compressive sensing

In the framework of single-pixel imaging, the imaging process in intensity is formulated as

The object is recovered off-line from the projections ${\left|E_n^{\mathit{in}}\right|}^2$ by solving the algebraical problem described in Eq. (1). To form a completely determined set of measurements, the rank of the sampling matrix M must equal the object data dimension N, as has been written in Eq. (1). However, compressive sensing allows the reconstruction of sparse signals Iⁱⁿ although the number of measurements M be lower than the number of pixels of the sampled object, i.e., M << N. This image sampling mechanism overcomes the fundamental tradeoff in imaging between speed and resolution and allows to speed up computational imaging. The point is that a sparse signal has only a small part of coefficients with a significant value when transformed in the appropriate base of functions. Therefore, many of the modes of the base provide little to no useful information and can be removed without substantial loss of image quality. Although it is not known at the sensing stage what coefficients have appreciable amplitude, compressive sensing algorithm recovers an undersampled signal ${\mathbf{I}}_{\mathit{rec}}^{\mathit{in}}$ that approaches the exact signal Iⁱⁿ with high probability from random undersampling of the matrix M. From a mathematical point of view, the system is undetermined and the object is recovered by solving the optimization problem min(‖Iⁱⁿ‖_l1) such that y(M) = M_usIⁱⁿ(N). In the above equation M_us is the undersampled version of the sampling matrix M, with dimension M × N, and ‖Iⁱⁿ‖_l1 is the l₁-norm of the object represented in an appropriate basis (in our case, the Walsh-Hadamard basis), which reflects the inherent sparsity that exists in natural objects. Given a sufficiently large number of samplings, the problem is rigorously solvable and the recovered signal ${\mathbf{I}}_{\mathit{rec}}^{\mathit{in}}$ approaches the exact signal Iⁱⁿ. If only k Hadamard projections of the signal Iⁱⁿ have a significant value, the number of helpful sampling modes scales as M ≥ k logN [20] and the compression ratio is given by CR = N/M.

3.2. Application to a microscopic sample

To test the quality of the images recovered by compressive sensing through a turbid medium, we conducted a series of experiments with the optical implementation shown in Fig. 3(a). The laser source and the photodiode were those employed in the preceding section. The object was a sample of stained onion cells, the Walsh-Hadamard patterns had 64 × 64 pixels and the optics of the system was adapted to adjust the spatial scale of the projecting patterns to the sample by use of an optical relay containing a microscope objective (Nikon 4× and NA = 0.1). We used the diffuser shown in Fig. 1(b) and included in the optical system the Stingray CCD camera. The sample was hidden by the scattering layer so that the image recorded by the CCD camera was a speckle pattern, as is shown in the small intensity map in Fig. 3(a). The code employed for compressive sensing was the function l1eq-pd of the l₁ -magic software package, which solves the standard basis pursuit problem using a primal-dual algorithm [29]. The quality of the undersampled image was tested using the standard peak signal-to-noise ratio, $\mathit{PSNR}=10\text{log}(I_{\mathit{max}}^2/\mathit{MSE})$, where I_max is the maximum pixel intensity value of the reference image and $\mathit{MSE}=\left(\frac{1}{N}\right){\Vert {\mathbf{I}}_{\mathit{ref}}^{\mathit{in}}-{\mathbf{I}}_{\mathit{rec}}^{\mathit{in}}\Vert }^2$ stands for the mean square error between the undersampled image, ${\mathbf{I}}_{\mathit{rec}}^{\mathit{in}}$, and the image recovered from the whole measurement set, ${\mathbf{I}}_{\mathit{ref}}^{\mathit{in}}$. Figure 3(b) illustrates that the time required for the sensing stage could be reduced by a factor of 2 while the PSNR is still higher than 20dB, which indicates high image fidelity. This is relevant to speed up the image retrieval. Moreover, for faster operation, the slow liquid crystal spatial light modulator can be replaced by a digital micromirror device or any equivalent component. Considering that the maximum full-image frame rate of commercially available digital micromirror devices is 22.7kHz, the capture of images of 64 × 64 pixels at the sensing stage could operate at a frame rate of 10Hz. Even in this situation, the above imaging rate is still unsatisfactory to overcome the dynamic nature of real scattering media whose speckle decorrelations are on the millisecond order.

 

Fig. 3 Disorder-assisted compressive microscopy. (a) Schematic diagram of the optical setup. LS laser source; L lens; BS beam splitter; SLM programmable spatial light modulator; D iris diaphram; MO microscope objective; OBJ sample (stained onion cells); SM scattering medium; CCD charge-coupled device camera; PD Photodiode. (b) Plot of the quality of the recovered images in dB as a function of the compression ratio. We show as insets the image of the onion cells retrieved from the photodetected signals generated by a randomized selection (24% and 50%) of the 4096 input modes, along with the unsampled image (CR = 1) for comparison.

Download Full Size | PDF

4. Image reconstruction through dynamic scattering media

Our method is also valid to recover the image through a scattering medium that changes its temporal properties due to the movement of the scattering centres. The critical point here is that our technique works without the need of a signal feedback. For a dynamic scattering medium, new realizations of the random walk are created as the time goes by and, as a consequence, the speckle intensity at any point on the detector plane changes with time, and the transmission matrix too. However, our approach addresses this situation in a rather different way. As long as the statistical properties of the medium remain stationary and the number of speckle grains impinging on the active area of the sensor is high enough, the photodetected current does not change with time and, as a result, the sample is reconstructed regardless of the diffuser movement (see Fig. 4(a)). In our experiment the diffuser was a polypropylene cover characterized by its orange peel texture. The diffuser was moved by means of a stream of air and a USAF resolution chart acted as the sample. The chaotic movement of the diffuser mimicked a dynamic scattering medium generating variable speckle. The key point was that the photodiode performed a spatial intensity integration in such a way that, under the above assumptions, the value of every measurement provided by the non-pixelated detector was effectively the same as in the static state. Figure 4(b) shows a snapshot of the dynamic speckle pattern after the moving diffuser. This image was recorded by use of a charge-coupled device camera like in Fig. 3(a). A plot of the measured intensity as a function of time for a single pixel of the camera is compared with that corresponding to a small region of 64×64 pixels. The uniformity of the intensity level in the latter case is clearly noticed and is the basis of our implementation.

 

Fig. 4 Image reconstruction through a dynamic scattering medium. (a) Image of the sample (a typical resolution chart) reconstructed from the photodetected signal showing the successful image retrieving ability of our single-pixel imaging setup through a moving diffuser. (b) Snapshot of the speckle field generated by the object placed in front of a moving diffuser. At the right side we plot the detected signal with an integration time smaller than the typical temporal drift of the speckle grains corresponding to both a pixel of the camera (top) and a region of 64 × 64 pixels (bottom).

Download Full Size | PDF

5. Discussion

Here we examine in greater detail the impact of the statistical properties of the scattering medium in the retrieved image process. As discussed before, the medium is characterized at the macroscopic scale through the transmission matrix K with complex coefficients k_mn. The individual quantity k_mn connects the nth input Hadamard mode with the mth output measurement mode expressed in the canonical basis. The measurement in our scheme of single-pixel detection is given by the summation of the signals provided by the different output modes that cover the active surface of the sensor, say S modes. Thus, when the media is fed with a complex signal that consists of N input modes, the photodetected signal is given by

 

Fig. 5 Charting the statistical properties of the scattering layer. (a) Histogram of the absolute value of the transmission matrix elements assessed by experimental means. The plot fairly matches the Rayleigh distribution, as we expected. (b) Histogram of the summations A_n evaluated from the data in (a). The small ratio between the standard deviation σ_A and the mean value 〈A〉 ensures that the integrated contributions coming from the different input modes are nearly equally weighted. (c) Relative dispersion of the A_n contributions in terms of the number of output modes integrated by the area of the sensor. (d) Histogram of the summations B_nn′ evaluated from the data in (a) assigning randomly a uniform phase distribution between 0 and 2π. Note that in order to make easier the comparison of results, the scale of the abscissa axis in the graphs (b) and (d) is the same.

Download Full Size | PDF

Two findings are clear from the histograms in Figs. 5(b) and (d). First, the A_n values are clearly clustered around a certain value 〈A〉. In mathematical terms, the quotient between the standard deviation σ_A and the mean value of the histogram turns out to be 0.13, in good agreement with the nearly zero value we expected. Likewise, the B_nn′ coefficient histogram (a nearly Gaussian distribution) is centred at origin, shows positive and negative values, and is far away to intersect the A_n coefficient distribution. The combination of all the above facts validates the assumptions we guessed to derive Eq. (4) from Eq. (3). Figure 5(c) shows that the relative dispersion of the A_n contributions increases when the number of output modes integrated by the detector is smaller and smaller. The stagnation value (0.13 in Fig. 5(b)) can be reduced by increasing the number of input modes taken into account. In our experiments in Fig. 3, the number of modes inside the active surface of the photodiode was about 10⁵, which is clearly enough to assume decorrelation between the different input modes.

6. Conclusions

We have demonstrated that computational techniques combined with single-pixel sensing enables image reconstruction behind arbitrary scattering media, in contrast to charge-coupled device cameras, where the pixelated structure of the sensor returns a noise-like speckle pattern. Our approach does not require a previous calibration of the disordered media and permits to retrieve images when we deal with dynamic scatterers. In contrast with techniques based on measuring the transmission matrix, our technique does not need to characterize the scattering medium, but operates on an intensity basis, thereby computing intensity distributions instead of complex fields. Moreover, the use of compressive sensing is limited to scenes that are sparse on the chosen basis. Different scenes may need different compression ratios (up to 1), which could entail higher acquisition times depending on the object under study. Our implementation is a first step to tackle the general problem of imaging objects completely embedded in a scattering medium. In parallel, our disordered-assisted single-pixel configuration shows straightforward applications for image transmission through multimode fibers or to look around corners. In addition, the operation principle of single-pixel imaging offers an ideal framework for using dedicated sensors, such as fiber spectrometers, beam polarimeters, and avalanche photodiodes, which can be employed to measure new physical imaging dimensions (wavelength, polarization, low-light intensity level, ...) of the sample under scrutiny.