Open Access
Add to BibSonomy
Abstract
Scattered light imaging techniques leveraging memory effects have been extensively investigated, yet most approaches are limited to operating in predominantly dark environments. The introduction of additional optical noise disrupts the fine structure of the original speckle pattern, undermining spatial correlation and resulting in imaging failure. In this study, we present a high-performance imaging method that integrates a lock-in process to overcome this limitation. Our experimental results demonstrate that the proposed technique enables successful imaging of targets in low signal-to-background ratio (SBR) environments, even at SBR levels as low as -28.0 dB. Furthermore, the method allows for the directional separation of targets with distinct modulation frequencies. This innovative approach has the potential to significantly expand the applicability of scattering imaging techniques by eliminating the constraints of dark field environments, thereby enhancing the convenience of in vivo microscopy and daytime astronomical observations.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Optical imaging is a powerful tool in the fields of biomicroscope and cosmic exploration. It enables us to observe and analyze the morphology and dynamics of biological samples and celestial bodies at high resolution. However, the imaging result can be significantly degraded due to scattering when light interacts with complex media. Various methods have been proposed to combat scattering in optical imaging, such as wavefront shaping [1–4], confocal microscopy [5], optical coherence tomography [6], and multiphoton microscopy [7,8]. These techniques rely on different principles and have unique advantages and limitations.
Recently, computational imaging methods based on the optical memory effect (OME) have shown their unique capabilities in imaging through scattering media. As the light information from the target degraded to form a complex and seemingly random speckle pattern after the scattering process, the OME refers to the fact that the speckle patterns still retain a high correlation when the incident light rotates within a small angle [9,10]. Within a specific range, the OME determines a local linear translation invariance property in the optical system. This correlation can be exploited to retrieve information about the original object by measuring the scattered light at different positions and times. The methods based on the OME demonstrate excellent real-time, full-color, high-speed, and even 3-dimensional imaging performance with a large field of view [11–17]. In particular, the phase retrieval algorithms show great potential for non-invasive imaging applications [18–21].
Imaging based on the OME requires acquiring scattered light with a high signal-to-background ratio (SBR). However, there will be strong ambient light interference in many scenes, which makes the application of these methods challenging. For example, when imaging deep layers of living tissue, general illumination light will produce strong surface reflection and backscattered light that interfere with pure speckle acquisition. The ambient interference noise can directly destroy the speckle's fine structure, resulting in image reconstruction failure. To address this issue, several research efforts are focused on achieving imaging recovery in bright-field scenarios. One approach involves using a deterministic phase modulator to generate a spatially incoherent light source, effectively removing ambient interference noise [22]. Another strategy utilizes Zernike-based background fitting, jointly with a modified low-rank and sparse decomposition method, which exhibits some ability to combat noise [23]. Additionally, deep learning-based imaging methods have demonstrated the ability to cope with background interference [24]. A plug-and-play algorithm has been created using the generalized alternating projection (GAP) optimization framework, the Fienup phase retrieval (FPR) method, and FFDNeT to restore objects through scattering media under non-darkroom environments [25].
In this paper, we present a novel method that enables the extraction of pure speckle patterns for scattering imaging under bright-field conditions, even at extremely low SBR. By employing the lock-in process, we extract the time-modulated speckle signal through the scattering media, which exhibits a remarkable similarity to the pure speckle observed in dark-field conditions. Utilizing these extracted patterns, we demonstrate the feasibility of speckle correlation imaging in bright-field scenarios.
Furthermore, our method allows for the separation of speckle signals with different modulation frequencies, enabling the imaging of distinct targets. In addition, given the prevalence of spontaneous motion in cells or biological tissues [26], or the possibility of exogenous modulation by techniques like ultrasound [27], our method finds broad applicability. For instance, we showcase imaging results obtained from vibration-modulated targets to illustrate the versatility of our proposed approach.
By effectively addressing the challenge of scattering imaging under strong background interference, our method provides valuable insights for resolving bright scenario issues in imaging through living tissues and conducting daytime astronomical observations.
2. Principle
In the OME range, the speckle pattern S formed by a target O on the object plane through the scattering media can be expressed as:
where $(m,n)$ and $(x,y)$ present the spatial coordinate on the image plane and object plane respectively, * indicates the two-dimension convolution, and PSF represents the point spread function of the imaging system.Under a high SBR scenario, on the one hand, we can obtain the PSF in the OME with reference objects or guide stars to realize imaging [13,28]; on the other hand, the autocorrelation of the PSF approximates the Dirac function $\delta $ based on the Wiener–Khinchin theorem [29]. The establishment of the following formula allows us to obtain an approximation of the target’s power spectrum and realize the image recovery with the phase retrieval algorithms:
where ${\bullet}$ denotes the correlation operator.Any additional object or light source other than the target introduces spatial noise, which disrupts speckle correlation. This makes image restoration a severe challenge no matter what approach is taken. Due to the distributive property of convolution and the noise robustness, all light that does not come from the target is noted as noise N. Considering the most complex time-varying noise cases, the received image I becomes:
where t indicates the time domain variable. Since the received generalized noise often has the randomness of space and time in application scenarios, such as the non-target tissue tremor, it cannot be filtered out by simple calculations such as the difference between frames. In turn, the target itself is usually naturally rhythmic, such as cellular deformation and the scintillation of starlight [30]. Or we can use external influences to modulate local motion in vivo tissue.Therefore, we introduce the principle of the lock-in amplifier to extract modulation information to break the SBR limitation. The lock-in amplifier is a typical instrument capable of extracting modulated signals from complex and strong ambient interference [31]. Considering its principle, when the optical signal of the target is modulated, Eq. (3) can be rewritten as:
Accordingly, there will be two outputs after the filter:
In the case of a target undergoing small periodic vibrations within the OME range, the resulting speckle pattern will exhibit corresponding movements. As a result, the intensity detected at individual pixels will exhibit periodic variations. Through a lock-in process, the extracted pattern can be interpreted as the speckle pattern formed by the target at a specific vibration state. Such a target can also be effectively recovered.
3. Results and discussion
To verify the effectiveness of the proposed method, we first conduct corresponding optical demonstration experiments. Figure 2 shows the schematic diagram of the experimental setup. A signal generator generates two sinusoidal signals of different frequencies $f_{1} = 20$Hz and $f_{2} = 15$Hz to drive two collimated LED light sources in modulation mode (LED: JCOPTIX LEM-530C2, nominal wavelength 527 nm; Driver: THORLABS LEDD1B). Two plates with hollow letters “U” and “S” are placed on the object planes O1 and O2 correspondingly, having the same object distance $v = 36$cm. The generated target signals are confirmed by the photoelectric probe that the waveforms are as expected. A diffuser (Newport 10DKIT-C1 - 1°) is placed at a distance of $u = 12$cm from the sCMOS (Hamamatsu C11440-22CU). An additional light source of the same wavelength introduces extra ambient interference light. By adjusting the power of the LEDs, it is possible to test the imaging capability in different SBR scenarios. A rotating diffuser influences the ambient interference light to form a spatiotemporal random noise distribution. Further, to demonstrate the robustness of the proposed method, an additional creased polyethylene film is attached to the surface of the rotating diffuser. It provides random refraction to increase the inhomogeneity of the random distribution. Such a way of introducing noise can be a good demonstration of what happens in the actual brightfield imaging process. In our experiments, the target light is in the same spectrum as the ambient interference light. When dealing with interference from other wavelengths, such as natural light, it’s useful to pre-emptively improve the signal purity by inserting an appropriate band-pass filter into the optical path.
In order to satisfy Nyquist-Shannon sampling theorem, only half of the sCMOS pixels are used so that the acquisition frequency can be stabilized at 60.01fps. The exposure time for each frame is set to 2 ms. Before the experiment, we pre-captured the pure speckle of the two targets separately. Then 1000 frames containing noise were acquired, and the lock-in process was calculated independently for the two corresponding frequencies. The autocorrelation of the speckle is a direct input into the phase recovery algorithm, which can reflect the power spectrum information. The effectiveness of the target signal extraction can be measured by the correlation between the autocorrelation of the lock-in results and the autocorrelation of the pure speckle. The higher the correlation, the closer the lock-in result is to the ideal value. Figure 3 shows a typical situation where the speckle correlation increases with the number of integration frames. The lock-in result reaches convergence after about 400 frames, with a final correlation of about 0.94. This indicates that the extracted speckle is highly consistent with the original pure speckle and the information loss during the extraction process is acceptable.
To evaluate the value of SBR, we acquired 1000 frames each from the pure modulated signal of the two targets and the pure ambient interference light, individually. The three image sequences are denoted as $I_{\textrm{S}}(m,n,i)$, $I_{\textrm{U}}(m,n,i)$, and $I_{\textrm{Noise}}(m,n,i)$, $i \in [1,1000]$, respectively. The SBR for the two targets can be calculated as follow:
In the following data processing, we choose an integration frame number of 800 to ensure that the lock-in results are up to convergence.
In practical application scenarios, convergence speed often depends on multiple reasons. The higher the SBR, the higher the sampling rate, and the closer the reference frequency is to the target frequency can all lead to faster convergence. In addition, factors such as the appropriate LPF parameters also affect the convergence.
Figure 4 shows the phase recovery results for the original and the lock-in speckles. Without the lock-in process, as in Fig. 4(a), the intense noise disrupts the structure of the scattered pattern, resulting in the autocorrelation result as in Fig. 4(b), where no valid information can be discerned. The corresponding recovery process, as in Fig. 4(c), is obviously unworkable. After the lock-in process, the expected speckles, as in Fig. 4(d)(g), can be separated and extracted. The corresponding autocorrelation, as in Fig. 4(e)(h), can be successfully recovered by the phase recovery algorithm, as shown in Fig. 4(f)(i). Thus, the feasibility of our proposed method is demonstrated.
Fig. 4. Pure speckles, lock-in speckles, and reconstructions. (a) A frame of speckle with noise; (d) and (g) lock-in speckles corresponding to “S” and “U”; (b), (e) and (h) autocorrelation patterns corresponding to (a), (d) and (g); (c), (f) and (i) phase recovery results corresponding to (b), (e) and (h).
Further, since we separated the speckles corresponding to the two targets and recovered the individual targets, it is straightforward to recover the spatial positions and relative intensities of the two targets by a simple calculation:
where ${O_{all}}$ is the new integrated reconstruction result of the two targets; ${O_1}$ corresponds to one of the phase recovery results; ${S_1}$ and ${S_2}$ denote the lock-in speckles of the two targets; F and ${F^{ - 1}}$ are the Fourier transform and inverse Fourier transform. As shown in Fig. 5, we successfully recovered the relative positions of “S” and “U” compared with the original case in Fig. 2. The ratio of the total intensity of “S” to the total intensity of “U” is 1.57, which is highly consistent with the original intensity ratio 1.42.The above experiments demonstrate that when the target light intensity can be modulated, imaging recovery through the scattering medium can also be achieved under the influence of strong ambient interference light. It is worth mentioning that, theoretically, the ultimate SBR that the proposed method can cope with depends on the dynamic range of the camera. Only a sufficiently large dynamic range can accurately acquire all light signals under extreme ambient interference.
Non-invasively modulating the vibrations of a target is also often a straightforward method to implement, for example, by using acoustic resonance. To further demonstrate the potential of our proposed method in practical applications, we used the motion vibration to modulate the target signal in the following experiment instead of modulating the light illumination intensity. Based on the experimental light path in Fig. 2, only one object plane is kept with its illumination. The light intensity of the illumination is no longer modulated. The vibration frequency is set to 20 Hz, and the amplitude is approximately 0.1 mm. The object and image distances remain $v = 36$cm and $u = 12$cm.
In an environment with a SBR of -14 dB, the speckle of the target can still be successfully separated using our proposed method, as shown in Fig. 6(b). Figures 6(c) and (e) show the autocorrelation pattern of the lock-in speckle and the corresponding phase recovery result, respectively. The image recovery result demonstrates good agreement with the original target image, as shown in Fig. 6(d).
Fig. 6. Imaging of a vibrating target. (a) Schematic diagram of the experimental setup. A vibration generator connected to the target; (b) lock-in speckle of the target; (c) autocorrelation pattern of the speckle; (d) original target image; (e) phase recovery result.
In practical applications, achieving modulation of the target intensity can be accomplished by manipulating the time series of the illumination light. For instance, in the context of multiphoton microscopy, control over the excitation beam allows for the generation of distinct repetitions, thereby enabling modulation. In fluorescence microscopy, selective modulation of the target fluorescence signal can also be achieved using “optical switches” [33]. Similarly, modulation of the target vibration can be attained through the reciprocating motion of the cell or tissue itself. Another possibility involves the introduction of ultrasonic focusing to induce vibrations in the target. These techniques provide means to effectively modulate the target signal and enable the application of our proposed method for extracting pure speckle patterns and facilitating imaging recovery.
Our paper focuses on demonstrating scenarios where the target frequency is known a priori. This is because the target frequency is naturally available when the modulation is artificially generated. When dealing with spontaneously generated modulated signals, various methods such as spectral analysis, time domain signal correlation, peak detection, or adaptive algorithms can be employed to determine the target frequency with relative ease. It should be noted that the efficacy of these methods relies on factors like the SBR and noise characteristics. Instances where the noise level is exceptionally high, or the target signal is exceedingly weak may pose challenges in accurately extracting the frequency without supplementary information or prior knowledge about the signal.
Besides, if the target receives undesirable modulation, resulting in an irregular signal, the bandwidth corresponding to the signal will increase accordingly. The lock-in process may not be able to properly demodulate and isolate the signal. In such cases, the lock-in process's filtering capabilities may suppress or distort the irregular signal, making it difficult or impossible to accurately extract the information of interest.
4. Conclusion
We introduce a novel approach based on the lock-in amplifier principle, enabling the extraction of pure speckle patterns even under bright-field conditions and their subsequent utilization for imaging recovery. Our experiments achieve imaging with the lowest SBR of -28.0 dB. The method's effectiveness is demonstrated by modulating both the intensity of the target signal and the target's motion. Since the separation of the speckle of different modulated targets can be achieved, the relative intensity and position of the targets can be effectively recovered compared to the simple autocorrelation phase recovery. Our proposed innovative approach can break through the limitations of strong ambient light interference, thus further extending the utility of scattering imaging techniques in multiple fields.
Funding
Guangdong Major Project of Basic and Applied Basic Research (2020B0301030009); National Natural Science Foundation of China (61991452, 12074444); National Key Research and Development Program of China (2020YFC2007102).
Acknowledgments
We would like to thank Dr. Haishan Liu for his help in tuning the code parameters.
Disclosures
The authors have no conflicts of interest to declare.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. I. M. Vellekoop and A. P. Mosk, “Focusing coherent light through opaque strongly scattering media,” Opt. Lett. 32(16), 2309–2311 (2007). [CrossRef]
2. A. P. Mosk, A. Lagendijk, G. Lerosey, and M. Fink, “Controlling waves in space and time for imaging and focusing in complex media,” Nat. Photonics 6(5), 283–292 (2012). [CrossRef]
3. P. Lai, L. Wang, J. W. Tay, and L. V. Wang, “Photoacoustically guided wavefront shaping for enhanced optical focusing in scattering media,” Nat. Photonics 9(2), 126–132 (2015). [CrossRef]
4. Z. Yu, H. Li, T. Zhong, J. H. Park, S. Cheng, C. M. Woo, Q. Zhao, J. Yao, Y. Zhou, X. Huang, W. Pang, H. Yoon, Y. Shen, H. Liu, Y. Zheng, Y. Park, L. V. Wang, and P. Lai, “Wavefront shaping: A versatile tool to conquer multiple scattering in multidisciplinary fields,” The Innovation 3(5), 100292 (2022). [CrossRef]
5. V. Ntziachristos, “Going deeper than microscopy: the optical imaging frontier in biology,” Nat. Methods 7(8), 603–614 (2010). [CrossRef]
6. M. B. Nasr, B. E. Saleh, A. V. Sergienko, and M. C. Teich, “Demonstration of dispersion-canceled quantum-optical coherence tomography,” Phys. Rev. Lett. 91(8), 083601 (2003). [CrossRef]
7. F. Helmchen and W. Denk, “Deep tissue two-photon microscopy,” Nat. Methods 2(12), 932–940 (2005). [CrossRef]
8. D. Debarre, E. J. Botcherby, T. Watanabe, S. Srinivas, M. J. Booth, and T. Wilson, “Image-based adaptive optics for two-photon microscopy,” Opt. Lett. 34(16), 2495–2497 (2009). [CrossRef]
9. S. Feng, C. Kane, P. A. Lee, and A. D. Stone, “Correlations and fluctuations of coherent wave transmission through disordered media,” Phys. Rev. Lett. 61(7), 834–837 (1988). [CrossRef]
10. I. I. Freund, M. Rosenbluh, and S. Feng, “Memory effects in propagation of optical waves through disordered media,” Phys. Rev. Lett. 61(20), 2328–2331 (1988). [CrossRef]
11. H. Zhuang, H. He, X. Xie, and J. Zhou, “High speed color imaging through scattering media with a large field of view,” Sci. Rep. 6(1), 32696 (2016). [CrossRef]
12. E. Edrei and G. Scarcelli, “Memory-effect based deconvolution microscopy for super-resolution imaging through scattering media,” Sci. Rep. 6(1), 33558 (2016). [CrossRef]
13. X. Xu, X. Xie, H. He, H. Zhuang, J. Zhou, A. Thendiyammal, and A. P. Mosk, “Imaging objects through scattering layers and around corners by retrieval of the scattered point spread function,” Opt. Express 25(26), 32829–32840 (2017). [CrossRef]
14. X. Xie, H. Zhuang, H. He, X. Xu, H. Liang, Y. Liu, and J. Zhou, “Extended depth-resolved imaging through a thin scattering medium with PSF manipulation,” Sci. Rep. 8(1), 4585 (2018). [CrossRef]
15. H. Liu, P. Lai, J. Gao, Z. Liu, J. Shi, and S. Han, “Alternative Interpretation of Speckle Autocorrelation Imaging Through Scattering Media,” Photonic Sens 12(3), 220308 (2022). [CrossRef]
16. J. Xie, X. Xie, Y. Gao, X. Xu, Y. Liu, and X. Yu, “Depth detection capability and ultra-large depth of field in imaging through a thin scattering layer,” J. Opt. 21(8), 085606 (2019). [CrossRef]
17. Y. Shui, T. Wang, J. Zhou, X. Luo, Y. Liu, and H. Liang, “Scattered light imaging beyond the memory effect using the dynamic properties of thick turbid media,” Adv. Photonics Nexus 2(02), 026010 (2023). [CrossRef]
18. J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491(7423), 232–234 (2012). [CrossRef]
19. O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6(8), 549–553 (2012). [CrossRef]
20. T. Lu, Y. Liu, H. Lin, Z. Chen, R. V. Vinu, and J. Pu, “Non-invasive imaging through dynamic scattering layers via speckle correlations,” Opt. Rev. 28(5), 557–563 (2021). [CrossRef]
21. D. Wang, S. K. Sahoo, X. Zhu, G. Adamo, and C. Dang, “Non-invasive super-resolution imaging through dynamic scattering media,” Nat. Commun. 12(1), 3150 (2021). [CrossRef]
22. M. Cua, E. H. Zhou, and C. Yang, “Imaging moving targets through scattering media,” Opt. Express 25(4), 3935–3945 (2017). [CrossRef]
23. W. Li, T. Xi, S. He, L. Liu, J. Liu, F. Liu, B. Wang, S. Wei, W. Liang, Z. Fan, Y. Sun, Y. Wang, and X. Shao, “Single-shot imaging through scattering media under strong ambient light interference,” Opt. Lett. 46(18), 4538–4541 (2021). [CrossRef]
24. S. Zheng, H. Wang, S. Dong, F. Wang, and G. Situ, “Incoherent imaging through highly nonstatic and optically thick turbid media based on neural network,” Photonics Res. 9(5), B220–B228 (2021). [CrossRef]
25. K. Ma, X. Wang, S. He, and L. Li, “Plug-and-play algorithm for imaging through scattering media under ambient light interference,” Opt. Lett. 48(7), 1754–1757 (2023). [CrossRef]
26. G. Beaune, C. Blanch-Mercader, S. Douezan, J. Dumond, D. Gonzalez-Rodriguez, D. Cuvelier, T. Ondarcuhu, P. Sens, S. Dufour, M. P. Murrell, and F. Brochard-Wyart, “Spontaneous migration of cellular aggregates from giant keratocytes to running spheroids,” Proc. Natl. Acad. Sci. U. S. A. 115(51), 12926–12931 (2018). [CrossRef]
27. R. Lencioni, D. Cioni, and C. Bartolozzi, “Tissue harmonic and contrast-specific imaging: back to gray scale in ultrasound,” Eur. Radiol. 12(1), 151–165 (2002). [CrossRef]
28. J. Schneider and C. M. Aegerter, “Guide star based deconvolution for imaging behind turbid media,” J Eur Opt Soc-Rapid 14(1), 21 (2018). [CrossRef]
29. O. Katz, P. Heidmann, M. Fink, and S. Gigan, “Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations,” Nat. Photonics 8(10), 784–790 (2014). [CrossRef]
30. A. H. Mikesell, A. A. Hoag, and J. S. Hall, “The Scintillation of Starlight*,” J. Opt. Soc. Am. 41(10), 689–695 (1951). [CrossRef]
31. J. H. Scofield, “Frequency-domain description of a lock-in amplifier,” Am. J. Phys. 62(2), 129–133 (1994). [CrossRef]
32. R. E. Best, N. V. Kuznetsov, G. A. Leonov, M. V. Yuldashev, and R. V. Yuldashev, “Tutorial on dynamic analysis of the Costas loop,” Annual Reviews in Control 42, 27–49 (2016). [CrossRef]
33. G. Marriott, S. Mao, T. Sakata, J. Ran, D. K. Jackson, C. Petchprayoon, T. J. Gomez, E. Warp, O. Tulyathan, H. L. Aaron, E. Y. Isacoff, and Y. Yan, “Optical lock-in detection imaging microscopy for contrast-enhanced imaging in living cells,” Proc. Natl. Acad. Sci. U. S. A. 105(46), 17789–17794 (2008). [CrossRef]
