Imaging through unknown scattering media based on physics-informed learning

Shuo Zhu; Enlai Guo; Enlai Guo; Jie Gu; Lianfa Bai; Jing Han; Jing Han

doi:10.1364/PRJ.416551

Photonics Research
Vol. 9,
Issue 5,
pp. B210-B219
(2021)
•https://doi.org/10.1364/PRJ.416551

Imaging through unknown scattering media based on physics-informed learning

Shuo Zhu, Enlai Guo, Jie Gu, Lianfa Bai, and Jing Han

Open Access

Get PDF
Email
Share
Get Citation
Copy Citation Text
Shuo Zhu, Enlai Guo, Jie Gu, Lianfa Bai, and Jing Han, "Imaging through unknown scattering media based on physics-informed learning," Photon. Res. 9, B210-B219 (2021)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article

Check for updates

Related Topics
Optics & Photonics Topics
?

The topics in this list come from the Optics and Photonics Topics applied to this article.

About this Article
History
- Original Manuscript: December 2, 2020
- Revised Manuscript: January 26, 2021
- Manuscript Accepted: February 21, 2021
- Published: April 15, 2021
Virtual Issues
Photonics Research Deep Learning in Photonics (2021)

Abstract

Imaging through scattering media is one of the hotspots in the optical field, and impressive results have been demonstrated via deep learning (DL). However, most of the DL approaches are solely data-driven methods and lack the related physics prior, which results in a limited generalization capability. In this paper, through the effective combination of the speckle-correlation theory and the DL method, we demonstrate a physics-informed learning method in scalable imaging through an unknown thin scattering media, which can achieve high reconstruction fidelity for the sparse objects by training with only one diffuser. The method can solve the inverse problem with more general applicability, which promotes that the objects with different complexity and sparsity can be reconstructed accurately through unknown scattering media, even if the diffusers have different statistical properties. This approach can also extend the field of view (FOV) of traditional speckle-correlation methods. This method gives impetus to the development of scattering imaging in practical scenes and provides an enlightening reference for using DL methods to solve optical problems.

Full Article | PDF Article

More Like This

Efficient color imaging through unknown opaque scattering layers via physics-aware learning

Shuo Zhu, Enlai Guo, Jie Gu, Qianying Cui, Chenyin Zhou, Lianfa Bai, and Jing Han
Opt. Express 29(24) 40024-40037 (2021)

Physical origin and boundary of scalable imaging through scattering media: a deep learning-based exploration

Xuyu Zhang, Shengfu Cheng, Jingjing Gao, Yu Gan, Chunyuan Song, Dawei Zhang, Songlin Zhuang, Shensheng Han, Puxiang Lai, and Honglin Liu
Photon. Res. 11(6) 1038-1046 (2023)

Deep speckle reassignment: towards bootstrapped imaging in complex scattering states with limited speckle grains

Shuo Zhu, Enlai Guo, Wenjun Zhang, Lianfa Bai, Honglin Liu, and Jing Han
Opt. Express 31(12) 19588-19603 (2023)

Previous Article Next Article

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.

Fig. 1. Speckle statistical characteristics analysis of the same object corresponding to different testing diffusers. (a) First row and second row are the speckle autocorrelation of the object within or exceeding the OME range, the third row is the cross-correlation with D1, respectively. (b)–(d) Intensity values of the white dash lines in the first, second, and third rows of (a), respectively. The color bar represents the normalized intensity. Scale bars: 875.52 µm.

Download Full Size | PDF

Fig. 2. Schematic of the physics-informed learning method for scalable scattering imaging.

Download Full Size | PDF

Fig. 3. Experimental setup for the scalable imaging. Different diffusers are employed to obtain speckle patterns with different scattering scenes. The OME range of this system is also measured by calculating the cross-correlation coefficient [21]. See Appendix B for details.

Download Full Size | PDF

Fig. 4. Testing results for generalization reconstruction of Group 1. Scale bars: 264.24 µm.

Download Full Size | PDF

Fig. 5. Testing results for generalization reconstruction of Group 2. Scale bars: 264.24 µm.

Download Full Size | PDF

Fig. 6. Testing results for generalization reconstruction of Group 3. Scale bars: 264.24 µm.

Download Full Size | PDF

Fig. 7. Testing results for generalization reconstruction of Group 4. Scale bars: 820.8 µm.

Download Full Size | PDF

Fig. 8. Generalization results for a single-character object with different scales and the scale of FOV is defined as the FOV/OME times. (a), (b) Results with different amounts of training diffusers, which are trained with one diffuser and three diffusers, respectively. (c) Reconstruction results with different scales and corresponding ground truth (GT).

Download Full Size | PDF

Fig. 9. Comparison results without or with this pre-processing step for imaging through an unknown diffuser. Three ground glasses are selected as the training diffusers and another diffuser for testing.

Download Full Size | PDF

Fig. 10. Results with different number of speckles via the physics-informed learning method. Three ground glasses are selected as the training diffusers and another diffuser for testing.

Download Full Size | PDF

Fig. 11. Generalization results of imaging exceeding OME range with different complexity objects.

Download Full Size | PDF

Tables (4)

Table 1. Quantitative Evaluation Results of the Objects within OME

View Table | View all tables in this article

Table 2. Quantitative Evaluation Results of Objects Extending the FOV 1.2 Times

View Table | View all tables in this article

Table 3. Objective Indicators with Different Number of Speckles via the Physics-Informed Learning Method

View Table | View all tables in this article

Table 4. Objective Indicators Corresponding to Fig. 11

View Table | View all tables in this article

Equations (11)

Equations on this page are rendered with MathJax. Learn more.

I = O * S,

I ⋆ I = (O * S) ⋆ (O * S) = (O ⋆ O) * (S ⋆ S),

I ⋆ I = (O ⋆ O) + C .

I ⋆ I = \sum_{i = 1}^{n} (O_{i} ⋆ O_{i}) + C^{'} .

R (x, y) = I (x, y) ⋆ I (x, y) = {FFT}^{- 1} {{| FFT {I (x, y)} |}^{2}} .

Loss = {Loss}_{NPCC} + {Loss}_{MSE},

{Loss}_{NPCC} = \frac{- 1 \times \sum_{x = 1}^{w} \sum_{y = 1}^{h} [i (x, y)] - \hat{i}] [I (x, y)] - \hat{I}]}{\sqrt{\sum_{x = 1}^{w} \sum_{y = 1}^{h} [i (x, y)] - \hat{i}]^{2}} \sqrt{\sum_{x = 1}^{w} \sum_{y = 1}^{h} [I (x, y)] - \hat{I}]^{2}}},

{Loss}_{MSE} = {Loss}_{I} = \sum_{x = 1}^{w} \sum_{y = 1}^{h} {| \tilde{i} (x, y) - I (x, y) |}^{2},

{PSF}_{i} ⋆ {PSF}_{j} \approx {\begin{matrix} δ_{i j}, & i = j \\ 0, & i \neq j \end{matrix} .

I ⋆ I = (\sum_{i = 1}^{n} O_{i} * {PSF}_{i}) ⋆ (\sum_{i = 1}^{n} O_{i} * {PSF}_{i}) = O_{1} ⋆ O_{1} + C_{1} + O_{2} ⋆ O_{2} + C_{2} + O_{3} ⋆ O_{3} + C_{3} + 2 (O_{1} ⋆ O_{2}) * ({PSF}_{1} ⋆ {PSF}_{2}) + 2 (O_{2} ⋆ O_{3}) * ({PSF}_{2} ⋆ {PSF}_{3}) + 2 (O_{1} ⋆ O_{3}) * ({PSF}_{1} ⋆ {PSF}_{3}) + \dots = \sum_{i = 1}^{n} (O_{i} ⋆ O_{i} + C_{i}) = \sum_{i = 1}^{n} (O_{i} ⋆ O_{i}) + C^{'} .

I ⋆ I = \sum_{i = 1}^{n} (O_{i} ⋆ O_{i}) + C^{'} .

Type	Training Set	Testing Set	MAE	SSIM	PSNR (dB)
Group 1	D1	Seen objects	0.0190	0.8477	23.41
	D1	Unseen objects	0.0288	0.8248	19.86
	D1, D2, D3	Seen objects	0.0033	0.9744	40.56
	D1, D2, D3	Unseen objects	0.0271	0.8088	20.58
Group 2	D1	Seen objects	0.0257	0.8608	20.75
	D1	Unseen objects	0.0548	0.6929	16.56
	D1, D2, D3	Seen objects	0.0072	0.9190	38.60
	D1, D2, D3	Unseen objects	0.0455	0.7678	16.76
Group 3	D1	Seen objects	0.1166	0.4672	15.59
	D1	Unseen objects	0.1234	0.5351	14.98
	D1, D2, D3	Seen objects	0.0323	0.8523	23.82
	D1, D2, D3	Unseen objects	0.0534	0.7410	20.67

Type	Training Set	Testing Set	MAE	SSIM	PSNR (dB)
Group 4	D1	Seen objects	0.0359	0.8439	17.53
	D1	Unseen objects	0.0432	0.6968	17.12
	D1, D2, D3	Seen objects	0.0085	0.9431	30.55
	D1, D2, D3	Unseen objects	0.0309	0.8706	18.90

Number of Speckles	MAE	SSIM	PSNR (dB)
$256 \times 256$	0.0054	0.9342	38.38
$512 \times 512$	0.0037	0.9539	44.53

Objects	Network	MAE	SSIM	PSNR (dB)
$5 \times$ OME single characters	U-Net	0.0232	0.8741	21.70
$5 \times$ OME double characters	U-Net	0.0361	0.8360	18.96
$1.2 \times$ OME faces	U-Net	0.0673	0.6972	18.40
$1.2 \times$ OME faces	PDSNet-L	0.0567	0.7050	21.16

Type	Training Set	Testing Set	MAE	SSIM	PSNR (dB)
Group 1	D1	Seen objects	0.0190	0.8477	23.41
	D1	Unseen objects	0.0288	0.8248	19.86
	D1, D2, D3	Seen objects	0.0033	0.9744	40.56
	D1, D2, D3	Unseen objects	0.0271	0.8088	20.58
Group 2	D1	Seen objects	0.0257	0.8608	20.75
	D1	Unseen objects	0.0548	0.6929	16.56
	D1, D2, D3	Seen objects	0.0072	0.9190	38.60
	D1, D2, D3	Unseen objects	0.0455	0.7678	16.76
Group 3	D1	Seen objects	0.1166	0.4672	15.59
	D1	Unseen objects	0.1234	0.5351	14.98
	D1, D2, D3	Seen objects	0.0323	0.8523	23.82
	D1, D2, D3	Unseen objects	0.0534	0.7410	20.67

Abstract

Cited By

Figures (11)

Tables (4)

Equations (11)

Photonics Research