Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Learning feature fusion for target detection based on polarimetric imaging

Open Access Open Access

Abstract

We propose a polarimetric imaging processing method based on feature fusion and apply it to the task of target detection. Four images with distinct polarization orientations were used as one parallel input, and they were fused into a single feature map with richer feature information. We designed a learning feature fusion method using convolutional neural networks (CNNs). The fusion strategy was derived from training. Meanwhile, we generated a dataset involving one original image, four polarization orientation images, ground truth masks, and bounding boxes. The effectiveness of our method was compared to that of conventional deep learning methods. Experimental results revealed that our method gets a 0.80 mean average precision (mAP) and a 0.09 miss rate (MR), which are both better than the conventional deep learning method.

© 2021 Optical Society of America

Full Article  |  PDF Article
More Like This
Mid-fusion of road scene polarization images on pretrained RGB neural networks

Khalid Omer and Meredith Kupinski
J. Opt. Soc. Am. A 38(4) 515-525 (2021)

Target classification of multislit streak tube imaging lidar based on deep learning

Wenhao Li, Shangwei Guo, Yu Zhai, Fei Liu, Zhengchao Lai, and Shaokun Han
Appl. Opt. 60(28) 8809-8817 (2021)

Polarimetric imaging detection using a convolutional neural network with three-dimensional and two-dimensional convolutional layers

Rui Sun, Xiaobing Sun, Feinan Chen, Qiang Song, and Hao Pan
Appl. Opt. 59(1) 151-155 (2020)

References

  • View by:

  1. P. Viola and M. J. Jones, “Robust real-time face detection,” Int. J. Comput. Vis. 57, 137–154 (2004).
    [Crossref]
  2. N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2005), pp. 886–893.
  3. D. Forsyth, “Object detection with discriminatively trained part-based models,” Computer 47, 6–7 (2014).
    [Crossref]
  4. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 779–788.
  5. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: single shot multibox detector,” presented at the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October8–16, 2016.
  6. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017).
    [Crossref]
  7. F. Goudail and J. S. Tyo, “When is polarimetric imaging preferable to intensity imaging for target detection?” J. Opt. Soc. Am. A 28, 46–53 (2011).
    [Crossref]
  8. G. P. Konnen, Polarized Light in Nature, 1st ed. (Cambridge University, 1985).
  9. M. Born, E. Wolf, and E. Hecht, “Principles of optics electromagnetic theory of propagation, interference and diffraction of light,” Phys. Today 53, 77–78 (2000).
    [Crossref]
  10. L. B. Wolff, “Polarization vision: a new sensory approach to image understanding,” Image Vis. Comput. 15, 81–93 (1997).
    [Crossref]
  11. P. J. Wu and J. T. Walsh, “Stokes polarimetry imaging of rat tail tissue in a turbid medium: degree of linear polarization image maps using incident linearly polarized light,” J. Biomed. Opt. 11, 014031 (2006).
    [Crossref]
  12. J. M. Bueno, J. J. Hunter, C. J. Cookson, M. L. Kisilak, and M. Campbell, “Improved scanning laser fundus imaging using polarimetry,” J. Opt. Soc. Am. A 24, 1337–1348 (2007).
    [Crossref]
  13. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006).
    [Crossref]
  14. M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
    [Crossref]
  15. P. Terrier, V. Devlaminck, and J. M. Charbois, “Segmentation of rough surfaces using a polarization imaging system,” J. Opt. Soc. Am. A 25, 423–430 (2008).
    [Crossref]
  16. G. Anna, N. Bertaux, F. Galland, F. Goudail, and D. Dolfi, “Joint contrast optimization and object segmentation in active polarimetric images,” Opt. Lett. 37, 3321–3323 (2012).
    [Crossref]
  17. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” presented at the NIPS’ 12: Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, December3–6, 2012.
  18. R. Sun, X. Sun, F. Chen, Q. Song, and H. Pan, “Polarimetric imaging detection using a convolutional neural network with three-dimensional and two-dimensional convolutional layers,” Appl. Opt. 59, 151–155 (2020).
    [Crossref]
  19. K. Usmani, G. Krishnan, T. O’Connor, and B. Javidi, “Deep learning polarimetric three-dimensional integral imaging object recognition in adverse environmental conditions,” Opt. Express 29, 12215–12228 (2021).
    [Crossref]
  20. B. Tavakoli, B. Javidi, and E. Watson, “Three dimensional visualization by photon counting computational integral imaging,” Opt. Express 16, 4426–4436 (2008).
    [Crossref]
  21. H. Li and X. Wu, “DenseFuse: a fusion approach to infrared and visible images,” IEEE Trans. Image Process. 28, 2614–2623 (2019).
    [Crossref]
  22. K. R. Prabhakar, V. S. Srikar, and R. V. Babu, “DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017), pp. 4714–4722.
  23. H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2Fusion: a unified unsupervised image fusion network,” IEEE Trans. Pattern Anal. Mach. Intell. (to be published).
  24. Y. Liu, X. Chen, R. K. Ward, and Z. J. Wang, “Image fusion with convolutional sparse representation,” IEEE Signal Process. Lett. 23, 1882–1886 (2016).
    [Crossref]
  25. Y. Liu, X. Chen, H. Peng, and Z. Wang, “Multi-focus image fusion with a deep convolutional neural network,” Inf. Fusion 36, 191–207 (2017).
    [Crossref]
  26. K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE Conference on Computer Vision (ICCV) (2017), pp. 2961–2969.
  27. T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 2117–2125.
  28. K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

2021 (1)

2020 (1)

2019 (1)

H. Li and X. Wu, “DenseFuse: a fusion approach to infrared and visible images,” IEEE Trans. Image Process. 28, 2614–2623 (2019).
[Crossref]

2017 (2)

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017).
[Crossref]

Y. Liu, X. Chen, H. Peng, and Z. Wang, “Multi-focus image fusion with a deep convolutional neural network,” Inf. Fusion 36, 191–207 (2017).
[Crossref]

2016 (1)

Y. Liu, X. Chen, R. K. Ward, and Z. J. Wang, “Image fusion with convolutional sparse representation,” IEEE Signal Process. Lett. 23, 1882–1886 (2016).
[Crossref]

2014 (1)

D. Forsyth, “Object detection with discriminatively trained part-based models,” Computer 47, 6–7 (2014).
[Crossref]

2012 (1)

2011 (1)

2008 (2)

2007 (1)

2006 (3)

J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45, 5453–5469 (2006).
[Crossref]

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

P. J. Wu and J. T. Walsh, “Stokes polarimetry imaging of rat tail tissue in a turbid medium: degree of linear polarization image maps using incident linearly polarized light,” J. Biomed. Opt. 11, 014031 (2006).
[Crossref]

2004 (1)

P. Viola and M. J. Jones, “Robust real-time face detection,” Int. J. Comput. Vis. 57, 137–154 (2004).
[Crossref]

2000 (1)

M. Born, E. Wolf, and E. Hecht, “Principles of optics electromagnetic theory of propagation, interference and diffraction of light,” Phys. Today 53, 77–78 (2000).
[Crossref]

1997 (1)

L. B. Wolff, “Polarization vision: a new sensory approach to image understanding,” Image Vis. Comput. 15, 81–93 (1997).
[Crossref]

Alouini, M.

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

Anguelov, D.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: single shot multibox detector,” presented at the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October8–16, 2016.

Anna, G.

Baarstad, I.

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

Babu, R. V.

K. R. Prabhakar, V. S. Srikar, and R. V. Babu, “DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017), pp. 4714–4722.

Belongie, S.

T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 2117–2125.

Berg, A. C.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: single shot multibox detector,” presented at the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October8–16, 2016.

Bertaux, N.

Born, M.

M. Born, E. Wolf, and E. Hecht, “Principles of optics electromagnetic theory of propagation, interference and diffraction of light,” Phys. Today 53, 77–78 (2000).
[Crossref]

Bourderionnet, J.

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

Bueno, J. M.

Campbell, M.

Charbois, J. M.

Chen, F.

Chen, X.

Y. Liu, X. Chen, H. Peng, and Z. Wang, “Multi-focus image fusion with a deep convolutional neural network,” Inf. Fusion 36, 191–207 (2017).
[Crossref]

Y. Liu, X. Chen, R. K. Ward, and Z. J. Wang, “Image fusion with convolutional sparse representation,” IEEE Signal Process. Lett. 23, 1882–1886 (2016).
[Crossref]

Chenault, D. B.

Cookson, C. J.

Dalal, N.

N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2005), pp. 886–893.

Devlaminck, V.

Divvala, S.

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 779–788.

Dolfi, D.

G. Anna, N. Bertaux, F. Galland, F. Goudail, and D. Dolfi, “Joint contrast optimization and object segmentation in active polarimetric images,” Opt. Lett. 37, 3321–3323 (2012).
[Crossref]

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

Dollar, P.

T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 2117–2125.

K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE Conference on Computer Vision (ICCV) (2017), pp. 2961–2969.

Erhan, D.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: single shot multibox detector,” presented at the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October8–16, 2016.

Farhadi, A.

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 779–788.

Forsyth, D.

D. Forsyth, “Object detection with discriminatively trained part-based models,” Computer 47, 6–7 (2014).
[Crossref]

Fu, C.-Y.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: single shot multibox detector,” presented at the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October8–16, 2016.

Galland, F.

Girshick, R.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017).
[Crossref]

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 779–788.

K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE Conference on Computer Vision (ICCV) (2017), pp. 2961–2969.

T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 2117–2125.

Gkioxari, G.

K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE Conference on Computer Vision (ICCV) (2017), pp. 2961–2969.

Goldstein, D. L.

Goudail, F.

G. Anna, N. Bertaux, F. Galland, F. Goudail, and D. Dolfi, “Joint contrast optimization and object segmentation in active polarimetric images,” Opt. Lett. 37, 3321–3323 (2012).
[Crossref]

F. Goudail and J. S. Tyo, “When is polarimetric imaging preferable to intensity imaging for target detection?” J. Opt. Soc. Am. A 28, 46–53 (2011).
[Crossref]

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

Grisard, A.

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

Guo, X.

H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2Fusion: a unified unsupervised image fusion network,” IEEE Trans. Pattern Anal. Mach. Intell. (to be published).

Hariharan, B.

T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 2117–2125.

He, K.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017).
[Crossref]

K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE Conference on Computer Vision (ICCV) (2017), pp. 2961–2969.

T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 2117–2125.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

Hecht, E.

M. Born, E. Wolf, and E. Hecht, “Principles of optics electromagnetic theory of propagation, interference and diffraction of light,” Phys. Today 53, 77–78 (2000).
[Crossref]

Hinton, G. E.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” presented at the NIPS’ 12: Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, December3–6, 2012.

Hunter, J. J.

Javidi, B.

Jiang, J.

H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2Fusion: a unified unsupervised image fusion network,” IEEE Trans. Pattern Anal. Mach. Intell. (to be published).

Jones, M. J.

P. Viola and M. J. Jones, “Robust real-time face detection,” Int. J. Comput. Vis. 57, 137–154 (2004).
[Crossref]

Kaspersen, P.

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

Kisilak, M. L.

Konnen, G. P.

G. P. Konnen, Polarized Light in Nature, 1st ed. (Cambridge University, 1985).

Krishnan, G.

Krizhevsky, A.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” presented at the NIPS’ 12: Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, December3–6, 2012.

Li, H.

H. Li and X. Wu, “DenseFuse: a fusion approach to infrared and visible images,” IEEE Trans. Image Process. 28, 2614–2623 (2019).
[Crossref]

Lin, T.-Y.

T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 2117–2125.

Ling, H.

H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2Fusion: a unified unsupervised image fusion network,” IEEE Trans. Pattern Anal. Mach. Intell. (to be published).

Liu, W.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: single shot multibox detector,” presented at the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October8–16, 2016.

Liu, Y.

Y. Liu, X. Chen, H. Peng, and Z. Wang, “Multi-focus image fusion with a deep convolutional neural network,” Inf. Fusion 36, 191–207 (2017).
[Crossref]

Y. Liu, X. Chen, R. K. Ward, and Z. J. Wang, “Image fusion with convolutional sparse representation,” IEEE Signal Process. Lett. 23, 1882–1886 (2016).
[Crossref]

Løke, T.

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

Ma, J.

H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2Fusion: a unified unsupervised image fusion network,” IEEE Trans. Pattern Anal. Mach. Intell. (to be published).

Normandin, X.

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

O’Connor, T.

Pan, H.

Peng, H.

Y. Liu, X. Chen, H. Peng, and Z. Wang, “Multi-focus image fusion with a deep convolutional neural network,” Inf. Fusion 36, 191–207 (2017).
[Crossref]

Prabhakar, K. R.

K. R. Prabhakar, V. S. Srikar, and R. V. Babu, “DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017), pp. 4714–4722.

Redmon, J.

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 779–788.

Reed, S.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: single shot multibox detector,” presented at the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October8–16, 2016.

Ren, S.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017).
[Crossref]

K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

Shaw, J. A.

Song, Q.

Srikar, V. S.

K. R. Prabhakar, V. S. Srikar, and R. V. Babu, “DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017), pp. 4714–4722.

Sun, J.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017).
[Crossref]

K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

Sun, R.

Sun, X.

Sutskever, I.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” presented at the NIPS’ 12: Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, December3–6, 2012.

Szegedy, C.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: single shot multibox detector,” presented at the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October8–16, 2016.

Tavakoli, B.

Terrier, P.

Triggs, B.

N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2005), pp. 886–893.

Tyo, J. S.

Usmani, K.

Viola, P.

P. Viola and M. J. Jones, “Robust real-time face detection,” Int. J. Comput. Vis. 57, 137–154 (2004).
[Crossref]

Walsh, J. T.

P. J. Wu and J. T. Walsh, “Stokes polarimetry imaging of rat tail tissue in a turbid medium: degree of linear polarization image maps using incident linearly polarized light,” J. Biomed. Opt. 11, 014031 (2006).
[Crossref]

Wang, Z.

Y. Liu, X. Chen, H. Peng, and Z. Wang, “Multi-focus image fusion with a deep convolutional neural network,” Inf. Fusion 36, 191–207 (2017).
[Crossref]

Wang, Z. J.

Y. Liu, X. Chen, R. K. Ward, and Z. J. Wang, “Image fusion with convolutional sparse representation,” IEEE Signal Process. Lett. 23, 1882–1886 (2016).
[Crossref]

Ward, R. K.

Y. Liu, X. Chen, R. K. Ward, and Z. J. Wang, “Image fusion with convolutional sparse representation,” IEEE Signal Process. Lett. 23, 1882–1886 (2016).
[Crossref]

Watson, E.

Wolf, E.

M. Born, E. Wolf, and E. Hecht, “Principles of optics electromagnetic theory of propagation, interference and diffraction of light,” Phys. Today 53, 77–78 (2000).
[Crossref]

Wolff, L. B.

L. B. Wolff, “Polarization vision: a new sensory approach to image understanding,” Image Vis. Comput. 15, 81–93 (1997).
[Crossref]

Wu, P. J.

P. J. Wu and J. T. Walsh, “Stokes polarimetry imaging of rat tail tissue in a turbid medium: degree of linear polarization image maps using incident linearly polarized light,” J. Biomed. Opt. 11, 014031 (2006).
[Crossref]

Wu, X.

H. Li and X. Wu, “DenseFuse: a fusion approach to infrared and visible images,” IEEE Trans. Image Process. 28, 2614–2623 (2019).
[Crossref]

Xu, H.

H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2Fusion: a unified unsupervised image fusion network,” IEEE Trans. Pattern Anal. Mach. Intell. (to be published).

Zhang, X.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

Appl. Opt. (2)

Computer (1)

D. Forsyth, “Object detection with discriminatively trained part-based models,” Computer 47, 6–7 (2014).
[Crossref]

IEEE Signal Process. Lett. (1)

Y. Liu, X. Chen, R. K. Ward, and Z. J. Wang, “Image fusion with convolutional sparse representation,” IEEE Signal Process. Lett. 23, 1882–1886 (2016).
[Crossref]

IEEE Trans. Image Process. (1)

H. Li and X. Wu, “DenseFuse: a fusion approach to infrared and visible images,” IEEE Trans. Image Process. 28, 2614–2623 (2019).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017).
[Crossref]

Image Vis. Comput. (1)

L. B. Wolff, “Polarization vision: a new sensory approach to image understanding,” Image Vis. Comput. 15, 81–93 (1997).
[Crossref]

Inf. Fusion (1)

Y. Liu, X. Chen, H. Peng, and Z. Wang, “Multi-focus image fusion with a deep convolutional neural network,” Inf. Fusion 36, 191–207 (2017).
[Crossref]

Int. J. Comput. Vis. (1)

P. Viola and M. J. Jones, “Robust real-time face detection,” Int. J. Comput. Vis. 57, 137–154 (2004).
[Crossref]

J. Biomed. Opt. (1)

P. J. Wu and J. T. Walsh, “Stokes polarimetry imaging of rat tail tissue in a turbid medium: degree of linear polarization image maps using incident linearly polarized light,” J. Biomed. Opt. 11, 014031 (2006).
[Crossref]

J. Opt. Soc. Am. A (3)

Opt. Express (2)

Opt. Lett. (1)

Phys. Today (1)

M. Born, E. Wolf, and E. Hecht, “Principles of optics electromagnetic theory of propagation, interference and diffraction of light,” Phys. Today 53, 77–78 (2000).
[Crossref]

Proc. SPIE (1)

M. Alouini, F. Goudail, A. Grisard, J. Bourderionnet, D. Dolfi, I. Baarstad, T. Løke, P. Kaspersen, and X. Normandin, “Active polarimetric and multispectral laboratory demonstrator: contrast enhancement for target detection,” Proc. SPIE 6396, 63960B (2006).
[Crossref]

Other (10)

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” presented at the NIPS’ 12: Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, December3–6, 2012.

K. R. Prabhakar, V. S. Srikar, and R. V. Babu, “DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017), pp. 4714–4722.

H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2Fusion: a unified unsupervised image fusion network,” IEEE Trans. Pattern Anal. Mach. Intell. (to be published).

G. P. Konnen, Polarized Light in Nature, 1st ed. (Cambridge University, 1985).

N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2005), pp. 886–893.

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 779–788.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: single shot multibox detector,” presented at the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, October8–16, 2016.

K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE Conference on Computer Vision (ICCV) (2017), pp. 2961–2969.

T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 2117–2125.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778.

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1.
Fig. 1. Polarimetric imaging schematic. The filter and polarizer are stacked together: 16 pixels comprise a group, and four polarization orientations are included for each color.
Fig. 2.
Fig. 2. Input formats of (a) the conventional deep learning method and (b) the proposed method.
Fig. 3.
Fig. 3. Specific process of fusion. Using a convolution with kernel size of ${4} \times {1}$, all the corresponding feature points are fused into one.
Fig. 4.
Fig. 4. Entire neural network architecture. C1, C2, C3, C4, C5, P2, P3, P4, P5, and P6 represent the feature maps of different scales obtained through each layer of the network, respectively.
Fig. 5.
Fig. 5. (a) Original polarization image taken by the camera, (b) four extracted images with distinct polarization orientations, and (c) calculated DoLP images.
Fig. 6.
Fig. 6. Comparison of the results obtained with four different models: (a) ground truth with masks and bounding boxes, (b) prediction results of the proposed method, (c) prediction results of the model trained with the monochrome image, (d) prediction results of the model trained with the RGB image, and (e) prediction results of the model trained with the DoLP image.
Fig. 7.
Fig. 7. Loss for the four models during training. The red lines represent the loss for the proposed model, and the other three color lines represent the loss for the three conventional deep learning models, respectively.

Tables (1)

Tables Icon

Table 1. Performance Comparison of the Proposed Model and the Conventional Deep Learning Modelsa

Equations (5)

Equations on this page are rendered with MathJax. Learn more.

{ S 0 = ( I 0 + I 45 + I 90 + I 135 ) / 2 S 1 = I 0 I 90 S 2 = I 45 I 135 ,
D o L P = S 1 2 + S 2 2 S 0 .
M R = 1 r e c a l l .
{ p r e c i s i o n = T P T P + F P r e c a l l = T P T P + F N .
F 1 = 2 × p r e c i s i o n × r e c a l l p r e c i s i o n + r e c a l l .

Metrics

Select as filters


Select Topics Cancel
© Copyright 2022 | Optica Publishing Group. All Rights Reserved