Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Multiscale feature pyramid network based on activity level weight selection for infrared and visible image fusion

Not Accessible

Your library or personal account may give you access

Abstract

At present, deep-learning-based infrared and visible image fusion methods have the problem of extracting insufficient source image features, causing imbalanced infrared and visible information in fused images. To solve the problem, a multiscale feature pyramid network based on activity level weight selection (MFPN-AWS) with a complete downsampling–upsampling structure is proposed. The network consists of three parts: a downsampling convolutional network, an AWS fusion layer, and an upsampling convolutional network. First, multiscale deep features are extracted by downsampling convolutional networks, obtaining rich information of intermediate layers. Second, AWS highlights the advantages of the ${l_1}$-norm and global pooling dual fusion strategy to describe the characteristics of target saliency and texture detail, and effectively balances the multiscale infrared and visible features. Finally, multiscale fused features are reconstructed by the upsampling convolutional network to obtain fused images. Compared with nine state-of-the-art methods via the publicly available experimental datasets TNO and VIFB, MFPN-AWS reaches more natural and balanced fusion results, such as better overall clarity and salient targets, and achieves optimal values on two metrics: mutual information and visual fidelity.

© 2022 Optica Publishing Group

Full Article  |  PDF Article
More Like This
Infrared and visible image fusion algorithm based on a cross-layer densely connected convolutional network

Ruixing Yu, Weiyu Chen, and Bing Zhu
Appl. Opt. 61(11) 3107-3114 (2022)

Unsupervised end-to-end infrared and visible image fusion network using learnable fusion strategy

Yili Chen, Minjie Wan, Yunkai Xu, Xiqing Cao, Xiaojie Zhang, Qian Chen, and Guohua Gu
J. Opt. Soc. Am. A 39(12) 2257-2270 (2022)

Adaptive brightness fusion method for intraoperative near-infrared fluorescence and visible images

Chong Zhang, Kun Wang, and Jie Tian
Biomed. Opt. Express 13(3) 1243-1260 (2022)

Data availability

Data underlying the results presented in this paper are available in Refs. [51,52,57].

51. I.-I. M. de EngenhariaI.-I. de Pesquisa da Marinha, and SMT/COPPE/Poli/UFRJ, “Visible-infrared database,” http://www02.smt.ufrj.br/~fusion/#VID1.1.

52. A. Toet, “TNO image fusion dataset,” figshare (2014), https://figshare.com/articles/TN_Image_Fusion_Dataset/1008029.

57. X. Zhang, P. Ye, and G. Xiao, “VIFB: a visible and infrared image fusion benchmark,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2020), pp. 104–105.

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (13)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (6)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (12)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All Rights Reserved