Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Unsupervised end-to-end infrared and visible image fusion network using learnable fusion strategy

Not Accessible

Your library or personal account may give you access

Abstract

Infrared and visible image fusion aims to reconstruct fused images with comprehensive visual information by merging the complementary features of source images captured by different imaging sensors. This technology has been widely used in civil and military fields, such as urban security monitoring, remote sensing measurement, and battlefield reconnaissance. However, the existing methods still suffer from the preset fusion strategies that cannot be adjustable to different fusion demands and the loss of information during the feature propagation process, thereby leading to the poor generalization ability and limited fusion performance. Therefore, we propose an unsupervised end-to-end network with learnable fusion strategy for infrared and visible image fusion in this paper. The presented network mainly consists of three parts, including the feature extraction module, the fusion strategy module, and the image reconstruction module. First, in order to preserve more information during the process of feature propagation, dense connections and residual connections are applied to the feature extraction module and the image reconstruction module, respectively. Second, a new convolutional neural network is designed to adaptively learn the fusion strategy, which is able to enhance the generalization ability of our algorithm. Third, due to the lack of ground truth in fusion tasks, a loss function that consists of saliency loss and detail loss is exploited to guide the training direction and balance the retention of different types of information. Finally, the experimental results verify that the proposed algorithm delivers competitive performance when compared with several state-of-the-art algorithms in terms of both subjective and objective evaluations. Our codes are available at https://github.com/MinjieWan/Unsupervised-end-to-end-infrared-and-visible-image-fusion-network-using-learnable-fusion-strategy.

© 2022 Optica Publishing Group

Full Article  |  PDF Article
More Like This
Multiscale feature pyramid network based on activity level weight selection for infrared and visible image fusion

Rui Xu, Gang Liu, Yuning Xie, Bavirisetti Durga Prasad, Yao Qian, and Mengliang Xing
J. Opt. Soc. Am. A 39(12) 2193-2204 (2022)

Infrared and visible image fusion algorithm based on a cross-layer densely connected convolutional network

Ruixing Yu, Weiyu Chen, and Bing Zhu
Appl. Opt. 61(11) 3107-3114 (2022)

Image fusion using a multi-level image decomposition and fusion method

Yu Tian, Wenjing Yang, and Ji Wang
Appl. Opt. 60(24) 7466-7479 (2021)

Data availability

Data that support the findings of this study are available at [56].

56. Y. Chen, M. Wan, Y. Xu, X. Cao, X. Zhang, Q. Chen, and G. Gu, “Unsupervised end-to-end infrared and visible image fusion network using learnable fusion strategy,” GitHub (2022), https://github.com/MinjieWan/Unsupervised-end-to-end-infrared-and-visible-image-fusion-network-using-learnable-fusion-strategy.

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (12)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (9)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (8)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All Rights Reserved