Abstract

In recent years, image fusion has emerged as an important research field due to its various applications. Images acquired by different sensors have significant differences in feature representation due to the different imaging principles. Taking visible and infrared image fusion as an example, visible images contain abundant texture details with high spatial resolution. In contrast, infrared images can obtain clear target contour information according to the principle of thermal radiation, and work well in all day/night and all weather conditions. Most existing methods employ the same feature extraction algorithm to get the feature information from visible and infrared images, ignoring the differences among these images. Thus, this paper proposes what we believe to be a novel fusion method based on a multi-level image decomposition method and deep learning fusion strategy for multi-type images. In image decomposition, we not only utilize a multi-level extended approximate low-rank projection matrix learning decomposition method to extract salient feature information from both visible and infrared images, but also apply a multi-level guide filter decomposition method to obtain texture information in visible images. In image fusion, a novel fusion strategy based on a pretrained ResNet50 network is presented to fuse multi-level feature information from both visible and infrared images into corresponding multi-level fused feature information, so as to improve the quality of the final fused image. The proposed method is evaluated subjectively and objectively in a large number of experiments. The experimental results demonstrate that the proposed method exhibits better fusion performance than other existing methods.

© 2021 Optical Society of America

Infrared and visible image fusion using salient decomposition based on a generative adversarial network

Lei Chen and Jun Han
Appl. Opt. 60(23) 7017-7026 (2021)

Infrared and visible image perceptive fusion through multi-level Gaussian curvature filtering image decomposition

Wei Tan, Huixin Zhou, Jiangluqi Song, Huan Li, Yue Yu, and Juan Du
Appl. Opt. 58(12) 3064-3073 (2019)

Multiscale feature pyramid network based on activity level weight selection for infrared and visible image fusion

Rui Xu, Gang Liu, Yuning Xie, Bavirisetti Durga Prasad, Yao Qian, and Mengliang Xing
J. Opt. Soc. Am. A 39(12) 2193-2204 (2022)

Infrared and visible image fusion via saliency analysis and local edge-preserving multi-scale decomposition

Xiaoye Zhang, Yong Ma, Fan Fan, Ying Zhang, and Jun Huang
J. Opt. Soc. Am. A 34(8) 1400-1410 (2017)

Infrared and visible image fusion using multiscale directional nonlocal means filter

Xiang Yan, Hanlin Qin, Jia Li, Huixin Zhou, Jing-guo Zong, and Qingjie Zeng
Appl. Opt. 54(13) 4299-4308 (2015)