基于交互融合的高精度图像融合算法的研究
作者:
作者单位:

1.南京航空航天大学电子信息工程学院 南京 210016;2.南京航空航天大学无人机研究院 南京 210016

作者简介:

王吉哲 1999年生,硕士研究生。
李 勃 1977年生,副研究员,硕士生导师。
马晨瑛 2000年生,硕士研究生。
周 鹏 1999年生,硕士研究生。
殷奇缘 1999年生,硕士研究生。

通讯作者:

李勃(libo70205830@nuaa.edu.cn)

中图分类号:

TP75;TP391

基金项目:

中国高校产学研创新基金(2021ZYA04004)


Research on high-precision image fusion algorithm based on interactive fusion
Author:
Affiliation:

1.College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China;2.Research Institute of Aerial Vehicles, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对可见光和SAR图像融合,提出了一种基于跨模态差分感知和注意力机制的交互融合(TDPAM Fusion)算法,能有效保留可见光图像中的纹理结构和SAR图像的细节信息。首先,采用跨模态差分感知融合(Cross-Modal Differential Perception Fusion,CMDAF)模块提取图像的互补信息,避免真值缺失并提高融合精度。其次,通过坐标注意力机制(Coordinate Attention,CA)提高特征提取的准确性和效率,增强语义信息的集成。最后通过交互融合算法(Interactive Fusion Module,IFM)将特征自适应融合。设计了相应的大型基准数据集,用于网络模型的训练和测试。实验结果表明:TDPAM Fusion融合算法可以获得包含清晰SAR信息的高质量可见光图像。此外,融合算法将互信息(Mutual Information,MI)、空间频率(Spatial Frequency,SF)、视觉保真度(Visual Fidelity,VIF)和相关系数(Correlation Coefficient,CC)等关键指标,分别提高了约6.41%、10.36%、14.25%和4.74%。

    Abstract:

    Aiming at the fusion of visible and SAR images, a novel interactive fusion algorithm based on Transmembrane Differential Perception and Attention Mechanism (TDPAM Fusion) is proposed, which can effectively preserve the texture structure of visible images and detail information of SAR images. Firstly, the Cross-Modal Differential Perception Fusion module is utilized to extract complementary information from images, which can avoid the missing of true values and improve the accuracy of fusion. Secondly, the coordinate attention mechanism is employed to enhance the accuracy and efficiency of feature extraction, and improve the integration of semantic information. Finally, a feature interaction fusion algorithm is used to adaptively fuse features from SAR and visible images. A corresponding large benchmark dataset is designed for model training and testing. Experimental results demonstrate that the fusion algorithm can obtain high-quality visible images with clear SAR information. In addition, the algorithm can improve key indicators such as mutual information, spatial frequency, visual fidelity, and correlation coefficient by approximately 6.41%, 10.36%, 14.25%, and 4.74%, respectively.

    相似文献
    引证文献
引用本文

王吉哲,李勃,马晨瑛,殷奇缘,周鹏.基于交互融合的高精度图像融合算法的研究[J].遥测遥控,2023,44(6):64-71.

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
    参考文献
历史
  • 收稿日期:2023-07-04
  • 最后修改日期:2023-08-16
  • 录用日期:
  • 在线发布日期: 2023-11-27
  • 出版日期:
  • 优先出版日期: 2023-11-27