《计算机应用》唯一官方网站 ›› 2026, Vol. 46 ›› Issue (8): 2577-2583.DOI: 10.11772/j.issn.1001-9081.2025070813

• 多媒体计算与计算机仿真 • 上一篇    下一篇

基于图像编辑代理的零样本遥感图像复合检索算法

张杰, 黄智勇, 王瑞锦, 张凤荔()   

  1. 电子科技大学 信息与软件工程学院,成都 610054
  • 收稿日期:2025-07-21 修回日期:2025-11-07 接受日期:2025-11-10 发布日期:2025-12-22 出版日期:2026-08-10
  • 通讯作者: 张凤荔
  • 作者简介:张杰(2001—),男,四川宜宾人,硕士研究生,CCF会员,主要研究方向:大语言模型、自然语言处理、多模态
    黄智勇(1982—),男,广西蒙山人,高级工程师,博士研究生,主要研究方向:信息化
    王瑞锦(1980—),男,甘肃天水人,副教授,博士,CCF会员,主要研究方向:联邦学习、网络与信息安全
    张凤荔(1963—),女,河南内乡人,教授,博士,博士生导师,CCF会员,主要研究方向:智能计算、联邦学习、网络与信息安全。
  • 基金资助:
    国家自然科学基金资助项目(U2333207)

Zero-shot composed image retrieval algorithm for remote sensing images based on image edit proxy

Jie ZHANG, Zhiyong HUANG, Ruijin WANG, Fengli ZHANG()   

  1. School of Information and Software Engineering,University of Electronic Science and Technology of China,Chengdu Sichuan 610054,China
  • Received:2025-07-21 Revised:2025-11-07 Accepted:2025-11-10 Online:2025-12-22 Published:2026-08-10
  • Contact: Fengli ZHANG
  • About author:ZHANG Jie, born in 2001, M. S. candidate. His research interests include large language models, natural language processing, multi-modality.
    HUANG Zhiyong, born in 1982, Ph. D.candidate, senior engineer. His research interests include informatization.
    WANG Ruijin, born in 1980, Ph. D., associate professor. His research interests include federated learning, network and information security.
  • Supported by:
    National Natural Science Foundation of China(U2333207)

摘要:

随着图像复合检索(CIR)技术的快速发展,研究者开始探索将它应用于遥感(RS)图像检索领域,以提高从RS图像库中检索出目标图像的准确性。然而,现有算法未能有效弥合图像与文本模态之间的语义鸿沟,同时也受限于RS领域缺乏适用于CIR模型训练的高质量标注数据集。针对这些挑战,提出一种基于图像编辑代理的零样本遥感图像复合检索(IEP4RS)算法,该算法利用图像编辑技术,生成与查询图像及文本描述对齐的代理图像,从而增强查询表征。IEP4RS算法基于查询图像与目标图像的文本描述生成图像编辑指令,然后将指令与查询图像输入图像编辑模型以生成代理图像,并通过融合代理图像与原始查询图像的特征,构建复合查询图像特征。该算法通过直接匹配图像特征,有效跨越了图像与文本模态之间的语义鸿沟,并采用零样本学习范式避免传统算法对标注数据集的依赖。在公开的RS CIR基准数据集PatternCom上的实验结果表明,IEP4RS算法显著提升了检索性能,在采用RemoteCLIP特征编码器时,IEP4RS算法在平均精度均值(mAP)上相较于基线算法WEICOM (WEIghted COMposed image retrieval)提升了9.74个百分点,而相较于主流的零样本图像复合检索(ZS-CIR)算法Pic2Word (mapping Pictures to Words for zero-shot composed image retrieval)、SEARLE (zero-Shot composEd imAge Retrieval with textuaL invErsion)和FREEDOM (composed image retrieval for training-FREE DOMain conversion)分别提升了11.79、7.81和3.99个百分点。

关键词: 图像复合检索, 遥感图像, 信息检索, 图像编辑, 零样本图像复合检索

Abstract:

With the rapid development of Composed Image Retrieval (CIR) technology, its application is explored in the field of Remote Sensing (RS) image retrieval to improve the accuracy of retrieving target images from RS image libraries. However, the existing algorithms fail to bridge the semantic gap between image and text modalities effectively and are limited by the lack of high-quality annotated datasets for training CIR models in the RS field. To address these challenges, a zero-shot composed image retrieval for Remote Sensing images based on Image Edit Proxy (IEP4RS) algorithm was proposed to use image editing techniques to generate proxy images aligned with the query image and text description, thereby enhancing the query representation. In IEP4RS algorithm, image editing instructions were generated based on the query image and the text description of the target image, these instructions, along with the query image, were then fed into an image editing model to produce a proxy image, and composite query image features were constructed by fusing the features of the proxy image and the original query image. In this algorithm, the semantic gap between image and text modalities was bridged effectively through direct image feature matching, and a zero-shot learning paradigm was used to avoid the dependency on annotated datasets required by traditional algorithms. Experimental results on the public RS CIR benchmark dataset, PatternCom, demonstrate that IEP4RS algorithm improves retrieval performance significantly. With RemoteCLIP as the feature encoder, compared to the baseline algorithm WEICOM (WEIghted COMposed image retrieval), IEP4RS algorithm has the mean Average Precision (mAP) improved by 9.74 percentage points. Furthermore, IEP4RS algorithm outperforms the mainstream zero-shot composed image retrieval (ZS-CIR) algorithms, achieving improvements of 11.79, 7.81, and 3.99 percentage points in mAP compared to Pic2Word (mapping Pictures to Words for zero-shot composed image retrieval), SEARLE (zero-Shot composEd imAge Retrieval with textuaL invErsion), and FREEDOM (composed image retrieval for training-FREE DOMain conversion), respectively.

Key words: Composed Image Retrieval (CIR), Remote Sensing (RS) image, information retrieval, image editing, Zero-Shot Composed Image Retrieval (ZS-CIR)

中图分类号: