关键词: 图像复合检索, 遥感图像, 信息检索, 图像编辑, 零样本图像复合检索

Abstract: With the rapid development of Composed Image Retrieval (CIR), its application isexplored in the field of Remote Sensing (RS) to improve the accuracy of retrieving target images from RS image databases. However, existing algorithms failed to effectively bridge the semantic gap between image and text modalities and were limited by the lack of high-quality annotated datasets for training CIR models in the RS domain. To address these challenges, a zero-shot algorithm termed IEP4RS (zero shot Composed Image Retrieval for Remote Sensing Based on Image Edit Proxy) was proposed. Image editing techniques were utilized by this algorithm to generate proxy images aligned with the query image and text description, thereby enhancing the query representation. Specifically, image editing instructions were first generated based on the query image and the text description of the target image. These instructions, along with the query image, were then fed into an image editing model to produce a proxy image. A composite query representation was constructed by fusing the features of this proxy image and the original query image. This algorithm effectively bridges the semantic gap between modalities through direct image feature matching. Adopting a zero-shot learning paradigm, the algorithm avoids the dependency on annotated datasets required by traditional approaches. Experimental results on the public remote sensing composed image retrieval benchmark dataset, PatternCom, demonstrate that the proposed IEP4RS algorithm significantly improves retrieval performance. Compared to the baseline WEICOM (WEIghted COMposed Image Retrieval Method), an improvement of 9.74 percentage points is observed. Furthermore, the algorithm outperforms mainstream zero-shot composed image retrieval methods, achieving improvements of 11.79, 7.81, and 3.99 percentage points over Pic2Word (Mapping Pictures to Words for Zero-shot Composed Image Retrieval), SEARLE (zero-Shot composEd imAge Retrieval with textuaL invErsion), and FREEDOM (Composed Image Retrieval for Training-FREE DOMain Conversion), respectively.

Key words: Composed Image Retrieval (CIR), remote sensing image, information retrieval, image editing, Zero-Shot Composed Image Retrieval (ZS-CIR)