Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (3): 901-908.DOI: 10.11772/j.issn.1001-9081.2023040412
• Multimedia computing and computer simulation • Previous Articles Next Articles
Meiyu CAI, Runzhe ZHU, Fei WU(), Kaiyu ZHANG, Jiale LI
Received:
2023-04-12
Revised:
2023-07-08
Accepted:
2023-07-13
Online:
2024-03-12
Published:
2024-03-10
Contact:
Fei WU
About author:
CAI Meiyu, born in 1998, M. S. candidate. Her research interests include visual positioning, scene matching and positioning.Supported by:
通讯作者:
吴飞
作者简介:
蔡美玉(1998—),女,山东德州人,硕士研究生,主要研究方向:视觉定位、景象匹配定位基金资助:
CLC Number:
Meiyu CAI, Runzhe ZHU, Fei WU, Kaiyu ZHANG, Jiale LI. Cross-view matching model based on attention mechanism and multi-granularity feature fusion[J]. Journal of Computer Applications, 2024, 44(3): 901-908.
蔡美玉, 朱润哲, 吴飞, 张开昱, 李家乐. 基于注意力机制和多粒度特征融合的跨视角匹配模型[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 901-908.
Add to citation manager EndNote|Ris|BibTeX
URL: http://www.joca.cn/EN/10.11772/j.issn.1001-9081.2023040412
数据集 | 样本数 | 类别数 | 学校数 | |
---|---|---|---|---|
训练集 | 43 253 | 701 | 33 | |
测试集 | Query_drone | 37 855 | 701 | 39 |
Query_satellite | 701 | 701 | ||
Gallery_drone | 51 355 | 951 | ||
Gallery_satellite | 951 | 951 |
Tab. 1 Dataset statistics
数据集 | 样本数 | 类别数 | 学校数 | |
---|---|---|---|---|
训练集 | 43 253 | 701 | 33 | |
测试集 | Query_drone | 37 855 | 701 | 39 |
Query_satellite | 701 | 701 | ||
Gallery_drone | 51 355 | 951 | ||
Gallery_satellite | 951 | 951 |
层 | 参数 |
---|---|
第1层 | 11,步长为1,填充为0 |
第2层 | 3×3,填充为1,groups为32 |
第3层 | 1×1,步长为1,填充为0 |
第4层 | 3×3,填充为1,groups为32 |
Tab. 2 CGAM model settings
层 | 参数 |
---|---|
第1层 | 11,步长为1,填充为0 |
第2层 | 3×3,填充为1,groups为32 |
第3层 | 1×1,步长为1,填充为0 |
第4层 | 3×3,填充为1,groups为32 |
库名 | 版本号 | 库名 | 版本号 |
---|---|---|---|
time | 1.7-25.1build1 | torchvision | 0.13.1 |
numpy | 1.21.5 | math | 10.3.0 |
pandas | 1.2.4 | timm | 0.6.7 |
torch | 1.12.1+ch113 | argparse | 1.1 |
sys | 1.5.12 |
Tab. 3 Configuration of Python packages
库名 | 版本号 | 库名 | 版本号 |
---|---|---|---|
time | 1.7-25.1build1 | torchvision | 0.13.1 |
numpy | 1.21.5 | math | 10.3.0 |
pandas | 1.2.4 | timm | 0.6.7 |
torch | 1.12.1+ch113 | argparse | 1.1 |
sys | 1.5.12 |
方法 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
IL | 58.23 | 62.91 | 74.47 | 59.45 |
LCM | 66.65 | 70.82 | 79.89 | 65.38 |
SFPN | 70.83 | 77.36 | 80.26 | 71.58 |
LPN | 75.93 | 79.14 | 86.45 | 74.79 |
PFFNet | 76.97 | 81.17 | 87.94 | 76.64 |
MMNet-distractors | 81.15 | 84.92 | — | — |
MMNET | 83.97 | 86.96 | 90.15 | 84.69 |
GAMF | 85.33 | 87.41 | 90.30 | 84.52 |
Tab. 4 Comparison results of different methods on University-1652 dataset
方法 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
IL | 58.23 | 62.91 | 74.47 | 59.45 |
LCM | 66.65 | 70.82 | 79.89 | 65.38 |
SFPN | 70.83 | 77.36 | 80.26 | 71.58 |
LPN | 75.93 | 79.14 | 86.45 | 74.79 |
PFFNet | 76.97 | 81.17 | 87.94 | 76.64 |
MMNet-distractors | 81.15 | 84.92 | — | — |
MMNET | 83.97 | 86.96 | 90.15 | 84.69 |
GAMF | 85.33 | 87.41 | 90.30 | 84.52 |
方法 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
Baseline | 72.96 | 76.40 | 85.16 | 74.53 |
Baseline+LB | 83.24 | 85.62 | 87.73 | 82.16 |
Baseline+LB+SGAM | 85.39 | 87.45 | 90.01 | 84.26 |
Baseline+LB+SGAM+CGAM | 85.33 | 87.41 | 90.30 | 84.52 |
Tab. 5 Test results of different modules on University-1652 dataset
方法 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
Baseline | 72.96 | 76.40 | 85.16 | 74.53 |
Baseline+LB | 83.24 | 85.62 | 87.73 | 82.16 |
Baseline+LB+SGAM | 85.39 | 87.45 | 90.01 | 84.26 |
Baseline+LB+SGAM+CGAM | 85.33 | 87.41 | 90.30 | 84.52 |
粒度等级 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
1 | 84.94 | 87.08 | 89.44 | 84.41 |
2 | 85.33 | 87.41 | 90.30 | 84.52 |
3 | 85.19 | 87.28 | 89.59 | 84.33 |
Tab. 6 Test results of different granularity levels on University-1652 dataset
粒度等级 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
1 | 84.94 | 87.08 | 89.44 | 84.41 |
2 | 85.33 | 87.41 | 90.30 | 84.52 |
3 | 85.19 | 87.28 | 89.59 | 84.33 |
粒度等级 | 方法 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|---|
R@1 | AP | R@1 | AP | ||
2 | 均匀划分 | 85.33 | 87.41 | 90.30 | 84.52 |
重叠窗口划分 | 70.14 | 73.84 | 80.88 | 70.64 | |
3 | 均匀划分 | 85.19 | 87.28 | 89.59 | 84.33 |
重叠窗口划分 | 66.23 | 70.28 | 78.32 | 65.87 |
Tab. 7 Test results of segmentation strategies with different granularity levels on University-1652 dataset
粒度等级 | 方法 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|---|
R@1 | AP | R@1 | AP | ||
2 | 均匀划分 | 85.33 | 87.41 | 90.30 | 84.52 |
重叠窗口划分 | 70.14 | 73.84 | 80.88 | 70.64 | |
3 | 均匀划分 | 85.19 | 87.28 | 89.59 | 84.33 |
重叠窗口划分 | 66.23 | 70.28 | 78.32 | 65.87 |
方法 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
LPN | 85.33 | 87.41 | 90.30 | 84.52 |
FPN | 78.97 | 81.93 | 85.02 | 78.88 |
Tab. 8 Test results of LPN and FPN on University-1652 dataset
方法 | Drone→Satellite | Satellite→Drone | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
LPN | 85.33 | 87.41 | 90.30 | 84.52 |
FPN | 78.97 | 81.93 | 85.02 | 78.88 |
1 | YANG M. Research on vehicle automatic driving target perception technology based on improved MSRPN algorithm [J]. Journal of Computational and Cognitive Engineering, 2022, 1(3):147-151. 10.47852/bonviewjcce20514 |
2 | 蒋三生,郭辉,王尚,等. 无人机农业植保应用研究新进展 [J].农业科学,2022, 12(11):1136-1142. |
JIANG S S, GUO H, WANG S, et al. New progress in the application of UAV in agricultural plant protection [J]. Hans Journal of Agricultural Sciences, 2020, 12(11):1136-1142. | |
3 | 申泽峰.无人机巡检系统在高速公路上的应用 [J].市政工程 2022, 7(7):119-121. |
SHEN Z F.Application of UAV inspection system on expressway [J]. Municipal Engineering, 2022, 7(7):119-121. | |
4 | ZHU R, YANG M, YIN L, et al. UAV’s status is worth considering: a fusion representations matching method for geo-localization [J].Sensors, 2023, 23(2):720. 10.3390/s23020720 |
5 | ZHENG Z, WEI Y, YANG Y. University-1652: a multi-view multi-source benchmark for drone-based geo-localization [C]// Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 1395-1403. 10.1145/3394171.3413896 |
6 | 上海工程技术大学. 一种用于无人机图像和卫星图像的跨视角景象匹配方法: CN202210889578.1 [P].2022-07-27. |
Shanghai University Of Engineering Science. A cross-perspective scene matching method for drone images and satellite images: CN202210889578.1 [P].2022-07-27. | |
7 | DING L, ZHOU J, MENG L, et al. A practical cross-view image matching method between UAV and satellite for UAV-based geo-localization [J].Remote Sensing, 2020, 13(1):47. 10.3390/rs13010047 |
8 | WANG T, ZHENG Z, YAN C, et al. Each part matters: Local patterns facilitate cross-view geo-localization [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(2):867-879. 10.1109/tcsvt.2021.3061265 |
9 | ZHUANG J, DAI M, CHEN X, et al. A faster and more effective cross-view matching method of UAV and satellite images for UAV geolocalization [J].Remote Sensing, 2021, 13(19):3979. 10.3390/rs13193979 |
10 | TIAN X, SHAO J, OUYANG D, et al. UAV-satellite view synthesis for cross-view geo-localization [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(7):4804-4815. 10.1109/tcsvt.2021.3121987 |
11 | 王泽宇,布树辉,黄伟,等.面向交通场景解析的局部和全局上下文注意力融合网络[J].计算机应用, 2023, 43(3):713-722. |
WANG Z Y, BU S H, HUANG W, et al. Local and global context attentive fusion network for traffic scene parsing [J].Journal of Computer Applications, 2023, 43(3):713-722. | |
12 | CHICCO D.Siamese neural networks: an overview [M]// Artificial Neural Networks: Methods in Molecular Biology 2190. New York: Humana Press, 2021: 73-94. 10.1007/978-1-0716-0826-5_3 |
13 | DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[EB/OL]. (2021-06-03) [2022-10-14]. . |
14 | 申志军,穆丽娜, 高静,等.细粒度图像分类综述[J].计算机应用, 2023; 43(1):51-60. |
SHEN Z J, MU L N, GAO J, et al. Review of fine-grained image categorization [J]. Journal of Computer Applications, 2023, 43(1):51-60. | |
15 | LU J, STEINERBERGER S. Neural collapse under cross-entropy loss [J]. Applied and Computational Harmonic Analysis, 2022,59: 224-241. 10.1016/j.acha.2021.12.011 |
16 | BOUTROS F, DAMER N, KIRCHBUCHNER F, et al. Self-restrained triplet loss for accurate masked face recognition [J]. Pattern Recognition, 2022,124:108473. 10.1016/j.patcog.2021.108473 |
17 | ZHU R, YIN L, YANG M, et al. SUES-200: a multi-height multi-scene cross-view image benchmark across drone and satellite [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(9): 4825-4839. 10.1109/tcsvt.2023.3249204 |
18 | LYU Z, ELSAID A, KARNS J, et al. An experimental study of weight initialization and Lamarckian inheritance on neuroevolution[C]// Proceedings of the 24th International Conference on Applications of Evolutionary Computation. Cham: Springer, 2021:584-600. 10.1007/978-3-030-72699-7_37 |
19 | HE S, WANG Y. Cross-view geo-localization via salient feature partition network [J]. Journal of Physics: Conference Series, 2021, 1914: 012009. 10.1088/1742-6596/1914/1/012009 |
20 | 周金坤,王先兰,穆楠,等.基于多视角多监督网络的无人机图像定位方法[J].计算机应用,2022,42(10):3191-3199. 10.11772/j.issn.1001-9081.2021081518 |
ZHOU J K, WANG X L, MU N, et al. Unmanned aerial vehicle image localization method based on multi-view and multi-supervision network [J]. Journal of Computer Applications,2022,42(10):3191-3199. 10.11772/j.issn.1001-9081.2021081518 | |
21 | 王嘉怡,陈子洋,袁小晨,等.面向跨视角地理定位的感知特征融合网络[J/OL].计算机工程与应用 [2023-04-01]. . 10.3778/j.issn.1002-8331.2209-0149 |
WANG J Y, CHEN Z Y, YUAN X C, et al. Perceptual feature fusion network for cross-view geo-localization [J/OL]. Journal of Computer Engineering and Applications [2023-04-01]. . 10.3778/j.issn.1002-8331.2209-0149 |
[1] | Pengfei ZHANG, Litao HAN, Hengjian FENG, Hongmei LI. Point cloud semantic segmentation based on attention mechanism and global feature optimization [J]. Journal of Computer Applications, 2024, 44(4): 1086-1092. |
[2] | Xianfeng YANG, Yilei TANG, Ziqiang LI. Aspect-level sentiment analysis model based on alternating‑attention mechanism and graph convolutional network [J]. Journal of Computer Applications, 2024, 44(4): 1058-1064. |
[3] | Boyue WANG, Yingxiang LI, Jiandan ZHONG. Segmentation network for day and night ground-based cloud images based on improved Res-UNet [J]. Journal of Computer Applications, 2024, 44(4): 1310-1316. |
[4] | Zexuan WAN, Chunli XIE, Quanrun LYU, Yao LIANG. Code clone detection based on dependency enhanced hierarchical abstract syntax tree [J]. Journal of Computer Applications, 2024, 44(4): 1259-1268. |
[5] | Rui TANG, Shibo YUE, Ruizhi ZHANG, Chuan LIU, Chuanlin PANG. Energy efficiency optimization mechanism for UAV-assisted and non-orthogonal multiple access-enabled data collection system [J]. Journal of Computer Applications, 2024, 44(4): 1209-1218. |
[6] | Xiangjie SUN, Qiang WEI, Yisen WANG, Jiang DU. Survey of code similarity detection technology [J]. Journal of Computer Applications, 2024, 44(4): 1248-1258. |
[7] | Weina DONG, Jia LIU, Xiaozhong PAN, Lifeng CHEN, Wenquan SUN. High-capacity robust image steganography scheme based on encoding-decoding network [J]. Journal of Computer Applications, 2024, 44(3): 772-779. |
[8] | Kui ZHAO, Huiqi QIU, Xu LI, Zhifei XU. Real-time pulmonary nodule detection algorithm combining attention and multipath fusion [J]. Journal of Computer Applications, 2024, 44(3): 945-952. |
[9] | Dapeng XU, Xinmin HOU. Feature selection method for graph neural network based on network architecture design [J]. Journal of Computer Applications, 2024, 44(3): 663-670. |
[10] | Yaoyao TANG, Yechen ZHU, Yangchuan LIU, Xin GAO. Research status and prospect of CT image ring artifact removal methods [J]. Journal of Computer Applications, 2024, 44(3): 890-900. |
[11] | Yuqiu LI, Liping HOU, Jian XUE, Ke LYU, Yong WANG. Remote sensing image recommendation method based on content interpretation [J]. Journal of Computer Applications, 2024, 44(3): 722-731. |
[12] | Zhiwen JING, Yujia ZHANG, Boting SUN, Hao GUO. Two-stage recommendation algorithm of Siamese graph convolutional neural network [J]. Journal of Computer Applications, 2024, 44(2): 469-476. |
[13] | Hualu QIU, Suzhen LIN, Yanbo WANG, Feng LIU, Dawei LI. Reconstruction algorithm for undersampled magnetic resonance images based on complex convolution dual-domain cascade network [J]. Journal of Computer Applications, 2024, 44(2): 580-587. |
[14] | Huzhen GAO, Changping DU, Yao ZHENG. Gimbal system control algorithm of unmanned aerial vehicle based on extended state observer [J]. Journal of Computer Applications, 2024, 44(2): 604-610. |
[15] | Xiang LIU, Bei HUA, Fei LIN, Hongyuan WEI. Design and implementation of component-based development framework for deep learning applications [J]. Journal of Computer Applications, 2024, 44(2): 526-535. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||