Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (10): 3191-3199.DOI: 10.11772/j.issn.1001-9081.2021081518
Special Issue: 多媒体计算与计算机仿真
• Multimedia computing and computer simulation • Previous Articles Next Articles
Jinkun ZHOU1, Xianlan WANG1, Nan MU2, Chen WANG3()
Received:
2021-08-26
Revised:
2021-12-14
Accepted:
2021-12-14
Online:
2022-01-07
Published:
2022-10-10
Contact:
Chen WANG
Supported by:
通讯作者:
王晨
作者简介:
周金坤(1995—),男,湖北荆州人,硕士研究生,主要研究方向:深度学习、计算机视觉基金资助:
CLC Number:
Jinkun ZHOU, Xianlan WANG, Nan MU, Chen WANG. Unmanned aerial vehicle image localization method based on multi-view and multi-supervision network[J]. Journal of Computer Applications, 2022, 42(10): 3191-3199.
周金坤, 王先兰, 穆楠, 王晨. 基于多视角多监督网络的无人机图像定位方法[J]. 《计算机应用》唯一官方网站, 2022, 42(10): 3191-3199.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2021081518
方法 | 无人机→卫星 | 卫星→无人机 | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
IL[ | 58.23 | 62.91 | 74.47 | 59.45 |
LCM[ | 66.65 | 70.82 | 79.89 | 65.38 |
SFPN[ | 70.83 | 77.36 | 80.26 | 71.58 |
LPN[ | 75.93 | 79.14 | 86.45 | 74.79 |
MMNet | 83.97 | 86.96 | 90.15 | 84.69 |
MMNet(distractors) | 81.15 | 84.92 | ― | ― |
Tab. 1 Comparison of the proposed method with state-of-the-art methods on University-1652 dataset
方法 | 无人机→卫星 | 卫星→无人机 | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
IL[ | 58.23 | 62.91 | 74.47 | 59.45 |
LCM[ | 66.65 | 70.82 | 79.89 | 65.38 |
SFPN[ | 70.83 | 77.36 | 80.26 | 71.58 |
LPN[ | 75.93 | 79.14 | 86.45 | 74.79 |
MMNet | 83.97 | 86.96 | 90.15 | 84.69 |
MMNet(distractors) | 81.15 | 84.92 | ― | ― |
方法 | 无人机→卫星 | 卫星→无人机 | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
MMNet | 83.97 | 86.96 | 90.15 | 84.69 |
GF(ID) | 64.02 | 69.14 | 79.61 | 63.08 |
LF(ID) | 78.53 | 82.61 | 88.30 | 76.89 |
JF(ID) | 79.65 | 84.46 | 88.82 | 79.11 |
GF(RRT) | 64.96 | 70.92 | 72.88 | 64.12 |
Tab. 2 Comparison results of different MMNet modules on University-1652 dataset
方法 | 无人机→卫星 | 卫星→无人机 | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
MMNet | 83.97 | 86.96 | 90.15 | 84.69 |
GF(ID) | 64.02 | 69.14 | 79.61 | 63.08 |
LF(ID) | 78.53 | 82.61 | 88.30 | 76.89 |
JF(ID) | 79.65 | 84.46 | 88.82 | 79.11 |
GF(RRT) | 64.96 | 70.92 | 72.88 | 64.12 |
采样策略 | 无人机→卫星 | 卫星→无人机 | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
批量挖掘[ | 82.93 | 83.53 | 87.46 | 82.05 |
MBM | 83.97 | 86.96 | 90.15 | 84.69 |
Tab. 3 Results of different sampling strategies in MMNet
采样策略 | 无人机→卫星 | 卫星→无人机 | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
批量挖掘[ | 82.93 | 83.53 | 87.46 | 82.05 |
MBM | 83.97 | 86.96 | 90.15 | 84.69 |
方法 | 无人机→卫星 | 卫星→无人机 | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
CL[ | 52.39 | 57.44 | 63.91 | 52.24 |
TL[ | 55.18 | 59.97 | 64.48 | 53.15 |
WSM[ | 53.21 | 58.03 | 65.62 | 54.47 |
RRT | 57.26 | 61.82 | 65.48 | 55.82 |
RRT+MBM | 59.93 | 64.96 | 67.70 | 58.21 |
Tab. 4 Comparison of RRT with other metric losses
方法 | 无人机→卫星 | 卫星→无人机 | ||
---|---|---|---|---|
R@1 | AP | R@1 | AP | |
CL[ | 52.39 | 57.44 | 63.91 | 52.24 |
TL[ | 55.18 | 59.97 | 64.48 | 53.15 |
WSM[ | 53.21 | 58.03 | 65.62 | 54.47 |
RRT | 57.26 | 61.82 | 65.48 | 55.82 |
RRT+MBM | 59.93 | 64.96 | 67.70 | 58.21 |
1 | WU Z C, NI M, HU Z W, et al. Mapping invasive plant with UAV-derived 3D mesh model in mountain area—a case study in Shenzhen Coast, China[J]. International Journal of Applied Earth Observation and Geoinformation, 2019, 77: 129-139. 10.1016/j.jag.2018.12.001 |
2 | DENG L, MAO Z H, LI X J, et al. UAV-based multispectral remote sensing for precision agriculture: a comparison between different cameras[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 146: 124-136. 10.1016/j.isprsjprs.2018.09.008 |
3 | YAN Y N, DENG L, LIU X L, et al. Application of UAV-based multi-angle hyperspectral remote sensing in fine vegetation classification[J]. Remote Sensing, 2019, 11(23): No.2753. 10.3390/rs11232753 |
4 | 赵爽,黄怀玉,胡一鸣,等. 基于深度学习的无人机航拍车辆检测[J]. 计算机应用, 2019, 39(S2):91-96. |
ZHAO S, HUANG H Y, HU Y M, et al. Vehicle detection in satellite imagery based on deep learning[J]. Journal of Computer Applications, 2019, 39(S2): 91-96. | |
5 | LIU W, YANG M Y, XIE M, et al. Accurate building extraction from fused DSM and UAV images using a chain fully convolutional neural network[J]. Remote Sensing, 2019, 11(24): No.2912. 10.3390/rs11242912 |
6 | ZHENG Z D, WEI Y C, YANG Y. University-1652: a multi-view multi-source benchmark for drone-based geo-localization[C]// Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 1395-1403. 10.1145/3394171.3413896 |
7 | VO N, JACOBS N, HAYS J. Revisiting IM2GPS in the deep learning era[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2640-2649. 10.1109/iccv.2017.286 |
8 | SATTLER T, HAVLENA M, SCHINDLER K, et al. Large-scale location recognition and the geometric burstiness problem[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1582-1590. 10.1109/cvpr.2016.175 |
9 | HAYS J, EFROS A A. IM2GPS: estimating geographic information from a single image[C]// Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2008: 1-8. 10.1109/cvpr.2008.4587784 |
10 | SHI Y J, YU X, CAMPBELL D, et al. Where am I looking at? joint location and orientation estimation by cross-view matching[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 4063-4071. 10.1109/cvpr42600.2020.00412 |
11 | TOKER A, ZHOU Q J, MAXIMOV M, et al. Coming down to earth: satellite-to-street view synthesis for geo-localization[C]// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 6484-6493. 10.1109/cvpr46437.2021.00642 |
12 | ZHU S J, YANG T J N, CHEN C. VIGOR: cross-view image geo-localization beyond one-to-one retrieval[C]// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 5316-5325. 10.1109/cvpr46437.2021.00364 |
13 | SHI Y J, YU X, LIU L, et al. Optimal feature transport for cross-view image geo-localization[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2020: 11990-11997. 10.1609/aaai.v34i07.6875 |
14 | LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. 10.1023/b:visi.0000029664.99615.94 |
15 | 孙鹏,肖经,赵海盟,等. 基于DSP的无人机遥感影像SIFT算法设计与实现[J]. 计算机应用, 2020, 40(4):1237-1242. 10.11772/j.issn.1001-9081.2019091689 |
SUN P, XIAO J, ZHAO H M, et al. Design and implementation of SIFT algorithm for UAV remote sensing image based on DSP platform[J]. Journal of Computer Applications, 2020, 40(4): 1237-1242. 10.11772/j.issn.1001-9081.2019091689 | |
16 | BAY H, TUYTELAARS T, GOOL L V. SURF: speeded up robust features[C]// Proceedings of the 2006 European Conference on Computer Vision, LNCS 3951. Berlin: Springer, 2006: 404-417. |
17 | ZHAI M H, BESSINGER Z, WORKMAN S, et al. Predicting ground-level scene layout from aerial imagery[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 4132-4140. 10.1109/cvpr.2017.440 |
18 | WORKMAN S, JACOBS N. On the location dependence of convolutional neural network features[C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2015: 70-78. 10.1109/cvprw.2015.7301385 |
19 | TIAN Y C, CHEN C, SHAH M. Cross-view image matching for geo-localization in urban environments[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 1998-2006. 10.1109/cvpr.2017.216 |
20 | VO N N, HAYS J. Localizing and orienting street views using overhead imagery[C]// Proceedings of the 2016 European Conference on Computer Vision, LNCS 9905. Cham: Springer, 2016: 494-509. |
21 | HU S X, FENG M D, NGUYEN R M H, et al. CVM-Net: cross-view matching network for image-based ground-to-aerial geo-localization[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7258-7267. 10.1109/cvpr.2018.00758 |
22 | ARANDJELOVIC R, GRONAT P, TORII A, et al. NetVLAD: CNN architecture for weakly supervised place recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 5297-5307. 10.1109/cvpr.2016.572 |
23 | REGMI K, SHAH M. Bridging the domain gap for ground-to-aerial image matching[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision . Piscataway: IEEE, 2019: 470-479. 10.1109/iccv.2019.00056 |
24 | WANG T C, LIU M Y, ZHU J Y, et al. High-resolution image synthesis and semantic manipulation with conditional GANs[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Piscataway: IEEE, 2018: 8798-8807. 10.1109/cvpr.2018.00917 |
25 | CAI S D, GUO Y L, KHAN S, et al. Ground-to-aerial image geo-localization with a hard exemplar reweighting triplet loss[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 8390-8399. 10.1109/iccv.2019.00848 |
26 | LIU L, LI H D. Lending orientation to neural networks for cross-view geo-localization[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 5617-5626. 10.1109/cvpr.2019.00577 |
27 | DING L R, ZHOU J, MENG L X, et al. A practical cross-view image matching method between UAV and satellite for UAV-based geo-localization[J]. Remote Sensing, 2020, 13(1): No.47. 10.3390/rs13010047 |
28 | HU S Y, CHANG X J. Multi-view drone-based geo-localization via style and spatial alignment[EB/OL]. (2020-07-09) [2021-05-28].. |
29 | WANG T Y, ZHENG Z D, YAN C G, et al. Each part matters: local patterns facilitate cross-view geo-localization[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(2): 867-879. 10.1109/tcsvt.2021.3061265 |
30 | SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10) [2021-05-28].. |
31 | HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Piscataway: IEEE, 2016: 770-778. 10.1109/cvpr.2016.90 |
32 | RADENOVIĆ F, TOLIAS G, CHUM O. Fine-tuning CNN image retrieval with no human annotation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(7): 1655-1668. 10.1109/tpami.2018.2846566 |
33 | SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: a unified embedding for face recognition and clustering[C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 815-823. 10.1109/cvpr.2015.7298682 |
34 | LIAO W T, YANG M Y, ZHAN N, et al. Triplet-based deep similarity learning for person re-identification[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops. Piscataway: IEEE, 2017: 385-393. 10.1109/iccvw.2017.52 |
35 | HERMANS A, BEYER L, LEIBE B. In defense of the triplet loss for person re-identification[EB/OL]. (2017-11-27) [2021-05-28].. 10.21203/rs.3.rs-1501673/v1 |
36 | YU B S, LIU T L, GONG M M, et al. Correcting the triplet selection bias for triplet loss[C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11210. Cham: Springer, 2018:71-86. |
37 | GORDO A, ALMAZÁN J, REVAUD J, et al. End-to-end learning of deep visual representations for image retrieval[J]. International Journal of Computer Vision, 2017, 124(2): 237-254. 10.1007/s11263-017-1016-8 |
38 | HE S J, WANG Y H. Cross-view geo-localization via salient feature partition network[J]. Journal of Physics: Conference Series, 2021, 1914: No.012009. 10.1088/1742-6596/1914/1/012009 |
[1] | Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969. |
[2] | Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974. |
[3] | Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918. |
[4] | Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703. |
[5] | Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877. |
[6] | Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557. |
[7] | Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625. |
[8] | Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650. |
[9] | Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263. |
[10] | Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072. |
[11] | Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP: defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086. |
[12] | Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318. |
[13] | Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199. |
[14] | Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215. |
[15] | Yuanjiong LIU, Maozheng HE, Yibin HUANG, Cheng QIAN. Ship identification model based on ResNet50 and improved attention mechanism [J]. Journal of Computer Applications, 2024, 44(6): 1935-1941. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||