基于场景图划分的无人机影像定位算法

doi:10.11772/j.issn.1001-9081.2020111795

计算机应用 ›› 2021, Vol. 41 ›› Issue (10): 3004-3009.DOI: 10.11772/j.issn.1001-9081.2020111795

所属专题：多媒体计算与计算机仿真

• 多媒体计算与计算机仿真 • 上一篇下一篇

基于场景图划分的无人机影像定位算法

张驰¹, 李铸洪², 刘舟³, 沈未名¹

1. 测绘遥感信息工程国家重点实验室(武汉大学), 武汉 430079;
2. 华中科技大学软件学院, 武汉 430074;
3. 武汉大学计算机学院, 武汉 430072

收稿日期:2020-11-17 修回日期:2021-02-27 出版日期:2021-10-10 发布日期:2021-10-27
通讯作者: 沈未名
作者简介:张驰(1997-),男,江苏徐州人,硕士研究生,主要研究方向:三维重建、深度学习;李铸洪(2002-),北京人,主要研究方向:计算机视觉;刘舟(1989-),湖北应城人,博士,主要研究方向:图像处理、计算机视觉;沈未名(1966-),男,湖北武汉人,教授,博士,主要研究方向:视频编码、多媒体通信、嵌入式多媒体系统、图像模式识别、计算机视觉。
基金资助:
国家自然科学基金资助项目（61771014）。

Unmanned aerial vehicle image positioning algorithm based on scene graph division

ZHANG Chi¹, LI Zhuhong², LIU Zhou³, SHEN Weiming¹

1. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing(Wuhan University), Wuhan Hubei 430079, China;
2. School of Software Engineering, Huazhong University of Science and Technology, Wuhan Hubei 430074, China;
3. School of Computer Science, Wuhan University, Wuhan Hubei 430072, China

Received:2020-11-17 Revised:2021-02-27 Online:2021-10-10 Published:2021-10-27
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61771014).

摘要/Abstract

摘要： 针对大规模长序列无人机（UAV）影像定位中存在的速度慢、误差漂移等问题，结合UAV影像的特点，提出了一种基于场景图划分的UAV影像定位算法。首先，利用全球定位系统（GPS）辅助信息缩小特征匹配的空间搜索范围，从而加速同名点的提取；之后结合视觉一致性和空间一致性来构建场景图，并利用归一化割（Ncut）对其进行划分；接着，对各组场景图进行增量重建；最后，利用光束法平差（BA）融合场景图从而计算出场景的三维模型。此外，在BA阶段，所提算法对代价函数进行扩充，即加入了GPS空间约束信息。在四个UAV影像数据集上的实验结果表明，与COLMAP等多种运动恢复结构（SFM）算法相比，所提算法的定位速度提升了50%，重投影误差减小了41%，定位误差控制在0.5m之内。此外，通过有无GPS辅助下的算法的实验对比，可以得知引入相对和绝对GPS约束的BA有效解决了误差漂移问题，避免了出现歧义性结果，并且极大地减小了定位误差。

关键词: 无人机影像定位, 场景图划分, 运动恢复结构, 光束法平差, 归一化割

Abstract: Due to the problems of slow speed and error drift in the positioning of large-scale long-sequence Unmanned Aerial Vehicle (UAV) images, a positioning algorithm of UAV images based on scene graph division was proposed according to the characteristics of UAV images. Firstly, the Global Positioning System (GPS) ancillary information was used to narrow the spatial search scope for feature matching, so as to accelerate the extraction of corresponding points. After that, visual consistency and spatial consistency were combined to construct the scene graphs, and Normalized Cut (Ncut) was used to divide them. Then, incremental reconstruction was performed to each group of scene graphs. Finally, all scene graphs were fused to establish a 3S scene model by Bundle Adjustment (BA). In addition, the GPS spatial constraint information was added to the cost function in the BA stage. In the experiments on four UAV image datasets, compared with COLMAP and other Structure From Motion (SFM) algorithms, the proposed algorithm has the positioning speed increased by 50%, the reprojection error decreased by 41%, and the positioning error was controlled within 0.5 m. Through the experimental comparison of algorithms with or without GPS assistance, it can be seen that BA with relative and absolute GPS constraints solves the problem of error drift, avoids the ambiguous results and greatly reduces positioning error.

Key words: Unmanned Aerial Vehicle (UAV) image positioning, scene graph division, Structure From Motion (SFM), Bundle Adjustment (BA), Normalized Cut (Ncut)

中图分类号:

TP391

张驰, 李铸洪, 刘舟, 沈未名. 基于场景图划分的无人机影像定位算法[J]. 计算机应用, 2021, 41(10): 3004-3009.

ZHANG Chi, LI Zhuhong, LIU Zhou, SHEN Weiming. Unmanned aerial vehicle image positioning algorithm based on scene graph division[J]. Journal of Computer Applications, 2021, 41(10): 3004-3009.

参考文献

[1] 李德仁, 李明. 无人机遥感系统的研究进展与应用前景[J]. 武汉大学学报(信息科学版), 2014, 39(5):505-513, 540.(LI D R, LI M. Research advance and application prospect of unmanned aerial vehicle remote sensing system[J]. Geomatics and Information Science of Wuhan University, 2014, 39(5):505-513, 540.)
[2] LU Y C, XUE Z C, XIA G S, et al. A survey on vision-based UAV navigation[J]. Geo-spatial Information Science, 2018, 21(1):21-32.
[3] XIANG T Z, XIA G S, ZHANG L P. Mini-unmanned aerial vehiclebased remote sensing:techniques, applications, and prospects[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7(3):29-63.
[4] 袁修孝, 高宇, 邹小容. GPS辅助空中三角测量在低空航测大比例尺地形测图中的应用[J]. 武汉大学学报(信息科学版), 2012, 37(11):1289-1293.(YUAN X X, GAO Y, ZOU X R. Application of GPS-supported aerotriangulation in large scale topographic mapping based on low-altitude photogrammetry[J]. Geomatics and Information Science of Wuhan University, 2012, 37(11):1289-1293.)
[5] 郭复胜, 高伟. 基于辅助信息的无人机图像批处理三维重建方法[J]. 自动化学报, 2013, 39(6):834-845.(GUO F S, GAO W. Batch reconstruction from UAV images with prior information[J]. Acta Automatica Sinica, 2013, 39(6):834-845.)
[6] RHEE S, KIM T. Investigation of 1:1000 scale map generation by stereo plotting using UAV images[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, XLⅡ-2/W6:319-324.
[7] 尹双双, 邓非. GPS改进的移动球形全景影像三维重建方法[J]. 测绘科学, 2020, 45(5):13-22.(YIN S S, DENG F. Improved incremental SFM algorithm for mobile spherical panoramic images based on GPS[J]. Science of Surveying and Mapping, 2020, 45(5):13-22.)
[8] 陈锐, 陈志, 张佳煜, 等. 基于优化三维重建技术的快速影像拼接[J]. 软件导刊, 2020, 19(7):219-222.(CHEN R, CHEN Z, ZHANG J Y, et al. Rapid image splice based on optimized 3D reconstruction technique[J]. Software Guide, 2020, 19(7):219-222.)
[9] LHUILLIER M. Incremental fusion of structure-from-motion and GPS using constrained bundle adjustments[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(12):2489-2495.
[10] IRSCHARA A, HOPPE C, BISCHOF H, et al. Efficient structure from motion with weak position and orientation priors[C]//Proceedings of the 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.Piscataway:IEEE, 2011:21-28.
[11] QU Y F, HUANG J Y, ZHANG X. Rapid 3D reconstruction for image sequence acquired from UAV camera[J]. Sensors, 2018, 18(1):No. 225.
[12] AGARWAL S, FURUKAWA Y, SNAVELY N, et al. Building Rome in a day[J]. Communications of the ACM, 2011, 54(10):105-112.
[13] SCHÖNBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:4104-4113.
[14] WU C C. Towards linear-time incremental structure from motion[C]//Proceedings of the 2013 International Conference on 3D Vision. Piscataway:IEEE, 2013:127-134.
[15] ZHENG E, WU C. Structure from motion using structure-less resection[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:2075-2083.
[16] CRANDALL D J, OWENS A, SNAVELY N, et al. SfM with MRFs:discrete-continuous optimization for large-scale structure from motion[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12):2841-2853.
[17] CUI Z, TAN P. Global structure-from-motion by similarity averaging[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:864-872.
[18] ZHU S, ZHANG R, ZHOU L, et al. Very large-scale global SfM by distributed motion averaging[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:4568-4577.
[19] SWEENEY C, SATTLER T, HÖLLERER T, et al. Optimizing the viewing graph for structure-from-motion[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:801-809.
[20] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[21] TUYTELAARS T, MIKOLAJCZYK K. Local invariant feature detectors:a survey[J]. Foundations and Trends in Computer Graphics and Vision, 2007, 3:177-280.
[22] FISCHLER M A, BOLLES R C. Random sample consensus:a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6):381-395.
[23] LEPETIT V, MORENO-NOGUER F, FUA P. EPnP:an accurate O(n) solution to the PnP problem[J]. International Journal of Computer Vision, 2009, 81(2):No. 155.
[24] SHI J B, MALIK J. Normalized cuts and image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8):888-905.
[25] MOULON P, MONASSE P, PERROT R, et al. OpenMVG:open multiple view geometry[C]//Proceedings of the 1st International Workshop on Reproducible Research in Pattern Recognition, LNCS 10214. Cham:Springer, 2017:60-74.
[26] mapillary. OpenSfM:open source structure-from-motion pipeline[CP/OL].[2021-02-24]. https://github.com/mapillary/opensfm.

基于场景图划分的无人机影像定位算法

Unmanned aerial vehicle image positioning algorithm based on scene graph division

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

[1]	杨阿华李学军刘涛李东岳. 基于直接解算与迭代优化的相对定向方法[J]. 计算机应用, 2014, 34(6): 1706-1710.
[2]	岳洪伟汪仁煌何最红. 基于Normalized cut的羽毛杆自动提取[J]. 计算机应用, 2012, 32(07): 1899-1901.
[3]	吴庆双付仲良孟庆祥. 基于序列图像的摄像机自标定方法 [J]. 计算机应用, 2011, 31(11): 3010-3014.