Virtual-real registration method of natural features based on binary robust invariant scalable keypoints and speeded up robust features

doi:10.11772/j.issn.1001-9081.2019091621

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (5): 1403-1408.DOI: 10.11772/j.issn.1001-9081.2019091621

• Virtual reality and multimedia computing • Previous Articles Next Articles

Virtual-real registration method of natural features based on binary robust invariant scalable keypoints and speeded up robust features

ZHOU Xiang^1,2, TANG Liyu^1,2, LIN Ding^1,2

1.Key Lab of Spatial Data Mining & Information Sharing, Ministry of Education (Fuzhou University), FuzhouFujian 350108, China
2.National-Local Joint Engineering Research Center of Geospatial Information Technology (Fuzhou University), FuzhouFujian 350108, China

Received:2019-09-24 Revised:2019-11-12 Online:2020-05-15 Published:2020-05-10
Contact: TANG Liyu, born in 1972, Ph. D., research fellow. Her research interests include geographic visualization and virtual geographic environment, virtual plant.
About author:ZHOU Xiang, born in 1995, M. S. candidate. His research interests include geographic visualization and virtual geographic environment. TANG Liyu, born in 1972, Ph. D., research fellow. Her research interests include geographic visualization and virtual geographic environment, virtual plant. LIN Ding, born in 1977, Ph. D., research assistant. Her research interests include geographic visualization and virtual geographic environment.
Supported by:
This work is partially supported by the National Key Research and Development Program of China (2018YFB1004905).

基于二进制鲁棒不变尺度关键点-加速稳健特征的自然特征虚实注册方法

周翔^1,2, 唐丽玉^1,2, 林定^1,2

1.空间数据挖掘与信息共享教育部重点实验室(福州大学)，福州 350108
2.地理空间信息技术国家地方联合工程研究中心(福州大学)，福州 350108

通讯作者: 唐丽玉(1972—)
作者简介:周翔(1995—)，男，安徽马鞍山人，硕士研究生，主要研究方向：地学可视化与虚拟地理环境；唐丽玉(1972—)，女，福建莆田人，研究员，博士，主要研究方向：地学可视化与虚拟地理环境、虚拟植物；林定(1977—)，女，福建福州人，助理研究员，博士，主要研究方向：地学可视化与虚拟地理环境。
基金资助:
国家重点研发计划项目（2018YFB1004905）。

Abstract

Abstract:

Concerning the problem that the accuracy and real-time effects of virtual-real registration in Augmented Reality (AR) based on vision are greatly affected by the changes of illumination, occlusion and perspective, which is easy to lead to failure of registration, a virtual-real registration method of natural features based on Binary Robust Invariant Scalable Keypoints-Speeded Up Robust Features (BRISK-SURF) algorithm was proposed. Firstly, Speeded Up Robust Features (SURF) feature extractor was used to detect the feature points. Then, Binary Robust Invariant Scalable Keypoints (BRISK) descriptor was used to describe the feature points in binary, and the feature points were matched accurately and efficiently by combining Hamming distance. Finally, the virtual-real registration was realized according to the homography relationship between images. Experiments were performed from the aspects of image feature matching and virtual-real registration. Results show that the average precision of BRISK-SURF algorithm is basically the same with that of SURF algorithm, is about 25% higher than that of BRISK algorithm, and the average recall of BRISK-SURF is increased by about 10% compared to that of BRISK algorithm; the result of the virtual-real registration method based on BRISK-SURF is close to the reference standard data with high precision and good real-time performance. The Experimental results illustrate that the proposed method has high recognition accuracy, registration precision and real-time effects for images with different illuminations, occlusions and perspectives. Besides, the interactive tourist resource presentation and experience system based on AR is realized by using the proposed method.

Key words: Augmented Reality (AR), Binary Robust Invariant Scalable Keypoints-Speeded Up Robust Features (BRISK-SURF) algorithm, homography matrix, natural feature, virtual-real registration

摘要：

针对基于视觉的增强现实(AR)中虚实注册的准确率和实时效果受光照、遮挡和视角变化影响大，易导致注册失败的问题，提出一种基于二进制鲁棒不变尺度关键点-加速稳健特征(BRISK-SURF)算法的自然特征虚实注册方法。首先,利用加速稳健特征(SURF)特征提取算子检测特征点；然后,采用二进制尺度旋转不变鲁棒(BRISK)特征描述算子对特征点进行二进制描述，结合汉明距离实现准确高速的特征匹配；最后，根据图像间的单应性关系实现虚实注册。从图像特征匹配和虚实注册两方面进行实验，结果显示BRISK-SURF算法的平均准确率与SURF算法基本保持一致，比BRISK算法提高了约25%，平均召回率提高了约10%；基于BRISK-SURF的注册方法的结果接近参考标准数据，精度较高，实时性较好。实验结果表明，所提方法对于光照、遮挡和视角情况不同的图像具有较高的识别准确度、注册精度和实时效果。另外，使用此方法实现了基于AR的交互式旅游资源呈现与体验系统。

关键词: 增强现实, 二进制尺度旋转不变鲁棒-加速稳健特征算法, 单应性矩阵, 自然特征, 虚实注册

CLC Number:

ZHOU Xiang, TANG Liyu, LIN Ding. Virtual-real registration method of natural features based on binary robust invariant scalable keypoints and speeded up robust features[J]. Journal of Computer Applications, 2020, 40(5): 1403-1408.

周翔, 唐丽玉, 林定. 基于二进制鲁棒不变尺度关键点-加速稳健特征的自然特征虚实注册方法[J]. 计算机应用, 2020, 40(5): 1403-1408.

References

1 LIESTØL G , MORRISON A . Views , alignment and incongruity in indirect augmented reality[C]// Proceedings of the 2013 IEEE International Symposium on Mixed and Augmented Reality — Arts, Media, and Humanities. Piscataway: IEEE, 2013: 23-28.
2 王涌天,刘越,胡晓明 . 户外增强现实系统关键技术及其应用的研究[J]. 系统仿真学报, 2003, 15(3): 329-333. (WANG Y T, LIU Y, HU X M. Study on key technique and application of outdoor AR system[J]. Journal of System Simulation, 2003, 15(3): 329-333.)
3 YUAN M L , ONG S K, NEE A Y C . Registration using natural features for augmented reality systems[J]. IEEE Transactions on Visualization and Computer Graphics, 2006, 12(4): 569-580.
4 李乾,高尚兵,潘志庚,等 . 基于无标记识别的增强现实方法研究[J]. 系统仿真学报, 2018, 30(7): 2608-2614. LI Q , GAO S B , PAN Z G , et al . Research on augmented reality method based on unmarked recognition[J]. Journal of System Simulation, 2018, 30(7): 2608-2614.
5 桂振文,刘越,陈靖,等 . 基于自然场景在线学习的跟踪注册技术[J]. 软件学报, 2016, 27(11): 2929-2945. GUI Z W , LIU Y , CHEN J , et al . Online learning of tracking and registration based on natural scenes[J]. Journal of Software, 2016, 27(11): 2929-2945.
6 LOWE D G . Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
7 BAY H, ESS A, TUYTELAARS T , et al . Speeded-Up Robust Features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3): 346-359.
8 韩晓微,岳高峰,谢英红,等 . 动态视角下自主目标识别与跟踪[J]. 仪器仪表学报, 2019, 40(3): 221-229. HAN X W , YUE G F , XIE Y H , et al . Autonomous target recognition and tracking under dynamic perspective[J]. Chinese Journal of Scientific Instrument, 2019, 40(3): 221-229.
9 陈苏婷,郭子烨,张艳艳 . 基于局部哈希学习的大面阵CCD航拍图像匹配方法[J]. 应用光学, 2019, 40(2): 259-264. (CHEN S T, GUO Z Y, ZHANG Y Y. Matching method of large array CCD aerial images based on local hashing learning[J]. Journal of Applied Optics, 2019, 40(2): 259-264.)
10 LEUTENEGGER S , CHLI M , SIEGWART R Y . BRISK: binary robust invariant scalable keypoints[C]// Proceedings of the 2011 International Conference on Computer Vision. Piscataway: IEEE, 2011:2548-2555.
11 HEINLY J , DUNN E , FRAHM J M . Comparative evaluation of binary features[C]// Proceedings of the 2012 European Conference on Computer Vision, LNCS 7573. Berlin: Springer, 2012: 759-773.
12 BEKELE D , TEUTSCH M , SCHUCHERT T . Evaluation of binary keypoint descriptors[C]// Proceedings of the 2013 IEEE International Conference on Image Processing. Piscataway: IEEE, 2013: 3652-3656.
13 BILLINGHURST M , DUNSER A . Augmented reality in the classroom[J]. Computer, 2012, 45(7): 56-63.
14 YANG S , MEI B , YUE X . Mobile augmented reality assisted chemical education: insights from elements 4D[J]. Journal of Chemical Education, 2018, 95(6): 1060-1062.
15 陈靖,王涌天,林精敦,等 . 基于增强现实技术的圆明园景观数字重现[J]. 系统仿真学报. 2010, 22(2): 424-428. CHEN J , WANG Y T , LIN J D , et al . Digital reconstruction of Yuan Ming Yuan based on augmented reality technology[J]. Journal of System Simulation, 2010, 22(2): 424-428.
16 FENU C , PITTARELLO F . Svevo tour: the design and the experimentation of an augmented reality application for engaging visitors of a literary museum[J]. International Journal of Human Computer Studies, 2018, 114: 20-35.
17 HAN D , DIECK M C T , JUNG T . User experience model for augmented reality applications in urban heritage tourism[J]. Journal of Heritage Tourism, 2018, 13(1): 46-61.
18 YU J , LU F , LU C . Key technology and application research on mobile augmented reality[C]// Proceedings of the 7th IEEE International Conference on Software Engineering and Service Science. Piscataway: IEEE, 2016: 547-550.
19 YU Y , GUO Y , WANG R , et al . Registration based on ORB and FREAK features for augmented reality systems[J]. Transactions of Tianjin University, 2017, 23(2): 192-200.
20 MUKHERJEE D , WU Q M J , WANG G . A comparative experimental study of image feature detectors and descriptors[J]. Machine Vision and Applications, 2015, 26(4): 443-466.
21 MIKOLAJCZYK K , SCHMID C . Performance evaluation of local descriptors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(10): 1615-1630.
22 MIKOLAJCZYK K , TUYTELAARS T , SCHMID C , et al . A comparison of affine region detectors[J]. International Journal of Computer Vision, 2005, 65(1/2): 43-72.

[1]	TAN Zhaoyi, CHEN Baifan. Enhanced vehicle 3D surround view based on coordinate inverse mapping [J]. Journal of Computer Applications, 2021, 41(4): 1165-1171.
[2]	SUN Qichang, MAI Yongfeng, CHEN Xiaojun. Fast calibration algorithm in surgical navigation system based on augmented reality [J]. Journal of Computer Applications, 2021, 41(3): 833-838.
[3]	LUO Bin, YU Bo. Computation offloading strategy based on particle swarm optimization in mobile edge computing [J]. Journal of Computer Applications, 2020, 40(8): 2293-2298.
[4]	LU Xiaoyong, YOU Bin, LIN Pei-Yu, CHEN Musheng. Augmented reality approach based on digital camera and temporal psycho-visual modulation [J]. Journal of Computer Applications, 2017, 37(8): 2298-2301.
[5]	LI Hua, WANG Xuyang, YANG Huamin, HAN Cheng. Illumination direction measurement based on halo analysis in high-dynamic range images [J]. Journal of Computer Applications, 2016, 36(5): 1387-1393.
[6]	LI Pengjun, LI Jianzeng, SONG Yao, ZHANG Yan, DU Yulong. Error analysis of unmanned aerial vehicle remote sensing images stitching based on simulation [J]. Journal of Computer Applications, 2015, 35(4): 1116-1119.
[7]	XUE Zhen-hua WANG Ping ZHANG Chu-han CAI Si-jia. Improvement of the adversarial optimization approach for outlier removal in image matching [J]. Journal of Computer Applications, 2012, 32(11): 3157-3160.
[8]	LI Hong-bo WU Liang-liang WU Yu. Shadow generation algorithm of augmented reality using adaptive sampling and fusion [J]. Journal of Computer Applications, 2012, 32(07): 1860-1863.

Virtual-real registration method of natural features based on binary robust invariant scalable keypoints and speeded up robust features

基于二进制鲁棒不变尺度关键点-加速稳健特征的自然特征虚实注册方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 8

Recommended Articles

Metrics