Architecture design of data fusion pipeline for unmanned systems

doi:10.11772/j.issn.1001-9081.2023081184

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (8): 2536-2543.DOI: 10.11772/j.issn.1001-9081.2023081184

• Computer software technology • Previous Articles Next Articles

Architecture design of data fusion pipeline for unmanned systems

Yi LIU, Guoli YANG, Qibin ZHENG(), Xiang LI, Yangsen ZHOU, Depeng CHEN

Advanced Institute of Big Data （Beijing），Beijing 100195，China

Received:2023-09-03 Revised:2023-10-08 Accepted:2023-10-17 Online:2024-08-22 Published:2024-08-10
Contact: Qibin ZHENG
About author:LIU Yi， born in 1990， Ph. D.， assistant research fellow. His research interests include intelligent data engineering， evolutionary algorithm.
YANG Guoli， born in 1987， Ph. D.， associate research fellow. His research interest include knowledge computing.
LI Xiang， born in 1988， Ph. D.， assistant research fellow. His research interest include big data.
ZHOU Yangsen， born in 1991， M. S.， engineer. His research interest include robot.
First author contact:CHEN Depen， born in 1995， M. S.， engineer. His research interest include robot.
Supported by:
National Science Foundation for Young Scientists of China(72201275);Young Elite Scientists Sponsorship Program by CAST(2022QNRC001)

无人系统数据融合流水线架构设计

刘艺, 杨国利, 郑奇斌(), 李翔, 周杨森, 陈德鹏

北京大数据先进技术研究院，北京 100195

通讯作者: 郑奇斌
作者简介:刘艺（1990—），男（回族），安徽蚌埠人，助理研究员，博士，主要研究方向：智能数据工程、演化算法
杨国利（1987—），男，河北石家庄人，副研究员，博士，主要研究方向：知识计算
郑奇斌（1990—），男，甘肃兰州人，助理研究员，博士，主要研究方向：数据工程、机器学习 zhengqb@aibd.ac.cn
李翔（1988—），男，四川资中人，助理研究员，博士，主要研究方向：大数据
周杨森（1991—），男，河北保定人，工程师，硕士，主要研究方向：机器人
陈德鹏（1995—），男，天津人，工程师，硕士，主要研究方向：机器人。
基金资助:
国家自然科学青年基金资助项目(72201275);第八届中国科协青年人才托举工程项目(2022QNRC001)

Abstract

Abstract:

Sensors are the basis for unmanned systems to perform intelligent actions. The fusion of multi-sensor data can enhance intelligent perception and autonomous decision-making capabilities of unmanned systems， and improve the reliability and robustness of these systems. Data fusion of unmanned systems encounters many challenges such as diverse sensor types， heterogeneous data formats， real-time needs of data fusion and analysis， as well as complex types and fast evolution of algorithm models. Traditional methods of developing fusion models through customization on front end and approaches based on fusion platform running on back end are difficult to apply in these cases. Therefore， a pipeline platform for data fusion was proposed. This platform has capabilities to support automatic data transformation， flexible algorithm combination， dynamic model configuration， and rapid iteration of functions to achieve dynamic and quick data fusion model construction and provide information service for different tasks. Based on the analysis of data fusion process and techniques， the pipeline framework and its key functions and components were characterized， the key technologies that urgently need breakthroughs were analyzed， the running way and actual case of the framework were given， and research directions for future development were pointed out.

Key words: data fusion, unmanned system, pipeline, sensing data, algorithm service

摘要：

传感器是无人系统智能化行动的基础，而通过融合多传感器的数据能增强无人系统的智能感知和自主决策能力，提升无人系统的可靠性和鲁棒性。无人系统的数据融合面临传感器类型多样、数据格式异构、数据融合分析的实时性强，以及算法模型种类复杂、更新演化快等挑战，传统定制化开发前端融合模型和基于后端融合平台的方法难以适用。因此，提出一种面向数据融合的流水线平台，以支持数据自动转换、算法灵活组合、模型高可配置、功能快速迭代，且能面向任务，动态、快速构建数据融合模型并提供信息服务。在剖析无人系统数据融合流程和技术体系的基础上，设计流水线框架及其关键功能构件，分析亟待突破的关键技术，给出框架的运行方式和实际案例，探讨未来的发展方向。

关键词: 数据融合, 无人系统, 流水线, 感知数据, 算法服务

CLC Number:

TP391

Yi LIU, Guoli YANG, Qibin ZHENG, Xiang LI, Yangsen ZHOU, Depeng CHEN. Architecture design of data fusion pipeline for unmanned systems[J]. Journal of Computer Applications, 2024, 44(8): 2536-2543.

刘艺, 杨国利, 郑奇斌, 李翔, 周杨森, 陈德鹏. 无人系统数据融合流水线架构设计[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2536-2543.

Figures/Tables 6

Fig. 1 Data fusion model of unmanned system

Fig. 2 Data fusion technical framework of unmanned system

Fig. 3 Data fusion pipeline framework of unmanned system

Fig. 4 Flowchart of applying data fusion pipeline of unmanned systems

Fig. 5 Data fusion pipeline of unmanned vehicle

Fig. 6 Data fusion positioning trajectory of unmanned vehicle

References 49

1	刘艺，李蒙蒙，郑奇斌，等. 视频目标跟踪算法综述［J］. 计算机科学与探索， 2022， 16（7）： 1504-1515.
	LIU Y， LI M M， ZHENG Q B， et al. Survey on video object tracking algorithms［J］. Journal of Frontiers of Computer Science and Technology， 2022， 16（7）： 1504-1515.
2	尹作重，刘继红，王佃鹏，等. 基于微服务的多机器人资源智能云服务平台架构［J］. 计算机集成制造系统， 2022， 28（7）： 2102-2111.
	YIN Z Z， LIU J H， WANG D P， et al. Multi-robot resource intelligent cloud service platform based on microservice［J］. Computer Integrated Manufacturing Systems， 2022， 28（7）： 2102-2111.
3	VÉLEZ-NICOLÁS M， GARCÍA-LÓPEZ S， BARBERO L， et al. Applications of Unmanned Aerial Systems （UASs） in hydrology： a review［J］. Remote Sensing， 2021， 13（7）： No.1359.
4	RAHMAN M F F， FAN S， ZHANG Y， et al. A comparative study on application of unmanned aerial vehicle systems in agriculture［J］. Agriculture， 2021， 11（1）： No.22.
5	RENAULT A. A model for assessing UAV system architectures［J］. Procedia Computer Science， 2015， 61： 160-167.
6	QIU S， ZHAO H， JIANG N， et al. Multi-sensor information fusion based on machine learning for real applications in human activity recognition： state-of-the-art and research challenges［J］. Information Fusion， 2022， 80（4）： 241-265.
7	LIU W， ZHANG T， HUANG S， et al. A hybrid optimization framework for UAV reconnaissance mission planning［J］. Computers and Industrial Engineering， 2022， 173： No.108653.
8	WAN L， LIU R， SUN L， et al. UAV swarm based radar signal sorting via multi-source data fusion： a deep transfer learning framework［J］. Information Fusion， 2022， 78： 90-101.
9	CASTANEDO F. A review of data fusion techniques［J］. The Scientific World Journal， 2013， 2013： No.704504.
10	DIEZ-OLIVAN A， DER SER J， GALAR D， et al. Data fusion and machine learning for industrial prognosis： trends and perspectives towards Industry 4.0［J］. Information Fusion， 2019， 50： 92-111.
11	LUO R C， KAY M G. Multisensor integration and fusion in intelligent systems［J］. IEEE Transactions on Systems， Man， and Cybernetics， 1989， 19（5）： 901-931.
12	AYED S B， TRICHILI H M， ALIMI A. Data fusion architectures： a survey and comparison［C］// Proceedings of the 15th International Conference on Intelligent Systems Design and Applications. Piscataway： IEEE， 2015： 277-282.
13	DASARATHY B V. Sensor fusion potential exploitation-innovative architectures and illustrative applications［J］. Proceedings of the IEEE， 1997， 85（1）： 24-38.
14	BLASCH E， STEINBERG A， DAS S， et al. Revisiting the JDL model for information exploitation［C］// Proceedings of the 16th International Conference on Information Fusion. Piscataway： IEEE， 2013： 129-136.
15	ULLAH I， YOUN H Y. Intelligent data fusion for smart IoT environment： a survey［J］. Wireless Personal Communications， 2020， 114（1）： 409-430.
16	ARAGÓN M E， CARLOS M R， GURROLA L C G， et al. A machine learning pipeline to automatically identify and classify roadway surface disruptions［C］// Proceedings of the 16th Mexican International Conference on Computer Science. New York： ACM， 2016： No.3.
17	ZUO S， LI J， LI Y， et al. Data processing pipeline for Tianlai experiment［J］. Astronomy and Computing， 2021， 34： No.100439.
18	DING X， CHEN L， GAO Y， et al. UlTraMan： a unified platform for big trajectory data management and analytics［J］. Proceedings of the VLDB Endowment， 2018， 11（7）： 787-799.
19	POOJARA S R， DEHURY C K， DEHURY P， et al. Serverless data pipeline approaches for IoT data in fog and cloud computing［J］. Future Generation Computer Systems， 2022， 130： 91-105.
20	REZIG E K， BRAHMAROUTU A， TATBUL N， et al. Debugging large-scale data science pipelines using dagger［J］. Proceedings of the VLDB Endowment， 2020， 13（12）： 2993-2996.
21	PEREIRA A， PROENÇA A. HEP-Frame： improving the efficiency of pipelined data transformation & filtering for scientific analyses［J］. Computer Physics Communications， 2021， 263： No.107844.
22	MISALE C， DROCCO M， TREMBLAY G， et al. PiCo： high-performance data analytics pipelines in modern C++［J］. Future Generation Computer Systems， 2018， 87： 392-403.
23	YANG C， FAN J， WU Z， et al. AutoML pipeline selection： efficiently navigating the combinatorial space［C］// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2020： 1446-1456.
24	DE PRADO M， SU J， SAEED R， et al. Bonseyes AI Pipeline — bringing AI to you： end-to-end integration of data， algorithms， and deployment tools［J］. ACM Transactions on Internet of Things， 2020， 1（4）： No.26.
25	FROST R， PAUL D， LI F. AI Pro： data processing framework for AI Models［C］// Proceedings of the IEEE 35th International Conference on Data Engineering. Piscataway： IEEE， 2019： 1980-1983.
26	REZIG E K， CAO L， STONEBRAKER M， et al. Data Civilizer 2.0： a holistic framework for data preparation and analytics［J］. Proceedings of the VLDB Endowment， 2019， 12（12）： 1954-1957.
27	OLEGHE O， SALONITIS K. A framework for designing data pipelines for manufacturing systems［J］. Procedia CIRP， 2020， 93： 724-729.
28	PERVAIZ F， VASHISTHA A， ANDERSON R. Examining the challenges in development data pipeline ［C］// Proceedings of the 2nd ACM SIGCAS Conference on Computing and Sustainable Societies. New York： ACM， 2019： 13-21.
29	ALLA S， ADARI S K. Introduction to MLFlow［M］// Beginning MLOps with MLFlow： Deploy Models in AWS SageMaker， Google Cloud， and Microsoft Azure. Berkeley： Apress， 2021： 125-227.
30	ZHAO J， ZHANG X， YAN J， et al. A wheat spike detection method in UAV images based on improved YOLOv5［J］. Remote Sensing， 2021， 13（16）： No.3095.
31	LI Y， YU L， LIU J， et al. NetDPO：（delta， gamma）-approximate pattern matching with gap constraints under one-off condition［J］. Applied Intelligence， 2022， 52（11）： 12155-12174.
32	丁玥，王涓，卢卫，等. 面向多源关系数据的融合［J］. 中国科学：信息科学， 2020， 50（5）： 649-661.
	DING Y， WANG J， LU W， et al. Multi-source relational data fusion［J］. SCIENTIA SINICA Informationis， 2020， 50（5）： 649-661.
33	NAFA Y， CHEN Q， CHEN Z， et al. Active deep learning on entity resolution by risk sampling［J］. Knowledge-Based Systems， 2022， 236： No.107729.
34	谭明超，刁兴春，曹建军. 实体分辨研究综述［J］. 计算机科学， 2014， 41（4）： 9-12， 20.
	TAN M C， DIAO X C， CAO J J. Survey on entity resolution［J］. Computer Science， 2014， 41（4）： 9-12， 20.
35	曹建军，常宸，陶嘉庆，等. 多源文本数据真值发现方法［J］. 国防科技大学学报， 2022， 44（4）： 172-179.
	CAO J J， CHANG C， TAO J Q， et al. Truth discovery method for multi-source text data［J］. Journal of National University of Defense Technology， 2022， 44（4）： 172-179.
36	YIN L， LUO J， LI K. An optimal image storage strategy for container-based edge computing in smart factory［J］. IEEE Internet of Things Journal， 2023， 10（8）： 7204-7214.
37	DUDCZYK J， CZYBA R， SKRZYPCZYK K. Multi-sensory data fusion in terms of UAV detection in 3D space［J］. Sensors， 2022， 22（12）： No.4323.
38	GUPTA A， FERNANDO X. Simultaneous Localization And Mapping （SLAM） and data fusion in unmanned aerial vehicles： recent advances and challenges［J］. Drones， 2022， 6（4）： No.85.
39	LIN C， CHENG Y， WANG X. A convolutional neural network particle filter for UUV target state estimation［J］. IEEE Transactions on Instrumentation and Measurement， 2022， 71： No.2508412.
40	LIU W， LIU Y， BUCKNALL R. Filtering based multi-sensor data fusion algorithm for a reliable unmanned surface vehicle navigation［J］. Journal of Marine Engineering and Technology， 2023， 22（2）： 67-83.
41	张卫民，刘灿灿，骆志刚. 科学工作流技术研究综述［J］. 国防科技大学学报， 2011， 33（3）： 56-65.
	ZHANG W M， LIU C C， LUO Z G. A review on scientific workflows［J］. Journal of National University of Defense Technology， 2011， 33（3）： 56-65.
42	文一凭，侯俊杰，谭铮，等. 面向交互式建模的科学工作流活动推荐方法［J］. 计算机集成制造系统， 2022， 28（10）： 3115-3121.
	WEN Y P， HOU J J， TAN Z， et al. Scientific workflow activity recommendation for interactive modeling［J］. Computer Integrated Manufacturing Systems， 2022， 28（10）： 3115-3121.
43	TSOUTSA P， FITSILIS P， ANTHOPOULOS L， et al. Nexus services in smart city ecosystems［J］. Journal of the Knowledge Economy， 2021， 12： 431-451.
44	李庚松，刘艺，秦伟，等. 面向算法选择的元学习研究综述［J］. 计算机科学与探索， 2023， 17（1）： 88-107.
	LI G S， LIU Y， QIN W， et al. Survey on meta-learning research of algorithm selection［J］. Journal of Frontiers of Computer Science and Technology， 2023， 17（1）： 88-107.
45	李庚松，刘艺，郑奇斌，等. 基于多目标混合蚁狮优化的算法选择方法［J］. 计算机研究与发展， 2023， 60（7）： 1533-1550.
	LI G S， LIU Y， ZHENG Q B， et al. Algorithm selection method based on multi-objective hybrid ant lion optimizer［J］. Journal of Computer Research and Development， 2023， 60（7）： 1533-1550.
46	KERSCHKE P， HOOS H H， NEUMANN F， et al. Automated algorithm selection： survey and perspectives［J］. Evolutionary Computation， 2019， 27（1）： 3-45.
47	RAUT A， SHIVHARE A， CHAURASIYA V K， et al. AEDS-IoT： adaptive clustering-based event detection scheme for IoT data streams［J］. Internet of Things， 2023， 22： No.100704.
48	李于锋，莫则尧，肖永浩，等. 超算环境科学工作流应用的容错［J］. 国防科技大学学报， 2020， 42（6）： 82-89.
	LI Y F， MO Z Y， XIAO Y H， et al. Fault tolerance in HPC scientific workflow application［J］. Journal of National University of Defense Technology， 2020， 42（6）： 82-89.
49	SOLÀ J. Quaternion kinematics for the error-state Kalman filter［EB/OL］. （2017-11-03）［2023-07-08］..

[1]	Wei LUO, Jinquan LIU, Zheng ZHANG. Dual vertical federated learning framework incorporating secret sharing technology [J]. Journal of Computer Applications, 2024, 44(6): 1872-1879.
[2]	Xiang LIU, Bei HUA, Fei LIN, Hongyuan WEI. Design and implementation of component-based development framework for deep learning applications [J]. Journal of Computer Applications, 2024, 44(2): 526-535.
[3]	Xiuli REN, Lei ZHANG. Improved multi-primary-node consensus mechanism based on practical Byzantine fault tolerance [J]. Journal of Computer Applications, 2022, 42(5): 1500-1507.
[4]	Haiwen XU, Jiacai SHI, Teng WANG. Departure flight delay prediction model based on deep fully connected neural network [J]. Journal of Computer Applications, 2022, 42(10): 3283-3291.
[5]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[6]	QU Jingyi, CAO Lei, CHEN Min, DONG Liang, CAO Yexiu. CliqueNet flight delay prediction model based on clique random connection [J]. Journal of Computer Applications, 2020, 40(8): 2420-2427.
[7]	XIONG Yong, ZHANG Jia, YU Jiajun, ZHANG Benren, LIANG Xuanzhuo, ZHU Qige. Intelligent layout optimization algorithm for 3D pipelines of ships [J]. Journal of Computer Applications, 2020, 40(7): 2164-2170.
[8]	ZHU Xiaojie, ZHAO Zihao, DU Yi. PiFlow: model driven big data pipeline framework [J]. Journal of Computer Applications, 2020, 40(6): 1638-1647.
[9]	CHEN Guihui, YI Xin, LI Zhongbing, QIAN Jiren, CHEN Wu. Third-party construction target detection in aerial images of pipeline inspection based on improved YOLOv2 and transfer learning [J]. Journal of Computer Applications, 2020, 40(4): 1062-1068.
[10]	LIN Zheng, LYU Xiafu. Autonomous obstacle avoidance of unmanned surface vessel based on improved fuzzy algorithm [J]. Journal of Computer Applications, 2019, 39(9): 2523-2528.
[11]	HUANG Fengyu, WU Yefu, CHEN Jingren, WU Bing. Traffic behavior spectrum analysis method based on regional road live data [J]. Journal of Computer Applications, 2019, 39(3): 907-912.
[12]	DING Jingquan, MA Bo, LI Xiao. Multi-view deep anomaly detection framework for vehicle refueling behaviors based on spatio-temporal data fusion [J]. Journal of Computer Applications, 2019, 39(11): 3370-3375.
[13]	QIU Jinyi, LUO Jun, LI Xiu, JIA Wei, NI Fuchuan, FENG Hui. Multi-scale grape image recognition method based on convolutional neural network [J]. Journal of Computer Applications, 2019, 39(10): 2930-2936.
[14]	WU Renbiao, LI Jiayi, QU Jingyi. Flight delay prediction model based on dual-channel convolutional neural network [J]. Journal of Computer Applications, 2018, 38(7): 2100-2106.
[15]	XU Shuang, ZHANG Qian, LI Yan, LIU Jiayong. Multi-source point of interest fusion algorithm based on distance and category [J]. Journal of Computer Applications, 2018, 38(5): 1334-1338.

Architecture design of data fusion pipeline for unmanned systems

无人系统数据融合流水线架构设计

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 49

Related Articles 15

Recommended Articles

Metrics