基于鲁棒Restless Bandits模型的多水下自主航行器任务分配策略

doi:10.11772/j.issn.1001-9081.2019020341

计算机应用 ›› 2019, Vol. 39 ›› Issue (10): 2795-2801.DOI: 10.11772/j.issn.1001-9081.2019020341

• 人工智能 • 下一篇

基于鲁棒Restless Bandits模型的多水下自主航行器任务分配策略

李鑫滨, 章寿涛, 闫磊, 韩松

工业计算机控制工程河北省重点实验室(燕山大学), 河北秦皇岛 066004

收稿日期:2019-03-04 修回日期:2019-05-18 出版日期:2019-10-10 发布日期:2019-07-03
通讯作者: 李鑫滨
作者简介:李鑫滨(1969-),男,北京人,教授,博士,主要研究方向:水声通信网络优化、多机器人控制与优化、水下目标追踪、机器学习;章寿涛(1994-),男,江西上饶人,硕士研究生,主要研究方向:机器学习、多机器人控制与优化;闫磊(1989-),男,河北秦皇岛人,博士研究生,主要研究方向:多机器人控制与优化、水下目标追踪;韩松(1989-),男,河北石家庄人,讲师,博士,主要研究方向:博弈论、多机器人控制与优化。
基金资助:
国家自然科学基金资助项目（61873224，61571387）。

Multiple autonomous underwater vehicle task allocation policy based on robust Restless Bandit model

LI Xinbin, ZHANG Shoutao, YAN Lei, HAN Song

Hebei Key Laboratory of Industrial Computer Control Engineering(Yanshan University), Qinhuangdao Hebei 066004, China

Received:2019-03-04 Revised:2019-05-18 Online:2019-10-10 Published:2019-07-03
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61873224, 61571387).

摘要/Abstract

摘要： 针对水下监测网络中多自主航行器（AUV）协同信息采集任务分配问题进行了研究。首先，为了同时考虑系统中目标传感器的节点状态与声学信道状态对AUV任务分配问题的影响，构建了水声监测网络系统的综合模型；其次，针对水下存在的多未知干扰因素并考虑了模型产生不精确的情况，基于强化学习理论将多AUV任务分配系统建模为鲁棒无休止赌博机问题（RBP）。最后，提出鲁棒Whittle算法求解所建立的RBP，从而求解得出多AUV的任务分配策略。仿真结果表明，在干扰环境下与未考虑干扰因素的分配策略相比，在系统分别选择1、2、3个目标时，鲁棒AUV分配策略对应的系统累计回报值参数的性能分别提升了5.5%、12.3%和9.6%，验证了所提方法的有效性。

关键词: 水声监测网络, 水下自主航行器任务分配, 鲁棒控制, 不确定模型, 无休止赌博机问题

Abstract: The problem of multiple Autonomous Underwater Vehicles (AUV) collaborative task allocation for information acquisition in the underwater detection network was researched. Firstly, a comprehensive model of underwater acoustic monitoring network system was constructed considering the influence of network system sensor nodes status and communication channel status synthetically. Secondly, because of the multi-interference factors under water, with the inaccuracy of the model generation considered, and the multi-AUV task allocation system was modeled as a robust Restless Bandits Problem (RBP) based on the theory of reinforce learning. Lastly, the robust Whittle algorithm was proposed to solve the RBP problem to get the task allocation policy of multi-AUV. Simulation results show that when the system selected 1, 2 and 3 targets, the system cumulative return performance of the robust allocation policy improves by 5.5%, 12.3% and 9.6% respectively compared with that of the allocation strategy without interference factors considered, proving the effectiveness of the proposed approaches.

Key words: underwater acoustic monitoring network, Autonomous Underwater Vehicles (AUV) task allocation, robust control, inaccuracy model, Restless Bandit Problem (RBP)

中图分类号:

TP181

李鑫滨, 章寿涛, 闫磊, 韩松. 基于鲁棒Restless Bandits模型的多水下自主航行器任务分配策略[J]. 计算机应用, 2019, 39(10): 2795-2801.

LI Xinbin, ZHANG Shoutao, YAN Lei, HAN Song. Multiple autonomous underwater vehicle task allocation policy based on robust Restless Bandit model[J]. Journal of Computer Applications, 2019, 39(10): 2795-2801.

参考文献

[1] LUDVIGSEN M, SØRENSEN A J. Towards integrated autonomous underwater operations for ocean mapping and monitoring[J]. Annual Reviews in Control, 2016, 42:145-157.
[2] 于文浩, 赵英. 基于MOOS的AUV通信系统的软件系统设计[J]. 中国新通信, 2017, 19(6):2-3. (YU W H, ZHAO Y. Software system design of AUV communication system based on MOOS[J]. China New Telecommunications, 2017, 19(6):2-3.)
[3] KULKARNI I S, POMPILI D. Task allocation for networked autonomous underwater vehicles in critical missions[J]. IEEE Journal on Selected Areas in Communications, 2010, 28(5):716-727.
[4] 李相民, 颜骥, 刘波, 等. 多智能体协同任务分配问题研究综述[J]. 计算机与数字工程, 2014, 42(12):2443-2450. (LI X M, YAN J, LIU B, et al. A survey of multi-Agents cooperative task allocation research[J]. Computer and Digital Engineering, 2014, 42(12):2443-2450.)
[5] FANG D, RAN X. A distributed sensor management algorithm based on auction[J]. Procedia Computer Science, 2017, 107:618-623.
[6] HU J, YANG J C. Application of distributed auction to multi-UAV task assignment in agriculture[J]. International Journal of Precision Agricultural Aviation, 2018, 1(1):44-50.
[7] SCHWARZROCK J, ZACARIAS I, BAZZAN A L C, et al. Solving task allocation problem in multi unmanned aerial vehicles systems using swarm intelligence[J]. Engineering Applications of Artificial Intelligence, 2018, 72:10-20.
[8] TKACH I, JEVTIC A, NOF S Y, et al. A modified distributed bees algorithm for multi-sensor task allocation[J]. Sensors, 2018, 18(3):NO.759.
[9] PUJOL-GONZALEZ M, CERQUIDES J, MESEGUER P, et al. Decentralized dynamic task allocation for UAVs with limited communication range[EB/OL].[2018-12-20]. https://arxiv.org/pdf/1809.07863v1.pdf.
[10] JANG I, SHIN H S, TSOURDOS A. A comparative study of game-theoretical and Markov-chain-based approaches to division of labour in a robotic swarm[J]. IFAC-PapersOnLine, 2018, 51(12):62-68.
[11] LUO Y, PU L, ZUBA M, et al. Challenges and opportunities of underwater cognitive acoustic networks[J]. IEEE Transactions on Emerging Topics in Computing, 2014, 2(2):198-211.
[12] FERRI G, BATES J, STINEO P, et al. Autonomous underwater surveillance networks:a task allocation framework to manage cooperation[C]//Proceedings of the 2018 MTS/IEEE Kobe Techno-Oceans. Piscataway:IEEE, 2018:1-10.
[13] LI J, ZHANG R, YANG Y. Multi-AUV autonomous task planning based on the scroll time domain quantum bee colony optimization algorithm in uncertain environment[J]. PLoS One, 2017, 12(11):No.e0188291.
[14] GITTINS J C. Bandit processes and dynamic allocation indices[J]. Journal of the Royal Statistical Society:Series B (Methodological), 1979, 41(2):148-177.
[15] WHITTLE P. Restless bandits:activity allocation in a changing world[J]. Journal of Applied Probability, 1988, 25(A):287-298.
[16] LI X, LIU J, YAN L, et al. Relay selection for underwater acoustic sensor networks:a multi-user multi-armed bandit formulation[J]. IEEE Access, 2018, 6:7839-7853.
[17] MUKHERJEE A, MISRA S, CHANDRA V S P, et al. Resource-optimized multi-armed bandit based offload path selection in edge UAV swarms[J]. IEEE Internet of Things Journal, 2019, 6(3):4889-4896.
[18] NIÑO-MORA J, VILLAR S S. Sensor scheduling for hunting elusive hiding targets via Whittle's restless bandit index policy[C]//Proceedings of the 2011 International Conference on Network Games, Control and Optimization. Piscataway:IEEE, 2011:1-8.
[19] FRYER R G, Jr., HARMS P. Two-armed restless bandits with imperfect information:stochastic control and indexability[J]. Mathematics of Operations Research, 2018, 43(2):399-427.
[20] LI J Y, KWON R H. Portfolio selection under model uncertainty:a penalized moment-based optimization approach[J]. Journal of Global Optimization, 2013, 56(1):131-164.
[21] CARO F, das GUPTA A. Robust control of the multi-armed bandit problem[EB/OL].[2019-02-10]. https://doi.org/10.1007/s10479-015-1965-7.
[22] KIM M J, LIM A E B. Robust multiarmed bandit problems[J]. Management Science, 2016, 62(1):264-285.
[23] TIBSHIRANI R. Regression shrinkage and selection via the Lasso[J]. Journal of the Royal Statistical Society:Series B (Methodological), 1996, 58(1):267-288.
[24] NILIM A, el GHAOUI L. Robust control of Markov decision processes with uncertain transition matrices[J]. Operations Research, 2005, 53(5):780-798.

基于鲁棒Restless Bandits模型的多水下自主航行器任务分配策略

Multiple autonomous underwater vehicle task allocation policy based on robust Restless Bandit model

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

编辑推荐

Metrics

[1]	黄浩, 唐昊, 周雷, 程文娟. 服务率不确定的单站点传送带给料加工站系统鲁棒优化控制[J]. 计算机应用, 2015, 35(7): 2067-2072.
[2]	李述清张胜修张煜东胡波. 连续搅拌反应釜过程的闭环增益成形PID控制器设计[J]. 计算机应用, 2011, 31(02): 483-484.