Resource allocation algorithm for low earth orbit satellites oriented to user demand

doi:10.11772/j.issn.1001-9081.2023050561

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (4): 1242-1247.DOI: 10.11772/j.issn.1001-9081.2023050561

• Network and communications • Previous Articles

Resource allocation algorithm for low earth orbit satellites oriented to user demand

Fatang CHEN, Miao HUANG, Yufeng JIN

School of Communications and Information Engineering，Chongqing University of Posts and Telecommunications，Chongqing 400065，China

Received:2023-05-09 Revised:2023-07-01 Accepted:2023-07-09 Online:2023-08-01 Published:2024-04-10
Contact: Miao HUANG
About author:CHEN Fatang， born in 1965， M. S.， research fellow. His research interests include physical layer algorithm of wireless communication.
HUANG Miao， born in 1998， M. S. candidate. His research interests include radio resource allocation of LEO satellite， resource scheduling of mobile communication MAC layer.
JIN Yufeng， born in 1999， M. S. candidate. His research interests include edge calculation and unloading of LEO satellite.
Supported by:
Chongqing Natural Science Foundation(cstc2021jcyj-msxmX0454)

面向用户需求的低轨卫星资源分配算法

陈发堂, 黄淼, 金宇峰

重庆邮电大学通信与信息工程学院，重庆 400065

通讯作者: 黄淼
作者简介:陈发堂（1965—），男，重庆人，研究员，硕士，主要研究方向：无线通信物理层算法
黄淼（1998—），男，四川达州人，硕士研究生，主要研究方向：低轨卫星无线资源分配、移动通信MAC层资源调度 S210101054@stu.cqupt.edu.cn
金宇峰（1999—），男，湖北荆州人，硕士研究生，主要研究方向：低轨卫星边缘计算与卸载。
基金资助:
重庆市自然科学基金资助项目(cstc2021jcyj?msxmX0454)

Abstract

Abstract:

In Low Earth orbit （LEO）satellite multi-beam communication scenario， the traditional fixed resource allocation algorithm can not meet the differences in channel capacity requirements of different users. In order to meet the requirements of users， the optimization model of minimum supply-demand difference of combining channel allocation， bandwidth allocation and power allocation was established， and Pattern Division Multiple Access technology （PDMA）was introduced to improve the utilization of channel resources. In view of the non-convex characteristic of the model， the optimal resource allocation strategy learned by the Q-learning algorithm was used to allocate the channel capacity suitable for each user， and a reward threshold was introduced to further improve the algorithm， speeding up the convergence and minimizing the difference between supply and demand when the algorithm converged. The simulation results show that the convergence speed of the improved algorithm is about 3.33 times that before improvement； the improved algorithm can meet larger user requirement， about 14% higher than the Q-learning algorithm before improvement， about 2.14 times that of the traditional fixed algorithm.

Key words: Low Earth Orbit (LEO) satellite, multi-beam, resource allocation, reinforcement learning, Pattern Division Multiple Access (PDMA)

摘要：

低轨（LEO）卫星多波束通信场景下，传统固定资源分配算法无法满足不同用户对信道容量的差异需求。以适应用户需求分配为主要目标，建立联合信道分配、带宽分配和功率分配的最小供需差优化模型，并引入图样分割多址接入技术（PDMA）提升信道资源的利用率。针对该模型的非凸特性，通过Q-learning算法学习资源分配最优策略为每个用户分配适合的信道容量，并引入奖励阈值进一步改进算法，加快算法的收敛，且使算法达到收敛时供需差异更小。仿真结果表明，改进后的算法收敛速度约是改进前的3.33倍：改进算法能满足更大的用户需求，比改进前Q-learning算法提升14%，是传统固定算法的2.14倍。

关键词: 低轨卫星, 多波束, 资源分配, 强化学习, 图样分割多址接入

CLC Number:

TN927

Fatang CHEN, Miao HUANG, Yufeng JIN. Resource allocation algorithm for low earth orbit satellites oriented to user demand[J]. Journal of Computer Applications, 2024, 44(4): 1242-1247.

陈发堂, 黄淼, 金宇峰. 面向用户需求的低轨卫星资源分配算法[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1242-1247.

Figures/Tables 8

Fig. 1 Beam coverage schematic diagram of LEO satellite

Fig. 2 Earth coordinate model

Fig. 3 Schematic diagram of beam interference

Tab. 1 Simulation parameters of LEO satellite environment and algorithm

仿真参数	仿真取值
卫星高度H	600 km
卫星经纬度 $(s l o n, s l a t)$	（30°E，105°N）
载波频率 $f$	20 GHz
最大波束功率 $P b$	30 W
最大波束带宽 $B b$	40 MHz
信道最大传输用户数 $N m a x$	4
单个波束用户数	1~50
单个用户需求 $D b, u$	0~20 Mbit·s^-1
$ξ b, c d B$ 的均值 $μ$	-2.70 dB
发射天线辐射增益 $Γ b, u (θ t)$	26 dB
接收天线辐射增益 $Γ b, u (θ r)$	14 dB
3 dB角度 $θ 3 d B b$	0.48°
高斯白噪声功率谱密度 $n 0$	-174 dBm·Hz^-1
初始化探索概率 $ε 0$	0.99
学习率 $α$	1
折扣因子 $γ$	0.90

Tab. 1 Simulation parameters of LEO satellite environment and algorithm

仿真参数	仿真取值
卫星高度H	600 km
卫星经纬度 $(s l o n, s l a t)$	（30°E，105°N）
载波频率 $f$	20 GHz
最大波束功率 $P b$	30 W
最大波束带宽 $B b$	40 MHz
信道最大传输用户数 $N m a x$	4
单个波束用户数	1~50
单个用户需求 $D b, u$	0~20 Mbit·s^-1
$ξ b, c d B$ 的均值 $μ$	-2.70 dB
发射天线辐射增益 $Γ b, u (θ t)$	26 dB
接收天线辐射增益 $Γ b, u (θ r)$	14 dB
3 dB角度 $θ 3 d B b$	0.48°
高斯白噪声功率谱密度 $n 0$	-174 dBm·Hz^-1
初始化探索概率 $ε 0$	0.99
学习率 $α$	1
折扣因子 $γ$	0.90

Fig. 4 Convergence performance of algorithm before and after improvement

Fig. 5 Relationship between user demand and allocated bandwidth under different algorithms

Fig. 6 Relationship between user demand and allocated power under different algorithms

Fig.7 Relationship between user demand and system supply under different algorithms

References 20

1	CHEN S， ZHAO J. The requirements， challenges， and technologies for 5G of terrestrial mobile telecommunication［J］. IEEE Communications Magazine， 2014，52（5）：36-43. 10.1109/mcom.2014.6815891
2	AGIWAL M， ROY A， SAXENA N. Next generation 5G wireless networks： a comprehensive survey［J］. IEEE Communications Surveys & Tutorials， 2016， 18（3）：1617-1655. 10.1109/comst.2016.2532458
3	柴蓉，邹飞，刘莎，等. 6G移动通信：愿景、关键技术和系统架构［J］. 重庆邮电大学学报（自然科学版）， 2021， 33（3）： 337-347. 10.3979/j.issn.1673-825X.202010270328
	CHAI R， ZOU F， LIU S， et al. 6G mobile communication： vision， key technologies and system architecture［J］. Journal of Chongqing University of Posts and Telecommunications （Natural Science Edition）， 2021， 33 （3）： 337-347. 10.3979/j.issn.1673-825X.202010270328
4	WANG W， TONG Y， LI L， et al. Near optimal timing and frequency offset estimation for 5G integrated LEO satellite communication system［J］. IEEE Access， 2019， 7： 113298-113310. 10.1109/access.2019.2935038
5	TANG J， BIAN D， LI G， et al. Resource allocation for LEO beam-hopping satellites in a spectrum sharing scenario［J］. IEEE Access， 2021， 9： 56468-56478. 10.1109/access.2021.3072059
6	LIAO Q， KANEKO M. Global energy efficiency optimization of a Ka-band multi-beam LEO satellite communication system［J］. IEEE Access， 2021， 9： 55232-55243. 10.1109/access.2021.3071475
7	梁承超，段瑞吉，麻世庆，等. 面向能效的低轨卫星联合跳波束调度和功率分配算法［J］. 电子与信息学报， 2023，45（2）： 436-445. 10.11999/JEIT220392
	LIANG C C， DUAN R J， MA S Q， et al. Joint beam hopping scheduling and power allocation of LEO satellites oriented energy efficiency［J］. Journal of Electronics & Information Technology， 2023，45（2）： 436-445. 10.11999/JEIT220392
8	徐东明，谭静茹，关文博.基于改进遗传算法的C-RAN网络动态无线资源分配［J］.电讯技术，2021，61（10）：1225-1232. 10.3969/j.issn.1001-893x.2021.10.005
	XU D M， TAN J R， GUAN W B. Dynamic wireless resource allocation based on improved genetic algorithm in C-RAN networks［J］. Telecommunication Engineering， 2021， 61（10）： 1225-1232. 10.3969/j.issn.1001-893x.2021.10.005
9	潘甦，张哓波.SCMA系统中基于等效频谱带宽的无线资源分配［J］.南京邮电大学学报（自然科学版），2018，38（4）：12-17. 10.14132/j.cnki.1673-5439.2018.04.002
	PAN S， ZHANG X B. Effective spectral bandwidth-based radio resource assignment for SCMA system［J］. Journal of Nanjing University of Posts and Telecommunications （Natural Science）， 2018， 38（4）： 12-17. 10.14132/j.cnki.1673-5439.2018.04.002
10	WAN R， ZHU L， LI T， et al. A NOMA-PSO based cooperative transmission method in satellite communication systems［C］// Proceedings of the 2017 9th International Conference on Wireless Communications and Signal Processing. Piscataway： IEEE， 2017： 1-6. 10.1109/wcsp.2017.8170912
11	WANG R， KANG W， LIU G， et al. Admission control and power allocation for NOMA-based satellite multi-beam network［J］. IEEE Access， 2020， 8： 33631-33643. 10.1109/access.2020.2973395
12	WANG J， LI D， ZHANG Z， et al. Traffic offloading and resource allocation for PDMA-based integrated satellite/terrestrial networks［C］// Proceedings of the 2022 IEEE 4th International Conference on Power， Intelligent Computing and Systems. Piscataway： IEEE， 2022： 259-262. 10.1109/icpics55264.2022.9873721
13	ZENG J， LIU B， SU X. Joint pattern assignment and power allocation in PDMA［C］// Proceedings of the 2017 IEEE 86th Vehicular Technology Conference. Piscataway： IEEE， 2017： 1-5. 10.1109/vtcfall.2017.8288406
14	LI S， SUN C， JIN X. Research on PDMA access technology for 5G communication［C］// Proceedings of the 2020 IEEE 20th International Conference on Communication Technology. Piscataway： IEEE， 2020： 519-523. 10.1109/icct50939.2020.9295661
15	LU K， WU S， YANG H. Optimized design pattern matrix of PDMA based on binary particle swarm optimization for 5G［C］// Proceedings of the 2020 IEEE 19th International Conference on Cognitive Informatics & Cognitive Computing. Piscataway：IEEE， 2020：220-224. 10.1109/iccicc50026.2020.9450225
16	LIN Z， LIN M， WANG J-B， et al. Joint beamforming and power allocation for satellite-terrestrial integrated networks with non-orthogonal multiple access［J］. IEEE Journal of Selected Topics in Signal Processing， 2019， 13（3）： 657-670. 10.1109/jstsp.2019.2899731
17	SHARMA S K， CHATZINOTAS S， OTTERSTEN B. Cognitive beamhopping for spectral coexistence of multibeam satellites［C］// Proceedings of the 2013 Future Network & Mobile Summit. Piscataway： IEEE， 2013： 1-10. 10.2514/6.2013-5624
18	陈前斌，麻世庆，段瑞吉，等.基于迁移深度强化学习的低轨卫星跳波束资源分配方案［J］. 电子与信息学报， 2023， 45（2）： 407-417. 10.11999/JEIT211457
	CHEN Q B， MA S Q， DUAN R J， et al. A novel beam hopping resource allocation scheme of low earth orbit satellite based on transfer deep reinforcement learning［J］. Journal of Electronics & Information Technology， 2023， 45（2）： 407-417. 10.11999/JEIT211457
19	3GPP. Study on New Radio （NR）to support non-terrestrial networks： TR 38.811 V15.4.0 ［R］. ［S.l.］： 3GPP， 2020. 10.1002/9781119236306.fmatter
20	麻世庆. 基于跳波束的低轨卫星资源分配策略研究［D］. 重庆：重庆邮电大学， 2022： 71-73. 10.1109/icc59986.2023.10420985
	MA S Q. Research on LEO satellite resource allocation strategy based on hopping beam ［D］. Chongqing： Chongqing University of Posts and Telecommunications， 2022：71-73. 10.1109/icc59986.2023.10420985

[1]	Hualiang LUO, Quanzhong LI, Qi ZHANG. Robust resource allocation optimization in cognitive wireless network integrating information communication and over-the-air computation [J]. Journal of Computer Applications, 2024, 44(4): 1195-1202.
[2]	Rui TANG, Shibo YUE, Ruizhi ZHANG, Chuan LIU, Chuanlin PANG. Energy efficiency optimization mechanism for UAV-assisted and non-orthogonal multiple access-enabled data collection system [J]. Journal of Computer Applications, 2024, 44(4): 1209-1218.
[3]	Xintong QIN, Zhengyu SONG, Tianwei HOU, Feiyue WANG, Xin SUN, Wei LI. Channel access and resource allocation algorithm for adaptive p-persistent mobile ad hoc network [J]. Journal of Computer Applications, 2024, 44(3): 863-868.
[4]	Fuqin DENG, Huifeng GUAN, Chaoen TAN, Lanhui FU, Hongmin WANG, Tinlun LAM, Jianmin ZHANG. Multi-robot reinforcement learning path planning method based on request-response communication mechanism and local attention mechanism [J]. Journal of Computer Applications, 2024, 44(2): 432-438.
[5]	Ziyang SONG, Junhuai LI, Huaijun WANG, Xin SU, Lei YU. Path planning algorithm of manipulator based on path imitation and SAC reinforcement learning [J]. Journal of Computer Applications, 2024, 44(2): 439-444.
[6]	Yuanchao LI, Chongben TAO, Chen WANG. Gait control method based on maximum entropy deep reinforcement learning for biped robot [J]. Journal of Computer Applications, 2024, 44(2): 445-451.
[7]	Jie LONG, Liang XIE, Haijiao XU. Integrated deep reinforcement learning portfolio model [J]. Journal of Computer Applications, 2024, 44(1): 300-310.
[8]	Yongjian MA, Xuhua SHI, Peiyao WANG. Constrained multi-objective evolutionary algorithm based on two-stage search and dynamic resource allocation [J]. Journal of Computer Applications, 2024, 44(1): 269-277.
[9]	Yu WANG, Tianjun REN, Zilin FAN. Air combat maneuver decision-making of unmanned aerial vehicle based on guided Minimax-DDQN [J]. Journal of Computer Applications, 2023, 43(8): 2636-2643.
[10]	Ziteng WANG, Yaxin YU, Zifang XIA, Jiaqi QIAO. Sparse reward exploration mechanism fusing curiosity and policy distillation [J]. Journal of Computer Applications, 2023, 43(7): 2082-2090.
[11]	Lei LI, Guofu ZHANG, Zhaopin SU, Feng YUE. Software testing resource allocation algorithm for dynamic changes in architecture [J]. Journal of Computer Applications, 2023, 43(7): 2261-2270.
[12]	Xiaolin LI, Yusang JIANG. Task offloading algorithm for UAV-assisted mobile edge computing [J]. Journal of Computer Applications, 2023, 43(6): 1893-1899.
[13]	Heping FANG, Shuguang LIU, Yongyi RAN, Kunhua ZHONG. Integrated scheduling optimization of multiple data centers based on deep reinforcement learning [J]. Journal of Computer Applications, 2023, 43(6): 1884-1892.
[14]	Xiaohui HUANG, Kaiming YANG, Jiahao LING. Order dispatching by multi-agent reinforcement learning based on shared attention [J]. Journal of Computer Applications, 2023, 43(5): 1620-1624.
[15]	Zhengkai DING, Qiming FU, Jianping CHEN, You LU, Hongjie WU, Nengwei FANG, Bin XING. Ultra-short-term photovoltaic power prediction by deep reinforcement learning based on attention mechanism [J]. Journal of Computer Applications, 2023, 43(5): 1647-1654.

Resource allocation algorithm for low earth orbit satellites oriented to user demand

面向用户需求的低轨卫星资源分配算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 20

Related Articles 15

Recommended Articles

Metrics