Abstract:

To solve the problem of joint power and channel resource allocation for integrated communication and sensing in multi-link transmission of next-generation Wi-Fi7 devices， a multi-link multi-agent reinforcement learning algorithm based on QMIX（Q-learning Mixing Network） for Joint Power Control and channel allocation （JPCQMIX） was proposed on the basis of special upper and lower Media Access Control （MAC） layer structure of Multi-Link Device （MLD）. In the algorithm， each lower-layer MAC， i.e.， each link， was regarded as an agent， and mixing network was set up in the upper-layer MAC to process all the local value functions of lower-layer MACs， thereby achieving the effect of centralized training. After the training， each lower-layer MAC entered the distributed execution mode and interacted with its local environment independently to perform power control and channel allocation decision making. Simulation results show that the proposed algorithm improves the communication throughput performance by 20.51% and 29.10% respectively compared with Multi-Agent Deep Q Network （MADQN） algorithm and the traditional heuristic Particle Swarm Optimization （PSO） algorithm. Meanwhile， the proposed algorithm demonstrates better robustness when facing with different sensing accuracy thresholds and different link minimum Signal-to-Interference-plus-Noise Ratio （SINR）. It can be seen that JPCQMIX enhances the system’s communication throughput under the condition of satisfying the sensing accuracy effectively.

Key words: Wi-Fi7, multi-link, integrated communication and sensing, multi-agent, deep reinforcement learning

摘要：

针对下一代Wi-Fi7设备中多链路传输时通信与感知一体化的功率和信道联合资源分配的问题，根据多链路设备（MLD）特殊的上下两层媒体接入控制层（MAC）结构，提出一种基于QMIX的联合功率控制与信道分配的多链路多智能体强化学习算法（JPCQMIX）。该算法将MLD的每个下层MAC即每条链路作为一个智能体，并在上层MAC中设置混合网络用来处理所有下层MAC的局部值函数，以达到中心式训练的效果。训练完成后，每个下层MAC进入分布式执行模式，并独立地与它的局部环境进行交互，以进行功率控制和信道分配决策。仿真结果表明，相较于多智能体深度Q网络（MADQN）算法和传统启发式粒子群优化（PSO）算法，所提算法在通信吞吐量性能上分别提高了20.51%和29.10%；同时，所提算法在面对不同感知精度阈值和不同链路最低信干噪比（SINR）时，鲁棒性更好。可见，JPCQMIX能有效提升系统在满足感知精度条件下的通信吞吐量。

关键词: Wi-Fi7, 多链路, 通信感知一体化, 多智能体, 深度强化学习