Abstract:

In view of the limitations of deep learning algorithms’ generalization and robustness， as well as the high computational cost of Full Parameter Fine-Tuning （FPFT） in object detection tasks in complex scenarios， a low-rank adaptive Parameter-Efficient Fine-Tuning （PEFT） algorithm based on YOLOv11 （You Only Look Once version 11） was proposed. Firstly， a Low-Rank Adaptation （LoRA） module was embedded into the backbone and neck networks of YOLOv11. Secondly， three low-rank decomposition algorithms， including LoRA， weight-Decomposed low-Rank Adaptation （DoRA） and Principal Singular values and Singular vectors Adaptation （PiSSA） were combined， and efficient parameter updates were achieved through weight decomposition and dynamic adjustment mechanisms. Finally， during the training process， most of the pre-trained weights of the YOLOv11 network were kept frozen， and only the low-rank matrices generated by the three low-rank decomposition algorithms in the LoRA module were trained， thereby reducing the trainable parameter size to 1.56% of the original algorithm. Experimental results on the COCO （Common Objects in COntext） dataset demonstrate that the proposed algorithm improves the precision， recall and mean Average Precision （mAP） at IoU （Intersection over Union） threshold of 0.5 by 4.18， 7.11 and 7.85 percentage points， respectively， compared with the baseline algorithm YOLOv11. It can be seen that the proposed algorithm provides an effective technical path for lightweight and efficient fine-tuning of large-scale detection algorithms in resource-constrained scenarios.

Key words: YOLOv11 (You Only Look Once version 11), Parameter?Efficient Fine?Tuning (PEFT), object detection, Low-Rank Adaptation (LoRA), deep learning

摘要：

针对复杂场景下目标检测任务中深度学习算法的泛化性和鲁棒性受限以及全参数微调（FPFT）计算成本高的问题，提出一种基于YOLOv11 （You Only Look Once version 11）的低秩自适应参数高效微调（PEFT）算法。首先，在YOLOv11的骨干和颈部网络中嵌入低秩自适应（LoRA）模块；其次，结合LoRA、权重分解低秩自适应（DoRA）和主奇异值与奇异向量自适应（PiSSA）这3种低秩分解算法，通过权重分解与动态调整机制实现参数的高效更新；最后，在训练过程中，将YOLOv11网络的绝大部分预训练权重保持冻结状态，仅对LoRA模块中由3种低秩分解算法生成的低秩矩阵进行训练，将可训练参数规模缩减至原算法的1.56%。COCO （Common Objects in COntext）数据集上的实验结果表明，所提算法相较于基线算法YOLOv11在精确度、召回率和交并比阈值为0.5时的平均精度均值（mAP）上分别提升了4.18、7.11和7.85个百分点。可见，所提算法为资源受限场景下的大型检测算法轻量化与高效微调提供了有效技术路径。

关键词: YOLOv11, 参数高效微调, 目标检测, 低秩自适应, 深度学习