基于改进实时检测Transformer的塔机上俯视场景小目标检测模型

doi:10.11772/j.issn.1001-9081.2023121796

《计算机应用》唯一官方网站 ›› 2024, Vol. 44 ›› Issue (12): 3922-3929.DOI: 10.11772/j.issn.1001-9081.2023121796

• 多媒体计算与计算机仿真 • 上一篇下一篇

基于改进实时检测Transformer的塔机上俯视场景小目标检测模型

庞玉东¹, 李志星¹(), 刘伟杰¹, 李天昊¹, 王宁宁²

^1.北京建筑大学机电与车辆工程学院，北京 102616
^2.洛阳理工学院智能制造学院，河南洛阳 471023

收稿日期:2023-12-26 修回日期:2024-03-15 接受日期:2024-03-18 发布日期:2024-04-10 出版日期:2024-12-10
通讯作者: 李志星
作者简介:庞玉东（1999—），男，山东枣庄人，硕士研究生，CCF会员，主要研究方向：计算机视觉、图像识别、目标检测
刘伟杰（2000—），男，吉林榆树人，硕士研究生，主要研究方向：计算机视觉、人工智能
李天昊（1998—），男，北京人，硕士研究生，主要研究方向：计算机视觉、人工智能
王宁宁（1986—），男，河南洛阳人，讲师，博士，主要研究方向：图像识别、磁流变液传动。
基金资助:
北京市属高校基本科研业务费资助项目(X21053);河南省高等学校重点科研项目(23A460020);河南省自然科学基金资助项目(242300420044)

Small target detection model in overlooking scenes on tower cranes based on improved real-time detection Transformer

Yudong PANG¹, Zhixing LI¹(), Weijie LIU¹, Tianhao LI¹, Ningning WANG²

^1.School of Mechanical，Electrical and Vehicle Engineering，Beijing University of Civil Engineering and Architecture，Beijing 102616，China
^2.School of Intelligent Manufacturing，Luoyang Institute of Science and Technology，Luoyang Henan 471023，China

Received:2023-12-26 Revised:2024-03-15 Accepted:2024-03-18 Online:2024-04-10 Published:2024-12-10
Contact: Zhixing LI
About author:PANG Yudong， born in 1999， M. S. candidate. His research interests include computer vision， image recognition， object detection.
LIU Weijie， born in 2000， M. S. candidate. His research interests include computer vision， artificial intelligence.
LI Tianhao， born in 1998， M. S. candidate. His research interests include computer vision， artificial intelligence.
WANG Ningning， born in 1986， Ph. D.， lecturer. His research interests include image recognition， magnetorheological fluid transmission.
Supported by:
Fundamental Research Funds for Universities in Beijing(X21053);Key Scientific Research Project of Higher Education Institutions of Henan Province(23A460020);Natural Science Foundation of Henan Province(242300420044)

摘要/Abstract

摘要：

针对塔机吊钩相互碰撞导致物体跌落以及塔机倒塌致使人员伤亡等一系列施工现场人员安全保障的问题，提出一种基于改进实时检测Transformer （Real-Time DEtection TRansformer， RT-DETR）的塔机上俯视场景小目标检测模型。首先，在原始模型中加入应用模型的重参数化思想设计的多路训练和单路推理结构以提升检测速度；其次，重新设计FasterNet Block中的卷积模块替换原始BackBone之中的BasicBlock以提升检测模型性能；再次，利用新的损失函数Inner-SIoU（Inner-Structured Intersection over Union）进一步提升模型精度与收敛速度；最后，进行消融实验与对比实验验证模型性能。结果表明，在检测塔机顶部俯视小目标图像时，所提模型的精度达到94.7%，高于原始RT-DETR模型6.1个百分点；所提模型的每秒检测帧数（FPS）达到59.7，检测速度相较于原模型提升了21%。在公共数据集COCO 2017上所提模型的平均精度（AP）比YOLOv5、YOLOv7和YOLOv8分别高2.4、1.5和1.3个百分点。可见所提模型满足塔机上俯视场景下的小目标检测精度和速度的要求。

关键词: 目标检测, RT-DETR, 小目标, Transformer, 计算机视觉, 注意力机制

Abstract:

In view of a series of problems of security guarantee of construction site personnel such as casualties led by falling objects and tower crane collapse caused by mutual collision of tower hooks， a small target detection model in overlooking scenes on tower cranes based on improved Real-Time DEtection TRansformer （RT-DETR） was proposed. Firstly， the multiple training and single inference structures designed by applying the idea of model reparameterization were added to the original model to improve the detection speed. Secondly， the convolution module in FasterNet Block was redesigned to replace BasicBlock in the original BackBone to improve performance of the detection model. Thirdly， the new loss function Inner-SIoU （Inner-Structured Intersection over Union） was utilized to further improve precision and convergence speed of the model. Finally， the ablation and comparison experiments were conducted to verify the model performance. The results show that， in detection of the small target images in overlooking scenes on tower cranes， the proposed model achieves the precision of 94.7%， which is higher than that of the original RT-DETR model by 6.1 percentage points. At the same time， the Frames Per Second （FPS） of the proposed model reaches 59.7， and the detection speed is improved by 21% compared with the original model. The Average Precision （AP） of the proposed model on the public dataset COCO 2017 is 2.4， 1.5， and 1.3 percentage points higher than those of YOLOv5， YOLOv7， and YOLOv8， respectively. It can be seen that the proposed model meets the precision and speed requirements for small target detection in overlooking scenes on tower cranes.

Key words: target detection, RT-DETR (Real-Time DEtection TRansformer), small target, Transformer, computer vision, attention mechanism

中图分类号:

TP391.41

庞玉东, 李志星, 刘伟杰, 李天昊, 王宁宁. 基于改进实时检测Transformer的塔机上俯视场景小目标检测模型[J]. 计算机应用, 2024, 44(12): 3922-3929.

Yudong PANG, Zhixing LI, Weijie LIU, Tianhao LI, Ningning WANG. Small target detection model in overlooking scenes on tower cranes based on improved real-time detection Transformer[J]. Journal of Computer Applications, 2024, 44(12): 3922-3929.

图/表 12

图1 RT-DETR模型结构

Fig. 1 RT-DETR model structure

图2 重参数化后的多路结构

Fig. 2 Multi-path structure after reparameterization

图3 结构重参数化的转化流程

Fig. 3 Structure reparameterization transformation flow

图4 残差路径设计

Fig. 4 Residual path design

图5 Faster Block结构

Fig. 5 Faster Block structure

图6 改进后的BasicBlock

Fig. 6 Improved BasicBlock

图7 SIoU中的角度损失和距离损失

Fig. 7 Angle loss and distance loss in SIoU

图8 OP数据集样图

Fig. 8 Sample images of OP dataset

表1 3种改进在RT-DETR模型上的消融实验结果

Tab. 1 Ablation experiment results of three improvements on RT-DETR model

重参数化	FasterNet Block	SIoU	Precision	Recall	mAP₅₀	mAP_50-90	FPS	GFLOPs
			0.886	0.882	0.894	0.513	49.3	63.3
√			0.888	0.869	0.887	0.482	60.6	22.5
	√		0.923	0.882	0.907	0.510	50.4	63.2
		√	0.931	0.885	0.918	0.529	50.1	65.0
√	√		0.914	0.882	0.906	0.516	54.9	25.1
√		√	0.902	0.891	0.897	0.516	51.8	24.9
	√	√	0.940	0.873	0.908	0.513	52.8	49.5
√	√	√	0.947	0.897	0.917	0.534	59.7	28.5

表2 不同检测模型在OP数据集上检测性能的对比结果

Tab. 2 Comparison results of detection performance of different detection models on OP dataset

模型	Precision	Recall	mAP₅₀	mAP_50-90
YOLOv5	0.841	0.808	0.725	0.419
YOLOv7	0.859	0.812	0.731	0.430
YOLOv8	0.872	0.828	0.752	0.431
DETR	0.860	0.870	0.885	0.493
RT-DETR	0.886	0.882	0.894	0.513
本文模型	0.947	0.897	0.917	0.534

表3 不同检测模型在COCO数据集上检测性能对比

Tab. 3 Comparison of detection performance of different detection models on COCO dataset

模型	AP	AP₅₀	AP₇₅
YOLOv5	0.444	0.631	0.456
YOLOv7	0.453	0.648	0.483
YOLOv8	0.455	0.639	0.488
DETR	0.435	0.618	0.442
RT-DETR	0.459	0.652	0.498
本文模型	0.468	0.662	0.510

图9 不同模型的检测结果可视化

Fig. 9 Visualization of different model detection results

参考文献 24

1	GIRSHICK R， DONAHUE J， DARRELL T， et al. Rich feature hierarchies for accurate object detection and semantic segmentation［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2014： 580-587.
2	REN S， HE K， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）：1137-1149.
3	REDMON J， FARHADI A. YOLOv3： an incremental improvement［EB/OL］. ［2023-12-10］..
4	BOCHKOVSKIY A， WANG C Y， LIAO H Y M. YOLOv4： optimal speed and accuracy of object detection［EB/OL］. ［2023-10-20］..
5	WANG C Y， BOCHKOVSKIY A， LIAO H Y M. YOLOv7： trainable bag-of-freebies sets new state-of-the-art for real-time object detectors［C］// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2023： 7464-7475.
6	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： unified， real-time object detection ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 779-788.
7	GE Z， LIU S， WANG F， et al. YOLOX： exceeding YOLO series in 2021［EB/OL］. ［2023-11-18］..
8	SALSCHEIDER N O. FeatureNMS： non-maximum suppression by learning feature embeddings［C］// Proceedings of the 25th International Conference on Pattern Recognition. Piscataway： IEEE， 2021：7848-7854.
9	LIU S， HUANG D， WANG Y. Adaptive NMS： refining pedestrian detection in a crowd［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 6452-6461.
10	ZHENG Z， WANG P， LIU W， et al. Distance-IoU loss： faster and better learning for bounding box regression ［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2020： 12993-13000.
11	周飞燕，金林鹏，董军. 卷积神经网络研究综述［J］. 计算机学报， 2017， 40（6）：1229-1251.
	ZHOU F Y， JIN L P， DONG J. Review of convolutional neural networks ［J］. Chinese Journal of Computers， 2017， 40（6）：1229-1251.
12	DING X， ZHANG X， HAN J， et al. Diverse branch block： building a convolution as an inception-like unit［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 10881-10890.
13	BAHDANAU D， CHO K， BENGIO Y. Neural machine translation by jointly learning to align and translate ［EB/OL］. ［2023-10-21］..
14	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2017： 6000-6010.
15	陈洛轩，林成创，郑招良，等. Transformer在计算机视觉场景下的研究综述［J］. 计算机科学， 2023， 50（12）：130-147.
	CHEN L X， LIN C C， ZHENG Z L， et al. Review of Transformer in computer vision［J］. Computer Science， 2023， 50（12）：130-147.
16	CHEN C F， PANDA R， FAN Q. RegionViT： regional-to-local attention for vision Transformers ［EB/OL］. ［2023-10-20］..
17	CARION N， MASSA F， SYNNAEVE G， et al. End-to-end object detection with Transformers ［C］// Proceedings of the 2020 European Conference on Computer Vision， LNCS 12346. Cham： Springer， 2020： 213-229.
18	ZHU X， SU W， LU L， et al. Deformable DETR： deformable Transformers for end-to-end object detection ［EB/OL］. ［2023-12-22］..
19	CHEN Q， CHEN X， WANG J， et al. Group DETR： fast DETR training with group-wise one-to-many assignment ［C］// Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2023： 6633-6642.
20	ZHAO Y， LV W， XU S， et al. DETRs beat YOLOs on real-time object detection［C］// Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2024： 16965-16974.
21	CHEN J， KAO S H， HE H， et al. Run， don't walk： chasing higher FLOPS for faster neural networks ［C］// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2023： 12021-12031.
22	ZHANG H， XU C， ZHANG S. Inner-IoU： more effective intersection over union loss with auxiliary bounding box ［EB/OL］. ［2023-12-20］..
23	GEVORGYAN Z. SIoU loss： more powerful learning for bounding box regression［EB/OL］. ［2023-12-20］..
24	KUHN H W. The Hungarian method for the assignment problem［J］. Naval Research Logistics， 2005， 52（1）： 7-21.

[1]	黄云川, 江永全, 黄骏涛, 杨燕. 基于元图同构网络的分子毒性预测[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2964-2969.
[2]	杨鑫, 陈雪妮, 吴春江, 周世杰. 结合变种残差模型和Transformer的城市公路短时交通流预测[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2947-2951.
[3]	潘烨新, 杨哲. 基于多级特征双向融合的小目标检测优化模型[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2871-2877.
[4]	赵志强, 马培红, 黑新宏. 基于双重注意力机制的人群计数方法[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2886-2892.
[5]	李金金, 桑国明, 张益嘉. APK-CNN和Transformer增强的多域虚假新闻检测模型[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2674-2682.
[6]	任烈弘, 黄铝文, 田旭, 段飞. 基于DFT的频率敏感双分支Transformer多变量长时间序列预测方法[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2739-2746.
[7]	秦璟, 秦志光, 李发礼, 彭悦恒. 基于概率稀疏自注意力神经网络的重性抑郁疾患诊断[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2970-2974.
[8]	方介泼, 陶重犇. 应对零日攻击的混合车联网入侵检测系统[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2763-2769.
[9]	李力铤, 华蓓, 贺若舟, 徐况. 基于解耦注意力机制的多变量时序预测模型[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2732-2738.
[10]	贾洁茹, 杨建超, 张硕蕊, 闫涛, 陈斌. 基于自蒸馏视觉Transformer的无监督行人重识别[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2893-2902.
[11]	张英俊, 李牛牛, 谢斌红, 张睿, 陆望东. 课程学习指导下的半监督目标检测框架[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2326-2333.
[12]	付帅, 郭小英, 白茹意, 闫涛, 陈斌. 改进的CloFormer模型与有序回归相结合的年龄评估方法[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2372-2380.
[13]	薛凯鹏, 徐涛, 廖春节. 融合自监督和多层交叉注意力的多模态情感分析网络[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2387-2392.
[14]	汪雨晴, 朱广丽, 段文杰, 李书羽, 周若彤. 基于交互注意力机制的心理咨询文本情感分类模型[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2393-2399.
[15]	高鹏淇, 黄鹤鸣, 樊永红. 融合坐标与多头注意力机制的交互语音情感识别[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2400-2406.

基于改进实时检测Transformer的塔机上俯视场景小目标检测模型

Small target detection model in overlooking scenes on tower cranes based on improved real-time detection Transformer

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 24

相关文章 15

编辑推荐

Metrics