Traffic sign recognition model in haze weather based on YOLOv5

doi:10.11772/j.issn.1001-9081.2021071305

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (9): 2876-2884.DOI: 10.11772/j.issn.1001-9081.2021071305

• Multimedia computing and computer simulation • Previous Articles Next Articles

Traffic sign recognition model in haze weather based on YOLOv5

Jinghan YIN¹^,², Shaojun QU¹^,²(), Zekai YAO¹^,², Xuanye HU¹^,², Xiaoyu QIN¹^,², Pujing HUA¹^,²

^1.College of Information Science and Engineering，Hunan Normal University，Changsha Hunan 410081，China
^2.College of Xiangjiang Artificial Intelligence，Hunan Normal University，Changsha Hunan 410081，China

Received:2021-07-19 Revised:2021-09-03 Accepted:2021-09-14 Online:2022-09-19 Published:2022-09-10
Contact: Shaojun QU
About author:YIN Jinghan， born in 2000. His research interests include object detection.
YAO Zekai， born in 2000. His research interests include image segmentation， object detection.
HU Xuanye， born in 1999. His research interests include embedded software.
QIN Xiaoyu， born in 2000. Her research interests include object detection.
HUA Pujing， born in 2000. His research interests include data mining.
Supported by:
the National Natural Science Foundation(12071126);Scientific Research Project of Hunan Provincial Department of Education(19C1149);Scientific Research Innovation Project of College of Xiangjiang Artificial Intelligence, Hunan Normal University(202031A12)

基于YOLOv5的雾霾天气下交通标志识别模型

尹靖涵¹^,², 瞿绍军¹^,²(), 姚泽楷¹^,², 胡玄烨¹^,², 秦晓雨¹^,², 华璞靖¹^,²

^1.湖南师范大学信息科学与工程学院，长沙 410081
^2.湖南师范大学湘江人工智能学院，长沙 410081

通讯作者: 瞿绍军
作者简介:尹靖涵（2000—），男，浙江丽水人，主要研究方向：目标检测；
姚泽楷（2000—），男，湖南双峰人，主要研究方向：图像分割、目标检测；
胡玄烨（1999—），男，河南汝州人，主要研究方向：嵌入式软件；
秦晓雨（2000—），女，山东聊城人，主要研究方向：目标检测；
华璞靖（2000—），男，江西景德镇人，主要研究方向：数据挖掘。
基金资助:
国家自然科学基金资助项目(12071126);湖南省教育厅科学研究项目(19C1149);湖南师范大学湘江人工智能学院科研创新项目(202031A12)

Abstract

Abstract:

Aiming at the problem of poor recognition precision and serious missed detection of small traffic signs in bad weather such as haze， rain and snow， a traffic sign recognition model in haze weather based on YOLOv5 （You Only Look Once version 5） was proposed. Firstly， the structure of YOLOv5 was optimized. By using contrary thinking， the problem of small object recognition difficulty was solved by reducing the depth of feature pyramid and limiting the maximum down sampling multiple. By adjusting the depth of residual module， the repeated overlapping of background features was suppressed. Secondly， the mechanisms such as data augmentation， K-means anchor and Global Non-Maximum Suppression （GNMS） were introduced into the model. Finally， the detection ability of the improved YOLOv5 facing the bad weather was verified on the Chinese traffic sign dataset TT100K， and the study on traffic sign recognition in the haze weather with the most obvious precision decline was focused on. Experimental results show that the F1-score， mean Average Precision @0.5 （mAP@0.5）， mean Average Precision @0.5：0.95 （mAP@0.5：0.95） of the improved YOLOv5 model reach 0.921 50， 95.3% and 75.2%， respectively. The proposed model can maintain high-precision recognition of traffic sign in bad weather， and has Frames Per Second （FPS） up to 50， meeting the requirement of real-time detection.

Key words: deep learning, object detection, You Only Look Once version 5 (YOLOv5), feature pyramid, haze weather, Traffic Sign Recognition (TSR)

摘要：

针对雾霾、雨雪等恶劣天气下小型交通标志识别精度低、漏检严重的问题，提出一种基于YOLOv5的雾霾天气下交通标志识别模型。首先，对YOLOv5的结构进行优化，采用逆向思维，通过削减特征金字塔深度、限制最高下采样倍数来解决小目标难以识别的问题，并通过调整残差模块的特征传递深度来抑制背景特征的重复叠加；其次，引入数据增强、K-means先验框、全局非极大值抑制（GNMS）等机制到模型；最后，在中国交通标志数据集TT100K上验证改进YOLOv5模型在面对恶劣天气时的检测能力，并对精度下降最显著的雾霾天气下的交通标志识别展开了重点研究。实验结果表明，改进YOLOv5模型的F1-score达0.921 50，平均精度均值@0.5 （mAP@0.5）达95.3%，平均精度均值@0.5：0.95 （mAP@0.5：0.95）达75.2%，且所提模型在恶劣天气下仍能进行交通标志的高精度识别，每秒检测帧数（FPS）达到50，满足实时检测的需求。

关键词: 深度学习, 目标检测, YOLOv5, 特征金字塔, 雾霾天气, 交通标志识别

CLC Number:

TP391.41

Jinghan YIN, Shaojun QU, Zekai YAO, Xuanye HU, Xiaoyu QIN, Pujing HUA. Traffic sign recognition model in haze weather based on YOLOv5[J]. Journal of Computer Applications, 2022, 42(9): 2876-2884.

尹靖涵, 瞿绍军, 姚泽楷, 胡玄烨, 秦晓雨, 华璞靖. 基于YOLOv5的雾霾天气下交通标志识别模型[J]. 《计算机应用》唯一官方网站, 2022, 42(9): 2876-2884.

Figures/Tables 21

Fig.1 Framework of YOLOv5 model

Fig.2 Main module structures of YOLOv5

Fig.3 Comparison of FPN and PAN before and after structural improvement

Fig.4 Improved residual module

Fig.5 Clustering scatter diagram

Fig.6 Curve of mAP@0.5 and FPS varying with img-size

Fig. 7 Variation curve ofmAP@0.5 with the number of iterations

Fig. 8 Variation curve of loss value with the number of iterations

Fig.9 Activation function and derivative function

Fig.10 TT100K dataset samples

Fig.11 Twenty-five kinds of traffic signs

Fig. 12 Traffic sign quantity distribution before augmentation

Fig. 13 Distribution of traffic sign quantity after augmentation

Fig.14 TT100K dataset samples after augmentation

Tab.1 Confusion matrix

预测值	真实值
预测值	Positive	Negative
Positive	TP	FP
Negative	FN	TN

Tab.2 Performance comparison of different YOLOv5 models

模型	F1-score	mAP@0.5/%	mAP@0.5：0.95/%	FPS
YOLOv5s	0.839 26	86.5	66.4	102
YOLOv5m	0.662 71	67.5	50.4	52
YOLOv5l	0.693 99	70.6	52.7	32
本文模型	0.921 50	95.3	75.2	50

Tab.3 F1-score comparison in different weather conditions

模型	F1-score
模型	测试集2	测试集3	测试集4
YOLOv5s	0.868 50	0.844 82	0.816 50
YOLOv5m	0.710 32	0.662 71	0.636 17
YOLOv5l	0.718 13	0.677 41	0.674 10
本文模型	0.941 42	0.924 97	0.914 00

Tab.4 mAP@0.5 comparison in different weather conditions

模型	mAP@0.5
模型	测试集2	测试集3	测试集4
YOLOv5s	90.6	87.9	84.1
YOLOv5m	74.9	70.0	64.0
YOLOv5l	75.5	71.5	68.6
本文模型	97.0	96.0	94.8

Tab.5 mAP@0.5：0.95 comparison in different weather conditions

模型	mAP@0.5：0.95
模型	测试集2	测试集3	测试集4
YOLOv5s	70.2	67.4	64.7
YOLOv5m	56.3	52.4	48.0
YOLOv5l	57.1	53.3	51.3
本文模型	77.4	75.0	75.0

Tab.6 Comparison of several models

组别	滤波器	F1-score	mAP@0.5/%	mAP@0.5：0.95/%
1	无	0.914 00	94.8	75.0
2	无	0.891 12	92.9	71.1
3	无	0.774 31	77.6	58.4
4	DCP	0.771 06	80.9	60.3
5	MSRCR	0.819 76	83.4	63.1
6	HE	0.746 09	78.2	57.9
7	AOD-Net	0.829 19	83.6	63.1
8	无	0.875 56	90.5	73.7
9	无	0.854 22	87.6	70.4

Fig.15 Model detection results

References 31

1	张震，程伟伟，吴磊，等. 基于不变矩和SVM的圆形交通标志识别方法研究［J］. 电子测量与仪器学报， 2017， 31（5）：773-779. 10.13382/j.jemi.2017.05.017
	ZHANG Z， CHENG W W， WU L， et al. Study on circular traffic signs recognition method based on invariant moments and SVM［J］. Journal of Electronic Measurement and Instrumentation， 2017， 31（5）： 773-779. 10.13382/j.jemi.2017.05.017
2	蔡俊，赵超，沈晓波，等. SIFT算法在指示类交通标志识别中的应用［J］. 太原学院学报（自然科学版）， 2020， 38（4）：45-50. 10.14152/j.cnki.2096-191X.2020.04.009
	CAI J， ZHAO C， SHEN X B， et al. Application of SIFT algorithm in indicating traffic signs recognition［J］. Journal of Taiyuan University （Natural Science Edition）， 2020， 38（4）： 45-50. 10.14152/j.cnki.2096-191X.2020.04.009
3	邓天民，方芳，周臻浩. 基于改进空间金字塔池化卷积神经网络的交通标志识别［J］. 计算机应用， 2020， 40（10）：2872-2880.
	DENG T M， FANG F， ZHOU Z H. Traffic sign recognition based on improved convolutional neural network with spatial pyramid pooling［J］. Journal of Computer Applications， 2020， 40（10）： 2872-2880.
4	周建平. 基于数据增强和双模型结构的交通标志检测与识别［D］. 广州：暨南大学， 2020：28-36.
	ZHOU J P. Traffic sign detection and recognition based on data augmentation and dual-model structure［D］. Guangzhou： Jinan University， 2020： 28-36.
5	赵坤，刘立，孟宇，等. 弱光照条件下交通标志检测与识别［J］. 工程科学学报， 2020， 42（8）：1074-1084.
	ZHAO K， LIU L， MENG Y， et al. Traffic signs detection and recognition under low-illumination conditions［J］. Chinese Journal of Engineering， 2020， 42（8）： 1074-1084.
6	KOU A L. Detection and recognition of traffic signs based on improved deep learning［J］. International Core Journal of Engineering， 2020， 6（12）： 208-213.
7	徐国整，周越，董斌，等. 基于改进Cascade R-CNN的交通标志牌识别［J］. 传感器与微系统， 2021， 40（5）：142-145， 153.
	XU G Z， ZHOU Y， DONG B， et al. Traffic signs recognition based on improved Cascade R-CNN［J］. Transducer and Microsystem Technologies， 2021， 40（5）： 142-145， 153.
8	张航. 自然场景下的交通标志识别算法研究［D］. 长沙：中南大学， 2006：53-94.
	ZHANG H. Study on algorithms for detection and recognition of traffic signs from natural scenes［D］. Changsha： Central South University， 2006： 53-94.
9	张国山，刘振. 雾霾天气下交通标志的检测与识别［J］. 天津工业大学学报， 2015， 34（5）：71-75. 10.3969/j.issn.1671-024x.2015.05.015
	ZHANG G S， LIU Z. Traffic sign detection and recognition under haze weather［J］. Journal of Tianjin Polytechnic University， 2015， 34（5）： 71-75. 10.3969/j.issn.1671-024x.2015.05.015
10	陈琼红，冀杰，种一帆，等. 基于AOD-Net和SSD的雾天车辆和行人检测［J］. 重庆理工大学学报（自然科学）， 2021， 35（5）：108-117.
	CHEN Q H， JI J， CHONG Y F， et al. Vehicle and pedestrian detection based on AOD-Net and SSD algorithm in hazy environment［J］. Journal of Chongqing University of Technology （Natural Science）， 2021， 35（5）： 108-117.
11	陈秀新，叶洋，于重重，等. 基于深度学习的雾霾天气下交通标志识别［J］. 重庆交通大学学报（自然科学版）， 2020， 39（12）：1-5. 10.3969/j.issn.1674-0696.2020.12.01
	CHEN X X， YE Y， YU C C， et al. Identification of traffic signs in haze weather based on deep learning［J］. Journal of Chongqing Jiaotong University （Natural Science）， 2020， 39（12）： 1-5. 10.3969/j.issn.1674-0696.2020.12.01
12	周小龙，陈小佳，陈胜勇，等. 弱监督学习下的目标检测算法综述［J］. 计算机科学， 2019， 46（11）： 49-57. 10.11896/jsjkx.181001899
	ZHOU X L， CHEN X J， CHEN S Y， et al. Weakly supervised learning-based object detection： a survey［J］. Computer Science， 2019， 46（11）： 49-57. 10.11896/jsjkx.181001899
13	JIAO L C， ZHANG F， LIU F， et al. A survey of deep learning-based object detection［J］. IEEE Access， 2019， 7： 128837-128868. 10.1109/access.2019.2939201
14	张泽苗，霍欢，赵逢禹. 深层卷积神经网络的目标检测算法综述［J］. 小型微型计算机系统， 2019， 40（9）： 1825-1831. 10.3969/j.issn.1000-1220.2019.09.004
	ZHANG Z M， HUO H， ZHAO F Y. Survey of object detection algorithm based on deep convolutional neural networks［J］. Journal of Chinese Computer Systems， 2019， 40（9）： 1825-1831. 10.3969/j.issn.1000-1220.2019.09.004
15	南晓虎，丁雷. 深度学习的典型目标检测算法综述［J］. 计算机应用研究， 2020， 37（S2）： 15-21.
	NAN X H， DING L. Review of typical target detection algorithms for deep learning［J］. Application Research of Computers， 2020， 37（S2）：15-21.
16	赵永强，饶元，董世鹏，等. 深度学习目标检测方法综述［J］. 中国图象图形学报， 2020， 25（4）： 629-654. 10.11834/jig.190307
	ZHAO Y Q， RAO Y， DONG S P， et al. Survey on deep learning object detection［J］. Journal of Image and Graphics， 2020， 25（4）： 629-654. 10.11834/jig.190307
17	陈飞，刘云鹏，李思远. 复杂环境下的交通标志检测与识别方法综述［J］. 计算机工程与应用，2021， 57（16）： 65-73.
	CHEN F， LIU Y P， LI S Y. Survey of traffic sign detection and recognition methods in complex environment［J］. Computer Engineering and Applications， 2021， 57（16）： 65-73.
18	廖璐明，张伟，王亚刚. 基于注意力机制的交通标志识别［J］. 小型微型计算机系统， 2021， 42（6）：1262-1266. 10.3969/j.issn.1000-1220.2021.06.021
	LIAO L M， ZHANG W， WANG Y G. Traffic sign recognition based on attention mechanism［J］. Journal of Chinese Computer Systems， 2021， 42（6）： 1262-1266. 10.3969/j.issn.1000-1220.2021.06.021
19	林雪，张志佳，刘立强，等. 面向城市路况交通标志识别的VGG网络剪枝与量化［J］. 小型微型计算机系统， 2021， 42（6）：1293-1296. 10.3969/j.issn.1000-1220.2021.06.026
	LIN X， ZHANG Z J， LIU L Q， et al. VGG network pruning and quantification for traffic sign recognition of urban road condition［J］. Journal of Chinese Computer Systems， 2021， 42（6）： 1293-1296. 10.3969/j.issn.1000-1220.2021.06.026
20	江金洪，鲍胜利，史文旭，等. 基于YOLO v3算法改进的交通标志识别算法［J］. 计算机应用， 2020， 40（8）：2472-2478.
	JIANG J H， BAO S L， SHI W X， et al. Improved traffic sign recognition algorithm based on YOLO v3 algorithm［J］. Journal of Computer Applications， 2020， 40（8）： 2472-2478.
21	SHEN L L， YOU L， PENG B， et al. Group multi-scale attention pyramid network for traffic sign detection［J］. Neurocomputing， 2021， 452： 1-14. 10.1016/j.neucom.2021.04.083
22	XIAO Q Y， YU W W. Traffic sign detection in complex environment based on improved YOLOv3［J］. International Core Journal of Engineering， 2021， 7（3）： 102-108.
23	YAO Z X， SONG X P， ZHAO L， et al. Real-time method for traffic sign detection and recognition based on YOLOv3-tiny with multiscale feature extraction［J］. Proceedings of the Institution of Mechanical Engineers， Part D： Journal of Automobile Engineering， 2021， 235（7）： 1978-1991. 10.1177/0954407020980559
24	陈名松，吴冉冉，张泽功，等. 基于改进CapsNet的交通标志分类模型［J］. 计算机应用研究， 2020， 37（S2）：367-368， 371.
	CHEN M S， WU R R， ZHANG Z G， et al. Traffic sign identification based on improved CapsNet［J］. Application Research of Computers， 2020， 37（S2）： 367-368， 371.
25	孙伟，杜宏吉，张小瑞，等. 基于CNN多层特征和ELM的交通标志识别［J］. 电子科技大学学报， 2018， 47（3）：343-349. 10.3969/j.issn.1001-0548.2018.03.004
	SUN W， DU H J， ZHANG X R， et al. Traffic sign recognition method based on multi-layer feature CNN and extreme learning machine［J］. Journal of University of Electronic Science and Technology of China， 2018， 47（3）： 343-349. 10.3969/j.issn.1001-0548.2018.03.004
26	王子键. 基于SSD模型的交通标志识别［D］. 长春：吉林大学， 2020：21-31.
	WANG Z J. Recognition of traffic signs based on SSD model［D］. Changchun： Jilin University， 2020： 21-31.
27	XU M D， ZHANG Z， HU H， et al. End-to-end semi-supervised object detection with soft teacher［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2021： 3040-3049. 10.1109/iccv48922.2021.00305
28	ZHANG H， WU C R， ZHANG Z Y， et al. ResNeSt： split-attention networks［EB/OL］. （2020-12-30）［2021-05-23］.. 10.1109/cvprw56347.2022.00309
29	LI L F， WANG Y R. Improved LeNet-5 convolutional neural network traffic sign recognition［J］. International Core Journal of Engineering， 2021， 7（4）： 114-121.
30	TAI S K， DEWI C， CHEN R C， et al. Deep learning for traffic sign recognition based on spatial pyramid pooling with scale analysis［J］. Applied Sciences， 2020， 10（19）： No.6997. 10.3390/app10196997
31	ZHOU K， ZHAN Y F， FU D M. Learning region-based attention network for traffic sign recognition［J］. Sensor， 2021， 21（3）： No.686. 10.3390/s21030686

[1]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[2]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[3]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[4]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[5]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[6]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[7]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[8]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[9]	Yeheng LI, Guangsheng LUO, Qianmin SU. Logo detection algorithm based on improved YOLOv5 [J]. Journal of Computer Applications, 2024, 44(8): 2580-2587.
[10]	Yingjun ZHANG, Niuniu LI, Binhong XIE, Rui ZHANG, Wangdong LU. Semi-supervised object detection framework guided by curriculum learning [J]. Journal of Computer Applications, 2024, 44(8): 2326-2333.
[11]	Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263.
[12]	Zhangjian JI, Na DU. Tiny target detection based on improved VariFocalNet [J]. Journal of Computer Applications, 2024, 44(7): 2200-2207.
[13]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[14]	Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP： defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086.
[15]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.

Traffic sign recognition model in haze weather based on YOLOv5

基于YOLOv5的雾霾天气下交通标志识别模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 21

References 31

Related Articles 15

Recommended Articles

Metrics