Semi-supervised object detection framework guided by curriculum learning

doi:10.11772/j.issn.1001-9081.2023081062

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (8): 2326-2333.DOI: 10.11772/j.issn.1001-9081.2023081062

• Artificial intelligence • Previous Articles Next Articles

Semi-supervised object detection framework guided by curriculum learning

Yingjun ZHANG¹, Niuniu LI¹(), Binhong XIE¹, Rui ZHANG¹, Wangdong LU²

^1.College of Computer Science and Technology，Taiyuan University of Science and Technology，Taiyuan Shanxi 030024，China
^2.Shanxi Tianhe Cloud Computing Company Limited，Lvliang Shanxi 033000，China

Received:2023-08-07 Revised:2023-10-10 Accepted:2023-10-17 Online:2023-12-18 Published:2024-08-10
Contact: Niuniu LI
About author:ZHANG Yingjun， born in 1969， M. S.， professor-level seniorengineer. His research interests include intelligent software， softwarearchitecture.
LI Niuniu ， born in 1998， M. S. candidate， His research interestsinclude semi-supervised object detection.
XIE Binhong ， born in 1971， M. S.， associate professor. Hisresearch interests include intelligent software， machine learning.
ZHANG Rui ， born in 1987， Ph. D.， associate professor. Hisresearch interests include intelligent information processing.
LU Wangdong ， born in 1970， M. S.， senior engineer. His researchinterests include signal and information systems.
Supported by:
This work is partially supported by Shanxi Provincial Basic ResearchProgram Project （20210302123216）； Lvliang Key Research andDevelopment Project for Introduction of High-level Scientific andTechnological Talents（ 2022RC08）

课程学习指导下的半监督目标检测框架

张英俊¹, 李牛牛¹(), 谢斌红¹, 张睿¹, 陆望东²

^1.太原科技大学计算机科学与技术学院，太原 030024
^2.山西天河云计算有限公司，山西吕梁 033000

通讯作者: 李牛牛
作者简介:张英俊（1969—），男，山西河津人，教授级高级工程师，硕士，主要研究方向：智能化软件、软件体系结构
李牛牛（1998—），男，山西吕梁人，硕士研究生，主要研究方向：半监督目标检测 dbvoid@163.com
谢斌红（1971—），男，山西运城人，副教授，硕士，主要研究方向：智能化软件、机器学习
张睿（1987—），男，山西太原人，副教授，博士，主要研究方向：智能信息处理
陆望东（1970—），男，山西吕梁人，高级工程师，硕士，主要研究方向：信号与信息系统。
基金资助:
山西省基础研究计划项目(20210302123216);吕梁市引进高层次科技人才重点研发项目(2022RC08)

Abstract

Abstract:

In order to enhance the quality of pseudo labels， address the issue of confirmation bias in Semi-Supervised Object Detection （SSOD）， and tackle the challenge of ignoring complexities in unlabeled data leading to erroneous pseudo labels in existing algorithms， an SSOD framework guided by Curriculum Learning （CL） was proposed. The framework consisted of two modules： the ICSD （IoU-Confidence-Standard-Deviation） difficulty measurer and the BP （Batch-Package） training scheduler. The ICSD difficulty measurer comprehensively considered information such as IoU （Intersection over Union） between pseudo-bounding boxes， confidence， class label， etc.，and the C_IOU （Checkpoint_IOU） method was introduced to evaluate the reliability of unlabeled data. The BP training scheduler designed two efficient scheduling strategies， starting from the perspectives of Batch and Package respectively， giving priority to unlabeled data with high reliability indicators to achieve full utilization of the entire unlabeled data set in the form of course learning. Extensive comparative experimental results on the Pascal VOC and MS-COCO datasets demonstrate that the proposed framework applies to existing SSOD algorithms and exhibits significant improvements in detection accuracy and stability.

Key words: semi-supervised learning, object detection, Curriculum Learning (CL), training strategy, difficulty measurer, training scheduler

摘要：

为了提高伪标签的质量，解决半监督目标检测（SSOD）中的确认偏差问题，并针对现有算法中忽视无标注数据复杂性导致错误伪标签的难点，提出一种课程学习（CL）指导下的SSOD框架，该框架主要由ICSD（IoU-Confidence-Standard-Deviation）难度测量器和BP（Batch-Package）训练调度器这2个模块组成。其中，ICSD难度测量器综合考虑了伪边界框之间的交并比（IoU）、置信度、类别标签等信息，并引入C_IOU（Checkpoint_IOU）方法评估无标注数据的可靠性；BP训练调度器设计2种高效调度策略，分别从Batch和Package角度出发，优先选择可靠性指标高的无标记数据，实现以CL的方式充分利用整个无标记数据集。在Pascal VOC和MS-COCO数据集上的广泛对比实验结果表明，所提框架不仅适用于现有的SSOD算法，而且检测精度和稳定性都得到显著提升。

关键词: 半监督学习, 目标检测, 课程学习, 训练策略, 难度测量器, 训练调度器

CLC Number:

TP391.41

Yingjun ZHANG, Niuniu LI, Binhong XIE, Rui ZHANG, Wangdong LU. Semi-supervised object detection framework guided by curriculum learning[J]. Journal of Computer Applications, 2024, 44(8): 2326-2333.

张英俊, 李牛牛, 谢斌红, 张睿, 陆望东. 课程学习指导下的半监督目标检测框架[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2326-2333.

Figures/Tables 8

Fig. 1 ICSD reliability index

Fig. 2 SSOD framework guided by CL

Fig. 3 Comparison of detection effects for images with different difficulties at different epochs

Tab. 1 AP50 comparison with different ? on Pascal VOC dataset

$∂$	Unbiased-teacher/%		STAC/%		CrossRectify/%
$∂$	BS	PS	BS	PS	BS	PS
0.50	80.58	80.63	76.13	76.71	80.69	80.73
0.70	81.30	81.52	77.65	77.80	81.90	82.02
0.75	81.49	81.76	78.02	78.21	82.44	82.63
0.80	81.34	81.48	77.72	77.91	81.81	81.98
0.90	80.57	80.54	76.28	77.11	80.78	80.94

Tab. 1 AP50 comparison with different ? on Pascal VOC dataset

$∂$	Unbiased-teacher/%		STAC/%		CrossRectify/%
$∂$	BS	PS	BS	PS	BS	PS
0.50	80.58	80.63	76.13	76.71	80.69	80.73
0.70	81.30	81.52	77.65	77.80	81.90	82.02
0.75	81.49	81.76	78.02	78.21	82.44	82.63
0.80	81.34	81.48	77.72	77.91	81.81	81.98
0.90	80.57	80.54	76.28	77.11	80.78	80.94

Tab. 2 Experimental results of Faster-RCNN-FPN based models （ResNet-50 backbone network） on Pascal VOC dataset （AP50）

算法	标注数据集	无标注数据集	对照组	实验组BS	实验组PS
CSD^［8］	VOC07	VOC12	77.50	78.15	78.58
STAC^［10］			77.50	78.02	78.21
co-rectify^［11］			79.20	79.86	80.13
Unbiased_teacher^［12］			80.50	81.49	81.76
CrossRectify^［14］			81.56	82.44	82.63

Tab. 3 Experimental results of Faster-RCNN-FPN based models （ResNet-50 backbone network） on MS-COCO dataset （AP50：90）

组别	算法	不同监督程度的AP_50：90值/%
组别	算法	1%	2%	5%	10%
对照组	CSD^［8］	10.51±0.06	13.93±0.12	18.63±0.07	22.46±0.08
	STAC^［10］	13.97±0.35	18.25±0.25	24.38±0.12	28.64±0.21
	co-rectify^［11］	18.05±0.15	22.45±0.15	26.75±0.05	30.40±0.05
	Unbiased_teacher ^［12］	20.75±0.12	24.30±0.07	28.27±0.11	31.50±0.10
	CrossRectify^［14］	21.90±0.11	26.70±0.07	31.70±0.04	34.89±0.07
实验组BS	CSD^［8］	11.45±0.13	15.03±0.09	19.59±0.06	23.58±0.10
	STAC^［10］	14.47±0.28	18.74±0.36	24.88±0.15	29.22±0.11
	co-rectify^［11］	18.86±0.06	23.21±0.09	27.63±0.12	31.35±0.08
	Unbiased_teacher ^［12］	21.73±0.04	25.31±0.12	29.39±0.25	32.62±0.06
	CrossRectify^［14］	22.86±0.20	27.71±0.15	32.63±0.30	35.81±0.05
实验组PS	CSD^［8］	11.56±0.13	15.07±0.12	19.71±0.10	23.76±0.06
	STAC^［10］	14.53±0.15	18.81±0.06	24.93±0.12	29.34±0.20
	co-rectify^［11］	18.98±0.15	23.46±0.21	27.79±0.25	31.59±0.05
	Unbiased_teacher ^［12］	22.01±0.35	25.44±0.04	29.54±0.12	32.96±0.12
	CrossRectify^［14］	22.97±0.12	27.89±0.06	32.76±0.11	35.89±0.10

Tab. 4 Comparision experiment results of ICSD difficulty measurer on Faster-RCNN-FPN based models （ResNet-50 backbone network）

算法	标注数据集	无标注数据集	mAP/%
DU^［22］	VOC07	VOC12	78.60
DU+ICSD	VOC07	VOC12	79.36

Fig. 4 Convergence curve of Unbiased-teacher

References 23

1	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： unified， real-time object detection［C］// Proceedings of the 2016 International Conference on Computer Vision and Pattern Recognition . Piscataway： IEEE， 2016： 779-788.
2	LIU W， ANGUELOV D， ERHAN D， et al. SSD： single shot MultiBox detector［C］// Proceedings of the 14th European Conference on Computer Vision. Cham： Springer， 2016： 21-37.
3	REN S， HE K， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149.
4	EVERINGHAM M， VAN GOOL L， WILLIAMS C K I， et al. The Pascal Visual Object Classes （VOC） challenge［J］. International Journal of Computer Vision， 2010， 88（2）： 303-338.
5	LIN T-Y， MAIRE M， BELONGIE S， et al. Microsoft COCO： common objects in context［C］// Proceedings of the 13th European Conference on Computer Vision. Cham： Springer， 2014： 740-755.
6	ROSENBERG C， HEBERT M， SCHNEIDERMAN H. Semi-supervised self-training of object detection models［C］// Proceedings of the 2005 7th IEEE Workshops on Applications of Computer Vision. Piscataway： IEEE， 2005： 29-36.
7	ARAZO E， ORTEGO D， ALBERT P， et al. Pseudo-labeling and confirmation bias in deep semi-supervised learning［C］// Proceedings of the 2020 International Joint Conference on Neural Networks. Piscataway： IEEE， 2020： 1-8.
8	JEONG J， LEE S， KIM J， et al. Consistency-based semi-supervised learning for object detection［C］// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2019： 10759-10768.
9	JEONG J， VERMA V， HYUN M， et al. Interpolation-based semi-supervised learning for object detection［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 11597-11606.
10	SOHN K， ZHANG Z， LI C-L， et al. A simple semi-supervised learning framework for object detection［EB/OL］. （2020-05-10）［2023-08-01］. .
11	ZHOU Q， YU C， WANG Z， et al. Instant-Teaching： an end-to-end semi-supervised object detection framework［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 4079-4088.
12	LIU Y-C， MA C-Y， KIRA Z. Unbiased teacher for semi-supervised object detection［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 9809-9818.
13	XU M， ZHANG Z， HU H， et al. End-to-end semi-supervised object detection with soft teacher［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2021： 3040-3049.
14	MA C， PAN X， YE Q， et al. CrossRectify： leveraging disagreement for semi-supervised object detection［J］. Pattern Recognition， 2023， 137： 109280.
15	REN Z， YEH R A， SCHWING A G. Not all unlabeled data are equal： learning to weight data in semi-supervised learning［C］// Proceedings of the 34th Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2020： 21786-21797.
16	BENGIO Y， LOURADOUR J， COLLOBERT R， et al. Curriculum learning［C］// Proceedings of the 26th Annual International Conference on Machine Learning. New York： ACM， 2009： 41-48.
17	TARVAINEN A， VALPOLA H. Mean teachers are better role models： weight-averaged consistency targets improve semi-supervised deep learning results［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2017： 1195-1204.
18	戴立伟，黄山.基于课程学习思想的目标检测增强算法［J］. 计算机辅助设计与图形学学报， 2021， 33（2）： 278-286.
	DAI L W， HUANG S. Object detection enhancement algorithm based on curriculum learning［J］. Journal of Computer-Aided Design & Computer Graphics， 2021， 33（2）： 278-286.
19	贾乐瑶，马盈仓，邢志伟.基于自步学习的自适应半监督聚类算法［J］. 西北大学学报（自然科学版），2022， 52（5）： 847-856.
	JIA L Y， MA Y C， XING Z W. An adaptive semi-supervised clustering algorithm based on self-paced learning［J］. Journal of Northwest University （Natural Science Edition）， 2022， 52（5）： 847-856.
20	古楠楠，孙湘南，刘伟.基于自步学习与稀疏自表达的半监督分类方法［J］. 系统科学与数学，2020，40（1）：191-208.
	GU N N， SUN X N， LIU W. Semi-supervised classification method based on self-paced learning and sparse self-expression［J］. Journal of Systems Science and Mathematical Sciences， 2020， 40（1）： 191-208.
21	PLATANIOS E A， STRETCU O， NEUBIG G， et al. Competence-based curriculum learning for neural machine translation［EB/OL］. （2019-03-23）［2023-08-01］. .
22	WANG Z， LI Y， GUO Y， et al. Data-uncertainty guided multi-phase learning for semi-supervised object detection［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 4566-4575.
23	YANG L， ZHUO W， QI L， et al. ST++： make self-training work better for semi-supervised semantic segmentation［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 4258-4267.

[1]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[2]	Yeheng LI, Guangsheng LUO, Qianmin SU. Logo detection algorithm based on improved YOLOv5 [J]. Journal of Computer Applications, 2024, 44(8): 2580-2587.
[3]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[4]	Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215.
[5]	Yan ZHOU, Yang LI. Rectified cross pseudo supervision method with attention mechanism for stroke lesion segmentation [J]. Journal of Computer Applications, 2024, 44(6): 1942-1948.
[6]	Yue LIU, Fang LIU, Aoyun WU, Qiuyue CHAI, Tianxiao WANG. 3D object detection network based on self-attention mechanism and graph convolution [J]. Journal of Computer Applications, 2024, 44(6): 1972-1977.
[7]	Yaping DENG, Yingjiang LI. Review of YOLO algorithm and its applications to object detection in autonomous driving scenes [J]. Journal of Computer Applications, 2024, 44(6): 1949-1958.
[8]	Huantong GENG, Zhenyu LIU, Jun JIANG, Zichen FAN, Jiaxing LI. Embedded road crack detection algorithm based on improved YOLOv8 [J]. Journal of Computer Applications, 2024, 44(5): 1613-1618.
[9]	Xiaogang SONG, Dongdong ZHANG, Pengfei ZHANG, Li LIANG, Xinhong HEI. Real-time object detection algorithm for complex construction environments [J]. Journal of Computer Applications, 2024, 44(5): 1605-1612.
[10]	Hongtian LI, Xinhao SHI, Weiguo PAN, Cheng XU, Bingxin XU, Jiazheng YUAN. Few-shot object detection via fusing multi-scale and attention mechanism [J]. Journal of Computer Applications, 2024, 44(5): 1437-1444.
[11]	Wei WANG, Chunhui ZHAO, Xinyao TANG, Liugang XI. 3D vehicle detection with adaptive horizon line constraints [J]. Journal of Computer Applications, 2024, 44(3): 909-915.
[12]	Xinye LI, Yening HOU, Yinghui KONG, Zhiqi YAN. Few-shot object detection combining feature fusion and enhanced attention [J]. Journal of Computer Applications, 2024, 44(3): 745-751.
[13]	Yuqiu LI, Liping HOU, Jian XUE, Ke LYU, Yong WANG. Remote sensing image recommendation method based on content interpretation [J]. Journal of Computer Applications, 2024, 44(3): 722-731.
[14]	Keyi FU, Gaocai WANG, Man WU. Few-shot object detection method based on improved region proposal network and feature aggregation [J]. Journal of Computer Applications, 2024, 44(12): 3790-3797.
[15]	Jiachen YU, Ye YANG. Irregular object grasping by soft robotic arm based on clipped proximal policy optimization algorithm [J]. Journal of Computer Applications, 2024, 44(11): 3629-3638.

Semi-supervised object detection framework guided by curriculum learning

课程学习指导下的半监督目标检测框架

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 23

Related Articles 15

Recommended Articles

Metrics