融合阈值寻优的卷积神经网络在图像标注中的应用

doi:10.11772/j.issn.1001-9081.2019111993

计算机应用 ›› 0, Vol. ›› Issue (): 1587-1592.DOI: 10.11772/j.issn.1001-9081.2019111993

融合阈值寻优的卷积神经网络在图像标注中的应用

曹建芳^1,2, 赵爱迪¹, 张自邦¹

1.太原科技大学计算机科学与技术学院，太原 030024
2.忻州师范学院计算机系，山西忻州 034000

收稿日期:2019-11-25 修回日期:2020-01-05 发布日期:2020-06-18 出版日期:2020-06-10
通讯作者: 曹建芳(1976—)
作者简介:曹建芳(1976—)，女，山西忻州人，教授，博士，CCF高级会员，主要研究方向：数字图像理解、大数据。赵爱迪(1993—)，女，安徽颍上人，硕士研究生，主要研究方向：深度学习、图像处理。张自邦(1995—)，男，河北临西人，硕士研究生，主要研究方向：深度学习、图像处理。
基金资助:
山西省自然科学基金资助项目(201701D121059)；山西省高等学校人文社会科学重点研究基地项目(20190130)；山西省艺术科学规划课题(2017F06);忻州市平台和人才专项(20180601)。

Application of convolutional neural network with threshold optimization in image annotation

CAO Jianfang^1,2, ZHAO Aidi¹, ZHANG Zibang¹

1. College of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan Shanxi 030024, China
2. Computer Department, Xinzhou Teachers University, Xinzhou Shanxi 034000, China

Received:2019-11-25 Revised:2020-01-05 Online:2020-06-18 Published:2020-06-10
Contact: CAO Jianfang, born in 1976, Ph. D., professor. Her research interests include digital image understanding, big data.
About author:CAO Jianfang, born in 1976, Ph. D., professor. Her research interests include digital image understanding, big data.ZHAO Aidi, born in 1993, M. S. candidate. Her research interests include deep learning, image processing.ZHANG Zibang, born in 1995, M. S. candidate. His research interests include deep learning, image processing.
Supported by:
Natural Science Foundation of Shanxi Province (201701D121059), the Program of Humanities and Social Sciences Key Research Base of Higher Education Institutions of Shanxi (20190130), the Art and Science Planning Project of Shanxi Province (2017F06), the Special Projects for Platforms and Talents of Xinzhou (20180601).

摘要/Abstract

摘要：

多标签图像标注在根据模型预测的概率，利用排名函数进行标注时会出现多标或少标的问题，提出了融合阈值寻优的卷积神经网络(CNN-THOP)模型，该模型由卷积神经网络(CNN)和阈值寻优构成。首先，通过CNN训练模型，利用该模型对图片进行预测，得到预测概率，其中在CNN中增加了批标准化层(BN)有效地加快了收敛。其次，利用该模型对测试集图片的预测概率进行阈值寻优，经过阈值寻优过程为每类标签得到一个最佳阈值，从而得到一组最佳阈值，只有当该类标签的预测概率大于等于该类标签的最佳阈值时，才会给图片标注该标签。在标注过程中，通过载入CNN模型和一组最佳阈值可以对所需标注的图像进行更加灵活的多标签标注。通过在自然场景图像数据集8 000张图片上的验证，结果表明，CNN-THOP较传统的基于排名的支持向量机法(Rank-SVM)在平均查准率上提升了约20个百分点，较基于均方误差函数的卷积神经网络(CNN-MSE)在平均召回率和F1值上分别提高了约6个百分点和4个百分点，且完全匹配度(CMD)达到了64.75%，验证了该方法在图像自动标注方面的有效性。

关键词: 图像自动标注, 多标签学习, 卷积神经网络, 阈值寻优, 批标准化

Abstract:

Ranking function based annotation may cause more or fewer labels according to the probability predicted by the model in multi-label image annotation. Therefore, a Convolutional Neural Network with THreshold OPtimization (CNN-THOP) model was proposed. The model consists of Convolutional Neural Network (CNN) and threshold optimization. Firstly, CNN was used to train a model, which was used to predict the image, so as to obtain the prediction probability, and Batch Normalization (BN) layer was added to the CNN to effectively accelerate the convergence. Secondly, threshold optimization was performed by the prediction probabilities of the test set images obtained by the proposed model. After the threshold optimization process, an optimal threshold was obtained for each kind of label, so as to obtain a set of optimal thresholds. Only when the prediction probability of this kind of label was greater than or equal to the best threshold of this kind of label, the image would be labeled with this label. In the labeling process, the CNN model and a set of optimal thresholds were added to achieve more flexible multi-label labeling of the image to be labeled. Through the verification on 8 000 images in the natural scene image dataset, experimental results show that CNN-THOP has about 20 percentage points improvement on average precision compared to Ranking Support Vector Machine (Rank-SVM), and is about 6 percentage points and 4 percentage points higher respectively than Convolutional Neural Network using Mean Square Error function (CNN-MSE) in average recall and F1 value respectively, and has the Complete Matching Degree (CMD) reached 64.75%, which proves that the proposed method is effective in automatic image annotation.

Key words: automatic image annotation, multi-label learning, Convolutional Neural Network (CNN), threshold optimization, Batch Normalization (BN)

中图分类号:

TP391.41

曹建芳, 赵爱迪, 张自邦. 融合阈值寻优的卷积神经网络在图像标注中的应用[J]. 计算机应用, 0, (): 1587-1592.

CAO Jianfang, ZHAO Aidi, ZHANG Zibang. Application of convolutional neural network with threshold optimization in image annotation[J]. Journal of Computer Applications, 0, (): 1587-1592.

参考文献

1 黎健成,袁春,宋友.基于卷积神经网络的多标签图像自动标注[J].计算机科学,2016,43(7):41-45. LIJ C, YUANC, SONGY. Multi-label images annotation based on convolutional neural network [J]. Computer Science, 2016, 43(7): 41-45.
2 吴伟,聂建云,高光来.一种基于改进的支持向量机多分类器图像标注方法[J].计算机工程与科学,2015,37(7):1338-1343. WUW, NIEJ Y, GAOG L. Improved SVM multiple classifiers for image annotation [J]. Computer Engineering and Science, 2015, 37(7): 1338-1343.
3 臧淼,徐惠民,张永梅.基于距离约束稀疏/组稀疏编码的自动图像标注[J].四川大学学报(工程科学版),2016,48(5):78-83. ZANGM, XUH M, ZHANGY M. Distance constraint sparse/group sparse coding for automatic image labeling [J]. Journal of Sichuan University (Engineering Science Edition), 2016, 48(5): 78-83.
4 杨晓玲,李志清,刘雨桐.基于多标签判别字典学习的图像自动标注[J].计算机应用,2018,38(5):1294-1298,1303. YANGX L, LIZ Q, LIUY T. Automatic image annotation based on multi-label discriminative dictionary learning [J]. Journal of Computer Applications, 2018, 38(5): 1294-1298, 1303.
5 张华忠,侯进.基于决策树C4.5集成算法的图像自动标注[J].计算机应用研究,2018,35(7):2222-2224. ZHANGH Z, HOUJ. Image annotation based on decision tree C4.5 ensemble algorithm [J]. Application Research of Computers, 2018, 35(7): 2222-2224.
6 READJ, PEREZ-CRUZF. Deep learning for multi-label classification [J]. Machine Learning, 2014, 85(3): 333-359.
7 WUF, WANGZ, ZHANGZ, et al. Weakly semi-supervised deep learning for multi-label image annotation [J]. IEEE Transactions on Big Data, 2015, 1(3): 109-122.
8 HINTONG E. A practical guide to training restricted Boltzmann machines [J]. Momentum, 2010, 9(1): 926-947.
9 柯逍,周铭柯,牛玉贞.融合深度特征和语义邻域的自动图像标注[J].模式识别与人工智能,2017,30(3):193-203. KEX, ZHOUM K, NIUY Z. Automatic image annotation combining semantic neighbors and deep features [J]. Pattern Recognition and Artificial Intelligence, 2017, 30(3): 193-203.
10 周铭柯,柯逍,杜明智.基于数据均衡的增进式深度自动图像标注[J].软件学报,2017,28(7):1862-1880. ZHOUM K, KEX, DUM Z. Enhanced deep automatic image annotation based on data equalization [J]. Journal of Software, 2017, 28(7): 1862-1880.
11 张蕾,蔡明.融合卷积神经网络与主题模型的图像标注[J].激光与光电子学进展,2019,56(20):201004-1-201004-7 ZHANG L,CAI M.Image annotation based on convolutional neural network and topic model [J]. Laser and Optoelectronics Progress, 2019, 56(20): 201004-1-201004-7.
12 黄冬梅,许琼琼,贺琪,等.融合多特征的深度学习标注方法[J].计算机工程与应用,2018,54(1):224-228. HUANGD M, XUQ Q, HEQ, et al. Multi-features fusion for image auto-annotation based on DBN model [J]. Computer Engineering and Applications, 2018, 54(1): 224-228.
13 杨阳,张文生,杨雪冰.基于Dropout深度网络的两步图像标注算法[J].计算机科学与探索,2015,9(12):1494-1505. YANGY, ZHANGW S, YANGX B. Two steps images annotation algorithm based on deep network with Dropout [J]. Journal of Frontiers of Computer Science and Technology, 2015, 9(12): 1494-1505.
14 KRIZHEVSKYA, SUTSKEVERI, HINTONG E. ImageNet classification with deep convolutional neural networks [C]// Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2012: 1097-1105.
15 HUBELD H, WIESELT N. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex [J]. The Journal of Physiology, 1962, 160(1): 106-154.
16 FUKUSHIMAK. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position [J]. Biological Cybernetics, 1980, 36(4): 193-202.
17 LECUNY, BOTTOUL, BENGIOY, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE,1998, 86(11): 2278-2324.
18 高耀东,侯凌燕,杨大利.基于多标签学习的卷积神经网络的图像标注方法[J].计算机应用,2017,37(1):228-232. GAOY D, HOUL Y, YANGD L. Automatic image annotation method using multi-label learning convolutional neural network [J]. Journal of Computer Applications, 2017, 37(1): 228-232.
19 LOFFES, SZEGEDYC. Batch normalization: accelerating deep network training by reducing internal covariate shift [C]// Proceedings of the 32nd International Conference on Machine Learning. New York: ACM, 2015: 448-456.
20 ZHANGM, ZHOUZ. ML-KNN: a lazy learning approach image to multi-label learning [J]. Pattern Recognition, 2007, 40(7): 2038-2048.
21 ELISSEEFFA, WESTONJ. A kernel method for multi-labelled classification[C]// Proceedings of the 14th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2001:681-687.
22 DE COMITéF, GILLERONR, TOMMASIM. Learning multi-label alternating decision tree from texts and data [C]// Proceedings of the 2003 International Workshop on Machine Learning and Data Mining in Pattern Recognition, LNCS 2734 . Berlin: Springer, 2003:35-49.
23 MURTHYV N, MAJIS, MANMATHAR. Automatic image annotation using deep learning representations [C]// Proceedings of the 5th ACM International Conference on Multimedia Retrieval. New York: ACM, 2015: 603-606.

[1]	秦璟, 秦志光, 李发礼, 彭悦恒. 基于概率稀疏自注意力神经网络的重性抑郁疾患诊断[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2970-2974.
[2]	李云, 王富铕, 井佩光, 王粟, 肖澳. 基于不确定度感知的帧关联短视频事件检测方法[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2903-2910.
[3]	张春雪, 仇丽青, 孙承爱, 荆彩霞. 基于两阶段动态兴趣识别的购买行为预测模型[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2365-2371.
[4]	陈虹, 齐兵, 金海波, 武聪, 张立昂. 融合1D-CNN与BiGRU的类不平衡流量异常检测[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2493-2499.
[5]	赵宇博, 张丽萍, 闫盛, 侯敏, 高茂. 基于改进分段卷积神经网络和知识蒸馏的学科知识实体间关系抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2421-2429.
[6]	王东炜, 刘柏辰, 韩志, 王艳美, 唐延东. 基于低秩分解和向量量化的深度网络压缩方法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 1987-1994.
[7]	高阳峄, 雷涛, 杜晓刚, 李岁永, 王营博, 闵重丹. 基于像素距离图和四维动态卷积网络的密集人群计数与定位方法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2233-2242.
[8]	沈君凤, 周星辰, 汤灿. 基于改进的提示学习方法的双通道情感分析模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1796-1806.
[9]	黄梦源, 常侃, 凌铭阳, 韦新杰, 覃团发. 基于层间引导的低光照图像渐进增强算法[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1911-1919.
[10]	李健京, 李贯峰, 秦飞舟, 李卫军. 基于不确定知识图谱嵌入的多关系近似推理模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1751-1759.
[11]	姚迅, 秦忠正, 杨捷. 生成式标签对抗的文本分类模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1781-1785.
[12]	席治远, 唐超, 童安炀, 王文剑. 基于双路时空网络的驾驶员行为识别[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1511-1519.
[13]	孙敏, 成倩, 丁希宁. 基于CBAM-CGRU-SVM的Android恶意软件检测方法[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1539-1545.
[14]	高文烁, 陈晓云. 基于节点结构的点云分类网络[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1471-1478.
[15]	陈天华, 朱家煊, 印杰. 基于注意力机制的鸟类识别算法[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1114-1120.

融合阈值寻优的卷积神经网络在图像标注中的应用

Application of convolutional neural network with threshold optimization in image annotation

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics