Automatic image annotation based on multi-label discriminative dictionary learning

doi:10.11772/j.issn.1001-9081.2017112650

Abstract

Abstract: Concerning the problem of semantic gap between low-level visual features and high-level semantics in automatic image annotation, based on traditional dictionary learning, a multi-label discriminative dictionary learning method was proposed to automatic image annotation. First of all, multiple types of features for each image were extracted, and a combination of a variety of features was used as input information of the input feature space to the dictionary learning. Then, a label consistency regularization term was designed to integrate the label information of the original samples into the initial input feature data, and the dictionary of label consistency and the label consistency regularization term were combined to learn the dictionary. Finally, the label sparse coding vector was obtained by the dictionary and sparse coding matrix to implement the semantic annotation for an unknown image. The performance of the annotation was tested on the Corel 5K data set. The average precision and average recall could reach 35% and 48% respectively, compared with the traditional Sparse Coding Method (MSC), which were increased by 10 percentage points and 16 percentage points respectively, and increased by 3 percentage points and 14 percentage points respectively than the method of Distance Constraint Sparse/Group Sparse Coding (DCSC/DCGSC) for automatic image lableing. Compared with the current image annotation methods, the experimental results show the proposed method can predict the semantic information for an unknown image properly, and has better annotation performance.

Key words: automatic image annotation, dictionary learning, feature representation, sparse coding, image retrieval

摘要： 针对图像自动标注中底层视觉特征与高层语义之间的语义鸿沟问题，在传统字典学习的基础上，提出一种基于多标签判别字典学习的图像自动标注方法。首先，为每幅图像提取多种类型特征，将多种特征组合作为字典学习输入特征空间的输入信息；然后，设计一个标签一致性正则化项，将原始样本的标签信息融入到初始的输入特征数据中，结合标签一致性判别字典和标签一致性正则化项进行字典学习；最后，通过得到的字典和稀疏编码矩阵求解标签稀疏编向量，实现未知图像的语义标注。在Corel 5K数据集上测试其标注性能，所提标注方法平均查准率和平均查全率分别可达到35%和48%；与传统的稀疏编码方法（MSC）相比，分别提高了10个百分点和16个百分点；与距离约束稀疏/组稀疏编码方法（DCSC/DCGSC）相比，分别提高了3个百分点和14个百分点。实验结果表明，所提方法能够较好地预测未知图像的语义信息，与当前几种流行的图像标注方法进行比较，所提方法具有较好的标注性能。

关键词: 图像自动标注, 字典学习, 特征表示, 稀疏编码, 图像检索

CLC Number:

TP391.41

YANG Xiaoling, LI Zhiqing, LIU Yutong. Automatic image annotation based on multi-label discriminative dictionary learning[J]. Journal of Computer Applications, 2018, 38(5): 1294-1298.

杨晓玲, 李志清, 刘雨桐. 基于多标签判别字典学习的图像自动标注[J]. 计算机应用, 2018, 38(5): 1294-1298.

References

[1] 刘梦迪, 陈燕俐, 陈蕾. 图像自动标注技术研究进展[J]. 计算机应用, 2016, 36(8):2274-2281.(LIU M D,CHEN Y L,CHEN L. Advances in automatic image annotation[J].Journal of Computer Applications, 2016, 36(8):2274-2281.)
[2] DATTA R, JOSHI D,LI J, et al. Image retrieval:ideas, influences, and trends of the new age[J]. ACM Computing Surveys, 2008, 40(2):1-60.
[3] WU J, SHEN H, LI Y D, et al. Learning a hybrid similarity measure for image retrieval[J]. Pattern Recognition, 2013, 46(11):2927-2939.
[4] CHANGE, GOHK, SYCHAY G, et al. CBSA:content-based soft annotation for multimodal image retrieval using Bayes point machines[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2003, 13(1):26-38.
[5] CUSANO C, CIOCCA G, SCHETTINI R. Image annotation using SVM[C]//Proceedings of SPIE 5304, Internet Imaging. Bellingham, WA:SPIE, 2003:330-338.
[6] WANG M, LI F, WANG M. Collaborative visual modeling for automatic image annotation via sparse model coding[J]. Nerocomputing, 2012,95(14):22-28.
[7] ZHANG W,TIAND, HU H, et al. Automatic image annotation via local sparse coding[C]//Proceedings of the 2013 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway, NJ:IEEE, 2013:1661-1665.
[8] GAO S, CHIA L T, TSANG W H, et al. Concurrent single-label image classification and annotation via efficient multi-layer group sparse coding[J]. IEEE Transactions on Multimedia, 2014, 16(3):762-771.
[9] LIU L, MA S, RUI L, et al. Locality constrained dictionary learning for non-linear dimensionality reduction and classification[J]. IET Computer Vision, 2017, 11(1):60-67.
[10] CARNEIRO G, CHAN A B, MORENO P J, et al. Supervised learning of semantic classes for image annotation and retrieval[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2007, 29(3):394-410.
[11] XIA Z, FENG X, PENG J, et al. Content-irrelevant tag cleansing via bi-layer clustering and peer cooperation[J]. Journal of Signal Processing Systems, 2015, 81(1):29-44.
[12] WANG C, YAN S, ZHANG L, et al. Multi-label sparse coding for automatic image annotation[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2009:1643-1650.
[13] 吴伟,高光来,聂建云.一种融合语义距离的最近邻图像标注方法[J].计算机科学,2015,42(1):297-302.(WU W, GAO G L, NIE J Y. Combination of nearest neighbor with semantic distance for image annotation[J].Computer Science,2015,42(1):297-302.)
[14] 臧淼, 徐惠民, 张永梅. 基于距离约束稀疏/组稀疏编码的自动图像标注[J].四川大学学报(工程科学版), 2016, 48(5):78-83.(ZANG M, XU H M, ZHANG Y M. Distance constraint sparse/group sparse coding for automatic image labeling[J]. Journal of Sichuan University (Engineering Science Edition), 2016, 48(5):78-83.)
[15] TANG J, HONG R, YAN S, et al. Image annotation by kNN-sparse graph-based label propagation over noisily tagged Web images[J]. ACM Transactions on Intelligent Systems & Technology, 2011, 2(2):1-15.
[16] AHARON M, ELAD M, BRUCKSTEIN A. K-SVD:an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11):4311-4322.
[17] JIANG Z, LIN Z, DAVISL S. Label consistent K-SVD:learning a discriminative dictionary for recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(11):2651-2664.
[18] LEE H, BATTLE A, RAINA R, et al. Efficient sparse coding algorithms[C]//Proceedings of the 19th International Conference on Neural Information Processing Systems. Cambridge, MA:MIT Press, 2006:801-808.
[19] GUILLAUMIN M, MENSINK T, VERBEEK J, et al. TagProp:Discriminative metric learning in nearest neighbor models for image auto-annotation[C]//Proceedings of the 2009 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2009:309-316.
[20] 范馨予,崔晓康.基于稀疏编码的图像自动标注[J].电子技术与软件工程,2017(4):83-84.(FAN X Y,CUI X K.Automatic image annotation based on sparse coding[J]. Electronic Technology and Software Engineering, 2017(4):83-84.)
[21] 吴寿昆,郭玉堂.基于Voronoi K阶邻近图的半监督学习自动图像标注[J].计算机应用与软件, 2016, 33(12):183-187.(WU S K, GUO Y T. Semi supervised learning automatic image annotation based on Voronoi K order adjacency graph[J]. Computer Applications and Software, 2016, 33(12):183-187.)
[22] MAKADIA A, PAVLOVIC V, KUNAR S. Baselines for image annotation[J].International Journal of Computer Vision, 2010, 90(1):88-105.
[23] CHEN M, ZHANG A, WEINBERGER K Q. Fast image tagging[C]//Proceedings of the 30th International Conference on International Conference on Machine Learning.[S.l.]:JMLR.org, 2013:1274-1282.
[24] SUN F, TANG J, LI H, et al. Multi-label image categorization with sparse factor representation[J]. IEEE Transactions on Image Processing, 2014, 23(3):1028-1037.