Abstract: Focusing on the shortcoming of the automatic image annotation, the lack of information caused by artificially selecting features, convolutional neural network was used to learn the characteristics of samples. Firstly, in order to adapt to the characteristics of multi label learning of automatic image annotation and increase the recall rate of the low frequency words, the loss function of convolutional neural network was improved and a Convolutional Neural Network of Multi-Label Learning (CNN-MLL) model was constructed. Secondly, the correlation between the image annotation words was used to improve the output of the network model. Compared with other traditional methods on the Technical Committee 12 of the International Association for Pattern Recognition (IAPR TC-12) benchmark image annotation database, the experimental result show that the Convolutional Neural Network using Mean Square Error function (CNN-MSE) method achieves the average recall rate of 12.9% more than the Support Vector Machine (SVM) method, the average accuracy of 37.9% more than the Back Propagation Neural Network (BPNN) method. And the average accuracy rate and average recall rate of marked results improved CNN-MLL method is 23% and 20% higher than those of the traditional CNN. The results show that the marked results improved CNN-MLL method can effectively avoid the information loss caused by the artificially selecting features, and increase the recall rate of the low frequency words.

Key words: automatic image annotation, multi-label learning, Convolution Neural Network (CNN), loss function

摘要： 针对图像自动标注中因人工选择特征而导致信息缺失的缺点，提出使用卷积神经网络对样本进行自主特征学习。为了适应图像自动标注的多标签学习的特点以及提高对低频词汇的召回率，首先改进卷积神经网络的损失函数，构建一个多标签学习的卷积神经网络（CNN-MLL）模型，然后利用图像标注词间的相关性对网络模型输出结果进行改善。通过在IAPR TC-12标准图像标注数据集上对比了其他传统方法，实验得出，基于采用均方误差函数的卷积神经网络（CNN-MSE）的方法较支持向量机（SVM）方法在平均召回率上提升了12.9%，较反向传播神经网络（BPNN）方法在平均准确率上提升了37.9%；基于标注结果改善的CNN-MLL方法较普通卷积神经网络的平均准确率和平均召回率分别提升了23%和20%。实验结果表明基于标注结果改善的CNN-MLL方法能有效地避免因人工选择特征造成的信息缺失同时增加了对低频词汇的召回率。

关键词: 图像自动标注, 多标签学习, 卷积神经网络, 损失函数