基于多模态信息融合的新闻图像人脸标注

doi:10.11772/j.issn.1001-9081.2017.10.3006

计算机应用 ›› 2017, Vol. 37 ›› Issue (10): 3006-3011.DOI: 10.11772/j.issn.1001-9081.2017.10.3006

• 应用前沿、交叉与综合 • 上一篇下一篇

基于多模态信息融合的新闻图像人脸标注

征察, 吉立新, 李邵梅, 高超

国家数字交换系统工程技术研究中心, 郑州 450000

收稿日期:2017-04-26 修回日期:2017-06-16 出版日期:2017-10-10 发布日期:2017-10-16
通讯作者: 征察(1994-),男,安徽宿州人,硕士研究生,主要研究方向:计算机视觉、跨媒体信息处理,E-mail:zcpi31415926@163.com
作者简介:征察(1994-),男,安徽宿州人,硕士研究生,主要研究方向:计算机视觉、跨媒体信息处理;吉立新(1969-),男,河南郑州人,研究员,博士,主要研究方向:通信与信息系统;李邵梅(1982-),女,湖北钟祥人,副研究员,博士,主要研究方向:数字图像处理、模式识别;高超(1982-),男,河南新郑人,讲师,博士,主要研究方向:计算机视觉、机器学习.
基金资助:
国家自然科学基金资助项目（61601513）。

Face annotation in news images based on multi-modal information fusion

ZHENG Cha, JI Lixin, LI Shaomei, GAO Chao

National Digital Switching System Engineering & Technological Research Center, Zhengzhou Henan 450000, China

Received:2017-04-26 Revised:2017-06-16 Online:2017-10-10 Published:2017-10-16
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61601513).

摘要/Abstract

摘要： 针对传统新闻图像中人脸标注方法主要依赖人脸相似度信息，分辨噪声和非噪声人脸能力以及非噪声人脸标注能力较差的问题，提出一种基于多模态信息融合的新闻图像人脸标注方法。首先根据人脸和姓名的共现关系，利用改进的K近邻算法，获得基于人脸相似度信息的人脸姓名匹配度；然后，分别从图像中提取人脸大小和位置的信息对人脸重要程度进行表征，从文本中提取姓名位置信息对姓名重要程度进行表征；最后，使用反向传播神经网络来融合上述信息完成人脸标签的推理，并提出一个标签修正策略来进一步改善标注结果。在Label Yahoo! News数据集上的测试效果表明，所提方法的标注准确率、精度和召回率分别达到了77.11%、73.58%和78.75%，与仅基于人脸相似度的算法相比，具有较好的分辨噪声和非噪声人脸能力以及非噪声人脸标注能力。

关键词: 新闻图像, 人脸标注, K近邻算法, 多模态信息, 反向传播神经网络

Abstract: The traditional face annotation methods for news images mainly rely on similarity information of the faces, and have poor ability to distinguish non-noise faces from noise faces and to annotate non-noise faces. Aiming at this issue, a face annotation method based on multi-modal information fusion was proposed. Firstly, according to the co-occurrence relations between faces and names, face-name match degrees based on face similarity were obtained by using a modified K-Nearest Neighbor (KNN) algorithm. After that, face importance degrees were characterized by the size and position information of faces extracted from images, and name importance degrees were characterized by the name position information extracted from images. Finally, Back Propagation (BP) neural network was applied to fuse the above information to infer labels of faces, and an annotation result correcting strategy was proposed to further improve the annotation results. Experimental results on Label Yahoo!News dataset demonstrate that the accuracy, precision and recall of the proposed method reach 77.11%, 73.58% and 78.75% respectively; compared with the methods only based on face similarity, the proposed method has outstanding ability to distinguish non-noise faces from noise faces and to annotate non-noise faces.

Key words: news image, faces annotation, K-Nearest Neighbor (KNN) algorithm, multi-modal information, Back Propagation (BP) neural network

中图分类号:

TP391.4

征察, 吉立新, 李邵梅, 高超. 基于多模态信息融合的新闻图像人脸标注[J]. 计算机应用, 2017, 37(10): 3006-3011.

ZHENG Cha, JI Lixin, LI Shaomei, GAO Chao. Face annotation in news images based on multi-modal information fusion[J]. Journal of Computer Applications, 2017, 37(10): 3006-3011.

参考文献

[1] TAIGMAN Y, YANG M, RANZATO M, et al. DeepFace: closing the gap to human-level performance in face verification [C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2014: 1701-1708.
[2] SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: a unified embedding for face recognition and clustering [C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2015: 815-823.
[3] OUYANG W, LOY C C, TANG X, et al. DeepID-Net: deformable deep convolutional neural networks for object detection [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 46(5): 2403-2412.
[4] 刘胜宇. 网络新闻图像中人脸标注技术研究[D]. 哈尔滨: 哈尔滨工业大学, 2011: 33-40. (LIU S Y. Research on annotation technology of face images in network news [D]. Harbin: Harbin Institute of Technology, 2011: 33-40.)
[5] FREY B J, DUECK D. Clustering by passing messages between data points [J]. Science, 2007, 315(5814): 972.
[6] LUO J, ORABONA F. Learning from candidate labeling sets [C]//NIPS 2010: Proceedings of the 23rd International Conference on Neural Information Processing Systems. New York: Curran Associates, 2010: 1504-1512.
[7] ZHANG M L, YU F. Solving the partial label learning problem: an instance-based approach [C]//Proceedings of the 2015 International Conference on Artificial Intelligence. Menlo Park: AAAI Press, 2015: 4048-4054.
[8] WANG D, HOI S C H, HE Y. A unified learning framework for auto face annotation by mining Web facial images [C]//Proceedings of the 2012 ACM International Conference on Information and Knowledge Management. New York: ACM, 2012: 1392-1401.
[9] WANG D, HOI S C H, HE Y, et al. Retrieval-based face annotation by weak label regularized local coordinate coding [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(3): 550-63.
[10] WANG D, HOI S C H, HE Y, et al. Mining weakly labeled Web facial images for search-based face annotation [J]. IEEE Transactions on Knowledge & Data Engineering, 2012, 26(1): 166-179.
[11] CHEN Z, FENG B, NGO C W, et al. Improving automatic name-face association using celebrity images on the Web [C]//Proceedings of the 2015 International Conference on Multimedia Retrieval. New York: ACM, 2015: 623-626.
[12] GUILLAUMIN M, VERBEEK J, SCHMID C. Multiple instance metric learning from automatically labeled bags of faces [C]//Proceedings of the 2010 European Conference on Computer Vision. Berlin: Springer, 2010: 634-647.
[13] XIAO S, TAN M, XU D. Weighted block-sparse low rank representation for face clustering in videos[C]//Proceedings of the 2014 European Conference on Computer Vision. Berlin: Springer, 2014: 123-138.
[14] XIAO S, XU D, WU J. Automatic face naming by learning discriminative affinity matrices from weakly labeled images [J]. IEEE Transactions on Neural Networks & Learning Systems, 2015, 26(10): 2440-2452.
[15] MATHIALAGAN C S, GALLAGHER A C, BATRA D. VIP: finding important people in images [C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2015: 4858-4866.
[16] BERG T L, BERG A C, EDWARDS J, et al. Names and faces in the news [C]//CVPR 2004: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2004: 848-854.
[17] GUILLAUMIN M, MENSINK T, VERBEEK J, et al. Face recognition from caption-based supervision[J]. International Journal of Computer Vision, 2012, 96(1): 64.
[18] CHAWLA N V, BOWYER K W, HALL L O, et al. SMOTE: synthetic minority over-sampling technique [J]. Journal of Artificial Intelligence Research, 2002, 16(1): 321-357.
[19] PANG L, NGO C W. Unsupervised celebrity face naming in Web videos [J]. IEEE Transactions on Multimedia, 2015, 17(6): 854-866.
[20] MARQUARDT D W. An algorithm for least-squares estimation of nonlinear parameters [J]. Journal of the Society for Industrial & Applied Mathematics, 2006, 11(2): 431-441.

基于多模态信息融合的新闻图像人脸标注

Face annotation in news images based on multi-modal information fusion

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 14

编辑推荐

Metrics

[1]	褚苏荣, 牛之贤, 宋春花, 牛保宁. 面向移动端的渐进网格简化算法[J]. 计算机应用, 2020, 40(3): 806-811.
[2]	吴小莉, 郑艺峰. 基于K近邻算法的噪声种类识别和强度估计[J]. 计算机应用, 2020, 40(1): 264-270.
[3]	韩颖, 赵萌, 陈胜勇, 王照锡. 基于最大截面特征的病变宫颈细胞核的自动筛查[J]. 计算机应用, 2019, 39(4): 1189-1195.
[4]	陈浩广, 王银河. 基于扩展BP神经网络的一类非线性系统自适应控制设计[J]. 计算机应用, 2017, 37(6): 1670-1673.
[5]	刘巧玲, 李劲, 肖人彬. 基于参数反演的网络舆情传播趋势预测——以新浪微博为例[J]. 计算机应用, 2017, 37(5): 1419-1423.
[6]	李世宝, 陈瑞祥, 刘建航, 陈海华, 丁淑妍, 龚琛. 基于反向传播神经网络改进的增益修改卡尔曼滤波算法[J]. 计算机应用, 2016, 36(5): 1196-1200.
[7]	郑奇斌, 刁兴春, 曹建军, 周星, 许永平. 结合局部敏感哈希的k近邻数据填补算法[J]. 计算机应用, 2016, 36(2): 397-401.
[8]	邵良杉, 郭雅婵. 基于Codebook的视频火焰识别算法[J]. 计算机应用, 2015, 35(5): 1483-1487.
[9]	罗丹, 罗海勇. 基于随机森林的跌倒检测算法[J]. 计算机应用, 2015, 35(11): 3157-3160.
[10]	张玲王玲吴桐. 基于改进的粒子群算法优化反向传播神经网络的热舒适度预测模型[J]. 计算机应用, 2014, 34(3): 775-779.
[11]	徐红艳党晓婉冯勇李军平. 基于BP神经网络的Deep Web实体识别方法[J]. 计算机应用, 2013, 33(03): 776-779.
[12]	马东昱孙龙清. 基于图像特征的籽棉品级分级模型的研究[J]. 计算机应用, 2010, 30(8): 2235-2238.
[13]	张宏斌贾志新. 基于逆向思维的系统可靠性精确分配模型[J]. 计算机应用, 2009, 29(09): 2468-2470.
[14]	张静俞辉. 一种多模态信息融合的视频检索模型[J]. 计算机应用, 2008, 28(1): 199-201,.