Multi-center convolutional feature weighting based image retrieval

doi:10.11772/j.issn.1001-9081.2018041100

Journal of Computer Applications ›› 2018, Vol. 38 ›› Issue (10): 2778-2781.DOI: 10.11772/j.issn.1001-9081.2018041100

Previous Articles Next Articles

Multi-center convolutional feature weighting based image retrieval

ZHU Jie¹, ZHANG Junsan², WU Shufang³, DONG Yukun¹, LYU Lin¹

1. Department of Information Management, the National Police University for Criminal Justice, Baoding Hebei 0710001, China;
2. College of Computer and Communication Engineering, China University of Petroleum(East China), Qingdao Shandong 266580, China;
3. College of Management, Hebei University, Baoding Hebei 071000, China

Received:2018-02-26 Revised:2018-05-28 Online:2018-10-10 Published:2018-10-13
Supported by:
This work is partially supported by the National Social Science Foundation of China (17BTQ068), the Youth Foundation Project of Hebei Social Science Foundation (QN2015099), the Youth Foundation Project of Hebei Natural Science Foundation (F2018511002), the China Postdoctoral Science Foundation (2017M621078), the Foundation of the National Police University for Criminal Justice (XYZ201602), the Funding Project of Midwest Colleges and Universities Promoting Comprehensive Strength of Hebei University.

基于多中心卷积特征加权的图像检索方法

朱杰¹, 张俊三², 吴树芳³, 董宇坤¹, 吕琳¹

1. 中央司法警官学院信息管理系, 河北保定 071000;
2. 中国石油大学 (华东) 计算机与通信工程学院, 山东青岛 266580;
3. 河北大学管理学院, 河北保定 071000

通讯作者: 吴树芳
作者简介:朱杰(1982-),男,河北保定人,副教授,博士,CCF会员,主要研究方向:机器学习、计算机视觉;张俊三(1978-),男,山东寿光人,副教授,博士,CCF会员,主要研究方向:信息检索、机器学习;吴树芳(1980-),女,河北邯郸人,副教授,博士,主要研究方向:信息检索、机器学习;董宇坤(1975-),女,河北保定人,副教授,博士,主要研究方向:数据挖掘、机器学习;吕琳(1978-),女,吉林长春人,讲师,博士,主要研究方向:数据挖掘、机器学习。
基金资助:
国家社会科学基金资助项目（17BTQ068）；河北省社会科学基金青年基金资助项目（QN2015099）；河北省自然科学基金青年基金资助项目（F2018511002）；中国博士后基金资助项目（2017M621078）；中央司法警官学院校级科研项目（XYZ201602）；河北大学中西部提升综合实力专项资金。

Abstract

Abstract: Deep convolutional features can provide rich semantic information for image content description. In order to highlight the object content in the image representation, the multi-center convolutional feature weighting method was proposed based on the relationship between high response positions and object regions. Firstly, the pre-trained deep network model was used to extract the deep convolutional features. Secondly, the activation map was obtained by summing the feature maps in all the channels and the positions with top few highest responses were considered as the centers of the object. Thirdly, the number of the centers was considered as the scale, and the descriptors corresponding to different positions were weighted based on the distances between these centers and the positions. Finally, the image representation for image retrieval was generated by merging the image features obtained based on different numbers of centers. Compared with Sum-pooled Convolutional (SPoC) algorithm and Cross-dimensional (CroW) algorithm, the proposed method can provide scale information and highlight the object content in the image representation, and achieves excellent retrieval results in the Holiday, Oxford and Paris image retrieval datasets.

Key words: multi-center, scale, deep feature weighting, image representation, image retrieval

摘要： 深度卷积特征能够为图像内容描述提供丰富的语义信息，为了在图像表示中突出对象内容，结合激活映射中较大响应值与对象区域的关系，提出基于多中心卷积特征加权的图像表示方法。首先，通过预训练深度模型提取出图像卷积特征；其次，通过不同通道特征映射求和得到激活映射，并将激活映射中有较大响应值的位置认为是对象的中心；再次，将中心数量作为尺度，结合激活映射中不同位置与中心的距离为对应位置的描述子加权；最后，合并不同中心数量下的图像特征，生成图像表示用于图像检索。与池化卷积（SPoC）算法和跨维度（CroW）算法相比，所提方法能够为图像表示提供尺度信息的同时突出对象内容，并在Holiday、Oxford和Paris图像集中取得了良好的检索结果。

关键词: 多中心, 尺度, 深度特征加权, 图像表示, 图像检索

CLC Number:

TP391

ZHU Jie, ZHANG Junsan, WU Shufang, DONG Yukun, LYU Lin. Multi-center convolutional feature weighting based image retrieval[J]. Journal of Computer Applications, 2018, 38(10): 2778-2781.

朱杰, 张俊三, 吴树芳, 董宇坤, 吕琳. 基于多中心卷积特征加权的图像检索方法[J]. 计算机应用, 2018, 38(10): 2778-2781.

References

[1] ZHANG L, RUI Y. Image search-from thousands to billions in 20 years[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2013, 9(1):1-20.
[2] ALZU'BI A, AMIRA A, RAMZAN N. Semantic content-based image retrieval:a comprehensive study[J]. Journal of Visual Communication and Image Representation, 2015, 32:20-54.
[3] ZHU L, SHEN J, XIE L. Unsupervised visual hashing with semantic assistant for content-based image retrieval[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(2):472-486.
[4] SIVIC J, ZISSERMAN A. Video google:a text retrieval approach to object matching in videos[C]//Proceedings of the Ninth IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2003:1470-1477.
[5] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[6] PHILBIN J, CHUM O, ISARD M. Object retrieval with large vocabularies and fast spatial matching[C]//Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2007:1-8.
[7] KRIZHEVSKY, A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[EB/OL].[2018-02-02]. http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf.
[8] RUSSAKOVSKY O, DENG J, SU H. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2014, 115(3):211-252.
[9] DONAHUE J, JIA Y, VINYALS O. DeCAF:a deep convolutional activation feature for generic visual recognition[EB/OL].[2018-02-10]. http://proceedings.mlr.press/v32/donahue14.pdf.
[10] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[11] TOLIAS G, SICRE R, JÉGOU H. Particular object retrieval with integral max-pooling of CNN activations[EB/OL].[2018-02-10]. https://arxiv.org/abs/1511.05879.
[12] RADENOVIC F, TOLIAS G, CHUM O. CNN image retrieval learns from BoW:unsupervised fine-tuning with hard examples[C]//ECCV 2016:Proceedings of the 2016 European Conference on Computer Vision. Berlin:Springer, 2016:3-20.
[13] BABENKO A, LEMPITSKY V. Aggregating local deep features for image retrieval[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2015:1269-1277.
[14] WEI X S, LUO J H, WU J. Selective convolutional descriptor aggregation for fine-grained image retrieval[J]. IEEE Transactions on Image Processing, 2017, 26(6):2868-2881.
[15] AZIZPOUR H, RAZAVIAN A S, SULLIVAN J. From generic to specific deep representations for visual recognition[C]//Proceedings of the 2015 International Conference on Computer Vision and Pattern Recognition Workshops. Washington, DC:IEEE Computer Society,2015:36-45.
[16] KALANTIDIS Y, MELLINA C, OSINDERO S. Cross-dimensional weighting for aggregated deep convolutional features[C]//ECCV 2016:Proceedings of the 2016 European Conference on Computer Vision. Berlin:Springer,2016:685-701.
[17] LAZEBNIK S, SCHMID C, PONCE J. Beyond bags of features:spatial pyramid matching for recognizing natural scene categories[C]//Proceedings of the 2006 International Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2006:2169-2178.
[18] JEGOU H, DOUZE M, SCHMID C. Hamming embedding and weak geometric consistency for large scale image search[C]//Proceedings of the 2008 European Conference on Computer Vision. Piscataway, NJ:IEEE, 2008:304-317.
[19] PHILBIN J, CHUM O, ISARD M, et al. Lost in quantization:improving particular object retrieval in large scale image databases[C]//Proceedings of the 2008 International Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2008:1-8.
[20] BABENKO A, SLESAREV A, CHIGORIN A. Neural codes for image retrieval[C]//ECCV 2014:Proceedings of the 2014 European Conference on Computer Vision. Berlin:Springer, 2014:584-599.
[21] RAZAVIAN A S, SULLIVAN J, CARLSSON S. Visual instance retrieval with deep convolutional networks[J]. ITE Transactions on Media Technology and Applications, 2016, 4(3):251-258.

Multi-center convolutional feature weighting based image retrieval

基于多中心卷积特征加权的图像检索方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	JI Zhangjian, REN Xingwang. Object tracking algorithm of fully-convolutional Siamese networks with rotation and scale estimation [J]. Journal of Computer Applications, 2021, 41(9): 2705-2711.
[2]	WANG Gaojie, YE Zhonglin, ZHAO Haixing, ZHU Yu, MENG Lei. Analysis of hypernetwork characteristics in Tang poems and Song lyrics [J]. Journal of Computer Applications, 2021, 41(8): 2432-2439.
[3]	LI Yafang, LIANG Ye, FENG Weiwei, ZU Baokai, KANG Yujian. Deep network embedding method based on community optimization [J]. Journal of Computer Applications, 2021, 41(7): 1956-1963.
[4]	YANG Su, OUYANG Zhi, DU Nisuo. Unsupervised parallel hash image retrieval based on correlation distance [J]. Journal of Computer Applications, 2021, 41(7): 1902-1907.
[5]	WU Lidan, XUE Yuyang, TONG Tong, DU Min, GAO Qinquan. Image colorization algorithm based on foreground semantic information [J]. Journal of Computer Applications, 2021, 41(7): 2048-2053.
[6]	NIU Kangli, CHEN Yuzhang, SHEN Junfeng, ZENG Zhangfan, PAN Yongcai, WANG Yichong. Dual-channel night vision image restoration method based on deep learning [J]. Journal of Computer Applications, 2021, 41(6): 1775-1784.
[7]	CAO Jianfang, TIAN Xiaodong, JIA Yiming, YAN Minmin. Application of improved DeepLabV3+ model in mural segmentation [J]. Journal of Computer Applications, 2021, 41(5): 1471-1476.
[8]	LONG Chaoqi, JIANG Yu, XIE Yu. Improved wavelet clustering algorithm based on peak grid [J]. Journal of Computer Applications, 2021, 41(4): 1122-1127.
[9]	DUAN Youxiang, ZHANG Hanxiao, SUN Qifeng, SUN Youkai. Image super-resolution reconstruction algorithm based on Laplacian pyramid generative adversarial network [J]. Journal of Computer Applications, 2021, 41(4): 1020-1026.
[10]	JIANG Li, HUANG Shijian, YAN Wenjuan. Human action recognition method based on low-rank action information and multi-scale convolutional neural network [J]. Journal of Computer Applications, 2021, 41(3): 721-726.
[11]	HU Lihua, ZUO Weijian, NIE Yaoyao. Feature matching method based on weighted similarity measurement [J]. Journal of Computer Applications, 2021, 41(2): 511-516.
[12]	DU Peide, YAN Hua. Crowd counting network based on multi-scale spatial attention feature fusion [J]. Journal of Computer Applications, 2021, 41(2): 537-543.
[13]	HAN Jiandong, LI Xiaoyu. Pedestrian re-identification method based on multi-scale feature fusion [J]. Journal of Computer Applications, 2021, 41(10): 2991-2996.
[14]	LU Rongxiu, CHEN Mingming, YANG Hui, ZHU Jianyong. Element component content dynamic monitoring system based on time sequence characteristics of solution images [J]. Journal of Computer Applications, 2021, 41(10): 3075-3081.
[15]	XIAO Yong, ZHENG Kaihong, ZHENG Zhenjing, QIAN Bin, LI Sen, MA Qianli. Multi-scale skip deep long short-term memory network for short-term multivariate load forecasting [J]. Journal of Computer Applications, 2021, 41(1): 231-236.