Image retrieval method based on deep residual network and iterative quantization hashing

doi:10.11772/j.issn.1001-9081.2021071135

Abstract

Abstract:

Focusing on the issue that the existing hashing image retrieval methods have weak expression ability， slow training speed， low retrieval precision， and difficulty in adapting to large-scale image retrieval， an image retrieval method based on Deep Residual Network and Iterative Quantitative Hashing （DRITQH） was proposed. Firstly， the deep residual network was used to perform multiple non-linear transformations on the image data to extract features of the image data and obtain high-dimensional feature vectors with semantic features. Then， Principal Component Analysis （PCA） was used to reduce the high-dimensional image features' dimensions. At the same time， to minimize the quantization error and obtain the best projection matrix， iterative quantization was used to binarize the generated feature vectors， the rotation matrix was updated and the data was mapped to the zero-center binary hypercube. Finally， the optimal binary hash code which was used to image retrieval in the Hamming space was obtained through performing the hash learning. Experimental results show that the retrieval precisions of DRITQH for four hash codes with different lengths on NUS-WIDE dataset are 0.789， 0.831， 0.838 and 0.846 respectively， which are 0.5， 3.8， 3.7 and 4.2 percentage points higher than those of Improved Deep Hashing Network （IDHN） respectively， and the average encoding time of the proposed method is 1 717 μs less than that of IDHN. DRITQH reduces the impact of quantization errors， improves training speed， and achieves higher retrieval performance in large-scale image retrieval.

Key words: image retrieval, deep residual network, iterative quantization, hashing code, quantization error

摘要：

针对现有的哈希图像检索方法表达能力较弱、训练速度慢、检索精度低，难以适应大规模图像检索的问题，提出了一种基于深度残差网络的迭代量化哈希图像检索方法（DRITQH）。首先，使用深度残差网络对图像数据进行多次非线性变换，从而提取图像数据的特征，并获得具有语义特征的高维特征向量；然后，使用主成分分析（PCA）对高维图像特征进行降维，同时运用迭代量化对生成的特征向量进行二值化处理，更新旋转矩阵，将数据映射到零中心二进制超立方体，从而最小化量化误差并得到最佳的投影矩阵；最后，进行哈希学习，以得到最优的二进制哈希码在汉明空间中进行图像检索。实验结果表明，DRITQH在NUS-WIDE数据集上，对4种哈希码的检索精度分别为0.789、0.831、0.838和0.846，与改进深度哈希网络（IDHN）相比分别提升了0.5、3.8、3.7和4.2个百分点，平均编码时间小了1 717 μs。DRITQH在大规模图像检索时减少了量化误差带来的影响，提高了训练速度，实现了更高的检索性能。

关键词: 图像检索, 深度残差网络, 迭代量化, 哈希码, 量化误差

CLC Number:

TN911.73

Liefa LIAO, Zhiming LI, Saisai ZHANG. Image retrieval method based on deep residual network and iterative quantization hashing[J]. Journal of Computer Applications, 2022, 42(9): 2845-2852.

廖列法, 李志明, 张赛赛. 基于深度残差网络的迭代量化哈希图像检索方法[J]. 《计算机应用》唯一官方网站, 2022, 42(9): 2845-2852.

Figures/Tables 9

References 34

1	余若晟，徐超，张帆. 大规模人脸检索系统设计与实现［J］. 计算机应用与软件， 2021， 38（3）：119-123. 10.3969/j.issn.1000-386x.2021.03.018
	YU R S， XU C， ZHANG F. Design and implementation of large-scale face retrieval system［J］. Computer Applications and Software， 2021， 38（3）：119-123. 10.3969/j.issn.1000-386x.2021.03.018
2	周晔，张军平. 基于多尺度深度学习的商品图像检索［J］. 计算机研究与发展， 2017， 54（8）：1824-1832. 10.7544/issn1000-1239.2017.20170197
	ZHOU Y， ZHANG J P. Multi-scale deep learning for product image search［J］. Journal of Computer Research and Development， 2017， 54（8）：1824-1832. 10.7544/issn1000-1239.2017.20170197
3	苏卓，柯司博，王若梅，等. 深度多模态融合服装风格检索［J］. 中国图象图形学报， 2021， 26（4）：857-871. 10.11834/jig.200193
	SU Z， KE S B， WANG R M， et al. Fashion style retrieval based on deep multimodal fusion［J］. Journal of Image and Graphics， 2021， 26（4）：857-871. 10.11834/jig.200193
4	秦品乐，李启，曾建潮，等. 基于多尺度密集网络的肺结节图像检索算法［J］. 计算机应用， 2019， 39（2）：392-397. 10.11772/j.issn.1001-9081.2018071451
	QIN P L， LI Q， ZENG J C， et al. Image retrieval algorithm for pulmonary nodules based on multi-scale dense network［J］. Journal of Computer Applications， 2019， 39（2）：392-397. 10.11772/j.issn.1001-9081.2018071451
5	苗建辉，栗志扬，周泽艳，等. 比特串划分多索引的近邻搜索算法［J］.计算机辅助设计与图形学学报， 2019， 31（5）： 771-779. 10.3724/sp.j.1089.2019.17341
	MIAO J H， LI Z Y， ZHOU Z Y， et al. Nearest neighbor search based on bit string partition and multiple index［J］. Journal of Computer-Aided Design and Computer Graphics， 2019， 31（5）： 771-779. 10.3724/sp.j.1089.2019.17341
6	杨粟，欧阳智，杜逆索. 基于相关度距离的无监督并行哈希图像检索［J］. 计算机应用， 2021， 41（7）：1902-1907. 10.3390/info12070285
	YANG S， OUYANG Z， DU N S. Unsupervised parallel hash image retrieval based on correlation distance［J］. Journal of Computer Applications， 2021， 41（7）： 1902-1907. 10.3390/info12070285
7	马敏耀，徐艺，刘卓. 隐私保护DNA序列汉明距离计算问题［J］. 计算机应用， 2019， 39（9）：2636-2640. 10.11772/j.issn.1001-9081.2019020247
	MA M Y， XU Y， LIU Z. Privacy preserving Hamming distance computing problem of DNA sequences［J］. Journal of Computer Applications， 2019， 39（9）： 2636-2640. 10.11772/j.issn.1001-9081.2019020247
8	SCHMIDHUBER J. Deep learning in neural networks： an overview［J］. Neural Networks， 2015， 61： 85-117. 10.1016/j.neunet.2014.09.003
9	刘颖，程美，王富平，等. 深度哈希图像检索方法综述［J］. 中国图象图形学报， 2020， 25（7）：1296-1317.
	LIU Y， CHENG M， WANG F P， et al. Deep Hashing image retrieval methods［J］. Journal of Image and Graphics， 2020， 25（7）：1296-1317.
10	ZHENG L， YANG Y， TIAN Q. SIFT meets CNN： a decade survey of instance retrieval［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018， 40（5）： 1224-1244. 10.1109/tpami.2017.2709749
11	XIA R K， PAN Y， LAI H J， et al. Supervised hashing for image retrieval via image representation learning［C］// Proceedings of the 28th AAAI conference on artificial intelligence. Palo Alto， CA： AAAI Press， 2014：2156-2162. 10.1609/aaai.v28i1.8952
12	LAI H J， PAN Y， LIU Y， et al. Simultaneous feature learning and hash coding with deep neural networks［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 3270-3278. 10.1109/cvpr.2015.7298947
13	ALZU’BI A， AMIRA A， RAMZAN N. Content-based image retrieval with compact deep convolutional features［J］. Neurocomputing， 2017， 249： 95-105. 10.1016/j.neucom.2017.03.072
14	CONJETI S， ROY A G， KATOUZIAN A， et al. Hashing with residual networks for image retrieval［C］// Proceedings of the 2017 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS 10435. Cham： Springer， 2017： 541-549.
15	LIN K， LU J W， CHEN C S， et al. Learning compact binary descriptors with unsupervised deep neural networks［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1183-1192. 10.1109/cvpr.2016.133
16	DO T T， DOAN A D， CHEUNG N M. Learning to hash with binary deep neural network［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS 9909. Cham： Springer， 2016： 219-234.
17	ZHANG H F， LIU L， LONG Y， et al. Unsupervised deep hashing with pseudo labels for scalable image retrieval［J］. IEEE Transactions on Image Processing， 2018， 27（4）： 1626-1638. 10.1109/tip.2017.2781422
18	GONG Y C， LAZEBNIK S， GORDO A， et al. Iterative quantization： a procrustean approach to learning binary codes for large-scale image retrieval［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2013， 35（12）： 2916-2929. 10.1109/tpami.2012.193
19	曹路，杨文强. 基于离散监督哈希的相似性检索算法［J］. 科学技术与工程， 2017， 17（26）：245-250. 10.3969/j.issn.1671-1815.2017.26.040
	CAO L， YANG W Q. Similarity search method based on discrete supervised hashing［J］. Science Technology and Engineering， 2017， 17（26）：245-250. 10.3969/j.issn.1671-1815.2017.26.040
20	LIU H M， WANG R P， SHAN S G， et al. Deep supervised hashing for fast image retrieval［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2064-2072. 10.1109/cvpr.2016.227
21	崔文成，徐盼盼，邵虹. 基于双线性迭代量化的哈希图像检索方法［J］. 计算机应用研究， 2020， 37（8）：2284-2287.
	CUI W C， XU P P， SHAO H. Hashing image retrieval based on bilinear iterative quantization［J］. Application Research of Computers， 2020， 37（8）：2284-2287.
22	LIU Z F， CHEN F， DUAN S K. Distributed fast supervised discrete hashing［J］. IEEE Access， 2019， 7： 90003-90011. 10.1109/access.2019.2924996
23	ZHANG Z， ZOU Q， LIN Y W， et al. Improved deep hashing with soft pairwise similarity for multi-label image retrieval［J］. IEEE Transactions on Multimedia， 2020， 22（2）： 540-553. 10.1109/tmm.2019.2929957
24	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
25	SZEGEDY C， LIU W， JIA Y Q， et al. Going deeper with convolutions［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015：1-9. 10.1109/cvpr.2015.7298594
26	KRIZHEVSKY A. Learning multiple layers of features from tiny images［R/OL］. （2009-04-08）［2021-04-20］..
27	CHUA T S， TANG J H， HONG R C， et al. NUS-WIDE： a real-world Web image database from national university of Singapore［C］// Proceedings of the 2009 ACM International Conference on Image and Video Retrieval. New York： ACM， 2009： No.48. 10.1145/1646396.1646452
28	RUSSAKOVSKY O， DENG J， SU H， et al. ImageNet large scale visual recognition challenge［J］. International Journal of Computer Vision， 2015， 115（3）： 211-252. 10.1007/s11263-015-0816-y
29	WEISS Y， TORRALBA A， FERGUS R. Spectral hashing［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2008：1753-1760.
30	ANDONI A， INDYK P， NGUYEN H L， et al. Beyond locality-sensitive hashing［C］// Proceedings of the 25th Annual ACM-SIAM Symposium on Discrete Algorithms. Philadelphia， PA： Society for Industrial and Applied Mathematics， 2014： 1018-1028. 10.1137/1.9781611973402.76
31	LIU W， WANG J， JI R R， et al. Supervised hashing with kernels［C］// Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2012： 2074-2081. 10.1109/cvpr.2012.6247912
32	ZHENG X T， ZHANG Y C， LU X Q. Deep balanced discrete hashing for image retrieval［J］. Neurocomputing， 2020， 403： 224-236. 10.1016/j.neucom.2020.04.037
33	LI Y Q， PEI W J， ZHA Y F， et al. Push for quantization： deep fisher hashing［C］// Proceedings of the 2019 British Machine Vision Conference. Durham： BMVA Press， 2019： No.938.
34	FAN L X， NG K W， JU C， et al. Deep polarized network for supervised learning of accurate binary hashing codes［C］// Proceedings of the 29th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2020： 825-831. 10.24963/ijcai.2020/115

方法	CIFAR-10				NUS-WIDE				ImageNet
方法	12 bit	24 bit	32 bit	48 bit	12 bit	24 bit	32 bit	48 bit	12 bit	24 bit	32 bit	48 bit
ITQ	0.158	0.169	0.172	0.175	0.452	0.468	0.472	0.477	0.244	0.352	0.428	0.482
LSH	0.127	0.135	0.140	0.149	0.390	0.391	0.389	0.390	0.152	0.163	0.187	0.425
SH	0.127	0.128	0.126	0.129	0.454	0.405	0.406	0.407	0.204	0.288	0.358	0.381
SDH	0.286	0.332	0.345	0.358	0.567	0.610	0.601	0.639	0.401	0.552	0.619	0.656
KSH	0.303	0.372	0.401	0.416	0.556	0.572	0.581	0.588	0.361	0.475	0.537	0.578
CNNH	0.439	0.517	0.512	0.523	0.611	0.618	0.625	0.618	0.518	0.550	0.627	0.554
DFH	0.752	0.773	0.791	0.802	0.775	0.816	0.825	0.844	0.631	0.698	0.726	0.747
IDHN	0.744	0.746	0.768	0.781	0.784	0.793	0.801	0.804	0.729	0.750	0.764	0.769
DBDH	0.767	0.790	0.779	0.782	0.802	0.832	0.836	0.841	0.618	0.728	0.745	0.761
DPN	0.755	0.759	0.789	0.769	0.762	0.793	0.809	0.827	0.684	0.740	0.756	0.756
DRITQH	0.789	0.801	0.822	0.827	0.789	0.831	0.838	0.846	0.714	0.763	0.776	0.781

方法	CIFAR-10				NUS-WIDE				ImageNet
方法	12 bit	24 bit	32 bit	48 bit	12 bit	24 bit	32 bit	48 bit	12 bit	24 bit	32 bit	48 bit
ITQ	0.158	0.169	0.172	0.175	0.452	0.468	0.472	0.477	0.244	0.352	0.428	0.482
LSH	0.127	0.135	0.140	0.149	0.390	0.391	0.389	0.390	0.152	0.163	0.187	0.425
SH	0.127	0.128	0.126	0.129	0.454	0.405	0.406	0.407	0.204	0.288	0.358	0.381
SDH	0.286	0.332	0.345	0.358	0.567	0.610	0.601	0.639	0.401	0.552	0.619	0.656
KSH	0.303	0.372	0.401	0.416	0.556	0.572	0.581	0.588	0.361	0.475	0.537	0.578
CNNH	0.439	0.517	0.512	0.523	0.611	0.618	0.625	0.618	0.518	0.550	0.627	0.554
DFH	0.752	0.773	0.791	0.802	0.775	0.816	0.825	0.844	0.631	0.698	0.726	0.747
IDHN	0.744	0.746	0.768	0.781	0.784	0.793	0.801	0.804	0.729	0.750	0.764	0.769
DBDH	0.767	0.790	0.779	0.782	0.802	0.832	0.836	0.841	0.618	0.728	0.745	0.761
DPN	0.755	0.759	0.789	0.769	0.762	0.793	0.809	0.827	0.684	0.740	0.756	0.756
DRITQH	0.789	0.801	0.822	0.827	0.789	0.831	0.838	0.846	0.714	0.763	0.776	0.781

[1]	Huaiqing HE, Jianqing YAN, Kanghua HUI. Lightweight face recognition method based on deep residual network [J]. Journal of Computer Applications, 2022, 42(7): 2030-2036.
[2]	Yaru HAN, Lianshan YAN, Tao YAO. Deep hashing retrieval algorithm based on meta-learning [J]. Journal of Computer Applications, 2022, 42(7): 2015-2021.
[3]	YANG Su, OUYANG Zhi, DU Nisuo. Unsupervised parallel hash image retrieval based on correlation distance [J]. Journal of Computer Applications, 2021, 41(7): 1902-1907.
[4]	REN Yiming, WANG Rangding, YAN Diqun, LIN Yuzhen. Speech steganalysis method based on deep residual network [J]. Journal of Computer Applications, 2021, 41(3): 774-779.
[5]	CHEN Lang, WANG Rangding, YAN Diqun, LIN Yuzhen. Audio steganography detection model combing residual network and extreme gradient boosting [J]. Journal of Computer Applications, 2021, 41(2): 449-455.
[6]	ZHONG Sha, HUANG Yuqing. Specified object tracking of unmanned aerial vehicle based on Siamese region proposal network [J]. Journal of Computer Applications, 2021, 41(2): 523-529.
[7]	LU Rongxiu, CHEN Mingming, YANG Hui, ZHU Jianyong. Element component content dynamic monitoring system based on time sequence characteristics of solution images [J]. Journal of Computer Applications, 2021, 41(10): 3075-3081.
[8]	YU Yingdong, YANG Yi, LIN Lan. Image style transfer network based on texture feature analysis [J]. Journal of Computer Applications, 2020, 40(3): 638-644.
[9]	Junhua GU, Feng WANG, Yongjun QI, Zheran SUN, Zepei TIAN, Yajuan ZHANG. Retrieval method of pulmonary nodule images based on multi-scale convolution feature fusion [J]. Journal of Computer Applications, 2020, 40(2): 561-565.
[10]	ZHANG Meiling, WU Junfeng, YU Hong, CUI Zhen, DONG Wanting. Fish image retrieval algorithm based on color four channels and spatial pyramid [J]. Journal of Computer Applications, 2019, 39(5): 1466-1472.
[11]	QIN Pingle, LI Qi, ZENG Jianchao, ZHANG Na, SONG Yulong. Image retrieval algorithm for pulmonary nodules based on multi-scale dense network [J]. Journal of Computer Applications, 2019, 39(2): 392-397.
[12]	CHEN Hongyu, DENG Dexiang, YAN Jia, FAN Ci'en. Image retrieval algorithm based on saliency semantic region weighting [J]. Journal of Computer Applications, 2019, 39(1): 136-142.
[13]	YANG Xiaoling, LI Zhiqing, LIU Yutong. Automatic image annotation based on multi-label discriminative dictionary learning [J]. Journal of Computer Applications, 2018, 38(5): 1294-1298.
[14]	WANG Peng, ZHANG Aofan, WANG Liqin, DONG Yongfeng. Image automatic annotation based on transfer learning and multi-label smoothing strategy [J]. Journal of Computer Applications, 2018, 38(11): 3199-3203.
[15]	ZHU Jie, ZHANG Junsan, WU Shufang, DONG Yukun, LYU Lin. Multi-center convolutional feature weighting based image retrieval [J]. Journal of Computer Applications, 2018, 38(10): 2778-2781.