基于多级语义的判别式跨模态哈希检索算法

doi:10.11772/j.issn.1001-9081.2020101607

计算机应用 ›› 2021, Vol. 41 ›› Issue (8): 2187-2192.DOI: 10.11772/j.issn.1001-9081.2020101607

所属专题：人工智能

基于多级语义的判别式跨模态哈希检索算法

刘芳名^1,2, 张鸿^1,2

1. 武汉科技大学计算机科学与技术学院, 武汉 430065;
2. 智能信息处理与实时工业系统湖北省重点实验室(武汉科技大学), 武汉 430065

收稿日期:2020-10-16 修回日期:2021-01-13 出版日期:2021-08-10 发布日期:2021-08-06
通讯作者: 刘芳名
作者简介:刘芳名(1996-),女,湖北黄冈人,硕士研究生,主要研究方向:跨模态检索、机器学习;张鸿(1979-),女,湖北襄阳人,教授,博士,主要研究方向:跨媒体检索、机器学习、数据挖掘。
基金资助:
国家自然科学基金资助项目（61373109）。

Cross-modal retrieval algorithm based on multi-level semantic discriminative guided hashing

LIU Fangming^1,2, ZHANG Hong^1,2

1. School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan Hubei 430065, China;
2. Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System(Wuhan University of Science and Technology), Wuhan Hubei 430065, China

Received:2020-10-16 Revised:2021-01-13 Online:2021-08-10 Published:2021-08-06
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61373109).

摘要/Abstract

摘要： 针对大多数跨模态哈希方法采用二进制矩阵表示相关程度，因此无法捕获多标签数据之间更深层的语义信息，以及它们忽略了保持语义结构和数据特征的判别性等问题，提出了一种基于多级语义的判别式跨模态哈希检索算法——ML-SDH。所提算法使用多级语义相似度矩阵发现跨模态数据中的深层关联信息，同时利用平等指导跨模态哈希表示在语义结构和判别分类中的关联关系，不仅实现了对蕴含高级语义信息的多标签数据进行编码的目的，而且构建的保留多级语义的结构能够确保最终学习的哈希码在保持语义相似度的同时又具有判别性。在NUS-WIDE数据集上，哈希码长度为32 bit时，所提算法在两个检索任务中的平均准确率（mAP）比深度跨模态哈希（DCMH）、成对关联哈希（PRDH）、平等指导判别式哈希（EGDH）算法分别高出了19.48，14.50，1.95个百分点和16.32，11.82，2.08个百分点。

关键词: 多级语义, 语义结构, 判别性哈希, 语义指导, 跨模态检索

Abstract: Most cross-modal hashing methods use binary matrix to represent the degree of correlation, which results in high-level semantic information cannot be captured in multi-label data, and those methods ignore maintaining the semantic structure and the discrimination of the data features. Therefore, a cross-modal retrieval algorithm named ML-SDH (Multi-Level Semantics Discriminative guided Hashing) was proposed. In the algorithm, multi-level semantic similarity matrix was used to discover the deeply correlated information in the cross-modal data, and equally guided cross-modal hashing was used to express the correlations in the semantic structure and discriminative classification. As the result, not only the purpose of encoding multi-label data of high-level semantic information was achieved, but also the distinguishability and semantic similarity of the final learned hash codes were ensured by the constructed multi-level semantic structure. On NUS-WIDE dataset, with the hash code length of 32 bit, the mean Average Precision (mAP) of the proposed algorithm in two retrieval tasks is 19.48,14.50,1.95 percentage points and 16.32,11.82,2.08 percentage points higher than those of DCMH (Deep Cross-Modal Hashing), PRDH (Pairwise Relationship guided Deep Hashing) and EGDH (Equally-Guided Discriminative Hashing) algorithms respectively.

Key words: multi-level semantic, semantic structure, discriminative hashing, semantic guidance, cross-modal retrieval

中图分类号:

TP391.3

刘芳名, 张鸿. 基于多级语义的判别式跨模态哈希检索算法[J]. 计算机应用, 2021, 41(8): 2187-2192.

LIU Fangming, ZHANG Hong. Cross-modal retrieval algorithm based on multi-level semantic discriminative guided hashing[J]. Journal of Computer Applications, 2021, 41(8): 2187-2192.

参考文献

[1] 张鸿, 吴飞, 张晓龙. 基于关系矩阵融合的多媒体数据聚类[J]. 计算机学报,2011,34(9):1705-1711.(ZHANG H,WU F, ZHANG X L. Multimedia data clustering based on correlation matrix fusion[J]. Chinese Journal of Computers,2011,34(9):1705-1711.)
[2] 黄育, 张鸿. 基于潜语义主题加强的跨媒体检索算法[J]. 计算机应用,2017,37(4):1061-1064,1110.(HUANG Y,ZHANG H. Cross-media retrieval based on latent semantic topic reinforce[J]. Journal of Computer Applications,2017,37(4):1061-1064, 1110.)
[3] GONG X L,HUANG L P,WANG F W. Deep semantic correlation learning based hashing for multimedia cross-modal retrieval[C]//Proceedings of the 2018 IEEE International Conference on Data Mining. Piscataway:IEEE,2018:117-126.
[4] PENG Y X,HUAGN X,ZHAO Y Z. An overview of cross-media retrieval:concepts,methodologies,benchmarks,and challenges[J]. IEEE Transactions on Circuits and Systems for Video Technology,2018,28(9):2372-2385.
[5] XU X,SHEN F M,YANG Y,et al. Learning discriminative binary codes for large-scale cross-modal retrieval[J]. IEEE Transactions on Image Processing,2017,26(5):2494-2507.
[6] WANG D, GAO X B, WANG X M, et al. Semantic topic multimodal hashing for cross-media retrieval[C]//Proceedings of the 24th International Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2015:3890-3896.
[7] DING G G,GUO Y C,ZHOU J L. Collective matrix factorization hashing for multimodal data[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2014:2083-2090.
[8] JIANG Q Y,LI W J. Deep cross-modal hashing[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:3270-3278.
[9] CRESWELL A,WHITE T,DUMOULIN V,et al. Generative adversarial networks:an overview[J]. IEEE Signal Processing Magazine,2018,35(1):53-65.
[10] ZHANG J,PENG Y X,YUAN M K. SCH-GAN:semi-supervised cross-modal hashing by generative adversarial network[J]. IEEE Transactions on Cybernetics,2020,50(2):489-502.
[11] LI C,DENG C,LI N,et al. Self-supervised adversarial hashing networks for cross-modal retrieval[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:4242-4251.
[12] SHI Y F, YOU X G, ZHENG F, et al. Equally-guided discriminative hashing for cross-modal retrieval[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2019:4767-4773.
[13] BRONSTEIN M M,BRONSTEIN A M,MICHEL F,et al. Data fusion through cross-modality metric learning using similaritysensitive hashing[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2010:3594-3601.
[14] ZHANG D Q,LI W J. Large-scale supervised multimodal hashing with semantic correlation maximization[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2014:2177-2183.
[15] LIN Z J,DING G G,HU M Q,et al. Semantics-preserving hashing for cross-view retrieval[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:3864-3872.
[16] YANG E K,DENG C,LIU W,et al. Pairwise relationship guided deep hashing for cross-modal retrieval[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2017:1618-1625.
[17] HUISKES M J,LEW M S. The MIR Flickr retrieval evaluation[C]//Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. New York:ACM, 2008:39-43.
[18] CHUA T S,TANG J H,HONG R C,et al. NUS-WIDE:a realworld web image database from National University of Singapore[C]//Proceedings of the 2009 ACM International Conference on Image and Video Retrieval. New York:ACM,2009:No. 48.
[19] CHATFIELD K,SIMONYAN K,VEDALDI A,et al. Return of the devil in the details:delving deep into convolutional nets[C]//Proceedings of the 2014 British Machine Vision Conference. Durham:BMVA Press,2014:No. 054.
[20] DENG J,DONNG W,SOCHER R,et al. ImageNet:a largescale hierarchical image database[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2009:248-255.

基于多级语义的判别式跨模态哈希检索算法

Cross-modal retrieval algorithm based on multi-level semantic discriminative guided hashing

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics