Journal of Computer Applications ›› 2017, Vol. 37 ›› Issue (11): 3249-3255.DOI: 10.11772/j.issn.1001-9081.2017.11.3249

Previous Articles     Next Articles

Clothing retrieval based on landmarks

CHEN Aiai, LI Lai, LIU Guangcan, LIU Qingshan   

  1. Jiangsu Key Laboratory of Big Data Analysis Technology(Nanjing University of Information Science & Technology), Nanjing Jiangsu 210044, China
  • Received:2017-05-11 Revised:2017-07-12 Online:2017-11-10 Published:2017-11-11
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61622305, 61502238, 61532009), the Natural Science Foundation of Jiangsu Province (BK20160040).

基于关键点的服装检索

陈嫒嫒, 李来, 刘光灿, 刘青山   

  1. 江苏省大数据分析技术重点实验室(南京信息工程大学), 南京 210044
  • 通讯作者: 陈嫒嫒
  • 作者简介:陈嫒嫒(1992-),女,江苏扬州人,硕士研究生,主要研究方向:模式识别、数据挖掘、深度学习;李来(1990-),男,江苏徐州人,硕士研究生,主要研究方向:图像检索、深度学习;刘光灿(1982-),男,湖南邵阳人,教授,博士,主要研究方向:模式识别、数据挖掘;刘青山(1975-),男,安徽合肥人,教授,博士生导师,博士,主要研究方向:图像分析、视频分析、机器学习。
  • 基金资助:
    国家自然科学基金资助项目(61622305,61502238,61532009);江苏省自然科学基金资助项目(BK20160040)。

Abstract: At present, the same or similar style clothing retrieval is mainly text-based or content-based. The text-based algorithms often require massive labled samples, and the shortages of exist label missing and annotation difference caused by artificial subjectivity. The content-based algorithms usually extract image features, such as color, shape, texture, and then measured the similarity, but it is difficult to deal with background color interference, and clothing deformation due to different angles, attitude, etc. Aiming at these problems, clothing retrieval based on landmarks was proposed. The proposed method used cascaded deep convolutional neural network to locate the key points and combined the low-level visual information of the key point region as well as the high-level semantic information of the whole image. Compared with traditional methods, the proposed method can effectively deal with the distortion of clothing and complex background interference due to angle of view and attitude. Meanwhile, it does not need huge labeled samples, and is robust to background and deformation. Experiments on two large scale datasets Fashion Landmark and BDAT-Clothes show that the proposed algorithm can effectively improve the precision and recall.

Key words: landmark, deep convolution neural network, cascade, clothing retrieval

摘要: 目前,同款或近似款式服装检索主要分为基于文本和基于内容两类。基于文本算法往往需要海量标注样本,且存在人工主观性带来的标注缺失和标注差异等问题;基于内容算法一般对服装图像的颜色、形状、纹理提取特征,进行相似性度量,但难以应对背景颜色干扰,以及视角、姿态引起的服装形变等问题。针对上述问题,提出一种基于关键点的服装检索方法。利用级联深度卷积神经网络为基础,定位服装关键点,融合关键点区域低层视觉信息以及整幅图像的高层语义信息。对比传统检索方法,所提算法能有效处理视角、姿态引起的服装形变和复杂背景的干扰;同时不需大量样本标定,且对背景、形变鲁棒。在Fashion Landmark数据集和BDAT-Clothes数据集上与常用算法进行对比实验。实验结果表明所提算法能有效提升检索的查准率和查全率。

关键词: 关键点, 深度卷积神经网络, 级联, 服装检索

CLC Number: