Journal of Computer Applications ›› 2017, Vol. 37 ›› Issue (9): 2585-2589.DOI: 10.11772/j.issn.1001-9081.2017.09.2585

Previous Articles     Next Articles

Deep belief networks based on sparse denoising auto encoders

ZENG An1, ZHANG Yinan1, PAN Dan2, Xiao-wei SONG3   

  1. 1. Faculty of Computer Science, Guangdong University of Technology, Guangzhou Guangdong 510006, China;
    2. Modern Education Technical Center, Guangdong Construction Polytechnic, Guangzhou Guangdong 510440, China;
    3. ImageTech Lab, Simon Fraser University, Vancouver V6B 5K3, Canada
  • Received:2017-03-28 Revised:2017-06-07 Online:2017-09-10 Published:2017-09-13
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61300107), the Natural Science Foundation of Guangdong, China (S2012010010212), the Science and Technology Program of Guangzhou (201504301341059,201505031501397).


曾安1, 张艺楠1, 潘丹2, Xiao-Wei Song3   

  1. 1. 广东工业大学 计算机学院, 广州 510006;
    2. 广东建设职业技术学院 现代教育技术中心, 广州 510440;
    3. 西蒙弗雷泽大学 影像技术实验室, 加拿大 温哥华 V6B 5K3
  • 通讯作者: 潘丹,
  • 作者简介:曾安(1978-),女,湖南新化人,教授,博士,CCF会员,主要研究方向:人工智能、数据挖掘;张艺楠(1993-),女,广东兴宁人,硕士研究生,主要研究方向:数据挖掘;潘丹(1975-),男,广东兴宁人,高级工程师,博士,主要研究方向:人工智能、数据挖掘、大数据;Xiao-Wei Song (1962-),女,北京人,研究员,博士,主要研究方向:脑科学、神经影像。
  • 基金资助:
    国家自然科学基金资助项目(61300107);广东省自然科学基金资助项目(S2012010010212);广州市科技计划资助项目(201504301341059, 201505031501397)。

Abstract: The conventional Deep Belief Network (DBN) often utilizes the method of randomly initializing the weights and bias of Restricted Boltzmann Machine(RBM) to initialize the network. Although it could overcome the problems of local optimality and long training time to some extent, it is still difficult to further achieve higher accuracy and better learning efficiency owing to the huge difference between reconstruction and original input resulting from random initialization. In view of the above-mentioned problem, a kind of DBN model based on Sparse Denoising AutoEncoder (SDAE) was proposed. The advantage of the advocated model was the feature extraction by SDAE. Firstly, SDAE was trained, and then, the obtained weights and bias were utilized to initialize DBN. Finally, DBN was trained. Experiments were performed on card game data set of Poker hand and handwriting data sets of MNIST and USPS to verify the performance of the proposed model. In Poker hand data set, compared with the conventional DBN, the error rate of the proposed model is lowered by 46.4%, the accuracy rate and the recall rate are improved by 15.56% and 14.12% respectively. The results exhibit that the proposed method is superior to other existing methods in recognition performance.

Key words: Deep Belief Network (DBN), Restricted Boltzmann Machine (RBM), Sparse Denoising AutoEncoder (SDAE), deep learning

摘要: 传统的深度置信网络(DBN)采用随机初始化受限玻尔兹曼机(RBM)的权值和偏置的方法初始化网络。虽然这在一定程度上克服了由BP算法带来的易陷入局部最优和训练时间长的问题,但随机初始化仍然会导致网络重构和原始输入的较大差别,这使得网络无论在准确率还是学习效率上都无法得到进一步提升。针对以上问题,提出一种基于稀疏降噪自编码器(SDAE)的深度网络模型,其核心是稀疏降噪自编码器对数据的特征提取。首先,训练稀疏降噪自编码;然后,用训练后得到的权值和偏置来初始化深度置信网络;最后,训练深度置信网络。在Poker Hand 纸牌游戏数据集和MNIST、USPS手写数据集上测试模型性能,在Poker Hand数据集下,方法的误差率比传统的深度置信网络降低46.4%,准确率和召回率依次提升15.56%和14.12%。实验结果表明,所提方法能有效地改善模型性能。

关键词: 深度置信网络, 受限玻尔兹曼机, 稀疏降噪自编码器, 深度学习

CLC Number: