Journal of Computer Applications ›› 2018, Vol. 38 ›› Issue (7): 1872-1876.DOI: 10.11772/j.issn.1001-9081.2018010001

Previous Articles     Next Articles

Mechanism of sparse restricted Boltzmann machine based on competitive learning

ZHOU Lijun, LIU Kai, LYU Haiyan   

  1. Aeronautical Basic College, Naval Aeronautical University, Yantai Shandong 264001, China
  • Received:2018-01-03 Revised:2018-03-02 Online:2018-07-10 Published:2018-07-12
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61531020, 61032001).


周立军, 刘凯, 吕海燕   

  1. 海军航空大学 航空基础学院, 山东 烟台 264001
  • 通讯作者: 周立军
  • 作者简介:周立军(1982-),男,湖南岳阳人,副教授,硕士,主要研究方向:智能信息处理、装备信息化、数据挖掘;刘凯(1986-),男,山东安丘人,讲师,博士,主要研究方向:智能信息处理、机器学习、数据挖掘;吕海燕(1983-),女,山东淄博人,讲师,硕士,主要研究方向:计算机仿真。
  • 基金资助:

Abstract: To resolve the problems of feature homogeneity in unsupervised training of Restricted Boltzmann Machine (RBM) and non-adaptiveness of Sparse Restricted Boltzmann Machine (SRBM), a new sparse mechanism method of RBM based on competitive learning was designed. Firstly, a distance measurement was designed based on the cosine value between the neuron weight vector and the input vector to evaluate the similarity. Secondly, the optimal matching implicit unit based on distance measurement was selected for different samples during training. Thirdly, the sparse penalty for other hidden units was calculated according to the activation state of the optimal matching hidden unit. Finally, the parameters were updated and the competitive sparseness was applied to the construction of Deep Boltzmann Machine (DBM) based on the deep model training process. The handwritten number recognition results show that, compared with the mechanism using the sum of squared errors as the regularization factor, the classification accuracy of DBM based on new sparse mechanism is improved by 0.74%, and the average sparsity measurement is increased by 5.6%, without the need to set sparse parameters. Therefore, the proposed sparse mechanism can improve the training efficiency of unsupervised training model, such as RBM, and can be applied into the construction of deep models.

Key words: Restricted Boltzmann Machine (RBM), Sparse RBM (SRBM), competitive learning, sparse representation, neuron

摘要: 针对受限玻尔兹曼机(RBM)无监督训练存在特征同质化问题以及现有稀疏受限玻尔兹曼机(SRBM)难以自适应稀疏的缺陷,提出了一种基于竞争学习的RBM稀疏机制方法。首先设计基于神经元权值向量与输入向量间夹角余弦值的距离度量,评估两者相似度;然后在训练过程中对不同样本选择出基于距离度量的最优匹配隐单元;其次根据最优匹配隐单元激活状态计算对其他隐单元的稀疏惩罚度;最后执行参数更新并依据深度模型训练过程,将竞争稀疏应用于深度玻尔兹曼机(DBM)的构建中。通过手写数字识别实验证明,与误差平方和正则化因子相比,基于该稀疏机制的DBM分类准确率提高了0.74%,平均稀疏度提高了5.6%,且无需设置稀疏参数,因此,该稀疏机制可提高RBM等无监督训练模型的训练效率,并应用于深度模型的构建中。

关键词: 受限玻尔兹曼机, 稀疏受限玻尔兹曼机, 竞争学习, 稀疏表示, 神经元

CLC Number: