基于自动编码器组合的深度学习优化方法

doi:10.11772/j.issn.1001-9081.2016.03.697

计算机应用 ›› 2016, Vol. 36 ›› Issue (3): 697-702.DOI: 10.11772/j.issn.1001-9081.2016.03.697

基于自动编码器组合的深度学习优化方法

邓俊锋^1,2, 张晓龙^1,2

1. 武汉科技大学计算机科学与技术学院, 武汉 430065;
2. 智能信息处理与实时工业系统湖北省重点实验室, 武汉 430065

收稿日期:2015-08-13 修回日期:2015-10-14 出版日期:2016-03-10 发布日期:2016-03-17
通讯作者: 张晓龙
作者简介:邓俊锋(1989-),男,湖北黄冈人,硕士研究生,主要研究方向:机器学习、数据挖掘;张晓龙(1963-),男,江西永新人,教授,博士生导师,主要研究方向:数据挖掘、机器学习、生物信息处理。
基金资助:
国家自然科学基金资助项目(61273225);国家科技支撑计划项目(2012BAC22B01)。

Deep learning algorithm optimization based on combination of auto-encoders

DENG Junfeng^1,2, ZHANG Xiaolong^1,2

1. School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan Hubei 430065, China;
2. Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan Hubei 430065, China

Received:2015-08-13 Revised:2015-10-14 Online:2016-03-10 Published:2016-03-17
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61273225) and the National Key Technology R&D Program (2012BAC22B01).

摘要/Abstract

摘要： 为了提高自动编码器算法的学习精度,更进一步降低分类任务的分类错误率,提出一种组合稀疏自动编码器(SAE)和边缘降噪自动编码器(mDAE)从而形成稀疏边缘降噪自动编码器(SmDAE)的方法,将稀疏自动编码器和边缘降噪自动编码器的限制条件加载到一个自动编码器(AE)之上,使得这个自动编码器同时具有稀疏自动编码器的稀疏性约束条件和边缘降噪自动编码器的边缘降噪约束条件,提高自动编码器算法的学习能力。实验表明,稀疏边缘降噪自动编码器在多个分类任务上的学习精度都高于稀疏自动编码器和边缘降噪自动编码器的分类效果;与卷积神经网络(CNN)的对比实验也表明融入了边缘降噪限制条件,而且更加鲁棒的SmDAE模型的分类精度比CNN还要好。

关键词: 深度学习, 自动编码器, 稀疏自动编码器, 降噪自动编码器, 卷积神经网络

Abstract: In order to improve the learning accuracy of Auto-Encoder (AE) algorithm and further reduce the classification error rate, Sparse marginalized Denoising Auto-Encoder (SmDAE) was proposed combined with Sparse Auto-Encoder (SAE) and marginalized Denoising Auto-Encoder (mDAE). SmDAE is an auto-encoder which was added the constraint conditions of SAE and mDAE and has the characteristics of SAE and mDAE, so as to enhance the ability of deep learning. Experimental results show that SmDAE outperforms both SAE and mDAE in the given classification tasks; comparative experiments with Convolutional Neural Network (CNN) show that SmDAE with marginalized denoising and a more robust model outperforms convolutional neural network.

Key words: deep learning, Auto-Encoder (AE), Sparse Auto-Encoder (SAE), Denoising Auto-Encoder (DAE), Convolutional Neural Network (CNN)

中图分类号:

TP392

邓俊锋, 张晓龙. 基于自动编码器组合的深度学习优化方法[J]. 计算机应用, 2016, 36(3): 697-702.

DENG Junfeng, ZHANG Xiaolong. Deep learning algorithm optimization based on combination of auto-encoders[J]. Journal of Computer Applications, 2016, 36(3): 697-702.

参考文献

[1] RUMELHART D E, HINTON G E, WILLIAMS R J. Learning representations by back-propagating errors [J]. Nature, 1986,323(9):533-536.
[2] BALDI P, HORNIK K. Neural networks and principal component analysis: learning from examples without local minima [J]. Neural Networks, 1989,2(1):53-58.
[3] BENGIO Y, LAMBLIN P, POPOVICI D, et al. Personal communications with Will Zou. learning optimization Greedy layer-wise training of deep networks [C]//Proceedings of the 20th Annual Conference on Neural Information Processing System. Cambridge, MA: MIT Press, 2006:153-160.
[4] BENGIO Y. Learning deep architectures for AI [J]. Foundations & Trends® in Machine Learning, 2009, 2(1): 1-127.
[5] VINCENT P, LAROCHELLE H, BENGIO Y, et al. Extracting and composing robust features with denoising autoencoders [C]//Proceedings of the 2008 25th International Conference on Machine Learning. New York: ACM, 2008: 1096-1103.
[6] CHEN M, WEINBERGER K, SHA F, et al. Marginalized denoising auto-encoders for nonlinear representations [C]//Proceedings of the 2014 31th International Conference on Machine Learning. New York: ACM, 2014: 1476-1484.
[7] VINCENT P, LAROCHELLE H, LAJOIE I, et al. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion [J]. Journal of Machine Learning Research, 2010,11(6): 3371-3408.
[8] LeCUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[9] FARABET C, COUPRIE C, NAJMAN L, et al. Learning hierarchical features for scene labeling [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8): 1915-1929.
[10] MOHAMED A, DAHL G E, HINTON G. Acoustic modeling using deep belief networks [J]. IEEE Transactions on Audio, Speech, and Language Processing, 2012, 20(1): 14-22.
[11] LeCUN Y, BOTTOU L, ORR G B, et al. Efficient BackProp [M]//ORR G B, MVLLER K-R. Neural Networks: Tricks of the Trade, LNCS 1524. Berlin: Springer, 1998:9-50.
[12] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets [J]. Neural Computation, 2006,18(7): 1527-1554.
[13] JAITLY N, HINTON G E. Using an autoencoder with deformable templates to discover features for automated speech recognition [EB/OL]. [2015-04-07]. http://www.cs.toronto.edu/~ndjaitly/jaitly-interspeech13.pdf.
[14] TSURUOKA Y, TSUJII J, ANANIADOU S. Stochastic gradient descent training for L1-regularized log-linear models with cumulative penalty [EB/OL]. [2015-04-07]. http://aye.comp.nus.edu.sg/~antho/P/P09/P09-1054.pdf.
[15] LE Q V, NGIAM J, COATES A, et al. On optimization methods for deep learning [EB/OL]. [2015-04-07]. http://ai.stanford.edu/~ang/papers/icml11-OptimizationForDeepLearning.pdf.

基于自动编码器组合的深度学习优化方法

Deep learning algorithm optimization based on combination of auto-encoders

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.
[2]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[3]	张永斌, 常文欣, 孙连山, 张航. 基于字典的域名生成算法生成域名的检测方法[J]. 计算机应用, 2021, 41(9): 2609-2614.
[4]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[5]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[6]	牟长宁, 王海鹏, 周丕宇, 侯鑫行. 基于图卷积神经网络的串联质谱从头测序[J]. 计算机应用, 2021, 41(9): 2773-2779.
[7]	王贺兵, 张春梅. 基于非对称卷积-压缩激发-次代残差网络的人脸关键点检测[J]. 计算机应用, 2021, 41(9): 2741-2747.
[8]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[9]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[10]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[11]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[12]	何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431.
[13]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[14]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[15]	黄程程, 董霄霄, 李钊. 基于二维Winograd算法的深流水线5×5卷积方法[J]. 计算机应用, 2021, 41(8): 2258-2264.