Denoising autoencoder deep convolution process neural network and its application in time-varying signal classification

doi:10.11772/j.issn.1001-9081.2019081435

Abstract

Abstract: To solve the problem of nonlinear time-varying signal classification, a Denoising AutoEncoder Deep Convolution Process Neural Network (DAE-DCPNN) was proposed, which combines the information processing mechanism of Process Neural Network (PNN) with convolution operation. The model consists of a time-varying signal input layer, a Convolution Process Neuron (CPN) hidden layer, a deep Denoising AutoEncoder (DAE) network structure and a softmax classifier. The inputs of CPN were time-series signals, and the convolution kernel was taken as a five-order array with gradient property. And convolution operation was carried out based on sliding window to realize the spatio-temporal aggregation of time-series signals and the extraction of process features. After the CPN hidden layer, the DAE deep network and the softmax classifier were stacked to realize the high-level extraction and classification of features of time-varying signals. The properties of DAE-DCPNN were analyzed, and the comprehensive training algorithm of the initial value assignment training based on each information unit and the overall optimization of model parameters was given. Taking 7 kinds of cardiovascular disease classification diagnosis based on 12-lead ElectroCardioGram （ECG） signals as an example, the experimental results verify the effectiveness of the proposed model and algorithm.

Key words: time-varying signal classification, Convolution Process Neuron (CPN), Denoising AutoEncoder (DAE), Convolution Process Neural Network (CPNN), feature extraction, ElectroCardioGram (ECG） signal classification

摘要： 针对非线性时变信号分类问题，将过程神经网络（PNN）的信息处理机制与卷积运算相结合，提出了一种降噪自编码器深度卷积过程神经网络（DAE-DCPNN）。该模型由时变信号输入层、卷积过程神经元（CPN）隐层、深度降噪自动编码器（DAE）网络结构和softmax分类器构成。CPN的输入为时序信号，卷积核取为具有梯度性质的5阶数组，基于滑动窗口进行卷积运算，实现时序信号的时空聚合和过程特征提取。在CPN隐层之后，栈式叠加DAE深度网络和softmax分类器，实现对时变信号特征高层次的提取和分类。分析了DAE-DCPNN的性质，给出了按各信息单元分别进行赋初值训练、模型参数整体调优的综合训练算法。以基于12导联心电图（ECG）信号对7种心血管疾病分类诊断为例，实验结果验证了所提模型和算法的有效性。

关键词: 时变信号分类, 卷积过程神经元, 降噪自编码器, 卷积过程神经网络, 特征提取, 心电图信号分类

CLC Number:

TP183

ZHU Zhe, XU Shaohua. Denoising autoencoder deep convolution process neural network and its application in time-varying signal classification[J]. Journal of Computer Applications, 2020, 40(3): 698-703.

朱喆, 许少华. 降噪自编码器深度卷积过程神经网络及在时变信号分类中的应用[J]. 计算机应用, 2020, 40(3): 698-703.

References

[1] LÄNGKVIST M,KARLSSON L,LOUTFI A. A review of unsupervised feature learning and deep learning for time-series modeling[J]. Pattern Recognition Letters,2014,42:11-24.
[2] GAO J,MURPHEY Y L,ZHU H. Multivariate time series prediction of lane changing behavior using deep neural network[J]. Applied Intelligence,2018,48(10):3523-3537.
[3] FENG Z,LIANG M,CHU F. Recent advances in time-frequency analysis methods for machinery fault diagnosis:a review with application examples[J]. Mechanical Systems and Signal Processing, 2013,38(1):165-205.
[4] 刘志刚, 许少华, 李盼池. 基于ELM和连续过程神经网络的抽油机工况诊断[J]. 计算机工程与科学,2017,39(10):1934-1940. (LIU Z G,XU S H,LI P C. Working condition diagnosis of oil pump machine based on continuous process neural network and extreme learning machine[J]. Computer Engineering and Science, 2017,39(10):1934-1940.)
[5] HOCHREITER S,SCHMIDHUBER J. Long short-term memory[J]. Neural Computation,1997,9(8):1735-1780.
[6] HÜSKEN M,STAGGE P. Recurrent neural networks for time series classification[J]. Neurocomputing,2003,50:223-235.
[7] SUTSKEVER I,HINTON G E. Learning multilevel distributed representations for high-dimensional sequences[J]. Journal of Machine Learning Research,2007,2:548-555.
[8] BENGIO Y,LAMBLIN P,POPOVICI D,et al. Greedy layer-wise training of deep networks[C]//Proceedings of the 19th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2006:153-160.
[9] SUTSKEVER I,HINTON G E,TAYLOR G W. The recurrent temporal restricted Boltzmann machine[C]//Proceedings of the 21st International Conference on Neural Information Processing Systems. New York:Curran Associates Inc.,2008:1601-1608.
[10] 何新贵, 梁久祯. 过程神经元网络若干理论问题[J]. 中国工程科学,2000,2(12):40-44.(HE X G,LIANG J Z. Some theoretical issues on procedure neural networks[J]. Engineering Sciences,2000,2(12):40-44.)
[11] 何新贵, 梁久祯, 许少华. 过程神经网络的训练及其应用[J]. 中国工程科学,2001,3(4):31-35. (HE X G,LIANG J Z,XU S H. Learning and application of procedure neural networks[J]. Engineering Sciences,2001,3(4):31-35.)
[12] XU S LIU K,LI X. A fuzzy process neural network model and its application in process signal classification[J]. Neurocomputing, 2019,335:1-8.
[13] LIU K,XU S,FENG N. A radial basis probabilistic process neural network model and corresponding classification algorithm[J]. Applied Intelligence,2019,49(6):2256-2265.
[14] WANG C C,KANG Y,SHEN P C,et al. Applications of fault diagnosis in rotating machinery by using time series analysis with neural network[J]. Expert Systems with Applications,2010,37(2):1696-1702.
[15] 丁刚, 钟诗胜. 基于过程神经网络的热平衡温度预测研究[J]. 宇航学报,2006,27(3):489-492,545. (DING G,ZHONG S S. Thermal equilibrium temperature prediction based on process neural network[J]. Journal of Astronautics,2006,27(3):489-492, 545.)
[16] 王兵, 李盼池, 许少华. 一种基于过程神经元网络辨识的PID控制模型及方法[J]. 计算机应用,2010,30(1):233-235. (WANG B,LI P C,XU S H. Pid control model and method based on process neural network identification[J]. Journal of Computer Applications,2010,30(1):233-235.)
[17] SAINATH T N,MOHAMED A R,KINGSBURY B,et al. Deep convolutional neural networks for LVCSR[C]//Proceedings of the 2013 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE,2013:8614-8618.
[18] KIRANYAZ S,INCE T,GABBOUJ M. Real-time patient-specific ECG classification by 1-D convolutional neural networks[J]. IEEE Transactions on Biomedical Engineering,2016,63(3):664-675.
[19] YANG J,BAI Y,LIN F,et al. A novel electrocardiogram arrhythmia classification method based on stacked sparse auto-encoders and softmax regression[J]. International Journal of Machine Learning and Cybernetics,2018,9(10):1733-1740.
[20] YILDIRIM Ö. A novel wavelet sequence based on deep bidirectional LSTM network model for ECG signal classification[J]. Computers in Biology and Medicine,2018,96:189-202.
[21] XIONG Z H,NASH M P,CHENG E,et al. ECG signal classification for the detection of cardiac arrhythmias using a convolutional recurrent neural network[J]. Physiological Measurement,2018, 39(9):No. 094006.
[22] VINCENT P,LAROCHELLE H,LAJOIE I,et al. Stacked denoising autoencoders:learning useful representations in a deep network with a local denoising criterion[J]. Journal of Machine Learning Research,2010,11:3371-3408.
[23] VALENCIA D,OREJUELA D,SALAZAR J,et al. Comparison analysis between rigrsure,sqtwolog,heursure and minimaxi techniques using hard and soft thresholding methods[C]//Proceedings of the XXI Symposium on Signal Processing,Images and Artificial Vision. Piscataway:IEEE,2016:1-5.
[24] 许少华, 何新贵. 基于函数正交基展开的过程神经网络学习算法[J]. 计算机学报,2004,27(5):645-650. (XU S H,HE X G. Learning algorithms of process neural networks based on orthogonal function basis expansion[J]. Chinese Journal of Computers, 2004,27(5):645-650.)
[25] 许少华, 何新贵, 尚福华. 基于基函数展开的双隐层过程神经元网络及其应用[J]. 控制与决策,2004,19(1):36-39. (XU S H,HE X G,SHANG F H. Research and application of process neural network with two hidden-layer based on expansion of basis function[J]. Control and Decision,2004,19(1):36-39.)