Anomaly detection based on synthetic minority oversampling technique and deep belief network

doi:10.11772/j.issn.1001-9081.2018010178

Abstract

Abstract: To solve low detection rate problem of intrusion for a small number of samples in mass unbalanced datasets, an anomaly detection based on Synthetic Minority Oversampling Technique (SMOTE) and Deep Belief Network (DBN), called SMOTE-DBN method, was proposed. Firstly, SMOTE technology was used to increase the number of samples in minority categories. Secondly, on the preprocessed balanced data set, the dimensionality of the preprocessed high-dimensional data was reduced by unsupervised Restricted Boltzmann Machine (RBM). Thirdly, the model parameters were finely tuned by Back Propagation (BP) algorithm to obtain the lower low-dimensional representation of the preprocessed data. Finally, softmax classifier was used to classify the optimal low-dimensional data. The simulation experiment results on KDD1999 show that, compared with DBN method and Support Vector Machine (SVM) method, the detection rate of SMOTE-DBN method is increased by 3.31 and 7.34 percentage points respectively, and the false alarm rate is decreased by 1.11 and 2.67 percentage points respectively.

Key words: Synthetic Minority Oversampling Technique (SMOTE), Deep Belief Network (DBN), Restricted Boltzmann Machine (RBM), Logistic Regression (LR), intrusion detection

摘要： 针对现有海量非平衡数据集中少数类别样本入侵检测率低的问题，提出了一种基于合成少数类过采样技术（SMOTE）和深度信念网络（DBN）的异常检测（SMOTE-DBN）方法。首先，用SMOTE技术增加了少数类别样本的样本数；然后在预处理后的较平衡数据集上，用非监督的受限玻尔兹曼机（RBM）对预处理后的高维数据进行特征降维；其次，用反向传播（BP）算法微调模型参数，获得预处理后数据的较优低维表示；最后通过softmax分类器对较优低维数据进行分类。KDD1999数据集仿真实验表明，SMOTE优化处理能够提高模型对少数类别样本的检测率，在相同数据集上，SMOTE-DBN方法与DBN方法、支持向量机（SVM）方法相比，检测率分别提高了3.31个百分点和7.34个百分点，误报率分别降低了1.11个百分点和2.67个百分点。

关键词: 合成少数类过采样技术, 深度信念网络, 受限玻尔兹曼机, 逻辑回归, 入侵检测

CLC Number:

TP393.08

SHEN Xueli, QIN Shujuan. Anomaly detection based on synthetic minority oversampling technique and deep belief network[J]. Journal of Computer Applications, 2018, 38(7): 1941-1945.

沈学利, 覃淑娟. 基于SMOTE和深度信念网络的异常检测[J]. 计算机应用, 2018, 38(7): 1941-1945.

References

[1] TSAI C F, HSU Y F, LIN C Y, et al. Intrusion detection by machine learning:a review[J]. Expert Systems with Applications, 2009, 36(10):11994-12000.
[2] DAS S, NENE M J. A survey on types of machine learning techniques in intrusion prevention systems[C]//Proceedings of the 2017 International Conference on Wireless Communications, Signal Processing and Networking. Piscataway, NJ:IEEE, 2017:2296-2299.
[3] CHAND N, MISHRA P, KRISHNA C R, et al. A comparative analysis of SVM and its stacking with other classification algorithm for intrusion detection[C]//Proceedings of the 2016 International Conference on Advances in Computing, Communication & Automation. Piscataway, NJ:IEEE, 2016:1-6.
[4] ABUROMMAN A A, REAZ M B I. Ensemble of binary SVM classifiers based on PCA and LDA feature extraction for intrusion detection[C]//Proceedings of the 2017 Advanced Information Management, Communicates, Electronic and Automation Control Conference. Piscataway, NJ:IEEE, 2017:636-640.
[5] TENG S, WU N, ZHU H, et al. SVM-DT-based adaptive and collaborative intrusion detection[J]. IEEE/CAA Journal of Automatica Sinica, 2017, 5(1):108-118.
[6] DENG C, QIAO H. Network security intrusion detection system based on incremental improved convolutional neural network model[C]//Proceedings of the 2017 International Conference on Communication and Electronics Systems. Piscataway, NJ:IEEE, 2017:1-5.
[7] 杨雅辉,黄海珍,沈晴霓,等.基于增量式GHSOM神经网络模型的入侵检测研究[J].计算机学报,2014,37(5):1216-1224.(YANG Y H, HUANG H Z, SHEN Q N, et al. Research on intrusion detection based on incremental GHSOM[J]. Chinese Journal of Computers, 2014, 37(5):1216-1224.)
[8] 杨昆朋.基于深度学习的入侵检测[D].北京:北京交通大学,2015:31-47.(YANG K P. Intrusion detection based on deep Learning[D]. Beijing:Beijing Jiaotong University, 2015:31-47.)
[9] GAO N, GAO L, HE Y, et al. Intrusion detection model based on deep belief nets[J]. Journal of Southeast University (English Edition), 2015, 31(3):339-346.
[10] ALOM M Z, BONTUPALLI V R, TAHA T M. Intrusion detection using deep belief networks[C]//Proceedings of the 2016 Aerospace and Electronics Conference. Piscataway, NJ:IEEE, 2016:339-344.
[11] 霍玉丹,谷琼,蔡之华,等.基于遗传算法改进的少数类样本合成过采样技术的非平衡数据集分类算法[J].计算机应用,2015,35(1):121-124.(HUO Y D, GU Q, CAI Z H, et al. Classification method for imbalance dataset based on genetic algorithm improved synthetic minority over-sampling technique[J]. Journal of Computer Applications, 2015, 35(1):121-124.)
[12] DEMIDOVA L, KLYUEVA I. SVM classification:optimization with the SMOTE algorithm for the class imbalance problem[C]//Proceedings of the 2017 Embedded Computing. Piscataway, NJ:IEEE, 2017:1-4.
[13] ALRAWASHDEH K, PURDY C. Toward an online anomaly intrusion detection system based on deep learning[C]//Proceedings of the 2017 International Conference on Machine Learning and Applications. Piscataway, NJ:IEEE, 2017:195-200.
[14] POTLURI S, DIEDRICH C. Accelerated deep neural networks for enhanced intrusion detection system[C]//Proceedings of the 2016 International Conference on Emerging Technologies and Factory Automation. Piscataway, NJ:IEEE, 2016:1-8.
[15] 金建国.聚类方法综述[J].计算机科学,2014,41(b11):288-293.(JIN J G. Review of clustering method[J]. Computer Science,2014, 41(b11):288-293.)
[16] HINTON G E, OSINDERO S, TEH Y W. A fast learning algo-rithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
[17] HINTON G. Training products of experts by minimizing contrastive divergence[J]. Neural Computation, 2002, 14(8):1771-1800.
[18] FISCHER A, IGEL C. An introduction to restricted Boltzmann machines[C]//CIARP 2012:Proceedings of the 17th Iberoamerican Congress on Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. Berlin:Springer, 2012:14-36.
[19] ZHANG H, LI B. Application of an improved multi-layer BP neural network algorithm in intrusion detection[C]//Proceedings of the 2016 Sixth International Conference on Instrumentation & Measurement, Computer, Communication and Control. Piscataway, NJ:IEEE, 2016:619-622.
[20] 孔令智.基于网络异常的入侵检测算法研究[D].北京:北京交通大学,2017:38-39.(KONG L Z. Research on intrusion detection algorithm based on network anomaly[D]. Beijing:Beijing Jiaotong University, 2017:38-39.)
[21] 陈虹,万广雪,肖振久.基于优化数据处理的深度信念网络模型的入侵检测方法[J].计算机应用,2017,37(6):1636-1643.(CHEN H, WAN G X, XIAO Z J. Intrusion detection method of deep belief network model based on optimization of data processing[J]. Journal of Computer Applications, 2017, 37(6):1636-1643.)
[22] STOLFO S J, FAN W, LEE W, et al. Cost-based modeling for fraud and intrusion detection:results from the JAM project[C]//DISCEX'00:Proceedings of the 2000 DARPA Information Survivability Conference and Exposition. Piscataway, NJ:IEEE, 2000:130-144.
[23] YIN C L, ZHU Y F, FEI J L, et al. A deep learning approach for intrusion detection using recurrent neural networks[J]. IEEE Access, 2017, 5:21954-21961.
[24] GAO N, GAO L, GAO Q, et al. An intrusion detection model based on deep belief networks[C]//Proceedings of the 2nd International Conference on Advanced Cloud and Big Data. Washington, DC:IEEE Computer Society, 2014:247-252.
[25] 高妮,贺毅岳,高岭.海量数据环境下用于入侵检测的深度学习方法[J].计算机应用研究,2018,35(4):1197-1200.(GAO N, HE Y Y, GAO L. Deep learning method for intrusion detection in massive data[J]. Applications Research of Computers, 2018,35(4):1197-1200.)