Real-time remaining life prediction method of Web software system based on self-attention-long short-term memory network

doi:10.11772/j.issn.1001-9081.2020091486

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (8): 2346-2351.DOI: 10.11772/j.issn.1001-9081.2020091486

Special Issue: 前沿与综合应用

• Frontier and comprehensive applications • Previous Articles Next Articles

Real-time remaining life prediction method of Web software system based on self-attention-long short-term memory network

DANG Weichao, LI Tao, BAI Shangwang, GAO Gaimei, LIU Chunxia

College of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan Shanxi 030024, China

Received:2020-09-24 Revised:2020-11-18 Online:2021-08-10 Published:2021-04-07
Supported by:
This work is partially supported by the Research Program of Application Foundation of Shanxi Province (201901D111266).

基于自注意力长短期记忆网络的Web软件系统实时剩余寿命预测方法

党伟超, 李涛, 白尚旺, 高改梅, 刘春霞

太原科技大学计算机科学与技术学院, 太原 030024

通讯作者: 李涛
作者简介:党伟超(1974-),男,山西运城人,副教授,博士,CCF会员,主要研究方向:智能计算、软件可靠性;李涛(1996-),男,山西长治人,硕士研究生,主要研究方向:软件可靠性;白尚旺(1964-),男,山西吕梁人,教授,硕士,主要研究方向:智能软件系统;高改梅(1978-),女,山西吕梁人,讲师,博士,CCF会员,主要研究方向:网络安全、密码学;刘春霞(1977-),女,山西大同人,副教授,硕士,CCF会员,主要研究方向:软件工程、数据库。
基金资助:
山西省应用基础研究计划项目（201901D111266）。

Abstract

Abstract: In order to predict the Remaining Useful Life (RUL) of Web software system in real time and accurately, taking into consideration the time sequence characteristics of the Web system health status performance indicators and the interdependence between the indicators, a real-time remaining life prediction method of Web software system based on Self-Attention-Long Short-Term Memory (Self-Attention-LSTM) network was proposed. Firstly, an accelerated life test platform was built to collect the performance indicators data reflecting the aging trend of the Web software system. Then, according to the time sequence characteristics of the performance indicators data, a Long Short-Term Memory (LSTM) recurrent neural network was constructed to extract the hidden layer characteristics of the performance indicators, and the self-attention mechanism was used to model the dependency relationship between the characteristics. Finally, the real-time RUL prediction value of the Web system was obtained. On three test sets, the proposed model was compared with the Back Propagation (BP) network and the conventional Recurrent Neural Network (RNN). Experimental results show that the Mean Absolute Error (MAE) of the model is 16.92% lower than that of LSTM on average, and the relative accuracy (Accuracy) of the model is 5.53% higher than that of LSTM on average, which verify the effectiveness of the RUL model of Self-Attention-LSTM network. It can be seen that the proposed method can provide technical support for optimizing the software rejuvenation decision of the Web system.

Key words: Web software system, Remaining Useful Life (RUL), Long Short-Term Memory (LSTM) network, self-attention mechanism, rejuvenation decision

摘要： 为了能够实时准确对Web软件系统的剩余使用寿命（RUL）进行预测，考虑Web系统健康状态性能指标的时序特性和指标间的相互依赖特性，提出了一种基于自注意力长短期记忆（Self-Attention-LSTM）网络的Web软件系统实时剩余寿命预测方法。首先，搭建加速寿命测试实验平台来收集反映Web软件系统老化趋势的性能指标数据；然后，根据该性能指标数据的时序特性来构建长短期记忆（LSTM）循环神经网络以提取性能指标的隐含层特征，并使用自注意力机制建模特征间的依赖关系；最后，得到系统RUL的实时预测值。在三组测试集上，把所提模型与反向传播（BP）网络和常规的循环神经网络（RNN）做了对比。实验结果表明，所提模型的平均绝对误差（MAE）比长短期记忆（LSTM）网络平均低16.92%，相对准确率（Accuracy）比LSTM网络平均高5.53%，验证了Self-Attention-LSTM网络剩余寿命预测模型的有效性。可见所提方法能为优化系统抗衰决策提供技术支撑。

关键词: Web软件系统, 剩余使用寿命, 长短期记忆网络, 自注意力机制, 抗衰决策

CLC Number:

TP301

DANG Weichao, LI Tao, BAI Shangwang, GAO Gaimei, LIU Chunxia. Real-time remaining life prediction method of Web software system based on self-attention-long short-term memory network[J]. Journal of Computer Applications, 2021, 41(8): 2346-2351.

党伟超, 李涛, 白尚旺, 高改梅, 刘春霞. 基于自注意力长短期记忆网络的Web软件系统实时剩余寿命预测方法[J]. 计算机应用, 2021, 41(8): 2346-2351.

References

[1] WU X X, ZHENG W, PU M C, et al. Invalid bug reports complicate the software aging situation[J]. Software Quality Control,2020,28(1):195-200.
[2] HUANG Y, KINTALA C, KOLETTIS N, et al. Software rejuvenation:analysis,module and applications[C]//Proceedings of the 25th International Symposium on Fault-Tolerant Computing. Piscataway:IEEE,1995:381-390.
[3] ZHENG J J, OKAMURA H, DOHI T. A transient interval reliability analysis for software rejuvenation models with phase expansion[J]. Software Quality Control,2020,28(1):173-194.
[4] YAN Y Q,GUO P. A practice guide of software aging prediction in a web server based on machine learning[J]. China Communications,2016,13(6):225-235.
[5] 梁佩. 基于WEB技术的软件老化趋势预测研究[D]. 哈尔滨:哈尔滨工程大学, 2013:15-25.(LIANG P. Research on the tendency of software aging for the WEB technology developed software[D]. Harbin:Harbin Engineering University,2013:15-25.)
[6] 谭宇宁, 党伟超, 潘理虎, 等. 基于SATLSTM的Web系统老化趋势预测[J]. 计算机应用与软件,2020,37(4):17-24.(TAN Y N,DANG W C,PAN L H,et al. Prediction of Web system aging trend based on SATLSTM[J]. Computer Applications and Software,2020,37(4):17-24.)
[7] 于震梁, 孙志礼, 曹汝男, 等. 基于支持向量机和卡尔曼滤波的机械零件剩余寿命预测模型研究[J]. 兵工学报,2018,39(5):991-997.(YU Z L,SUN Z L,CAO R N,et al. Research on remaining useful life predictive model of machine parts based on SVM and Kalman filter[J]. Acta Armamentarii,2018,39(5):991-997.)
[8] 秦松, 贾银山. 支持向量机算法的并行实现技术研究[J]. 微处理机,2010,31(5):42-45.(QIN S,JIA Y S. The research of parallel algorithm for Support Vector Machines[J]. Microprocessor,2010,31(5):42-45.)
[9] AHMAD S,LAVIN A,PURDY S,et al. Unsupervised real-time anomaly detection for streaming data[J]. Neurocomputing,2017, 262:134-147.
[10] 闫楚良, 郝云霄, 刘克格. 基于遗传算法优化的BP神经网络的材料疲劳寿命预测[J]. 吉林大学学报(工学版),2014,44(6):1710-1715. (YAN C L,HAO Y X,LIU K G. Fatigue life prediction of materials based on BP neural networks optimized by genetic algorithm[J]. Journal of Jilin University(Engineering and Technology Edition),2014,44(6):1710-1715.)
[11] 王佳炜, 王召斌, 黄周霖. 果蝇算法优化的BP神经网络在电磁继电器贮存寿命预测中的应用[J]. 电器与能效管理技术, 2019(2):19-24. (WANG J W,WANG Z B,HUANG Z L. Application of BP neural network optimized by drosophila algorithm in storage life prediction of electromagnetic relay[J]. Electrical and Energy Management Technology, 2019(2):19-24.)
[12] ZHANG A S,WANG H L,LI S B,et al. Transfer learning with deep recurrent neural networks for remaining useful life estimation[J]. Applied Sciences,2018,8(12):No. 2416.
[13] 胡天中, 余建波. 基于多尺度分解和深度学习的锂电池寿命预测[J]. 浙江大学学报(工学版),2019,53(10):1852-1864. (HU T Z,YU J B. Life prediction of lithium-ion batteries based on multiscale decomposition and deep learning[J]. Journal of Zhejiang University(Engineering Edition),2019,53(10):1852-1864.)
[14] WU Y T,YUAN M,DONG S P,et al. Remaining useful life estimation of engineered systems using vanilla LSTM neural networks[J]. Neurocomputing,2018,275:167-179.
[15] LAI G K,CHANG W C,YANG Y M,et al. et al. Modeling longand short-term temporal patterns with deep neural networks[C]//Proceedings of the 41th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM,2018:95-104.
[16] CHENG J P,DONG L,LAPATA M. Long short-term memorynetworks for machine reading[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2016:551-561.
[17] LI W J,QI F,TANG M,et al. Bidirectional LSTM with selfattention mechanism and multi-channel features for sentiment classification[J]. Neurocomputing,2020,387:63-77.
[18] 周治平, 张威. 结合视觉属性注意力和残差连接的图像描述生成模型[J]. 计算机辅助设计与图形学学报,2018,30(8):156-162.(ZHOU Z P,ZHANG W. An image caption generation model based on visual concept attention and residual connection[J]. Journal of Computer-Aided Design and Computer Graphics,2018, 30(8):156-162.)
[19] LUONG T,PHAM H,MANNING C D. Effective approaches to attention-based neural machine translation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2015:1412-1421.
[20] YIN Y C,COOLEN F P A,COOLEN-MATURI T. An imprecise statistical method for accelerated life testing using the powerWeibull model[J]. Reliability Engineering and System Safety, 2017,167:158-167.
[21] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research,2014,15:1929-1958.

Real-time remaining life prediction method of Web software system based on self-attention-long short-term memory network

基于自注意力长短期记忆网络的Web软件系统实时剩余寿命预测方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[2]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[3]	ZHANG Yuanjun, ZHANG Xihuang. Dynamic network representation learning model based on graph convolutional network and long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1857-1864.
[4]	XU Meng, WANG Yakun. Remaining useful life prediction of DA40 aircraft carbon brake pads based on bidirectional long short-term memory network [J]. Journal of Computer Applications, 2021, 41(5): 1527-1532.
[5]	LI Wenhui, ZENG Shangyou, WANG Jinjin. Image description generation algorithm based on improved attention mechanism [J]. Journal of Computer Applications, 2021, 41(5): 1262-1267.
[6]	ZHANG Xinhuan, LIU Hongjie, SHI Junqing, MAO Chengyuan, MENG Guolian. LSTM and artificial neural network for urban bus travel time prediction based on spatiotemporal eigenvectors [J]. Journal of Computer Applications, 2021, 41(3): 875-880.
[7]	QIAN Bin, ZHENG Kaihong, CHEN Zipeng, XIAO Yong, LI Sen, YE Chunzhuang, MA Qianli. Time series imputation model based on long-short term memory network with residual connection [J]. Journal of Computer Applications, 2021, 41(1): 243-248.
[8]	XIAO Yong, ZHENG Kaihong, ZHENG Zhenjing, QIAN Bin, LI Sen, MA Qianli. Multi-scale skip deep long short-term memory network for short-term multivariate load forecasting [J]. Journal of Computer Applications, 2021, 41(1): 231-236.
[9]	MA Tingting, JI Tianjiao, YANG Guanyu, CHEN Yang, XU Wenbo, LIU Hongtu. Incidence trend prediction of hand-foot-mouth disease based on long short-term memory neural network [J]. Journal of Computer Applications, 2021, 41(1): 265-269.
[10]	FU Kui, LIANG Shaoqing, LI Bing. Commodity recommendation model based on improved deep Q network structure [J]. Journal of Computer Applications, 2020, 40(9): 2613-2621.
[11]	RONG Wei, JIANG Zheyuan, XIE Zhao, WU Kewei. Clustering relational network for group activity recognition [J]. Journal of Computer Applications, 2020, 40(9): 2507-2513.
[12]	CHEN Jiawei, HAN Fang, WANG Zhijie. Aspect-based sentiment analysis with self-attention gated graph convolutional network [J]. Journal of Computer Applications, 2020, 40(8): 2202-2206.
[13]	LI Shengwu, ZHANG Xuande. Multi-domain convolutional neural network based on self-attention mechanism for visual tracking [J]. Journal of Computer Applications, 2020, 40(8): 2219-2224.
[14]	QIU Yuan, Chang Xiangmao, QIU Qian, PENG Cheng, SU Shanting. Stream data anomaly detection method based on long short-term memory network and sliding window [J]. Journal of Computer Applications, 2020, 40(5): 1335-1339.
[15]	FU Hongliang, LEI Peizhi. Speech deception detection algorithm based on denoising autoencoder and long short-term memory network [J]. Journal of Computer Applications, 2020, 40(2): 589-594.