Multivariate linear regression forecasting model based on MapReduce

doi:10.11772/j.issn.1001-9081.2014.07.1862

Journal of Computer Applications ›› 2014, Vol. 34 ›› Issue (7): 1862-1866.DOI: 10.11772/j.issn.1001-9081.2014.07.1862

Previous Articles Next Articles

Multivariate linear regression forecasting model based on MapReduce

DAI Liang¹,²,XU Hongke¹,²,CHEN Ting²,³,QIAN Chao¹,²,LIANG Dianpeng⁴

1. School of Electronic and Control Engineering, Chang'an University, Xi'an Shaanxi 710064, China;
2. Shaanxi Road Traffic Detection and Equipment Engineering Research Center, Xi'an Shaanxi 710064,China;
3. School of Information Engineering, Chang'an University, Xi'an Shaanxi 710064, China;
4. IBM China Systems and Technology Laboratory, Xi'an Shaanxi 710068, China

Received:2014-01-17 Revised:2014-03-16 Online:2014-08-01 Published:2014-07-01
Contact: DAI Liang

基于MapReduce的多元线性回归预测模型

代亮¹,²,许宏科¹,²,陈婷²,³,钱超¹,²,粱殿鹏⁴

1. 长安大学电子与控制工程学院，西安 710064;
2. 陕西省道路交通智能检测与装备工程技术研究中心,西安 710064；
3. 长安大学信息工程学院，西安 710064
4. IBM中国系统与科技开发中心,西安 710068

通讯作者: 代亮
作者简介:代亮(1981-)，男，陕西西安人，讲师，博士，CCF会员，主要研究方向：并行计算、海量数据并行处理；许宏科(1963-)，男，陕西宝鸡人，教授，博士生导师，博士，主要研究方向：智能交通系统；陈婷(1982-)，女，陕西西安人，讲师, 博士，主要研究方向：分布式系统、并行计算；钱超(1984-)，男，江苏徐州人，讲师，博士，主要研究方向：交通数据挖掘与分析；梁殿鹏(1977-)，男，甘肃武威人，高级工程师，硕士，主要研究方向：并行计算、负载均衡。
基金资助:
国家自然科学基金资助项目;教育部创新团队发展计划项目;交通运输部基础研究项目;陕西省自然科学基础研究计划项目;中央高校基本科研业务费专项资金资助项目;中国博士后科学基金面上资助项目

Abstract

Abstract:

According to the characteristics of traditional multivariate linear regression method for long processing time and limited memory, a parallel multivariate linear regression forecasting model was designed based on MapReduce for the time-series sample data. The model was composed of three MapReduce processes which were used to solve the eigenvector and standard orthogonal vector of cross product matrix composed by historical data, to forecast the future parameter of the eigenvalues and eigenvectors matrix, and to estimate the regression parameters in the next moment respectively. Experiments were designed and implemented to the validity effectiveness of the proposed parallel multivariate linear regression forecasting model. The experimental results show multivariate linear regression prediction model based on MapReduce has good speedup and scaleup, and suits for analysis and forecasting of large data.

摘要：

针对传统的多元线性回归预测方法处理时间长且受内存限制的特点，对时序样本数据设计了基于MapReduce的并行多元线性回归预测模型。模型由三组MapReduce过程组成，分别求解由历史数据所构成叉积矩阵的特征向量和标准正交特征向量，用来预测未来参数的特征值和特征向量矩阵和未来时刻回归参数的估计量。设计并实现了实验来验证提出的并行多元线性回归预测模型的有效性。实验结果表明，基于MapReduce的多元线性回归预测模型具有较好的加速比和可扩展性，适合于大规模时序数据的分析和预测。

CLC Number:

TP311

DAI Liang XU Hongke CHEN Ting QIAN Chao LIANG Dianpeng. Multivariate linear regression forecasting model based on MapReduce[J]. Journal of Computer Applications, 2014, 34(7): 1862-1866.

代亮许宏科陈婷钱超粱殿鹏. 基于MapReduce的多元线性回归预测模型[J]. 计算机应用, 2014, 34(7): 1862-1866.

References

［1］SEGHOUANE A K. New AIC corrected variants for multivariate linear regression model selection［J］. IEEE Transactions on Aerospace and Electronic Systems, 2011, 47(2):1154-1165.
［2］WANG H, MENG J. Predictive modeling on multivariate linear regression［J］. Journal of Beijing University of Aeronautics and Astronautics, 2007,33(4):500-504. (王惠文,孟洁.多元线性回归的预测建模方法［J］.北京航空航天大学学报,2007,33(4):500-504.)
［3］FU Y, WANG X. Recognition for change-point of traffic flow binary linear regressions based on projective transform method［J］. Journal of Computer Applications,2010, 30(1) 263-265. (付宇,王晓原. 基于投影变换的交通流二元线性回归变点识别方法［J］. 计算机应用,2010, 30(1) 263-265.)
［4］AKZ O, KARSLIGIL M E. Severity detection of traffic accidents at intersections based on vehicle motion analysis and multiphase linear regression［C］// Proceedings of the 13th International IEEE Conference on Intelligent Transportation Systems. Piscataway: IEEE, 2010: 474-479.
［5］ZHONG L, SUN X, CHEN Y, et al. Accident prediction model of freeway［J］. Journal of Beijing University of Technology, 2009,35(7): 966-971. (钟连德, 孙小端, 陈永胜，等. 高速公路事故预测模型［J］. 北京工业大学学报, 2009,35(7): 966-971.)
［6］SIMON P W, MATTHEW G K, FRED L M. Statistical and econometric methods for transportation data analysis［M］. 2nd ed. London: Chapman and Hall/CRC, 2010.
［7］SHEN L O, LU C X, ZHAO, et al. Discrete fourier transformation for seasonal-factor pattern classification and assignment［J］. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(2): 511-516.
［8］WANG S, CUI L J, LIU D C, et al. Vehicle identification via sparse representation［J］. IEEE Transactions on Intelligent Transportation Systems, 2012, 13(2): 955-962.
［9］CHEN Q, DENG Q . Cloud computing and its key techniques［J］. Journal of Computer Applications, 2009,29(9):2562-2567. (陈全, 邓倩妮. 云计算及其关键技术［J］. 计算机应用, 2009,29(9):2562-2567.)
［10］NANDI A, YU C, BOHANNON P, et al. Data cube materialization and mining over MapReduce［J］. IEEE Transactions on Knowledge and Data Engineering, 2012, 24(20):1747-1759.
［11］WANG Z K, AGRAWAL D, TAN K L. COSAC: a framework for combinatorial statistical analysis on cloud［J］. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(9):2010-2023.
［12］ZHANG X, GONG K, ZHAO G. Parallel K-Medoids algorithm based on MapReduce［J］. Journal of Computer Applications, 2013, 33(4): 1023-1025. (张雪萍,龚康莉,赵广才. 基于MapReduce的K-Medoids并行算法［J］. 计算机应用, 2013, 33(4): 1023-1025.)
［13］LU Q, CHENG X. Parallelization of decision tree algorithm based on MapReduce［J］. Journal of Computer Applications, 2012, 32(9): 2463-2465. (陆秋,程小辉. 基于MapReduce的决策树算法并行化［J］. 计算机应用, 2012, 32(9): 2463-2465.)
［14］LIU Y, JING N, CHEN L, et al. Algorithm for processing k-nearest join based on R-tree in MapReduce［J］. Journal of Software, 2013,24(8):1836-1851.(刘义,景宁,陈荦,等. MapReduce框架下基于R-树的k-近邻连接算法［J］. 软件学报,2013,24(8):1836-1851.)
［15］GUO Q, LI Y, LIU T, et al. Correlation-based performance analysis for full-system MapReduce optimization［C］// Proceedings of the 2013 IEEE International Conference on Big Data. Washington, DC: IEEE Computer Society, 2013: 753-761.
［16］BORIS S K. Introduction to modern traffic flow theory and control［M］. Berlin: Springer-Verlag, 2009.

[1]	. Research advances in blockchain consensus mechanisms and improvement algorithms [J]. Journal of Computer Applications, 0, (): 0-0.
[2]	. Blockchain-based identity management system for internet of things [J]. Journal of Computer Applications, 0, (): 0-0.
[3]	Jinhui CAI, Zhongxu YIN, Guoxiao ZONG, Junru LI. Integrated method of inference and taint analysis for nested branch breakthrough [J]. Journal of Computer Applications, 2024, 44(12): 3823-3830.
[4]	Bo LI, Jianqiang HUANG, Dongqiang HUANG, Xiaoying WANG. Adaptive computing optimization of sparse matrix-vector multiplication based on heterogeneous platforms [J]. Journal of Computer Applications, 2024, 44(12): 3867-3875.
[5]	Ziqian CHEN, Kedi NIU, Zhongyuan YAO, Xueming SI. Review of blockchain lightweight technology applied to internet of things [J]. Journal of Computer Applications, 2024, 44(12): 3688-3698.
[6]	Tingting GAO, Zhongyuan YAO, Miao JIA, Xueming SI. Overview of on-chain and off-chain consistency protection technologies [J]. Journal of Computer Applications, 2024, 44(12): 3658-3668.
[7]	Miao JIA, Zhongyuan YAO, Weihua ZHU, Tingting GAO, Xueming SI, Xiang DENG. Progress and prospect of zero-knowledge proof enabling blockchain [J]. Journal of Computer Applications, 2024, 44(12): 3669-3677.
[8]	Kedi NIU, Min LI, Zhongyuan YAO, Xueming SI. Review of blockchain consensus algorithms for internet of things [J]. Journal of Computer Applications, 2024, 44(12): 3678-3687.
[9]	. Heterogeneous Environment Aware Streaming Graph Partitioning for Power-Law Graphs [J]. Journal of Computer Applications, 0, (): 0-0.
[10]	. Source code vulnerability detection method based on Transformer-GCN [J]. Journal of Computer Applications, 0, (): 0-0.
[11]	. Partial label regression algorithm integrating feature attention and residual connection [J]. Journal of Computer Applications, 0, (): 0-0.
[12]	. Cloud-edge collaborative data storage and retrieval architecture for industrial scenarios [J]. Journal of Computer Applications, 0, (): 0-0.
[13]	Jinxing TU, Zhixiong LI, Jianqiang HUANG. Dynamic partition algorithm for diagonal sparse matrix vector multiplication based on GPU [J]. Journal of Computer Applications, 2024, 44(11): 3521-3529.
[14]	Li ZENG, Jingru YANG, Gang HUANG, Xiang JING, Chaoran LUO. Survey on hypergraph application methods： issues， advances， and challenges [J]. Journal of Computer Applications, 2024, 44(11): 3315-3326.
[15]	. Two-stage data selection method for classifier with high performance and low energy consumption [J]. Journal of Computer Applications, 0, (): 0-0.

Multivariate linear regression forecasting model based on MapReduce

基于MapReduce的多元线性回归预测模型

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics