基于MapReduce的多元线性回归预测模型

doi:10.11772/j.issn.1001-9081.2014.07.1862

计算机应用 ›› 2014, Vol. 34 ›› Issue (7): 1862-1866.DOI: 10.11772/j.issn.1001-9081.2014.07.1862

基于MapReduce的多元线性回归预测模型

代亮¹,²,许宏科¹,²,陈婷²,³,钱超¹,²,粱殿鹏⁴

1. 长安大学电子与控制工程学院，西安 710064;
2. 陕西省道路交通智能检测与装备工程技术研究中心,西安 710064；
3. 长安大学信息工程学院，西安 710064
4. IBM中国系统与科技开发中心,西安 710068

收稿日期:2014-01-17 修回日期:2014-03-16 出版日期:2014-07-01 发布日期:2014-08-01
通讯作者: 代亮
作者简介:代亮(1981-)，男，陕西西安人，讲师，博士，CCF会员，主要研究方向：并行计算、海量数据并行处理；许宏科(1963-)，男，陕西宝鸡人，教授，博士生导师，博士，主要研究方向：智能交通系统；陈婷(1982-)，女，陕西西安人，讲师, 博士，主要研究方向：分布式系统、并行计算；钱超(1984-)，男，江苏徐州人，讲师，博士，主要研究方向：交通数据挖掘与分析；梁殿鹏(1977-)，男，甘肃武威人，高级工程师，硕士，主要研究方向：并行计算、负载均衡。
基金资助:
国家自然科学基金资助项目;教育部创新团队发展计划项目;交通运输部基础研究项目;陕西省自然科学基础研究计划项目;中央高校基本科研业务费专项资金资助项目;中国博士后科学基金面上资助项目

Multivariate linear regression forecasting model based on MapReduce

DAI Liang¹,²,XU Hongke¹,²,CHEN Ting²,³,QIAN Chao¹,²,LIANG Dianpeng⁴

1. School of Electronic and Control Engineering, Chang'an University, Xi'an Shaanxi 710064, China;
2. Shaanxi Road Traffic Detection and Equipment Engineering Research Center, Xi'an Shaanxi 710064,China;
3. School of Information Engineering, Chang'an University, Xi'an Shaanxi 710064, China;
4. IBM China Systems and Technology Laboratory, Xi'an Shaanxi 710068, China

Received:2014-01-17 Revised:2014-03-16 Online:2014-07-01 Published:2014-08-01
Contact: DAI Liang

摘要/Abstract

摘要：

针对传统的多元线性回归预测方法处理时间长且受内存限制的特点，对时序样本数据设计了基于MapReduce的并行多元线性回归预测模型。模型由三组MapReduce过程组成，分别求解由历史数据所构成叉积矩阵的特征向量和标准正交特征向量，用来预测未来参数的特征值和特征向量矩阵和未来时刻回归参数的估计量。设计并实现了实验来验证提出的并行多元线性回归预测模型的有效性。实验结果表明，基于MapReduce的多元线性回归预测模型具有较好的加速比和可扩展性，适合于大规模时序数据的分析和预测。

Abstract:

According to the characteristics of traditional multivariate linear regression method for long processing time and limited memory, a parallel multivariate linear regression forecasting model was designed based on MapReduce for the time-series sample data. The model was composed of three MapReduce processes which were used to solve the eigenvector and standard orthogonal vector of cross product matrix composed by historical data, to forecast the future parameter of the eigenvalues and eigenvectors matrix, and to estimate the regression parameters in the next moment respectively. Experiments were designed and implemented to the validity effectiveness of the proposed parallel multivariate linear regression forecasting model. The experimental results show multivariate linear regression prediction model based on MapReduce has good speedup and scaleup, and suits for analysis and forecasting of large data.

中图分类号:

TP311

代亮许宏科陈婷钱超粱殿鹏. 基于MapReduce的多元线性回归预测模型[J]. 计算机应用, 2014, 34(7): 1862-1866.

DAI Liang XU Hongke CHEN Ting QIAN Chao LIANG Dianpeng. Multivariate linear regression forecasting model based on MapReduce[J]. Journal of Computer Applications, 2014, 34(7): 1862-1866.

参考文献

［1］SEGHOUANE A K. New AIC corrected variants for multivariate linear regression model selection［J］. IEEE Transactions on Aerospace and Electronic Systems, 2011, 47(2):1154-1165.
［2］WANG H, MENG J. Predictive modeling on multivariate linear regression［J］. Journal of Beijing University of Aeronautics and Astronautics, 2007,33(4):500-504. (王惠文,孟洁.多元线性回归的预测建模方法［J］.北京航空航天大学学报,2007,33(4):500-504.)
［3］FU Y, WANG X. Recognition for change-point of traffic flow binary linear regressions based on projective transform method［J］. Journal of Computer Applications,2010, 30(1) 263-265. (付宇,王晓原. 基于投影变换的交通流二元线性回归变点识别方法［J］. 计算机应用,2010, 30(1) 263-265.)
［4］AKZ O, KARSLIGIL M E. Severity detection of traffic accidents at intersections based on vehicle motion analysis and multiphase linear regression［C］// Proceedings of the 13th International IEEE Conference on Intelligent Transportation Systems. Piscataway: IEEE, 2010: 474-479.
［5］ZHONG L, SUN X, CHEN Y, et al. Accident prediction model of freeway［J］. Journal of Beijing University of Technology, 2009,35(7): 966-971. (钟连德, 孙小端, 陈永胜，等. 高速公路事故预测模型［J］. 北京工业大学学报, 2009,35(7): 966-971.)
［6］SIMON P W, MATTHEW G K, FRED L M. Statistical and econometric methods for transportation data analysis［M］. 2nd ed. London: Chapman and Hall/CRC, 2010.
［7］SHEN L O, LU C X, ZHAO, et al. Discrete fourier transformation for seasonal-factor pattern classification and assignment［J］. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(2): 511-516.
［8］WANG S, CUI L J, LIU D C, et al. Vehicle identification via sparse representation［J］. IEEE Transactions on Intelligent Transportation Systems, 2012, 13(2): 955-962.
［9］CHEN Q, DENG Q . Cloud computing and its key techniques［J］. Journal of Computer Applications, 2009,29(9):2562-2567. (陈全, 邓倩妮. 云计算及其关键技术［J］. 计算机应用, 2009,29(9):2562-2567.)
［10］NANDI A, YU C, BOHANNON P, et al. Data cube materialization and mining over MapReduce［J］. IEEE Transactions on Knowledge and Data Engineering, 2012, 24(20):1747-1759.
［11］WANG Z K, AGRAWAL D, TAN K L. COSAC: a framework for combinatorial statistical analysis on cloud［J］. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(9):2010-2023.
［12］ZHANG X, GONG K, ZHAO G. Parallel K-Medoids algorithm based on MapReduce［J］. Journal of Computer Applications, 2013, 33(4): 1023-1025. (张雪萍,龚康莉,赵广才. 基于MapReduce的K-Medoids并行算法［J］. 计算机应用, 2013, 33(4): 1023-1025.)
［13］LU Q, CHENG X. Parallelization of decision tree algorithm based on MapReduce［J］. Journal of Computer Applications, 2012, 32(9): 2463-2465. (陆秋,程小辉. 基于MapReduce的决策树算法并行化［J］. 计算机应用, 2012, 32(9): 2463-2465.)
［14］LIU Y, JING N, CHEN L, et al. Algorithm for processing k-nearest join based on R-tree in MapReduce［J］. Journal of Software, 2013,24(8):1836-1851.(刘义,景宁,陈荦,等. MapReduce框架下基于R-树的k-近邻连接算法［J］. 软件学报,2013,24(8):1836-1851.)
［15］GUO Q, LI Y, LIU T, et al. Correlation-based performance analysis for full-system MapReduce optimization［C］// Proceedings of the 2013 IEEE International Conference on Big Data. Washington, DC: IEEE Computer Society, 2013: 753-761.
［16］BORIS S K. Introduction to modern traffic flow theory and control［M］. Berlin: Springer-Verlag, 2009.

[1]	祁祥洲邢红杰. 基于中心核对齐的多核单类支持向量机[J]. 计算机应用, 0, (): 0-0.
[2]	陈浩杰，范江亭，刘勇. 分布式强化学习解决动态旅行商问题[J]. 计算机应用, 0, (): 0-0.
[3]	郭一阳于炯杜旭升杨少智曹铭. 基于自编码器与集成学习的离群点检测算法[J]. 计算机应用, 0, (): 0-0.
[4]	王周恺, 张炯, 马维纲, 王怀军. 面向高速列车监测数据的并行解压缩算法[J]. 计算机应用, 2021, 41(9): 2586-2593.
[5]	李卓, 宋子晖, 沈鑫, 陈昕. 边缘计算支持下的移动群智感知本地差分隐私保护机制[J]. 计算机应用, 2021, 41(9): 2678-2686.
[6]	赵津宋文爱邰隽杨吉江王青李晓丹雷毅邱悦. 儿童阻塞性睡眠呼吸暂停计算机人脸辅助诊断综述[J]. 计算机应用, 0, (): 0-0.
[7]	张妮韩萌王乐李小娟程浩东. 基于正负效用划分的高效用模式挖掘方法综述[J]. 计算机应用, 0, (): 0-0.
[8]	武鹏, 吴尽昭. 基于线性误差断言的推理方法[J]. 计算机应用, 2021, 41(8): 2199-2204.
[9]	王梓森, 梁英, 刘政君, 谢小杰, 张伟, 史红周. 科研项目同行评议专家学术专长匹配方法[J]. 计算机应用, 2021, 41(8): 2418-2426.
[10]	赵全, 汤小春, 朱紫钰, 毛安琪, 李战怀. 大规模短时间任务的低延迟集群调度框架[J]. 计算机应用, 2021, 41(8): 2396-2405.
[11]	康军, 黄山, 段宗涛, 李宜修. 时空轨迹序列模式挖掘方法综述[J]. 计算机应用, 2021, 41(8): 2379-2385.
[12]	陈静, 毛莺池, 陈豪, 王龙宝, 王子成. 基于改进单点多盒检测器的大坝缺陷目标检测方法[J]. 计算机应用, 2021, 41(8): 2366-2372.
[13]	马华, 陈跃鹏, 唐文胜, 娄小平, 黄卓轩. 面向工作者能力评估的众包任务分配方法的研究进展综述[J]. 计算机应用, 2021, 41(8): 2232-2241.
[14]	孙蕊, 韩萌, 张春砚, 申明尧, 杜诗语. 含负项top-k高效用项集挖掘算法[J]. 计算机应用, 2021, 41(8): 2386-2395.
[15]	李莉吴怡杨祉坤陈云鹏. 基于分区型区块链医疗电子病历共享方案[J]. , 0, (): 0-0.

基于MapReduce的多元线性回归预测模型

Multivariate linear regression forecasting model based on MapReduce

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics