基于选择性加载策略的电能质量数据处理

doi:10.11772/j.issn.1001-9081.2016.05.1434

计算机应用 ›› 2016, Vol. 36 ›› Issue (5): 1434-1438.DOI: 10.11772/j.issn.1001-9081.2016.05.1434

基于选择性加载策略的电能质量数据处理

赵霞^1,2, 林天华², 马素霞³, 齐林海³

1. 北京航空航天大学自动化科学与电气工程学院, 北京 100191;
2. 河北经贸大学信息技术学院, 石家庄 050061;
3. 华北电力大学控制与计算机工程学院, 北京 102206

收稿日期:2015-10-14 修回日期:2016-01-01 出版日期:2016-05-10 发布日期:2016-05-09
通讯作者: 赵霞
作者简介:赵霞(1979-),女,河北定州人,副教授,博士,CCF会员,主要研究方向:软件工程、非线性控制;林天华(1979-),男,福建上杭人,副教授,硕士,主要研究方向:软件工程、软件架构、信息系统;马素霞(1964-),女,山东章丘人,教授,主要研究方向:软件工程、软件体系结构、软件构件技术;齐林海(1964-),男,北京人,副教授,主要研究方向:数据挖掘、信息系统、商务智能。
基金资助:
河北省教育厅高等学校科学技术研究项目(YQ2013038);河北省自然科学基金资助项目(F2015207009);河北经贸大学科研基金资助项目(2013KYY17)。

Method for processing power quality data based on selective reloading

ZHAO Xia^1,2, LIN Tianhua², MA Suxia³, QI Linhai³

1. School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China;
2. School of Information Technology, Hebei University of Economics and Business, Shijiazhuang Hebei 050061, China;
3. School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China

Received:2015-10-14 Revised:2016-01-01 Online:2016-05-10 Published:2016-05-09
Supported by:
This work is partly supported by the Science Technology Research Projects for the High school of Hebei Education Department (YQ2013038), the Natural Science Foundation of Hebei Province(F2015207009) and the Science Research Project of Hebei University of Economics and Business(2013KYY17).

摘要/Abstract

摘要： 根据电能质量系统中监测数据海量化的趋势,提出了一种基于部分存储和选择性加载的数据处理算法,彻底解决了现有数据处理算法中重复排序和多余处理的问题。在计算日指标时,根据存储率存储部分日排序数据;在计算周(月、季、年)指标时,利用多路归并算法将存储的部分日排序数据合并,计算出临时95概率大值(CP95);根据临时CP95确定需要重载的日数据,对部分存储的日数据和重载数据重新排序以计算稳态指标。部分存储的日排序数据可以重复利用,有效解决了传统处理方案中的重复排序问题;排序过程中只需读取部分日排序数据和少量重载数据,有效解决了传统处理方案中冗余处理问题。与传统的数据处理方法做测试对比,结果表明:日采样数据较小时,性能提升3倍以上;日采样数据超过2880时,性能提升15倍以上。数据量越大,性能提升越明显。所提方案已在山西、河北等监测系统中成功应用,实践证明所提方案正确、有效。

关键词: 电能质量, 海量数据, 多路归并, 存储率, 重载率

Abstract: The monitoring data in the power quality monitoring system increased quickly. A new method based on partial storage and selective reloading was proposed, which can solve the problem of repetitive sorting and redundant processing in the tradition methods. In the calculation of the daily index, daily data was sorted and stored partly based on saving rate. In the calculation of week (month, season or year) index, the partly saved daily data in a week (month, season or year) were merged by the multiple merge algorithm to calculate a temporary 95 percentile (CP95), which could be used to determine which daily data should be reloaded. Besides the reloaded data, all other needed data were reordered to calculate the steady index. The sorting process only needed part of the stored daily data and a small amount of reloaded data, so the redundant processing problem in traditional processing method was solved effectively. Compared with the traditional data processing method, the experimental results show the efficiency can be increased more than 3 times using the proposed method when daily sampling data is relatively small. When the number of daily sampling data is more than 2880, the efficiency can be increased more than 15 times. The larger the amount of sampling data is, the more obviously the performance improves.The method has been applied in the monitoring system of Shanxi, Hebei and other provinces successfully. It is proved in practice that the method is correct and effective.

Key words: power quality, mass data, multiple merge, store rate, reload rate

中图分类号:

TP319

赵霞, 林天华, 马素霞, 齐林海. 基于选择性加载策略的电能质量数据处理[J]. 计算机应用, 2016, 36(5): 1434-1438.

ZHAO Xia, LIN Tianhua, MA Suxia, QI Linhai. Method for processing power quality data based on selective reloading[J]. Journal of Computer Applications, 2016, 36(5): 1434-1438.

参考文献

[1] MELHORN C J S, MAITRA A, SUNDERMAN W, et al.Distribution system power quality assessment phase Ⅱ:voltage sag and interruption analysis[C]//PCIC2005:Industry Applications Society 52nd Annual Petroleum and Chemical Industry Conference. Piscataway, NJ:IEEE, 2005:113-120.
[2] BOLLEN M. Understanding Power Quality Problems[M]. New York:IEEE, 2000:50-100.
[3] FUCHS E F, MASOUM M A S. Power Quality in Power Systems and Electrical Machines[M]. London:Elsevier Academic Press, 2011:1-400.
[4] 王玲, 康健, 邹宏亮, 等.实时电能质量监测系统的构建及应用[J]. 电力系统保护与控制, 2011, 39(2):108-111. (WANG L, KANG J, ZOU H L, et al.Construction and application of real-time power quality monitoring system[J]. Power System Protection and Control, 2011, 39(2):108-111.)
[5] 陈锦铭, 袁晓冬.电能质量在线监测平台的设计与开发[J].电力信息化, 2011, 9(3):60-64.(CHEN J M, YUAN X D. Design and development of an online power quality monitoring platform[J].Electric Power Information Technology, 2011, 9(3):60-64.)
[6] 何维国, 谢伟, 张健.上海电网电能质量监测系统[J].华东电力, 2010, 38(4):497-499.(HE W G, XIE W, ZHANG J.Power quality monitoring system of Shanghai grid[J].East China Electric Power, 2010, 38(4):497-499.)
[7] 全国电压电流等级和频率标准化技术委员会. GB/T 12325-2008, 电能质量供电电压偏差[S]. 北京:中国标准出版社, 2008:3.(Standard voltages, current ratings and frequencies. GB/T 12325-2008, Power quality-deviation of supply voltage[S]. Beijing:China Standard Press, 2008:3.).
[8] IEEE Standards Coordinating Committee 22 on Power Quality. IEEE Std1159.3-2003 IEEE recommended practice for the transfer of power quality data[S]. New York:IEEE, 2004:1-200.
[9] 马友忠, 慈祥, 孟小峰. 海量高维向量的并行Top-k连接查询[J]. 计算机学报, 2015, 38(1):86-98.(MA Y Z, CI X, MENG X F. Parallel Top-k join on massive high-dimensional vectors[J]. Chinese Journal of Computers, 2015, 38(1):86-98.)
[10] 王有为, 王伟平, 孟丹. 基于统计方法的Hive数据仓库查询优化实现[J]. 计算机研究与发展, 2015, 52(6):1452-1462. (WANG Y W, WANG W P, MENG D.Query optimization by statistical approach for Hive data warehouse[J]. Journal of Computer Research and Development, 2015, 52(6):1452-1462.)
[11] 刘亚秋, 李海涛, 景维鹏. 基于Hadoop的海量嘈杂数据决策树算法的实现[J]. 计算机应用, 2015, 35(4):1143-1147. (LIU Y Q, LI H T, JING W P.Implementation of decision tree algorithm dealing with massive noisy data based on Hadoop[J].Journal of Computer Applications, 2015, 35(4):1143-1147.)
[12] 张逸, 杨洪耕.海量电能质量数据交换格式文件快速解析方案[J].电力自动化设备, 2013, 33(12):116-121. (ZHANG Y, YANG H G.Fast parsing of massive PQDIF files[J].Electric Power Automation Equipment, 2013, 33(12):116-121.)
[13] 张逸, 杨洪耕, 叶茂清.基于分布式文件系统的海量电能质量监测数据管理方案[J].电力系统自动化, 2014, 38(2):102-108. (ZHANG Y, YANG H G, YE M Q. A data management scheme of massive power quality monitoring data based on distributed file system[J].Automation of Electric Power Systems, 2014, 38(2):102-108.)
[14] 魏雪琴.电能质量数据压缩研究[D].合肥:安徽大学, 2013:5-25. (WEI X Q. The research on power quality data compression[D].Hefei:Anhui University, 2013:5-25.)
[15] 赵东旭. 电能质量数据中心建设关键问题研究[D]. 北京:华北电力大学, 2012:10-20.(ZHAO D X. Research on key issues in power quality data center construction[D]. Beijing:North China Electric Power University, 2012:10-20.)
[16] 王淑祥, 马素霞. 基于Hadoop的海量电能质量监测数据分析平台研究[J].中国科技信息, 2013, 25(13):79-80.(WANG S X, MA S X. Study on massive power quality monitoring data analysis platform based on Hadoop[J]. China Science and Technology Information, 2013, 25(13):79-80.)
[17] 李斌, 郭雅娟, 陈锦铭, 等. 电能质量监测系统95概率大值的top-k 优化研究[J]. 电力信息化, 2013, 11(1):20-24. (LI B, GUO Y J, CHEN J M, et al.Research on the top-k optimization method of 95% probability values for the power quality monitoring system[J].Electric Power Information Technology, 2013, 11(1):20-24.)
[18] 白宇, 郭显娥. 深度优先稳定原地归并排序的高效算法[J].计算机应用, 2013, 33(4):1039-1042.(BAI Y, GUO X E. Efficient algorithm of depth-first stable in-place merge sort[J].Journal of Computer Applications, 2013, 33(4):1039-1042.)
[19] 孙琳琳, 侯秀萍, 朱波, 等. 基于多线程归并排序算法设计[J]. 吉林大学学报(信息科学版), 2015, 33(1):105-110.(SUN L L, HOU X P, ZHU B, et al.Merge sort algorithm design based on multi thread[J].Journal of Jilin University (Information Science Edition), 2015, 33(1):105-110.)
[20] 姜忠华, 徐文丽, 刘家文, 等.智能归并排序[J].电子设计工程, 2011, 19(21):53-55.(JIANG Z H, XU W L, LIU J W, et al. Intelligently merge sort[J].Electronic Design Engineering, 2011, 19(21):53-55.)
[21] HOSSAIN N, ALMA M G R, AMIRUZZAMAN M, et al. An efficient merge sort technique that reduces both times and comparisons[C]//2004 ICTTA:Proceedings of the 2004 International Conference on Information and Communication Technologies:from Theory to Applications. Piscataway, NJ:IEEE, 2004:857-858.
[22] KAZMI S Z H, KAZMI S S, KAZMI S S H, et al. Chunk sort[J]. International Journal of Computer Science and Information Security, 2010, 8(4):162.
[23] 赵霞, 林天华.分布式电能质量信息系统[J].科技导报, 2013, 31(19):58-61.(ZHAO X, LIN T H. Design and implementation of distributed electric power quality information system[J].Science & Technology Review, 2013, 31(19):58-61.)

基于选择性加载策略的电能质量数据处理

Method for processing power quality data based on selective reloading

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	唐林川, 邓思宇, 吴彦学, 温柳英. 基于pHash分块局部探测的海量图像查重算法[J]. 计算机应用, 2019, 39(9): 2789-2794.
[2]	陈羽中, 郭松荣, 陈宏, 李婉华, 郭昆, 黄启成. 基于并行分类算法的电力客户欠费预警[J]. 计算机应用, 2016, 36(6): 1757-1761.
[3]	刘超, 胡成玉, 姚宏, 梁庆中, 颜雪松. 面向海量非结构化数据的非关系型存储管理机制[J]. 计算机应用, 2016, 36(3): 670-674.
[4]	梁俊杰, 甘文婷, 余敦辉. 基于位置编码索引树的个性化推荐算法[J]. 计算机应用, 2016, 36(2): 419-423.
[5]	白鹤翔, 王健, 李德玉, 陈千. 基于粗糙集的非监督快速属性选择算法[J]. 计算机应用, 2015, 35(8): 2355-2359.
[6]	孙远帅陈垚官新均林琛. 基于Hadoop的大矩阵乘法处理方法[J]. 计算机应用, 2013, 33(12): 3339-3344.
[7]	张永浮盼盼张玉婷. 基于分层聚类及重采样的大规模数据分类[J]. 计算机应用, 2013, 33(10): 2801-2803.
[8]	周典瑞周莲英. 海量数据的相似重复记录检测算法[J]. 计算机应用, 2013, 33(08): 2208-2211.
[9]	蒋新华廖律超邹复民. 基于浮动车移动轨迹的新增道路自动发现算法[J]. 计算机应用, 2013, 33(02): 579-582.
[10]	徐翔邹复民廖律超朱铨. 基于GemFire的海量数据计算性能实验分析[J]. 计算机应用, 2013, 33(01): 226-229.
[11]	夏秀峰赵龙. 基于三层存储模型的RFID数据压缩存储方法[J]. 计算机应用, 2012, 32(03): 625-628.
[12]	黄海峰张珂珩张鸿季学纯陈鹏. 电力系统动态信息数据库关键技术[J]. 计算机应用, 2011, 31(06): 1681-1684.
[13]	王灿秦志光冯朝胜彭静. 面向重复数据消除的备份数据加密方法[J]. 计算机应用, 2010, 30(07): 1763-1766.
[14]	王春马纯永陈戈. 基于GPGPU的海量山地地形数据的实时绘制算法[J]. 计算机应用, 2009, 29(08): 2105-2108.
[15]	李涛夏浪何怡刚. 叠加暂态电能质量扰动的分形熵分析[J]. 计算机应用, 2009, 29(08): 2288-2290.