基于分组特性的冗余流量消除算法

doi:10.11772/j.issn.1001-9081.2014.06.1541

计算机应用 ›› 2014, Vol. 34 ›› Issue (6): 1541-1545.DOI: 10.11772/j.issn.1001-9081.2014.06.1541

基于分组特性的冗余流量消除算法

郑鸿¹,²,邢玲¹,²,马强¹,²

1. 特殊环境机器人技术四川省重点实验室, 四川绵阳 621010
2. 西南科技大学信息工程学院,四川绵阳 621010

收稿日期:2013-12-13 修回日期:2014-03-01 发布日期:2014-07-02 出版日期:2014-06-01
通讯作者: 郑鸿
作者简介:郑鸿(1987-),男,四川成都人,硕士研究生,主要研究方向:冗余流量抑制;邢玲(1978-),女,四川成都人,副教授,博士,主要研究方向:大尺度冗余流量测量及其特性分析;马强(1982-),男,四川成都人,讲师,博士,主要研究方向:冗余抑制的体系结构。
基金资助:
国家自然科学基金资助项目

Redundancy traffic elimination algorithm based on packet feature

ZHENG Hong¹,²,XING Ling¹,²,MA Qiang¹,²

1. Robot Technology Used for Special Environment Key Laboratory of Sichuan Province, Mianyang Sichuan 621010, China
2. School of Information Engineering, Southwest University of Science and Technology, Mianyang Sichuan 621010, China;

Received:2013-12-13 Revised:2014-03-01 Online:2014-07-02 Published:2014-06-01
Contact: ZHENG Hong

摘要/Abstract

摘要：

针对大量数据片段冗余传输造成网络传输效率降低的问题,在协议无关冗余消除技术的基础上,提出了一种基于分组特性的冗余流量消除(PFRTE)算法。该算法以对数据包大小分组为基础,动态统计和分析网络流量的双峰特性和分组特性,以对冗余消除贡献大的数据包为阈值,利用滑动窗口计算数据块的边界点,对两个边界点间的数据块计算其指纹并进行指纹匹配。对重复传输的数据块进行简单编码,用编码数据替换重复传输的数据片段,再对消除冗余流量的数据包进行传输。与基于最大值选择和基于静态查找表选择等冗余流量消除算法相比,PFRTE算法动态统计网络流量分组特性,能够大大减少服务器端和客户端的CPU运行时间,同时冗余消除字节节省了8%~40%,证明了该算法的有效性。

Abstract:

Concerning the low efficiency of network transmission caused by redundant traffic, an algorithm named Packet Feature based Redundancy Traffic Elimination (PFRTE) was proposed based on the protocol-independent traffic redundancy elimination technique. Based on the grouping of packet size, PFRTE dynamically analyzed statistical bimodal characteristics and packet features of network traffic and regarded the size of the packet with the greatest capability of redundancy elimination as the threshold. It decided the boundary points by using sliding window method and calculated the fingerprint of block data within two boundary points. PFRTE encoded the redundant blocks in a simple way and transfered the encoded data instead of redundant data. The experimental results show that, compared with redundant traffic elimination algorithm based on maximum selection and static lookup table selection, PFRTE has the advantage of analyzing the redundancy statistics of network traffic dynamically, and the CPU consumption reduces both at server and client. Meanwhile, the algorithm is also effective with rate of redundancy elimination bytes saving of 8%-40%.

中图分类号:

TP393

郑鸿邢玲马强. 基于分组特性的冗余流量消除算法[J]. 计算机应用, 2014, 34(6): 1541-1545.

ZHENG Hong XING Ling MA Qiang. Redundancy traffic elimination algorithm based on packet feature[J]. Journal of Computer Applications, 2014, 34(6): 1541-1545.

参考文献

[1]MARTYNOV M. Challenges for high-speed protocol-independent redundancy eliminating systems[C]// ICCCN 2009: Proceedings of the 18th Internatonal Conference on Computer Communications and Networks. Piscataway: IEEE Press, 2009: 1-6.
[2]TANG H, LIN X, HAN C. Duplicate elimination algorithm for data streams with SKIP Bloom filter[J]. Journal on Communications, 2012, 33:(2):7-14.(唐海娜, 林小拉, 韩春静. 基于移动指针的数据流冗余消除算法[J]. 通信学报, 2012, 33(2): 7-14.)
[3]WOLMAN A, VOELKER G, SHARMA N, et al. Organization-based analysis of Web-object sharing and caching [C]// USITS 1999: Proceedings of the 2nd Conference on USENIX Symposium on Internet Technologies and Systems. Berkeley: USENIX Association, 1999,2:3.
[4]SPRING N T, WETHERALL D. A protocol-independent technique for eliminating redundant network traffic[J]. ACM SIGCOMM Computer Communication Review, 2000, 30(4): 87-95.
[5]ANAND A, SEKAR V, AKELLA A. SmartRE: an architecture for coordinated network-wide redundancy elimination[J]. ACM SIGCOMM Computer Communication Review, 2009, 39(4): 87-98.
[6]AGARWAL B, AKELLA A, ANAND A, et al. EndRE: an end-system redundancy elimination service for enterprises[C]// NSDI 2010: Proceedings of the 7th USENIX Conference on Networked Systems Design and Implementation. Berkeley: USENIX Association, 2010: 419-432.
[7]THOMPSON K, MILLER G J, WILDER R. Wide-area Internet traffic patterns and characteristics[J]. IEEE Network, 1997, 11(6): 10-23.
[8]HALEPOVIC E, WILLIAMSON C, GHADERI M. Enhancing redundant network traffic elimination[J]. Computer Networks, 2012, 56(2): 795-809.
[9]LU G, JIN Y, DU D H C. Frequency based chunking for data de-duplication[C]// Proceedings of the 2010 IEEE International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems. Piscataway: IEEE Press, 2010: 287-296.
[10]WILLIAMSON C. Internet traffic measurement[J]. IEEE Internet Computing, 2001, 5(6): 70-74.
[11]YAMAMOTO S, NAKAO A. P2P packet cache router for network-wide traffic redundancy elimination[C]// Proceedings of the 2012 International Conference on Computing, Networking and Communications. Piscataway: IEEE Press, 2012: 830-834.
[12]BRESLAU L, CAO P, FAN L, et al. Web caching and Zipf-like distributions: evidence and implications[C]// INFOCOM 1999: Proceedings of the 18th Annual Joint Conference of the IEEE Computer and Communications Societies. Piscataway: IEEE Press, 1999, 1: 126-134.
[13]COCERES R, DANZIG P B, JAMIN S, et al. Characteristics of wide-area TCP/IP conversations[J]. ACM SIGCOMM Computer Communication Review, 1991, 21(4): 101-112.

[1]	宗学军韩冰王国刚宁博伟何戡连莲. TDRFuzzer:基于自适应动态区间策略的工业控制协议模糊测试方法[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[2]	张明, 付乐, 王海峰. 面向边缘计算的并发数据流接转控制模型[J]. 《计算机应用》唯一官方网站, 2024, 44(12): 3876-3883.
[3]	郑智强王锐棋范子静何发镁姚叶鹏汪秋云姜政伟. DNS隧道检测技术研究综述[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[4]	张立孝马垚杨玉丽于丹陈永乐. 基于命名实体识别的大规模物联网二进制组件识别[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[5]	刘羿希, 何俊, 吴波, 刘丙童, 李子玉. DevSecOps中软件安全性测试技术综述[J]. 《计算机应用》唯一官方网站, 2024, 44(11): 3470-3478.
[6]	庞川林, 唐睿, 张睿智, 刘川, 刘佳, 岳士博. D2D通信系统中基于图卷积网络的分布式功率控制算法[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2855-2862.
[7]	方介泼, 陶重犇. 应对零日攻击的混合车联网入侵检测系统[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2763-2769.
[8]	张一鸣曹腾飞. 基于本地漂移和多样性算力的联邦学习优化算法[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[9]	徐航, 杨智, 陈性元, 韩冰, 杜学绘. 基于自适应敏感区域变异的覆盖引导模糊测试[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2528-2535.
[10]	陈虹, 齐兵, 金海波, 武聪, 张立昂. 融合1D-CNN与BiGRU的类不平衡流量异常检测[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2493-2499.
[11]	耿海军董赟胡治国池浩田杨静尹霞. 基于Attention-1DCNN-CE的加密流量分类方法[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[12]	邓淼磊阚雨培孙川川徐海航樊少珺周鑫. 基于深度学习的网络入侵检测系统综述[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[13]	陈瑞龙胡涛卜佑军伊鹏胡先君乔伟. 面向加密恶意流量检测模型的堆叠集成对抗防御方法[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[14]	吴中岱, 韩德志, 蒋海豹, 冯程, 韩冰, 陈重庆. 海洋船舶通信网络安全综述[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2123-2136.
[15]	姚梓豪, 栗远明, 马自强, 李扬, 魏良根. 基于机器学习的多目标缓存侧信道攻击检测模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1862-1871.

基于分组特性的冗余流量消除算法

Redundancy traffic elimination algorithm based on packet feature

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics