Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (8): 2354-2358.DOI: 10.11772/j.issn.1001-9081.2019010203

• Network and communications • Previous Articles     Next Articles

Detection method for network-wide persistent flow based on sketch data structure

ZHOU Aiping1,2, ZHU Chengang3   

  1. 1. School of Computer Science and Technology, Taizhou University, Taizhou Jiangsu 225300, China;
    2. Key Laboratory of Computer Network and Information Integration of Ministry of Education(Southeast University), Nanjing Jiangsu 211189, China;
    3. School of Computer Science and Engineering, Southeast University, Nanjing Jiangsu 211189, China
  • Received:2019-02-13 Revised:2019-03-19 Online:2019-08-14 Published:2019-08-10
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61802274), the Open Project Foundation of Key Laboratory of Computer Network and Information Integration of Ministry of Education (Southeast University) (K93-9-2017-01), the Scientific Research Foundation for Advanced Talents of Taizhou (QD2016027).


周爱平1,2, 朱琛刚3   

  1. 1. 泰州学院 计算机科学与技术学院, 江苏 泰州 225300;
    2. 计算机网络和信息集成教育部重点实验室(东南大学), 南京 211189;
    3. 东南大学 计算机科学与技术学院, 南京 211189
  • 通讯作者: 周爱平
  • 作者简介:周爱平(1982-),男,江苏泰州人,讲师,博士,CCF会员,主要研究方向:网络测量、数据挖掘;朱琛刚(1982-),男,江苏南京人,博士研究生,主要研究方向:大数据分析、网络测量。
  • 基金资助:

Abstract: Persistent flow is an important feature of hidden network attack. It does not generate a large amount of traffic and it occurs regularly in a long period, so that it brings a large challenge for traditional detection methods. Network attacks have invisibility, single monitors have heavy load and limited information. Aiming at the above problems, a method to detect network-wide persistent flows was proposed. Firstly, a sketch data structure was designed and was deployed on each monitor. Secondly, when the network flow arrived at a monitor, the summary information was extracted from network data stream and one bit in the sketch data structure was updated. Thirdly, at the end of measurement period, the summary information from other monitors was synthesized by the main monitor. Finally, the approximate estimation of flow persistence was presented. A bit vector was constructed for each flow by some simple computing, flow persistence was estimated by using probability statistical method, and the persistent flows were detected based on revised persistence estimation. The experiments were conducted on real network traffic, and their results show that compared with the algorithm of Tracing Long Duration flows (TLF), the proposed method increases the accuracy by 50% and reduces the false positive rate, false negative rate by 22%, 20% respectively. The results illustrate that the method of detecting network-wide persistent flows can effectively monitor network traffic in high-speed networks.

Key words: network measurement, persistent flow detection, network attack, sketch data structure, probabilistic statistical method

摘要: 持续流是隐蔽的网络攻击过程中显现的一种重要特征,它不产生大量流量且在较长周期内有规律地发生,给传统的检测方法带来极大挑战。针对网络攻击的隐蔽性、单监测点的重负荷和信息有限的问题,提出全网络持续流检测方法。首先,设计一种概要数据结构,并将其部署在每个监测点;其次,当网络流到达监测点时,提取流的概要信息并更新概要数据结构的一位;然后,在测量周期结束时,主监测点将来自其他监测点的概要信息进行综合;最后,提出流持续性的近似估计,通过一些简单计算为每个流构建一个位向量,利用概率统计方法估计流持续性,使用修正后的持续性估计检测持续流。通过真实的网络流量进行实验,结果表明,与长持续时间流检测算法(TLF)相比,所提方法的准确性提高了50%,误报率和漏报率分别降低了22%和20%,说明全网络持续流检测方法能够有效监测高速网络流量。

关键词: 网络测量, 持续流检测, 网络攻击, 概要数据结构, 概率统计方法

CLC Number: