Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (4): 984-989.DOI: 10.11772/j.issn.1001-9081.2020081290

Special Issue: CCF第35届中国计算机应用大会(CCF NCCA 2020)

• The 35 CCF National Conference of Computer Applications (CCF NCCA 2020) • Previous Articles     Next Articles

Message aggregation technology of runtime system for graph computing

ZHANG Lufei1,2, SUN Rujun2, QIN Fang2   

  1. 1. Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China;
    2. State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi Jiangsu 214125, China
  • Received:2020-08-24 Revised:2020-10-16 Online:2021-04-10 Published:2020-11-25
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61972180).


张鲁飞1,2, 孙茹君2, 秦芳2   

  1. 1. 清华大学 计算机科学与技术系, 北京 100084;
    2. 数学工程与先进计算国家重点实验室, 江苏 无锡 214125
  • 通讯作者: 张鲁飞
  • 作者简介:张鲁飞(1986—),男,湖北襄阳人,工程师,博士,主要研究方向:人工智能、操作系统;孙茹君(1990—),女,甘肃兰州人,助理研究员,博士,主要研究方向:人工智能、运行时系统;秦芳(1975—),女,湖南零陵人,高级工程师,硕士,主要研究方向:操作系统。
  • 基金资助:

Abstract: The main communication mode of graph computing applications is spatiotemporally random point-to-point fine-grained communication. However, existing high-performance computer network systems perform poorly when dealing with a large number of fine-grained communications, which affect the overall performance. The communication optimization in application layer can improve the performance of graph computing application effectively, but this brings great burden to application developers. Therefore, a structure-dynamic message aggregation technique was proposed and implemented, which produced a lot of intermediate points in the communication path by building virtual topologies, so as to greatly improve the effect of message aggregation. By contrast, the traditional message aggregation strategy generally performed only at the communication source or destination with limited aggregation chances. In addition, this proposed technique adapted different kinds of hardware conditions and application features by flexibly adjusting the structure and configuration of the virtual topology. At the same time, the runtime system with message aggregation for graph computing was proposed and implemented, which allowed the runtime system to dynamically select parameters when executing iterations, so as to reduce the burden of developers. Experimental results on a system with 256 nodes show that typical graph computing application performance can achieve more than 100% improvement after optimized by the proposed message aggregation technique.

Key words: graph computing, runtime system, scalability, communication model, message aggregation

摘要: 图计算应用的通信模式以时空随机的点对点细粒度通信为主,但现有高性能计算机的网络系统应对大量细粒度通信时表现不佳,进而影响整体性能。虽然在应用层进行通信优化可以有效提升图计算应用性能,但这会给应用开发人员带来很大的负担,因此提出并实现结构动态的消息聚合技术,通过构建虚拟拓扑的方法在通信路径上增加中间点从而提升消息聚合的效果。传统的消息聚合策略一般仅在通信源或者目的地上进行,聚合机会有限,而所提技术通过灵活调整虚拟拓扑的结构和配置适应了不同硬件条件和应用特征。同时,还提出并实现了面向图计算的有消息聚合的运行时系统,这使得在程序迭代执行时可以动态选择参数,从而减少开发人员负担。在256节点规模的系统上实验的结果显示,使用所提消息聚合技术优化后的典型图计算应用的性能可得到100%以上的提升。

关键词: 图计算, 运行时系统, 可扩展性, 通信模型, 消息聚合

CLC Number: