Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (9): 2802-2809.DOI: 10.11772/j.issn.1001-9081.2023091252

• Advanced computing • Previous Articles     Next Articles

Optimization of tensor virtual machine operator fusion based on graph rewriting and fusion exploration

Na WANG1, Lin JIANG1(), Yuancheng LI1, Yun ZHU2   

  1. 1.College of Computer Science and Technology,Xi’an University of Science and Technology,Xi’an Shaanxi 710600,China
    2.School of Electronic Engineering,Xi’an University of Posts and Telecommunications,Xi’an Shaanxi 710121,China
  • Received:2023-09-18 Revised:2023-11-14 Accepted:2023-11-20 Online:2024-03-15 Published:2024-09-10
  • Contact: Lin JIANG
  • About author:WANG Na, born in 1994, M. S. candidate. Her research interests include reconfigurable compilation optimization, deep learning
    LI Yuancheng, born in 1981, Ph. D., lecturer. His research interests include computer architecture, parallel computing, artificial intelligence.
    ZHU Yun, born in 1981, M. S., lecturer. Her research interests include integrated circuit design and simulation.
  • Supported by:
    Scientific and Technological Innovation 2030 — Major Project of “New Generation of Artificial Intelligence”(2022ZD0119005);National Natural Science Foundation of China(61834005);Shaanxi Natural Science Foundation(2020JM-525)

基于图形重写和融合探索的张量虚拟机算符融合优化

王娜1, 蒋林1(), 李远成1, 朱筠2   

  1. 1.西安科技大学 计算机科学与技术学院,西安 710600
    2.西安邮电大学 电子工程学院,西安 710121
  • 通讯作者: 蒋林
  • 作者简介:王娜(1994—),女,陕西渭南人,硕士研究生,主要研究方向:可重构编译优化、深度学习
    李远成(1981—),男,河南开封人,讲师,博士,CCF会员,主要研究方向:计算机体系结构、并行计算、人工智能
    朱筠(1981—),女,陕西西安人,讲师,硕士,主要研究方向:集成电路设计及仿真。
  • 基金资助:
    科技创新2030——“新一代人工智能”重大项目(2022ZD0119005);国家自然科学基金资助项目(61834005);陕西省自然科学基金资助项目(2020JM-525)

Abstract:

In the process of computation-intensive neural networks using the Tensor Virtual Machine (TVM) operator fusion, there are problems such as excessive access counts and low memory resource utilization dure to layer-by-layer exploration of computational graphs. Therefore, an optimization method for TVM operator fusion based on graph rewriting and fusion exploration was proposed. Firstly, an analysis was conducted on the mapping types of operators. Secondly, the computational graph was rewritten based on operation laws to simplify its structure, thereby reducing the generation of intermediate results, and then lowering memory resource consumption and enhancing fusion efficiency. Thirdly, a fusion exploration algorithm was employed to identify operators with lower fusion costs for prioritized fusion, thereby avoiding data redundancy and register spilling. Finally, neural network operator fusion was implemented on the CPU, and the fusion acceleration performance was tested. Experimental results indicate that the proposed method can reduce the numbers of computational graph layers and operators effectively, and decrease memory access frequency and data to be transferred. Compared to the TVM operator fusion method, the proposed method has an average reduction of 18% in computational graph layers and the inference speed is increased by an average of 23% during the fusion process, confirming the effectiveness of the method in optimizing computational graph fusion process.

Key words: operator fusion, graph rewriting, Tensor Virtual Machine (TVM), neural network, fusion exploration

摘要:

针对计算密集型神经网络在使用张量虚拟机(TVM)算符融合过程中对计算图进行逐层查找导致访问次数过多、内存资源利用率低等问题,提出一种基于图形重写和融合探索的TVM算符融合优化方法。首先,对运算符的映射类型进行分析;其次,基于运算定律对计算图进行重写,简化计算图结构以减少中间结果生成,降低内存资源消耗并提升融合效率;再次,采用融合探索算法寻找融合代价较小的算符优先进行融合,避免数据冗余和寄存器溢出;最后,在CPU上实现神经网络算符融合,并测试融合加速性能。实验结果表明,所提方法可有效减少计算图层数和算符个数,降低访存频率和数据传输量。与TVM算符融合方法相比,所提方法在融合过程中的计算图层数平均减少18%,推理速度平均提升23%,验证了该方法在优化计算图融合过程中的有效性。

关键词: 算符融合, 图形重写, 张量虚拟机, 神经网络, 融合探索

CLC Number: