Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (2): 342-346.DOI: 10.11772/j.issn.1001-9081.2019081406

• DPCS 2019 • Previous Articles     Next Articles

Inference delay optimization of branchy neural network model based on edge computing

Qi FAN1,2, Zhuo LI1,2(), Xin CHEN2   

  1. 1.Beijing Key Laboratory of Internet Culture and Digital Dissemination Research (Beijing Information Science & Technology University),Beijing 100101,China
    2.School of Computer Science,Beijing Information Science & Technology University,Beijing 100101,China
  • Received:2019-07-31 Revised:2019-09-05 Accepted:2019-09-23 Online:2020-02-26 Published:2020-02-10
  • Contact: Zhuo LI
  • About author:FAN Qi, born in 1995, M. S. candidate. His research interests include edge computing.
    CHEN Xin, born in 1965, Ph. D., professor. His research interests include network performance evaluation, network security.
  • Supported by:
    the National Natural Science Foundation of China(61502040);the Beijing Municipal Program for Top Talent Cultivation(CIT&TCD201804055);the Qinxin Talent Program of Beijing Information Science and Technology University, the Open Program of Beijing Key Laboratory of Internet Culture and Digital Dissemination Research(ICDDXN001)

基于边缘计算的分支神经网络模型推断延迟优化

樊琦1,2, 李卓1,2(), 陈昕2   

  1. 1.网络文化与数字传播北京市重点实验室(北京信息科技大学),北京 100101
    2.北京信息科技大学 计算机学院,北京 100101
  • 通讯作者: 李卓
  • 作者简介:樊琦(1995—),男,山西运城人,硕士研究生,主要研究方向:边缘计算
    陈昕(1965—),男,江西南昌人,教授,博士,CCF会员,主要研究方向:网络性能评价、网络安全。
  • 基金资助:
    国家自然科学基金资助项目(61502040);北京市属高校高水平教师队伍建设支持计划青年拔尖人才培育计划资助项目(CIT&TCD201804055);北京信息科技大学“勤信人才”培养计划资助项目;网络文化与数字传播北京市重点实验室开放课题资助项目(ICDDXN001)

Abstract:

Aiming at the long delay of inference tasks in Deep Neural Network (DNN) on cloud servers, a branchy neural network deployment model based on edge computing was proposed. The distributed deployment problem of DNNs in edge computing scenarios was analyzed, and was proved to be NP-hard. A Deployment algorithm based on Branch and Bound (DBB) was designed to select appropriate edge computing nodes to reduce inference delay. And a Selection Node Exit (SNE) algorithm was designed and implemented to select the appropriate edge computing nodes for different tasks to exit the inference task. The simulation results show that, compared with the approach of deploying neural network model on the cloud, the branchy neural network model based on edge computing reduces the inference delay by 36% on average.

Key words: edge computing, branchy neural network, Deep Neural Network (DNN), inference delay, deployment problem

摘要:

针对云服务器上深度神经网络(DNN)模型推断任务延迟过高的问题,提出基于边缘计算的分支神经网络部署模型。分析了边缘计算场景中深度神经网络的分布式部署问题,证明该问题是NP-难的。设计了一种基于分支定界思想的部署算法(DBB),选择合适的边缘计算节点部署模型以减少推断任务的延迟。设计并实现了选择节点退出(SNE)算法,为不同任务选择合适的边缘计算节点来退出推断任务。仿真实验结果表明,与在云端部署神经网络模型的方法相比,基于边缘计算的分支神经网络模型的推断延迟平均降低了36%。

关键词: 边缘计算, 分支神经网络, 深度神经网络, 推断延迟, 部署问题

CLC Number: