《计算机应用》唯一官方网站 ›› 2023, Vol. 43 ›› Issue (1): 176-184.DOI: 10.11772/j.issn.1001-9081.2021112018

所属专题: 先进计算

• 先进计算 • 上一篇    下一篇

云平台下基于截止时间的自适应调度策略

吴仁彪, 张振驰, 贾云飞, 乔晗   

  1. 天津市智能信号与图像处理重点实验室(中国民航大学),天津 300300
  • 收稿日期:2021-11-29 修回日期:2022-05-06 发布日期:2023-01-12
  • 通讯作者: 吴仁彪(1966—),男,湖北武汉人,教授,博士生导师,博士,主要研究方向:自适应信号处理,现代谱分析及其在雷达、卫星导航和空管中的应用rbwu@cauc.edu.cn
  • 作者简介:张振驰(1996—),男,湖北天门人,硕士研究生,主要研究方向:大数据、分布式计算;贾云飞(1979—),男,河北石家庄人,副教授,博士,主要研究方向:云计算、民航运输大数据工程;乔晗(1997—),女,吉林辽源人,硕士研究生,主要研究方向:自然语言处理;
  • 基金资助:
    天津市研究生科研创新项目(2020YJSS008)。

Adaptive scheduling strategy based on deadline under cloud platform

WU Renbiao, ZHANG Zhenchi, JIA Yunfei, QIAO Han   

  1. Tianjin Key Laboratory of Advanced Signal Processing (Civil Aviation University of China), Tianjin 300300, China
  • Received:2021-11-29 Revised:2022-05-06 Online:2023-01-12
  • Contact: WU Renbiao, born in 1966, Ph. D., professor. His research interests include adaptive signal processing, modern spectrum analysis and its applications in radar, satellite navigation and air traffic control.
  • About author:ZHANG Zhenchi, born in 1996, M. S. candidate. His research interests include big data, distributed computing;JIA Yunfei, born in 1979, Ph. D., associate professor. His research interests include cloud computing, civil aviation transportation big data engineering;QIAO Han, born in 1997, M. S. candidate. Her research interests include natural language processing;
  • Supported by:
    This work is partially supported by Tianjin Research Innovation Project for Postgraduate Students (2020YJSS008).

摘要: 针对在共享集群中进行任务调度时,无法兼顾任务的响应速度与任务完成时间的问题,提出一种基于截止时间的自适应调度算法。该算法以用户提交的截止时间为依据,根据任务的执行进度自适应地分配适当的计算资源。不同于传统调度方式里由用户提交固定资源参数,该算法在资源约束的情况下会对优先级高的任务进行抢占式调度以保证服务质量(QoS),并在抢占过程结束后额外分配资源补偿被抢占的任务。在Spark平台进行的任务调度实验结果显示,与另一种资源协调者(YARN)框架下的调度算法相比,所提算法能严格地控制短任务的响应速度,并使长作业的任务完成时间缩短35%。

关键词: 云平台任务调度, 服务质量, 自适应, 任务抢占, Spark

Abstract: Aiming at the problem that the response speed and the completion time of the task cannot be taken into account at the same time when scheduling tasks in a shared cluster, an adaptive scheduling algorithm based on deadline was proposed. In the algorithm, based on the deadline submitted by the user, the appropriate computing resources were allocated adaptively according to the execution progress of the tasks. Different from that fixed resource parameters were submitted by users in the traditional scheduling methods, in this algorithm, tasks with high priority would be executed with preemptive scheduling under resource constraints. Preemptive scheduling was used to ensure the Quality of Service (QoS), and additional resources would be allocated to compensate the preempted tasks after the preemption process. The task scheduling experimental results on the Spark platform show that compared with the scheduling algorithm under Yet Another Resource Negotiator (YARN) framework, the proposed algorithm can control the response speed of short tasks strictly and shorten the task completion time of long jobs by 35%.

Key words: cloud platform task scheduling, Quality of Service (QoS), adaptive, task preemption, Spark

中图分类号: