关键词: 云平台任务调度, 服务质量, 自适应, 任务抢占, Spark

Abstract: Aiming at the problem that the response speed and the completion time of the task cannot be taken into account at the same time when scheduling tasks in a shared cluster， an adaptive scheduling algorithm based on deadline was proposed. In the algorithm， based on the deadline submitted by the user， the appropriate computing resources were allocated adaptively according to the execution progress of the tasks. Different from that fixed resource parameters were submitted by users in the traditional scheduling methods， in this algorithm， tasks with high priority would be executed with preemptive scheduling under resource constraints. Preemptive scheduling was used to ensure the Quality of Service （QoS）， and additional resources would be allocated to compensate the preempted tasks after the preemption process. The task scheduling experimental results on the Spark platform show that compared with the scheduling algorithm under Yet Another Resource Negotiator （YARN） framework， the proposed algorithm can control the response speed of short tasks strictly and shorten the task completion time of long jobs by 35%.

Key words: cloud platform task scheduling, Quality of Service (QoS), adaptive, task preemption, Spark