面向动态负载的集群容器部署方法

doi:10.11772/j.issn.1001-9081.2020121913

计算机应用 ›› 2021, Vol. 41 ›› Issue (6): 1581-1588.DOI: 10.11772/j.issn.1001-9081.2020121913

所属专题： 2020年全国开放式分布与并行计算学术年会（DPCS 2020）

• 2020年全国开放式分布与并行计算学术年会（DPCS 2020） • 上一篇下一篇

面向动态负载的集群容器部署方法

尹飞¹, 龙玲莉¹, 孔峥¹, 邵涵², 李鑫³, 钱柱中²

1. 江苏方天电力技术有限公司, 南京 211102;
2. 南京大学计算机科学与技术系, 南京 210023;
3. 南京航空航天大学计算机科学与技术学院, 南京 211106

收稿日期:2020-11-04 修回日期:2021-04-11 出版日期:2021-06-10 发布日期:2021-06-23
通讯作者: 李鑫
作者简介:尹飞(1978-),男,安徽马鞍山人,高级工程师,主要研究方向:电力信息化、云计算;龙玲莉(1984-),女,湖北武汉人,工程师,硕士,主要研究方向:电力信息化、云计算;孔峥(1986-),男,江苏扬州人,助理工程师,主要研究方向:电力信息化、云计算;邵涵(1994-),女,安徽黄山人,博士研究生,研究方向:分布式系统、计算机网络;李鑫(1987-),男,江西南昌人,副教授,博士,CCF会员,主要研究方向:云计算、边缘计算;钱柱中(1980-),男,江苏常熟人,教授,博士,CCF会员,主要研究方向:分布式系统、数据中心网络。
基金资助:
国家自然科学基金资助项目（61802182）。

Deployment method of dockers in cluster for dynamic workload

YIN Fei¹, LONG Lingli¹, KONG Zheng¹, SHAO Han², LI Xin³, QIAN Zhuzhong²

1. Jiangsu Frontier Electric Technology Company Limited, Nanjing Jiangsu 211102, China;
2. Department of Computer Science and Technology, Nanjing University, Nanjing Jiangsu 210023, China;
3. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing Jiangsu 211106, China

Received:2020-11-04 Revised:2021-04-11 Online:2021-06-10 Published:2021-06-23
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61802182).

摘要/Abstract

摘要： 针对集群负载动态变化引发容器频繁迁移的问题，提出了一种基于资源预留的容器部署方法。首先，设计了基于马尔可夫链模型的单容器资源需求动态变化描述机制，用于刻画单容器的资源需求情况；其次，基于单容器马尔可夫链模型分析了多容器资源动态变化情况，以刻画容器资源需求态势；随后，基于多容器马尔可夫链提出了面向动态负载的容器部署与资源预留算法；最后，基于容器资源需求特征的分析对所提算法的性能进行了优化。基于国产软硬件环境构建了仿真实验环境，仿真结果表明，在资源冲突率方面，所提方法的性能接近最优的峰值配置策略RP，但所需宿主机数量、容器动态迁移次数明显比其更少；在资源利用率方面，所提方法的宿主机使用数量略多于最优的谷值配置策略RV，但动态迁移次数更少，资源冲突率更低；相较于峰谷配置策略RVP，所提方法在综合性能方面更佳。

关键词: 集群, 容器, 动态负载, 整合与迁移, 马尔可夫链

Abstract: Aiming at the problem of frequent migration of containers triggered by dynamic changes of cluster workload, a container deployment method based on resource reservation was proposed. Firstly, a dynamic change description mechanism of single-container resource demand based on Markov chain model was designed to describe the resource demand situation of single container. Secondly, the dynamic change of multi-container resource was analyzed based on the single-container Markov chain model to describe the container resource demand state. Thirdly, a container deployment and resource reservation algorithm for dynamic workload was proposed based on the multi-container Markov chain. Finally, the performance of the proposed algorithm was optimized based on the analysis of container resource demand characteristics. The simulation experimental environment was constructed based on the domestic software and hardware environment, and the simulation results show that in terms of resource conflict rate, the performance of the proposed method has the performance close to the optimal peak allocation strategy named Resource with Peak (RP), but its number of required hosts and container dynamic migration number are significantly less; in terms of resource utilization rate, the proposed method has the number of hosts used slightly more than the optimal valley allocation strategy named Resource with Valley (RV), but has less dynamic migration number and lower resource conflict rate; compared with the peak and valley allocation strategy named Resource with Valley and Peak (RVP), the proposed method has better comprehensive performance.

Key words: cluster, docker, dynamic workload, consolidation and migration, Markov chain

中图分类号:

TP315

尹飞, 龙玲莉, 孔峥, 邵涵, 李鑫, 钱柱中. 面向动态负载的集群容器部署方法[J]. 计算机应用, 2021, 41(6): 1581-1588.

YIN Fei, LONG Lingli, KONG Zheng, SHAO Han, LI Xin, QIAN Zhuzhong. Deployment method of dockers in cluster for dynamic workload[J]. Journal of Computer Applications, 2021, 41(6): 1581-1588.

参考文献

[1] XIE G,ZENG G,LI R,et al. Quantitative fault-tolerance for reliable workflows on heterogeneous IaaS clouds[J]. IEEE Transactions on Cloud Computing,2020,8(4):1223-1236.
[2] SHARIFFDEEN R S,MUNASINGHE D T S P,BHATHIYA H S, et al. Workload and resource aware proactive auto-scaler for PaaS cloud[C]//Proceedings of the 2016 IEEE 9th International Conference on Cloud Computing. Piscataway:IEEE,2016:11-18.
[3] KLIMOVIC A,WANG Y,STUEDI P,et al. Pocket:elastic ephemeral storage for serverless analytics[C]//Proceedings of the 2018 13th USENIX Symposium on Operating Systems Design and Implementation. Berkeley:USENIX Association,2018:427-444.
[4] YI B,WANG X,LI K,et al. A comprehensive survey of network function virtualization[J]. Computer Networks, 2018, 133:212-262.
[5] HUANG Z,WU S,JIANG S,et al. FastBuild:accelerating docker image building for efficient development and deployment of container[C]//Proceedings of the 2019 35th Symposium on Mass Storage Systems and Technologies. Piscataway:IEEE, 2019:28-37.
[6] AJIRO Y,TANAKA A. Improving packing algorithms for server consolidation[C]//Proceedings of the 2007 33rd International Computer Measurement Group Conference. Sicklerville:CMG, 2007:399-406.
[7] RAMPERSAUD S, GROSU D. Sharing-aware online virtual machine packing in heterogeneous resource clouds[J]. IEEE Transactions on Parallel and Distributed Systems,2017,28(7):2046-2059.
[8] ARDELEAN D,DIWAN A,ERDMAN C. Performance analysis of cloud applications[C]//Proceedings of the 2018 15th USENIX Symposium on Networked Systems Design and Implementation. Berkeley:USENIX Association,2018:405-417.
[9] CORTEZ E,BONDE A,MUZIO A,et al. Resource central:understanding and predicting workloads for improved resource management in large cloud platforms[C]//Proceedings of the 2017 26th Symposium on Operating Systems Principles. New York:ACM,2017:153-167.
[10] LIU L,XU H. Elasecutor:elastic executor scheduling in data analytics systems[C]//Proceedings of the 2018 ACM Symposium on Cloud Computing. New York:ACM,2018:107-120.
[11] SILVA FILHO M C,MONTEIRO C C,INÁCIO P R M,et al. Approaches for optimizing virtual machine placement and migration in cloud environments:a survey[J]. Journal of Parallel and Distributed Computing,2018,111:222-250.
[12] VMware. Distributed resource scheduler and management[EB/OL].[2020-06-18]. https://www.vmware.com/products/vsphere.html.
[13] IBM. IBM system planning tool for POWER processor-based systems[EB/OL].[2020-06-17]. https://www.ibm.com/support/pages/node/632621.
[14] WANG M,MENG X,ZHANG L. Consolidating virtual machines with dynamic bandwidth demand in data centers[C]//Proceedings of the 2011 IEEE International Conference on Computer Communications. Piscataway:IEEE,2011:71-75.
[15] BREITGAND D,EPSTEIN A. Improving consolidation of virtual machines with risk-aware bandwidth oversubscription in compute clouds[C]//Proceedings of the 2012 IEEE International Conference on Computer Communications. Piscataway:IEEE, 2012:2861-2865.
[16] RODRIGUES T G,SUTO K,NISHIYAMA H,et al. Hybrid method for minimizing service delay in edge cloud computing through VM migration and transmission power control[J]. IEEE Transactions on Computers,2017,66(5):810-819.
[17] TSAKALOZOS K,VERROIOS V,ROUSSOPOULOS M,et al. Live VM migration under time-constraints in share-nothing IaaSclouds[J]. IEEE Transactions on Parallel and Distributed Systems,2017,28(8):2285-2298.
[18] LE F, NAHUM E M. Experiences implementing live VM migration over the WAN with multi-path TCP[C]//Proceedings of the 2019 IEEE International Conference on Computer Communications. Piscataway:IEEE,2019:1090-1098.
[19] MI N,CASALE G,CHERKASOVA L,et al. Injecting realistic burstiness to a traditional client-server benchmark[C]//Proceedings of the 2009 6th International Conference on Autonomic Computing. New York:ACM,2009:149-158.
[20] BAARZI A F, ZHU T, URGAONKAR B. BurScale:using burstable instances for cost-effective auto-scaling in the public cloud[C]//Proceedings of the 2019 ACM Symposium on Cloud Computing. New York:ACM,2019:126-138.
[21] YU L,CHEN L,CAI Z,et al. Stochastic load balancing for virtual resource management in datacenters[J]. IEEE Transactions on Cloud Computing,2020,8(2):459-472.
[22] JIN H,PAN D,XU J,et al. Efficient VM placement with multiple deterministic and stochastic resources in data centers[C]//Proceedings of the 2012 IEEE Global Communications Conference. Piscataway:IEEE,2012:2505-2510.
[23] ZHANG S,QIAN Z,LUO Z,et al. Burstiness-aware resource reservation for server consolidation in computing clouds[J]. IEEE Transactions on Parallel and Distributed Systems,2016,27(4):964-977.
[24] ZHAO N,ALBAHAR H,ABRAHAM S,et al. DupHunter:flexible high-performance deduplication for docker registries[C]//Proceedings of the 2020 USENIX Annual Technical Conference. Berkeley:USENIX Association,2020:769-783.
[25] 李启锐, 彭志平, 崔得龙, 等. 容器云环境虚拟资源配置策略的优化[J]. 计算机应用, 2019, 39(3):784-789.(LI Q R,PENG Z P,CUI D L,et al. Optimization of virtual resource deployment strategy in container cloud[J]. Journal of Computer Applications, 2019,39(3):784-789.)

面向动态负载的集群容器部署方法

Deployment method of dockers in cluster for dynamic workload

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	沈忱, 邰凌翔, 彭煜玮. 面向自动参数调优的动态负载匹配方法[J]. 计算机应用, 2021, 41(3): 657-661.
[2]	律睿慜, 梅莉琳, 邢红姹, 孟磊, 昃跃峰. 基于马尔可夫链的书画时序感量化方法[J]. 计算机应用, 2021, 41(1): 295-299.
[3]	杜旭升, 于炯, 叶乐乐, 陈嘉颖. 基于图上随机游走的离群点检测算法[J]. 计算机应用, 2020, 40(5): 1322-1328.
[4]	刘福鑫, 李劲巍, 王熠弘, 李琳. 基于Kubernetes的云原生海量数据存储系统设计与实现[J]. 计算机应用, 2020, 40(2): 547-552.
[5]	赵旭浩, 王轶群, 刘健, 徐春晖. 基于能量约束的自主水下航行器任务规划算法[J]. 计算机应用, 2019, 39(9): 2529-2534.
[6]	李启锐, 彭志平, 崔得龙, 何杰光. 容器云环境虚拟资源配置策略的优化[J]. 计算机应用, 2019, 39(3): 784-789.
[7]	王志伟, 杨超. 基于流量控制的Docker容器网络带宽控制机制[J]. 计算机应用, 2019, 39(12): 3628-3632.
[8]	毛莺池, 郝帅, 平萍, 戚荣志. 基于组合双向拍卖的云资源调度方法[J]. 计算机应用, 2019, 39(1): 1-7.
[9]	黄凯, 孟庆永, 谢雨来, 冯丹, 秦磊华. 基于Docker swarm集群的动态加权调度策略[J]. 计算机应用, 2018, 38(5): 1399-1403.
[10]	付朝江, 王天奇, 林悦荣. 基于有效并行求解策略的显式有限元分析并行算法[J]. 计算机应用, 2018, 38(4): 1072-1077.
[11]	彭丽苹, 吕晓丹, 蒋朝惠, 彭成辉. 基于Docker的云资源弹性调度策略[J]. 计算机应用, 2018, 38(2): 557-562.
[12]	唐俊勇, 田鹏辉, 王辉. 基于马尔可夫链与服务质量的网络可用性[J]. 计算机应用, 2018, 38(12): 3518-3523.
[13]	王日宏, 崔兴梅. 融合集群度与距离均衡优化的K-均值聚类算法[J]. 计算机应用, 2018, 38(1): 104-109.
[14]	王梓懿, 安俊秀, 王鹏. 基于多尺度量子谐振子算法的相空间概率聚类算法[J]. 计算机应用, 2017, 37(8): 2218-2222.
[15]	王中强, 陈继德, 彭舰, 黄飞虎, 仝博. 基于改进马尔可夫链的航线预测算法[J]. 计算机应用, 2017, 37(7): 2124-2128.