HSWAP: numerical simulation workflow management platform suitable for high performance computing environment

doi:10.11772/j.issn.1001-9081.2018122606

Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (6): 1569-1576.DOI: 10.11772/j.issn.1001-9081.2018122606

• 2018 National Annual Conference on High Performance Computing (HPC China 2018) • Previous Articles Next Articles

HSWAP: numerical simulation workflow management platform suitable for high performance computing environment

ZHAO Shicao, XIAO Yonghao, DUAN Bowen, LI Yufeng

Institute of Computer Application, China Academy of Engineering Physics, Mianyang Sichuan 621900, China

Received:2018-12-12 Revised:2019-03-01 Online:2019-06-10 Published:2019-06-17
Supported by:
This work is partially supported by the National Key R&D Program of China (2018YFB0703900).

HSWAP:适用于高性能计算环境的数值模拟工作流管理平台

赵士操, 肖永浩, 段博文, 李于锋

中国工程物理研究院计算机应用研究所, 四川绵阳 621900

通讯作者: 赵士操
作者简介:赵士操(1986-),男,陕西西安人,高级工程师,硕士,CCF会员,主要研究方向:科学工作流、软件架构设计、高性能数值模拟应用;肖永浩(1981-),男,甘肃永靖人,高级工程师,博士,CCF会员,主要研究方向:并行计算、科学工作流;段博文(1990-),男,黑龙江望奎人,工程师,硕士,CCF会员,主要研究方向:高性能计算服务;李于锋(1982-),男,河南光山人,高级工程师,博士研究生,CCF会员,主要研究方向:并行计算、调度算法。
基金资助:
国家重点研发计划项目（2018YFB0703900）。

Abstract

Abstract: Concerning the construction of integrated application of "modeling, computation, analysis, optimization" workflow under High Performance Computing (HPC) environment, HPC Simulation Workflow Application Platform (HSWAP) supporting numerical simulation software encapsulation and numerical simulation workflow interaction design was developed. Firstly, based on the modeling of runtime characteristics of numerical simulation activities, the component model was built. Then, the control and data dependency relationships between simulation activities were represented by the workflow, creating a formal numerical simulation workflow model. The formed workflow model was able to be automatically parsed in the platform to adapt to HPC resources. Therefore, HSWAP platform could be used for automatic generation and scheduling of a batch of related numerical simulation tasks, screening technical details of HPC resources from domain users. The platform provided Web Portal services, which supports the push of interactive interfaces of graphical numerical simulation programs. The platform is already deployed and applied at Supercomputing Center and with this platform, the integration of numerical simulation workflows with up to 10 numerical simulation softwares and 20 computing task nodes can be completed in 2 person-month.

Key words: scientific workflow, numerical simulation, software integration, High Performance Computing (HPC), Web service

摘要： 针对高性能计算（HPC）环境中的"建模、计算、分析、优化"一体化应用构建的问题，设计了支持数值模拟软件封装和数值模拟工作流交互设计的数值模拟工作流管理平台——HSWAP。首先，基于对数值模拟活动的运行特征共性建模构建组件模型；然后，利用工作流表达数值模拟活动间的控制、数据依赖关系，建立形式化的数值模拟工作流模型，所形成的工作流模型可在平台中自动解析并适配高性能计算资源，从而实现批量关联数值模拟任务的自动生成与调度，为领域用户屏蔽高性能计算资源的使用细节。平台提供Web Portal服务，支持图形数值模拟程序的交互界面推送。目前该平台已在超算中心实际生产环境得到部署应用，可在2人月内完成包含10个以下数值模拟软件、20个以内计算任务节点的数值模拟工作流的集成。

关键词: 科学工作流, 数值模拟, 软件集成, 高性能计算, Web服务

CLC Number:

TP391.9

ZHAO Shicao, XIAO Yonghao, DUAN Bowen, LI Yufeng. HSWAP: numerical simulation workflow management platform suitable for high performance computing environment[J]. Journal of Computer Applications, 2019, 39(6): 1569-1576.

赵士操, 肖永浩, 段博文, 李于锋. HSWAP:适用于高性能计算环境的数值模拟工作流管理平台[J]. 计算机应用, 2019, 39(6): 1569-1576.

References

[1] 曹荣强,王小宁,卢莎莎,等.基于Portlet的高性能计算应用集成组件[J].科研信息化技术与应用,2017,8(2):18-30.(CAO R Q, WANG X N, LU S S, et al. High performance computing application integration toolkits based on Portlet[J]. e-Science Technology & Application, 2017, 8(2):18-30.)
[2] TOWNS J, COCKERILL T, DAHAN M, et al. XSEDE:accelerating scientific discovery[J]. Computing in Science & Engineering, 2014, 16(5):62-74.
[3] 迟学斌,肖海力,王小宁,等.国家重点研发计划助力国家高性能计算环境服务化建设迈上新台阶[J].科研信息化技术与应用,2016,7(4):84-88.(CHI X B, XIAO H L, WANG X N, et al. A further promoting toward building China national grid supported by the national key research and development program of China[J]. e-Science Technology & Application, 2016, 7(4):84-88.)
[4] TAYLOR I J, DEELMAN E, GANNON D B, et al. Workflows for e-Science:Scientific Workflows for Grids[M]. Berlin:Springer, 2007:9-16.
[5] DEELMAN E, SINGH G, SU M, et al. Pegasus:a framework for mapping complex scientific workflows onto distributed systems[J]. Scientific Programming, 2005, 13(3):219-237.
[6] BERMAN F. From TeraGrid to knowledge grid[J]. Communications of the ACM, 2001, 44(11):27-28.
[7] FOSTER I, KESSELMAN C. Globus:a metacomputing infrastructure toolkit[J]. The International Journal of Supercomputer Applications and High Performance Computing, 1997, 11(2):115-128.
[8] 吴响,邓笋根,陆忠华.国内外科学工作流综述研究[J].科研信息化技术与应用,2014,5(5):86-95.(WU X, DENG S G, LU Z H. A review of the study on the scientific workflow[J]. e-Science Technology & Application, 2014, 5(5):86-95.)
[9] ATKINSON M, GESING S, MONTAGNAT J, et al. Scientific workflows:past, present and future[J]. Future Generation Computer Systems, 2017, 75:216-227.
[10] YU J, BUYYA R. A taxonomy of scientific workflow systems for grid computing[J]. ACM SIGMOD Record, 2005, 34(3):44-49.
[11] DEELMAN E, PETERKA T, ALTINTAS I, et al. The future of scientific workflows[J]. The International Journal of High Performance Computing Applications, 2018, 32(1):159-175.
[12] 李于锋,莫则尧,肖永浩,等.超算环境科学工作流应用平台的引擎设计和资源调度[J].计算机应用研究,2019,36(7):1-7.(LI Y F, MO Z Y, XIAO Y H,et al. Engine design and resource scheduling of scientific workflow application platform in supercomputing[J]. Application Research of Computers, 2019, 36(7):1-7.)
[13] LU S, PAI D, HUA J, et al. A task abstraction and mapping approach to the shimming problem in scientific workflows[C]//Proceedings of the 2009 IEEE International Conference on Services Computing. Piscataway, NJ:IEEE, 2009:284-291.
[14] HUERTA E A, HAAS R, JHA S, et al. Supporting high-performance and high-throughput computing for experimental science[J]. Computing and Software for Big Science, 2019, 3(1):5.
[15] JAIN A, ONG S P, CHEN W, et al. FireWorks:a dynamic workflow system designed for high-throughput applications[J]. Concurrency and Computation:Practice and Experience, 2015, 27(17):5037-5059.

HSWAP: numerical simulation workflow management platform suitable for high performance computing environment

HSWAP:适用于高性能计算环境的数值模拟工作流管理平台

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	LI Yan, CHEN Qiaoping. Internet rumor propagation model considering non-supportive comments [J]. Journal of Computer Applications, 2021, 41(4): 1128-1135.
[2]	WANG Peng, ZHOU Yan. MPI big data processing for high performance applications [J]. Journal of Computer Applications, 2018, 38(12): 3496-3499.
[3]	DENG Yun, LI Chaoqing, CHEN Xiaohui. Design of remote wireless monitoring system for smart home based on Internet of things [J]. Journal of Computer Applications, 2017, 37(1): 159-165.
[4]	XIE Qi, CUI Mengtian. Web service recommendation for user group [J]. Journal of Computer Applications, 2016, 36(6): 1579-1582.
[5]	LI Shaolei, ZHENG Jianjing, SHANG Mengmeng. Numerical simulation of flight vehicle multi-body separation based on unstructured mesh [J]. Journal of Computer Applications, 2016, 36(6): 1741-1744.
[6]	HE Zhengqiu, ZHANG Yelin, XU Junkui, SUN Danhui. Research on access control policy for Web service [J]. Journal of Computer Applications, 2015, 35(8): 2184-2188.
[7]	NI Zhiwei, FANG Qinghua, LI Rongrong, LI Yiming. Improved ant colony optimization for QoS-based Web service composition optimization [J]. Journal of Computer Applications, 2015, 35(8): 2238-2243.
[8]	DUAN Ju, CHEN Wanghu, WANG Runping, YU Maoyi, WANG Shikai. Execution optimization policy of scientific workflow based on cluster aggregation under cloud environment [J]. Journal of Computer Applications, 2015, 35(6): 1580-1584.
[9]	HOU Jinkui, WANG Lei. Formal model supporting Web service composition and verification [J]. Journal of Computer Applications, 2015, 35(6): 1773-1779.
[10]	TIAN Qiang XIA Yongying FU Xiaodong LI Changzhi WANG Wei. Reliability optimization approach for Web service composition based on cost benefit coefficient [J]. Journal of Computer Applications, 2014, 34(3): 683-689.
[11]	XUE Peng REN Pengfei WANG Hua. Washout control and stability analysis for cyclic traffic flows [J]. Journal of Computer Applications, 2014, 34(2): 597-600.
[12]	ZHANG Xuan LIU Cong WANG Lixia ZHAO Qian YANG Shuai. Trustworthy Web service recommendation based on collaborative filtering [J]. Journal of Computer Applications, 2014, 34(1): 213-217.
[13]	LI` Chunyan ZHANG Xuejie. Performance evaluation on open source cloud platform for high performance computing [J]. Journal of Computer Applications, 2013, 33(12): 3580-3585.
[14]	WANG Wei FU Xiaodong XIA Yongying TIAN Qiang LI Changzhi. Score distribution method for Web service composition [J]. Journal of Computer Applications, 2013, 33(11): 3252-3256.
[15]	WEI Haoyang ZENG Guosun DING Chunling. Parallel computing and numerical analysis of laminar diffusion combustion on GPU [J]. Journal of Computer Applications, 2013, 33(09): 2428-2431.