《计算机应用研究》|Application Research of Computers

超算环境科学工作流应用平台的引擎设计和资源调度

Engine design and resource scheduling of scientific workflow application platform in supercomputing

免费全文下载 (已被下载 次)  
获取PDF全文
作者 李于锋,莫则尧,肖永浩,赵士操,段博文
机构 1.中国工程物理研究院计算机应用研究所,四川 绵阳 621900;2.北京应用物理与计算数学研究所,北京 100094
统计 摘要被查看 次,已被下载
文章编号 1001-3695(2019)06-026-1723-04
DOI 10.19734/j.issn.1001-3695.2018.05.0263
摘要 围绕超算资源的易用性和多类软件的集成以及协作需求,开发了超算环境下的科学工作流应用平台,设计了异步并发的流程执行引擎,采取调度算法和调度器、引擎相分离的设计策略,给出了资源调度方案。提出了局部资源池化技术和资源预约算法,并比较分析了五种常用调度算法的性能,给出了算法选择的建议。实际应用表明设计的引擎能够支撑复杂工作流的灵活执行方式,给出的资源调度方案能够满足超算环境下工作流应用的高效执行。
关键词 科学工作流; 高性能计算; 资源调度; 工作流引擎
基金项目 国家重点研发计划资助项目(2016YFB0201504,2018YFB0703903)
本文URL http://www.arocmag.com/article/01-2019-06-026.html
英文标题 Engine design and resource scheduling of scientific workflow application platform in supercomputing
作者英文名 Li Yufeng, Mo Zeyao, Xiao Yonghao, Zhao Shicao, Duan Bowen
机构英文名 1.Institute of Computer Application,Chinese Academy of Engineering Physics,Mianyang Sichuan 621900,China;2.Institute of Applied Physics & Computational Mathematics,Beijing 100094,China
英文摘要 In general, lots of software need to be cooperated together for particular object in scientific experiments and engineering domains. This paper described a new scientific workflow application platform in HPC environment. This platform contained an engine with high concurrency and asynchronous framework to process workflow application. Scheduler, planner and engine were decoupled from each other, which allowed that the three components could develop independently: scheduler for scheduling algorithms implementation, planner for collecting the scheduling information and engine for workflow driver. Scheduler and planner used resource advance reservation and local pooling mechanism to increase the performance of workflow execution. This paper also implemented and compared five scheduling algorithms as to their performance on testing graph set, and got some useful advices of algorithm selection. Real application shows that engine can support various execution strategies and resource scheduling solution can help increase efficiency of workflow execution in supercomputing environment.
英文关键词 scientific workflow; high performance computing; resource scheduling; workflow engine
参考文献 查看稿件参考文献
 
收稿日期 2018/5/9
修回日期 2018/6/21
页码 1723-1726,1736
中图分类号 TP391
文献标志码 A