《计算机应用研究》|Application Research of Computers

Heron环境下基于实例重分配的传输负载优化策略

Transmission load optimization strategy based on instance reallocation in Heron

免费全文下载 (已被下载 次)  
获取PDF全文
作者 刘宇,于炯,蒲勇霖,李梓杨,张译天
机构 1.新疆大学 软件学院,乌鲁木齐 830091;2.新疆大学 信息科学与工程学院,乌鲁木齐 830046
统计 摘要被查看 次,已被下载
文章编号 1001-3695(2021)01-039-0198-06
DOI 10.19734/j.issn.1001-3695.2019.12.0638
摘要 作为新一代大数据流式计算框架,Heron忽略了任务实例之间不同通信方式的差异以及节点资源利用率不均衡的问题导致系统性能下降。针对这一问题,设计了节点资源限制模型、通信开销优化模型和实例数据流关系模型,并在此基础上提出了Heron环境下基于实例重分配的传输负载优化策略(transmission load optimization strategy based on instance reallocation in Heron,TLIR-Heron)。该策略包括节点资源限制算法和实例重分配算法,通过判定实例重分配条件并执行重分配算法将节点间数据流转换为节点内数据流,从而降低通信开销。实验结果表明,在三组拓扑测试下,TLIR-Heron相较于Heron默认调度策略能够降低节点间通信开销和系统的计算延迟,并提升了计算节点资源利用的均衡性。
关键词 大数据; 流式计算; Apache Heron; 资源限制; 通信开销
基金项目 国家自然科学基金资助项目(61862060,61462079,61562086,61562078)
国家科技部科技支撑基金资助项目(2015BAH02F01)
新疆大学博士生科技创新资助项目(XJUBSCX-201902)
本文URL http://www.arocmag.com/article/01-2021-01-039.html
英文标题 Transmission load optimization strategy based on instance reallocation in Heron
作者英文名 Liu Yu, Yu Jiong, Pu Yonglin, Li Ziyang, Zhang Yitian
机构英文名 1.School of Software,Xinjiang University,Urumqi 830091,China;2.College of Information Science & Engineering,Xinjiang University,Urumqi 830046,China
英文摘要 As a new platform in big data stream computing, Apache Heron ignores the difference in communication modes between task instances and the unbalance of processing load among nodes, which leads to the decline system performance. To address the problem, this paper designed the model of node resource limitation, the model of communication overhead optimization and the model of data stream relationships among instances, as the foundation to propose the TLIR-Heron. The strategy was composed of the node resource limitation algorithm and the instance reallocation algorithm. By judging the criteria for instance reallocation and executing instance reallocation algorithm, this strategy transformed the inter-node data streams into intra-node data streams and minimized the communication overhead of the system. The experimental results show that under the three sets of benchmarks, TLIR-Heron reduces the communication overhead between nodes and the response latency of the system compared with the default scheduling strategy, and improves the balance of resource utilization of computing nodes.
英文关键词 big data; stream computing; Apache Heron; resource limitation; communication overhead
参考文献 查看稿件参考文献
 
收稿日期 2019/12/1
修回日期 2020/2/3
页码 198-203
中图分类号 TP301.4
文献标志码 A