《计算机应用研究》|Application Research of Computers

基于HBase的列存储压缩策略的选择优化

Optimization of compression strategy selection method based on classification of HBase data

免费全文下载 (已被下载 次)  
获取PDF全文
作者 孙靖超,芦天亮
机构 中国人民公安大学 信息技术与网络安全学院,北京 100076
统计 摘要被查看 次,已被下载
文章编号 1001-3695(2019)05-029-1419-05
DOI 10.19734/j.issn.1001-3695.2017.12.0841
摘要 为解决现有列存储数据库压缩策略在压缩过程中遇到的数据离散程度大、分类粒度小、配套分类算法缺陷导致的学习成本高、压缩效率难以保证的问题,提出了一种基于排序的列区混合压缩策略。首先根据HBase特点设计了一种对各列数据进行排序的方法加强数据紧密度,然后根据数据特点分别使用混级区压缩策略和混级列压缩策略进行压缩策略推荐,在TPC-DS标准数据集上与前人策略进行比较。实验结果显示,该方法在压缩率、压缩/解压时间方面均有优异的表现,从而证明了其有效性。
关键词 列存储; 数据压缩; HBase; 压缩策略选择方法
基金项目 国家重点研发计划“网络空间安全”重点专项资助项目(2017YFB0802804)
国家自然科学基金资助项目(61602489)
赛尔网络下一代网络技术创新项目(NGII20160405)
本文URL http://www.arocmag.com/article/01-2019-05-029.html
英文标题 Optimization of compression strategy selection method based on classification of HBase data
作者英文名 Sun Jingchao, Lu Tianliang
机构英文名 School of Information Technology & Network Security,People's Public Security University of China,Beijing 100076,China
英文摘要 In order to solve the problem of high learning cost and low compression efficiency caused by large data dispersion, small classification granularity and the defect of applied classification algorithm encountered in the compression process of the existing column-based database compression strategy, this paper designed a sorted-based hybrid compression strategy of column-based compression and sector-based compression. Firstly, it designed a method to sort the data in each column according to the characteristics of HBase to strengthen the data compaction. Secondly, according to the characteristics of the data, it applied the hybrid column-based compression strategy and the hybrid sector-based compression strategy respectively to recommend the compression algorithm. This paper conducted experiments on TPC-DS standard data and the results demonstrate that the proposed strategy has excellent performance in both compression rate and compression/decompression time.
英文关键词 column-oriented storage; data compression; HBase; selection method of compression strategy
参考文献 查看稿件参考文献
 
收稿日期 2017/12/8
修回日期 2018/1/23
页码 1419-1423
中图分类号 TP274
文献标志码 A