《计算机应用研究》|Application Research of Computers

Text-CRNN+attention架构下的多类别文本信息分类

Multi-category text information classification with Text-CRNN+attention architecture

免费全文下载 (已被下载 次)  
获取PDF全文
作者 卢健,马成贤,杨腾飞,周嫣然
机构 西安工程大学 电子信息学院,西安 710048
统计 摘要被查看 次,已被下载
文章编号 1001-3695(2020)06-018-1693-04
DOI 10.19734/j.issn.1001-3695.2018.12.0858
摘要 迄今为止,传统机器学习方法依赖人工提取特征,复杂度高;深度学习网络本身特征表达能力强,但模型可解释性弱导致关键特征信息丢失。为此,以网络层次结合的方式设计了CRNN并引入attention机制,提出一种Text-CRNN+attention模型用于文本分类。首先利用CNN处理局部特征的位置不变性,提取高效局部特征信息;然后在RNN进行序列特征建模时引入attention机制对每一时刻输出序列信息进行自动加权,减少关键特征的丢失,最后完成时间和空间上的特征提取。实验结果表明,提出模型较其他模型准确率提升了2%~3%;在提取文本特征时,该模型既保证了数据的局部相关性又起到强化序列特征的有效组合能力。
关键词 文本分类; 卷积神经网络; 循环神经网络; convolutional recurrent neural network; 注意力机制
基金项目 国家自然科学基金资助项目(51607133)
陕西省教育厅专项科学研究计划项目(17JK0332)
陕西省科技厅科技发展计划项目(2011K06-01)
西安市碑林区应用技术研发项目(GX1807)
本文URL http://www.arocmag.com/article/01-2020-06-018.html
英文标题 Multi-category text information classification with Text-CRNN+attention architecture
作者英文名 Lu Jian, Ma Chengxian, Yang Tengfei, Zhou Yanran
机构英文名 School of Electronic & Information,Xi'an Polytechnic University,Xi'an 710048,China
英文摘要 In view of the current research process, traditional machine learning methods rely on manual feature extraction with high complexity. Deep learning network has strong feature expression ability, but the model is weak in interpretability, leading to the loss of key feature information. For this reason, this paper designed the CRNN in the way of network level combination, introduced attention mechanism and proposed a Text-CRNN+attention model. Firstly, it used CNN to deal with the position invariance of local features and extracted efficient local feature information. Then, it introduced attention mechanism to automatically weigh the output sequence information at each time to reduce the loss of key features when RNN was used to model the sequence features. It completed the feature extraction in time and space. The experimental results show that the accuracy of the proposed model is 2 to 3 percentage points higher than that of other models. When dealing with text data, the model not only guarantees the local correlation of data, but also strengthens the effective combination ability of sequence features.
英文关键词 text classification; convolutional neural network(CNN); recurrent neural network(RNN); convolutional recurrent neural network(CRNN); attention mechanism
参考文献 查看稿件参考文献
 
收稿日期 2018/12/1
修回日期 2019/2/26
页码 1693-1696,1701
中图分类号 TP393.04
文献标志码 A