《计算机应用研究》|Application Research of Computers

深层自动编码机的文本分类算法改进

Text-classification algorithm improvements based on deep autoencoder

免费全文下载 (已被下载 次)  
获取PDF全文
作者 胡侯立,魏维,谢青松
机构 西安通信学院,西安 710106
统计 摘要被查看 次,已被下载
文章编号 1001-3695(2015)04-0992-04
DOI 10.3969/j.issn.1001-3695.2015.04.007
摘要 自动编码机作为一种新兴的深层神经网络学习算法,在高维数据的降维和图像重构中取得了很好的效果。针对该方法在文本分类中重构出大量的对学习没有帮助的含噪数据,提出一种利用原型数据监督学习的改进模型,称做深层原型自动编码机,该方法改善了无监督学习的不足。并且,通过建立多个实例对应一个原型模型,可以大大降低算法对于原型数量的需求,提升了算法的运行效率,而且更加有利于原型学习在多种不同的数据上展开。实验证明该方法可以增加文本分类的准确率。
关键词 自动编码机;无监督学习;深层原型自动编码机;原型分类器
基金项目
本文URL http://www.arocmag.com/article/01-2015-04-007.html
英文标题 Text-classification algorithm improvements based on deep autoencoder
作者英文名 HU Hou-li, WEI Wei, XIE Qing-song
机构英文名 Xi'an Communication Institute of PLA, Xi'an 710106, China
英文摘要 As a new neural network algorithm, deep autoencoder achieved significant results in dimensionality reduction and image reconstruction at high-dimensional level. This paper introduced autoencoder in text classification, and found that this method could reconstruct a lot of noise data which was helpless to learning data. So it presented an improvement model using supervised learning by prototype data, which was called deep prototype autoencoder. This method improved the lack of unsupervised learning. Moreover, this approach created a model that multiple instances corresponding to one prototype data, greatly reduced the number of prototypes that algorithm needed, and enhanced the operational efficiency of the algorithm, while was more conducive to prototype learning in a variety of data. Experiments show that this method can increase the accuracy of text classification.
英文关键词 autoencoder; unsupervised learning; deep prototype autoencoder; prototype classifier
参考文献 查看稿件参考文献
  [1] 来斯惟, 徐立恒, 陈玉博, 等. 基于表示学习的中文分词算法探索[J] . 中文信息学报, 2013, 27(5):37-46.
[2] MNIH A, HINTON G. A scalable hierarchical distributed language model[C] //Advances in Neural Information Processing Systems. 2009:1081-1088.
[3] BENGIO Y. Learning deep architectures for AI[M] //Foundations and Trends in Machine Learning. [S. l. ] :Now Publishers Inc, 2009.
[4] HINTON G, OSINDERO S, TEH Y W. Fast learning algorithm for deep belief nets[J] . Neural Computation, 2006, 18(7):1527-1554.
[5] HINTON G, SALAKHUTINOV R. Reducing the dimensionality of data with neural networks[J] . Science, 2006, 5(2):504-507.
[6] DAHL G E, YU Dong, LI D, et al. Large vocabulary continuous speech recognition with context-dependent DBN-HMMS[C] //Proc of IEEE International Conference on Acoustics, Speech and Signal Processing. 2011:4688-4691.
[7] MONAMED A, SAINATH T N, DAHL G E, et al. Deep belief networks using discriminative features for phone recognition[C] //Proc of IEEE International Conference on Acoustics, Speech and Signal Processing. 2011:5060-5063.
[8] ARNOLD L, PAUGAM MOIDY H, SEBAG M. Unsupervised layer-wise model selection in deep neural networks[J] . Journal of Machine Learning Research, 2010, 6(3):610-634.
[9] 张开旭, 周昌乐. 基于自动编码器的中文词汇特征无监督学习[J] . 中文信息学报, 2013, 27(5):49-56.
[10] ERAHAN D, BENGIO Y, COURVILLE A, et al. Why does unsupervised pre-traininghelp deep learning?[J] . Journal of Machine Learning Research, 2010, 11(5):625-660.
[11] EEHAN D, MANZAGOL P, BENGIO Y, et al. The difficulty of training deep architectures and the effect of unsupervised pretraining[C] //Proc of the 12th International Conference on Artificial Intelligence and Statistics. 2009:153-160.
[12] RANZATO M A, BOUREAU Y L, LeCUN Y. Sparse feature learning for deep belief networks[C] //Advances in Neural Information Processing Systems. 2007:784-795.
[13] VINCENT P, LAROCHELLE H, BENGIO Y, et al. Extracting and composing robustfeatures with denoising autoencoders[C] //Proc of the 25th International Conference on Machine Learning. 2008:1096-1103.
[14] UTGOFF P, STRACUZZI D. Many-layered learning[J] . Neural Computation, 2002, 14(10):2497-2529.
[15] 孙志军, 薛磊, 许阳明, 等. 深度学习研究综述[J] . 计算机应用研究, 2012, 29(8):2806-2810.
[16] BENGIO Y, LAMBIN P, POPVICI D, et al. Greedy layer-wise trai-ning of deep networks[C] //Proc of the 20th Annual Conference on Neural Information Processing Systems. 2007:119-153.
[17] SEO S, BODE M, OBERMAYER K. Soft nearest prototype classification[J] . IEEE Trans on Neural Networks, 2003, 14(2):390-398.
[18] 殷力昂. 一种在深度结构中学习原型的分类方法[D] . 上海:上海交通大学, 2012.
[19] HINTON G, SALAKHUTDINOV R. Supporting online material for reducing the dimensionality of data with neural networks[J] . Science, 2006, 313(5786):504-507.
收稿日期 2014/2/21
修回日期 2014/4/5
页码 992-995
中图分类号 TP391.1;TP181
文献标志码 A