《计算机应用研究》|Application Research of Computers

一种基于词义和词频的向量空间模型改进方法

Method based on word meaning and word frequency to improve vector space model

免费全文下载 (已被下载 次)  
获取PDF全文
作者 邓晓衡,杨子荣,关培源
机构 中南大学 软件学院,长沙 410075
统计 摘要被查看 次,已被下载
文章编号 1001-3695(2019)05-023-1390-06
DOI 10.19734/j.issn.1001-3695.2017.12.0752
摘要 文本内容较多时,传统的向量空间模型(VSM)建模可能产生维数爆炸现象,效率低下且难以保证分类效果。针对VSM高维现象,利用词义和词频降低文本建模维度的方法提高效率和准确度,提出一种多义词判别优化的同义词聚类方法,结合上下文判别多义词的词义后,根据特征项词义相似度进行加权,合并词义相近的特征项。新方法使特征向量维度大大降低,多义词判别提高了文本特征提取的准确性。与其他文本特征提取和文本分类方法进行比较,结果表明,该算法在效率和准确度上有明显提高。
关键词 文本分类; 特征选择; 卡方分布; 向量空间模型
基金项目 中南大学研究生创新基金资助项目(2017zzts732)
本文URL http://www.arocmag.com/article/01-2019-05-023.html
英文标题 Method based on word meaning and word frequency to improve vector space model
作者英文名 Deng Xiaoheng, Yang Zirong, Guan Peiyuan
机构英文名 School of Software,Central South University,Changsha 410075,China
英文摘要 When the text content is more, the traditional VSM model may produce the dimension explosion phenomenon, the efficiency is low and the classification effect is difficult to guarantee. Aiming at the phenomenon of VSM, this paper proposed a method to reduce the dimension of text modeling by means of word meaning and frequency, in order to improve efficiency and accuracy. This paper proposed a synonym clustering method for polysemy discriminant optimization, combining with the context distinguishing word meaning, weighted by the similarity of the word meaning, and merging the feature items with similar meanings. The new method greatly reduced the dimension of eigenvector, and polysemy improved the accuracy of feature extraction. Compared with other text feature extraction and text categorization methods, the results show that the algorithm has a significant improvement in efficiency and accuracy.
英文关键词 text categorization; feature selection; chi-square; vector space model
参考文献 查看稿件参考文献
 
收稿日期 2017/12/1
修回日期 2018/1/24
页码 1390-1395
中图分类号 TP391.1
文献标志码 A