Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (6): 1781-1785.DOI: 10.11772/j.issn.1001-9081.2023050662

Special Issue: 人工智能

• Artificial intelligence • Previous Articles     Next Articles

Generative label adversarial text classification model

Xun YAO1, Zhongzheng QIN1, Jie YANG2()   

  1. 1.School of Computer Science and Artificial Intelligence,Wuhan Textile University,Wuhan Hubei 430200,China
    2.School of Computer and Information Technology,University of Wollongong Australia,Wollongong New South Wales 2552,Australia
  • Received:2023-05-29 Revised:2023-09-04 Accepted:2023-09-22 Online:2023-09-26 Published:2024-06-10
  • Contact: Jie YANG
  • About author:YAO Xun, born in 1969, Ph. D., lecturer. His research interests include image processing, computer vision, natural language processing.
    QIN Zhongzheng, born in 1999, M. S. candidate. His research interests include text classification.

生成式标签对抗的文本分类模型

姚迅1, 秦忠正1, 杨捷2()   

  1. 1.武汉纺织大学 计算机与人工智能学院, 武汉 430200
    2.伍伦贡大学 计算机与信息技术学院, 澳大利亚 新南威尔士州 伍伦贡市 2522
  • 通讯作者: 杨捷
  • 作者简介:姚迅(1969—),男,湖北武汉人,讲师,博士,主要研究方向:图像处理、计算机视觉、自然语言处理
    秦忠正(1999—),男,湖北随州人,硕士研究生,主要研究方向:文本分类;

Abstract:

Text classification is a fundamental task in Natural Language Processing (NLP), aiming to assign text data to predefined categories. The combination of Graph Convolutional neural Network (GCN) and large-scale pre-trained model BERT (Bidirectional Encoder Representations from Transformer) has achieved excellent results in text classification tasks. Undirected information transmission of GCN in large-scale heterogeneous graphs produces information noise, which affects the judgment of the model and reduce the classification ability of the model. To solve this problem, a generative label adversarial model, the Class Adversarial Graph Convolutional Network (CAGCN) model, was proposed to reduce the interference of irrelevant information during classification and improve the classification performance of the model. Firstly, the composition method in TextGCN (Text Graph Convolutional Network) was used to construct the adjacency matrix, which was combined with GCN and BERT models as a Class Generator (CG). Secondly, the pseudo-label feature training method was used in the model training to construct a clueter. The cluster and the class generator were jointly trained. Finally, experiments were carried out on several widely used datasets. Experimental results show that the classification accuracy of CAGCN model is 1.2, 0.1, 0.5, 1.7 and 0.5 percentage points higher than that of RoBERTaGCN model on the widely used classification datasets 20NG, R8, R52, Ohsumed and MR, respectively.

Key words: text classification, Graph Convolutional neural Network (GCN), BERT (Bidirectional Encoder Representations from Transformer), pseudo-label, heterogeneous graph

摘要:

文本分类是自然语言处理(NLP)中的一项基础任务,目的是将文本数据分配至预先定义的类别。图卷积神经网络(GCN)与大规模的预训练模型BERT(Bidirectional Encoder Representations from Transformer)的结合在文本分类任务中取得了良好的效果。大规模异构图中GCN的无向的信息传递产生信息噪声影响模型的判断,造成模型分类能力下降,针对这一问题,提出一种生成式标签对抗模型,即类对抗图卷积网络(CAGCN)模型,以降低分类时无关信息的干扰,提升模型的分类性能。首先,采用TextGCN(Text Graph Convolutional Network)中的构图法构建邻接矩阵,结合GCN和BERT模型作为类生成器(CG);其次,在模型训练时采用伪标签特征训练法,并构建聚类器与类生成器联合训练;最后,在多个广泛使用的数据集上进行实验。实验结果表明,在泛用的分类数据集20NG、R8、R52、Ohsumed和MR上,CAGCN模型的分类准确率比RoBERTaGCN模型分别提高了1.2、0.1、0.5、1.7和0.5个百分点。

关键词: 文本分类, 图卷积神经网络, BERT, 伪标签, 异构图

CLC Number: