基于多层次语义特征的英文作文自动评分方法

doi:10.11772/j.issn.1001-9081.2020101572

计算机应用 ›› 2021, Vol. 41 ›› Issue (8): 2205-2211.DOI: 10.11772/j.issn.1001-9081.2020101572

所属专题：人工智能

基于多层次语义特征的英文作文自动评分方法

周险兵¹, 樊小超^1,2, 任鸽¹, 杨勇¹

1. 新疆师范大学计算机科学技术学院, 乌鲁木齐 830054;
2. 大连理工大学计算机科学与技术学院, 辽宁大连 116024

收稿日期:2020-10-12 修回日期:2021-01-22 发布日期:2021-01-27 出版日期:2021-08-10
通讯作者: 杨勇
作者简介:周险兵(1995-),男,湖北黄冈人,硕士研究生,CCF会员,主要研究方向:自然语言处理、作文评分;樊小超(1982-),男(锡伯族),新疆塔城人,讲师,博士,主要研究方向:文本情感分析;任鸽(1986-),女,河南兰考人,讲师,硕士,主要研究方向:数据挖掘、网络信息安全;杨勇(1979-),男,陕西汉中人,副教授,博士,主要研究方向:自然语言处理、软件工程。
基金资助:
国家自然科学基金资助项目（62066044）；新疆维吾尔自治区高等学校科研计划项目（XJEDU2016S066）。

Automated English essay scoring method based on multi-level semantic features

ZHOU Xianbing¹, FAN Xiaochao^1,2, REN Ge¹, YANG Yong¹

1. College of Computer Science and Technology, Xinjiang Normal University, Urumqi Xinjiang 830054, China;
2. School of Computer Science and Technology, Dalian University of Technology, Dalian Liaoning 116024, China

Received:2020-10-12 Revised:2021-01-22 Online:2021-01-27 Published:2021-08-10
Supported by:
This work is partially supported by the National Natural Science Foundation of China (62066044), the Scientific Research Program of Colleges and Universities in Xinjiang Uygur Autonomous Region (XJEDU2016S066).

摘要/Abstract

摘要： 作文自动评分（AES）技术能够自动地对作文进行分析和评分，其已成为自然语言处理技术在教育领域应用的热点研究问题之一。针对目前AES方法割裂了深层和浅层语义特征，忽视了多层次语义融合对作文评分影响的问题，提出了一种基于多层次语义特征的神经网络（MLSF）模型进行AES。首先，采用卷积神经网络（CNN）捕获局部语义特征，并采用混合神经网络捕获全局语义特征，以从深层次获取作文的语义特征；其次，利用篇章级的作文主题向量来获取主题层特征，同时针对深度学习模型难以挖掘的语法错误和语言丰富程度特征，构造了少量人工特征以从浅层获取作文的语言学特征；最后，通过特征融合对作文进行自动评分。实验结果表明，所提出模型在Kaggle ASAP竞赛公开数据集的所有子集上性能均有显著提升，该模型的平均二次加权的卡帕值（QWK）达到79.17%，验证了该模型在AES任务中的有效性。

关键词: 英文作文, 作文自动评分, 多层语义特征, 深层语义理解, 特征融合, 自然语言处理

Abstract: The Automated Essay Scoring (AES) technology can automatically analyze and score the essay, and has become one of the hot research problems in the application of natural language processing technology in the education field. Aiming at the current AES methods that separate deep and shallow semantic features, and ignore the impact of multi-level semantic fusion on essay scoring, a neural network model based on Multi-Level Semantic Features (MLSF) was proposed for AES. Firstly, Convolutional Neural Network (CNN) was used to capture local semantic features, and the hybrid neural network was used to capture global semantic features, so that the essay semantic features were obtained from a deep level. Secondly, the feature of the topic layer was obtained by using the essay topic vector of text level. At the same time, aiming at the grammatical errors and language richness features that are difficult to mine by deep learning model, a small number of artificial features were constructed to obtain the linguistic features of the essay from the shallow level. Finally, the essay was automatically scored through the feature fusion. Experimental results show that the proposed model improves the performance significantly on all subsets of the public dataset of the Kaggle ASAP (Automated Student Assessment Prize) champion, with the average Quadratic Weighted Kappa (QWK) of 79.17%, validating the effectiveness of the model in the AES tasks.

Key words: English essay, Automated Essay Scoring (AES), Multi-Level Semantic Feature (MLSF), deep semantic understanding, feature fusion, natural language processing

中图分类号:

TP391.1

周险兵, 樊小超, 任鸽, 杨勇. 基于多层次语义特征的英文作文自动评分方法[J]. 计算机应用, 2021, 41(8): 2205-2211.

ZHOU Xianbing, FAN Xiaochao, REN Ge, YANG Yong. Automated English essay scoring method based on multi-level semantic features[J]. Journal of Computer Applications, 2021, 41(8): 2205-2211.

参考文献

[1] CHEN Y Y,LIU C L,CHANG T H,et al. An unsupervised automated essay scoring system[J]. IEEE Intelligent Systems, 2010,25(5):61-67.
[2] YANNAKOUDAKIS H, BRISCOE T, MEDLOCK B. A new dataset and method for automatically grading ESOL texts[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics, 2011:180-189.
[3] CHEN H B,XU J G,HE B. Automated essay scoring by capturing relative writing quality[J]. The Computer Journal,2014,57(9):1318-1330.
[4] TAGHIPOUR K,NG H T. A neural approach to automated essay scoring[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2016:1882-1891.
[5] ALIKANIOTIS D,YANNAKOUDAKIS H,REI M. Automatic text scoring using neural networks[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2016:715-725.
[6] COZMA M,BUTNARU A M,IONCSEU R T. Automated essay scoring with string kernels and word embeddings[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2018:503-509.
[7] BUTNARU A M,IONESCU R T. From image to text classification:a novel approach based on clustering word embeddings[J]. Procedia Computer Science,2017,112:1783-1792.
[8] 梁茂成, 文秋芳. 国外作文自动评分系统评述及启示[J]. 外语电化教学,2007(5):18-24.(LIANG M C,WEN Q F. A critical review and implications of some automated essay scoring systems[J]. Computer-Assisted Foreign Language Education,2007(5):18-24.)
[9] 彭星源, 柯登峰, 赵知. 基于词汇评分的汉语作文自动评分[J]. 中文信息学报,2012,26(2):102-108.(PENG X Y,KE D F, ZHAO Z. Automatic Chinese essay scoring based on word scores[J]. Journal of Chinese Information Processing,2012,26(2):102-108.)
[10] 刘明杨, 秦兵, 刘挺. 基于文采特征的高考作文自动评分[J]. 智能计算机与应用,2016,6(1):1-4,8.(LIU M Y,QIN B, LIU T. Automated Chinese composition scoring based on the literary feature[J]. Intelligent Computer and Applications,2016, 6(1):1-4,8.)
[11] 周明, 贾艳明, 周彩兰, 等. 基于篇章结构的英文作文自动评分方法[J]. 计算机科学,2019,46(3):234-241.(ZHOU M,JIA Y M,ZHOU C L,et al. English automated essay scoring methods based on discourse structure[J]. Computer Science,2019,46(3):234-241.)
[12] 余立清. 英语命题作文自动评分系统的研究与实现[D]. 武汉:华中师范大学,2019:1-33. (XU L Q. Research and implementation of the automated English essay scoring system[D]. Wuhan:Central China Normal University,2019:1-33.)
[13] PAGE E B. Grading essays by computer:progress report[C]//Proceedings of the 1967 Invitational Conference on Testing Problems. Princeton,NJ:Educational Testing Service,1967:87-100.
[14] ZUPANC K, BOSNIĆ Z. Automated essay evaluation with semantic analysis[J]. Knowledge-Based Systems,2017,120:118-132.
[15] PRAMUKANTORO E S,FAUZI M A. Comparative analysis of string similarity and corpus-based similarity for automatic essay scoring system on e-learning gamification[C]//Proceedings of the 2016 International Conference on Advanced Computer Science and Information Systems. Piscataway:IEEE,2016:149-155.
[16] 付瑞吉, 王栋, 王士进, 等. 面向作文自动评分的优美句识别[J]. 中文信息学报,2018,32(6):88-97.(FU R J,WANG D, WANG S J,et al. Elegart sentence recognition for automated essay scoring[J]. Journal of Chinese Information Processing, 2018,32(6):88-97.)
[17] 黄凯. 英语作文自动评分关键技术的研究与实现[D]. 武汉:华中师范大学, 2019:1-59. (HUANG K. Research and implementation of the key technology of English automatic essay scoring[D]. Wuhan:Central China Normal University,2019:1-59.)
[18] DONG F,ZHANG Y. Automatic features for essay scoring-an empirical study[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2016:1072-1077.
[19] DONG F, ZHANG Y, YANG J. Attention-based recurrent convolutional neural network for automatic essay scoring[C]//Proceedings of the 21st Conference on Computational Natural Language Learning. Stroudsburg, PA:Association for Computational Linguistics,2017:153-162.
[20] JIN C C,HE B,HUI K,et al. TDNN:a two-stage deep neural network for prompt-independent automated essay scoring[C]//Proceeding of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2018:1088-1097.
[21] TAY Y,PHAN M C,TUAN L A,et al. SkipFlow:incorporating neural coherence features for end-to-end automatic text scoring[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2018:5948-5955.
[22] RODRIGUEZ P U,JAFARI A,ORMEROD C M. Language models and automated essay scoring[EB/OL]. (2019-09-18)[2020-11-11]. https://arxiv.org/pdf/1909.09482.pdf.
[23] 刘浩坤. 英语作文自动评分算法的研究与设计[D]. 合肥:中国科学技术大学,2018:1-50.(LIU H K. Research and design of the automated English essay scoring algorithm[D]. Hefei:University of Science and Technology of China,2018:1-50.)
[24] 崔建鹏. 自动作文评分的关键技术研究[D]. 成都:电子科技大学,2020:1-62.(CUI J P. Research on key technologies of automated essay scoring[D]. Chengdu:University of Electronic Science and Technology of China,2020:1-62.)
[25] LIU J W,XU Y,ZHU Y G. Automated essay scoring based on two-stage learning[EB/OL]. (2019-12-20)[2020-11-11]. https://arxiv.org/pdf/1901.07744.pdf.
[26] FARAG Y, YANNAKOUDAKIS H, BRISCOE T. Neural automated essay scoring and coherence modeling for adversarially crafted input[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg,PA:Association for Computational Linguistics,2018:263-271.
[27] 陈建林. 大规模英语考试作文评分标准效度验证[J]. 中国考试,2016(1):29-38. (CHEN J L. Validating TEM-8 rating scale via MFRM and TAPs based evidence[J]. China Examinations, 2016(1):29-38.)
[28] 刘建达, 黄亚萍. 词汇特征量对CET作文得分的影响[J]. 外语电化教学,2011(3):11-15. (LIU J D,HUANG Y P. The relationship between four vocabulary measures and CET-4/6 essay scores[J]. Computer-Assisted Foreign Language Education,2011(3):11-15.)
[29] HOCHREITER S,SCHMIDHUBER J. Long short-term memory[J]. Neural Computation,1997,9(8):1735-1780.
[30] LE Q,MIKOLOV T. Distributed representations of sentences and documents[C]//Proceedings of the 31st International Conference on Machine Learning. New York:JMLR. org,2014:1188-1196.

基于多层次语义特征的英文作文自动评分方法

Automated English essay scoring method based on multi-level semantic features

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	潘烨新, 杨哲. 基于多级特征双向融合的小目标检测优化模型[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2871-2877.
[2]	帅奇, 王海瑞, 朱贵富. 基于双向对比训练的中文故事结尾生成模型[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2683-2688.
[3]	张全梅, 黄润萍, 滕飞, 张海波, 周南. 融合异构信息的自动国际疾病分类编码方法[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2476-2482.
[4]	刘瑞华, 郝子赫, 邹洋杨. 基于多层级精细特征融合的步态识别算法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2250-2257.
[5]	陆潜慧, 张羽, 王梦灵, 吴庭伟, 单玉忠. 基于改进循环池化网络的核电装备质量文本分类模型[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2034-2040.
[6]	黄梦源, 常侃, 凌铭阳, 韦新杰, 覃团发. 基于层间引导的低光照图像渐进增强算法[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1911-1919.
[7]	刘耀, 李雨萌, 宋苗苗. 基于业务流程的认知图谱[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1699-1705.
[8]	于右任, 张仰森, 蒋玉茹, 黄改娟. 融合多粒度语言知识与层级信息的中文命名实体识别模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1706-1712.
[9]	刘越, 刘芳, 武奥运, 柴秋月, 王天笑. 基于自注意力机制与图卷积的3D目标检测网络[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1972-1977.
[10]	李鑫, 孟乔, 皇甫俊逸, 孟令辰. 基于分离式标签协同学习的YOLOv5多属性分类[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1619-1628.
[11]	韩贵金, 张馨渊, 张文涛, 黄娅. 基于多特征融合的自监督图像配准算法[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1597-1604.
[12]	李鸿天, 史鑫昊, 潘卫国, 徐成, 徐冰心, 袁家政. 融合多尺度和注意力机制的小样本目标检测[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1437-1444.
[13]	高龙涛, 李娜娜. 基于方面感知注意力增强的方面情感三元组抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1049-1057.
[14]	杨先凤, 汤依磊, 李自强. 基于交替注意力机制和图卷积网络的方面级情感分析模型[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1058-1064.
[15]	蒋占军, 吴佰靖, 马龙, 廉敬. 多尺度特征和极化自注意力的Faster-RCNN水漂垃圾识别[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 938-944.