Boundary-aware approach to machine reading comprehension

doi:10.11772/j.issn.1001-9081.2023081178

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (7): 2004-2010.DOI: 10.11772/j.issn.1001-9081.2023081178

• Artificial intelligence • Previous Articles Next Articles

Boundary-aware approach to machine reading comprehension

Qing LIU¹^,²^,³, Yanping CHEN¹^,²^,³(), Anqi ZOU¹^,²^,³, Ruizhang HUANG¹^,²^,³, Yongbin QIN¹^,²^,³

^1.Text Computing and Cognitive Intelligence Engineering Research Center of Ministry of Education，Guizhou University，Guiyang Guizhou 550025，China
^2.State Key Laboratory of Public Big Data （Guizhou University），Guiyang Guizhou 550025，China
^3.College of Computer Science and Technology，Guizhou University，Guiyang Guizhou 550025，China

Received:2023-09-01 Revised:2023-09-20 Accepted:2023-10-09 Online:2024-07-18 Published:2024-07-10
Contact: Yanping CHEN
About author:LIU Qing， born in 1996， M. S. candidate. Her research interests include natural language processing， machine reading comprehension.
ZOU Anqi， born in 1996， Ph. D. candidate. His research interests include natural language processing， intelligent question-answering.
HUANG Ruizhang， born in 1979， Ph. D.， professor. Her research interests include big data and data mining， information extraction.
QIN Yongbin， born in 1980， Ph. D.， professor. His research interests include big data management and application， multi-source data fusion and application.
First author contact:CHEN Yanping， born in 1980， Ph. D.， professor. His research interests include artificial intelligence， natural language processing.
Supported by:
National Natural Science Foundation of China(62166007);Key Technology Research and Development Program of Guizhou Province(［2022］277)

面向机器阅读理解的边界感知方法

刘青¹^,²^,³, 陈艳平¹^,²^,³(), 邹安琪¹^,²^,³, 黄瑞章¹^,²^,³, 秦永彬¹^,²^,³

^1.贵州大学文本计算与认知智能教育部工程研究中心, 贵阳 550025
^2.公共大数据国家重点实验室(贵州大学), 贵阳 550025
^3.贵州大学计算机科学与技术学院, 贵阳 550025

通讯作者: 陈艳平
作者简介:刘青（1996—），女，湖南衡阳人，硕士研究生，主要研究方向：自然语言处理、机器阅读理解；
邹安琪（1996—），男，贵州安顺人，博士研究生，主要研究方向：自然语言处理、智能问答；
黄瑞章（1979—），女，天津人，教授，博士，CCF会员，主要研究方向：大数据与数据挖掘、信息提取；
秦永彬（1980—），男，山东烟台人，教授，博士，CCF高级会员，主要研究方向：大数据管理与应用、多源数据融合与应用。
第一联系人：陈艳平（1980—），男，贵州长顺人，教授，博士，CCF会员，主要研究方向：人工智能、自然语言处理；
基金资助:
国家自然科学基金资助项目(62166007);贵州省科技支撑计划项目(［2022］277)

Abstract

Abstract:

Existing methods for answer acquisition based on pre-trained language models may suffer from inaccuracies in predicting boundaries， a boundary-aware approach for span-based extraction Machine Reading Comprehension （MRC） is proposed to mitigate this issue. Firstly， special characters were introduced to mark the question boundary during the question input stage， enhancing the semantic information of the question to improve boundary perception. Secondly， during the answer prediction stage， an answer boundary regressor was constructed to facilitate semantic interaction between the perceived question boundary and the output of the predicted answer boundary. Lastly， the biased predicted answer boundary was further adjusted based on the post-interaction semantic information to calibrate the predicted answers. Experimental results demonstrate that when compared to the SpanBERT （Span-based Bidirectional Encoder Representation from Transformers）， the proposed method improves the F1 value by 0.2 percentage points and the Exact Match （EM） value by 0.9 percentage points on the public dataset SQuAD （Stanford Question Answering Dataset）1.1， it achieved improvements of 0.7 percentage points in both F1 score and EM value on the HotpotQA （Hotpot Question Answering） dataset， and it improved the F1 score by 2.8 percentage points and the EM value by 3.3 percentage points on the NewsQA （News Question Answering） dataset. The effectiveness of this method is rooted in its capacity to enhance the model’s perception of question boundary information and to accomplish the calibration of predicted answer boundary. Consequently， it results in an enhancement of system accuracy in applications such as intelligent question answering and intelligent customer service when dealing with text data comprehension and analysis.

Key words: Machine Reading Comprehension (MRC), question boundary-awareness, answer boundary regression, span extraction

摘要：

针对现有的基于预训练语言模型的答案获取方法存在预测边界不够准确的问题，提出一种面向片段抽取式机器阅读理解（MRC）的边界感知方法。首先，在问题输入阶段引入特殊字符标记问题边界，通过增强问题语义信息的方式实现对问题边界的感知；其次，在答案预测阶段，构建答案边界回归器，实现感知的问题边界语义信息与输出的预测答案边界语义信息的语义交互；最后，通过交互后的语义信息进一步调整存在偏差的预测答案边界，实现对预测答案的校准。实验结果表明，与SpanBERT （Span-based Bidirectional Encoder Representation from Transformers）相比，该方法在公共数据集SQuAD（Stanford Question Answering Dataset）1.1上的F1值提升了0.2个百分点、精确匹配（EM）值提升了0.9个百分点；在HotpotQA（Hotpot Question Answering）数据集上的F1值和EM值都提升了0.7个百分点；在NewsQA（News Question Answering）数据集上的F1值提升了2.8个百分点、EM值提升了3.3个百分点。可见，该方法能有效增强对问题边界信息的感知并且实现对预测答案边界的校准，有利于更好地理解和分析文本数据，在智能问答、智能客服等领域的应用中提高系统的准确性。

关键词: 机器阅读理解, 问题边界感知, 答案边界回归, 片段抽取

CLC Number:

TP391.1

Qing LIU, Yanping CHEN, Anqi ZOU, Ruizhang HUANG, Yongbin QIN. Boundary-aware approach to machine reading comprehension[J]. Journal of Computer Applications, 2024, 44(7): 2004-2010.

刘青, 陈艳平, 邹安琪, 黄瑞章, 秦永彬. 面向机器阅读理解的边界感知方法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2004-2010.

Figures/Tables 7

References 19

1	YOU C， CHEN N， ZOU Y. Knowledge distillation for improved accuracy in spoken question answering ［C］// Proceedings of the 2021 IEEE International Conference on Acoustics， Speech and Signal Processing. Piscataway： IEEE， 2021： 7793-7797.
2	奚雪峰，周国栋.面向自然语言处理的深度学习研究［J］.自动化学报， 2016， 42（10）： 1445-1465.
	XI X F， ZHOU G D. A survey on deep learning for natural language processing ［J］. Acta Automatica Sinica， 2016， 42（10）： 1445-1465.
3	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre-training of deep bidirectional Transformers for language understanding ［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1（Long and Short Papers）. Stroudsburg： ACL， 2019： 4171-4186.
4	JOSHI M， CHEN D， LIU Y， et al. SpanBERT： improving pre-training by representing and predicting spans ［J］. Transactions of the Association for Computational Linguistics， 2020， 8： 64-77.
5	RAJPURKAR P， ZHANG J， LOPYREV K， et al. SQuAD： 100000+ questions for machine comprehension of text ［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2016： 2383-2392.
6	YANG Z， QI P， ZHANG S， et al. HotpotQA： a dataset for diverse， explainable multi-hop question answering ［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2018： 2369-2380.
7	TRISCHLER A， WANG T， YUAN X， et al. NewsQA： a machine comprehension dataset ［C］// Proceedings of the 2 nd Workshop on Representation Learning for NLP. Stroudsburg： ACL， 2017： 191-200.
8	FISCH A， TALMOR A， JIA R， et al. MRQA 2019 shared task： evaluating generalization in reading comprehension ［C］// Proceedings of the 2 nd Workshop on Machine Reading for Question Answering. Stroudsburg： ACL， 2019： 1-13.
9	LIU S， ZHANG X， ZHANG S， et al. Neural machine reading comprehension： methods and trends ［J］. Applied Sciences， 2019， 9（18）： 3698.
10	WANG S， JIANG J. Machine comprehension using match-LSTM and answer pointer ［EB/OL］. （2016-11-07）［2023-09-06］. .
11	张虎，王宇杰，谭红叶，等.基于MHSA和句法关系增强的机器阅读理解方法研究［J］.自动化学报， 2022， 48（11）： 2718-2728.
	ZHANG H， WANG Y J， TAN H Y， et al. Research on machine reading comprehension method based on MHSA and syntactic relations enhancement ［J］. Acta Automatica Sinica， 2022， 48（11）： 2718-2728.
12	HU M， WEI F， PENG Y， et al. Read + verify： machine reading comprehension with unanswerable questions ［C］// Proceedings of the 33 rd AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2019： 6529-6537.
13	赵加坤，戴梦瑶，刘江宁，等.面向片段抽取式机器阅读理解的注意力网络［J］.计算机与数字工程， 2022， 50（2）： 350-355.
	ZHAO J K， DAI M Y， LIU J N， et al. Attention networks for fragment extractive machine reading comprehension ［J］. Computer and Digital Engineering， 2022， 50（2）： 350-355.
14	LIU Y， OTT M， GOYAL N， et al. RoBERTa： a robustly optimized BERT pretraining approach ［EB/OL］. （2019-07-26）［2023-09-06］. .
15	RAM O， KIRSTAIN Y， BERANT J， et al. Few-shot question answering by pretraining span selection ［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing s （Volume 1： Long Papers）. Stroudsburg： ACL， 2021： 3066-3079.
16	YASUNAGA M， LESKOVEC J， LIANG P. LinkBERT： pre-training language models with document links ［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2022： 8003-8016.
17	DHINGRA B， LIU H， YANG Z， et al. Gated-attention readers for text comprehension ［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2017： 1832-1846.
18	ZHANG Z， YANG J， ZHAO H. Retrospective reader for machine reading comprehension ［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2021： 14506-14514.
19	ZHANG W， REN F. ELMo+gated self-attention network based on BiDAF for machine reading comprehension ［C］// Proceedings of the IEEE 11th International Conference on Software Engineering and Service Science. Piscataway： IEEE， 2020： 1-6.

参数	设定值
学习率	2×10^-5
最大输入	512
长文本跨步	128
最大问题长度	64
最大答案长度	30
Batch size	16（SQuAD1.1）， 8（HotpotQA与NewsQA）
Epoch	4

参数	设定值
学习率	2×10^-5
最大输入	512
长文本跨步	128
最大问题长度	64
最大答案长度	30
Batch size	16（SQuAD1.1）， 8（HotpotQA与NewsQA）
Epoch	4

数据集	方法	F1值	EM值
SQuAD1.1	Match-LSTM^［10］	73.7	64.7
	ELMo+Gated Self-BiDAF^［19］	83.1	74.5
	Human Perf.^［5］	90.5	80.3
	BERT_BASE^［3］	88.5	80.8
	LinkBERT_BASE^*［16］	90.8	84.1
	RoBERTa_BASE^［14］	91.5	84.6
	SpanBERT_BASE^*	92.1	85.4
	本文方法	92.3	86.3
HotpotQA	BERT_BASE	76.0	—
	SpanBERT_BASE^*	79.3	63.4
	本文方法	80.0	64.1
NewsQA	Match-LSTM	49.6	34.4
	BERT_BASE	65.7	—
	SpanBERT_BASE^*	68.5	53.5
	本文方法	71.3	56.8

数据集	方法	F1值	EM值
SQuAD1.1	Match-LSTM^［10］	73.7	64.7
	ELMo+Gated Self-BiDAF^［19］	83.1	74.5
	Human Perf.^［5］	90.5	80.3
	BERT_BASE^［3］	88.5	80.8
	LinkBERT_BASE^*［16］	90.8	84.1
	RoBERTa_BASE^［14］	91.5	84.6
	SpanBERT_BASE^*	92.1	85.4
	本文方法	92.3	86.3
HotpotQA	BERT_BASE	76.0	—
	SpanBERT_BASE^*	79.3	63.4
	本文方法	80.0	64.1
NewsQA	Match-LSTM	49.6	34.4
	BERT_BASE	65.7	—
	SpanBERT_BASE^*	68.5	53.5
	本文方法	71.3	56.8

方法	F1值	EM值
SpanBERT_BASE^*	92.1	85.4
单边校准	92.3	86.3
双边校准	92.2	85.6

Boundary-aware approach to machine reading comprehension

面向机器阅读理解的边界感知方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 19

Related Articles 15

Recommended Articles

Metrics

方法	EM
SpanBERT_BASE^*	85.36
SpanBERT_BASE^*+ （a）	85.51
SpanBERT_BASE^*+ （b）	85.73
SpanBERT_BASE^*+ （b） - Bi-LSTM	85.66
SpanBERT_BASE^*+ （a） + （b）	86.30

[1]	ZHANG Qing, YANG Fan, FANG Yuhan. Chinese Spelling Correction Algorithm Based on Multi-Modal Information Fusion [J]. Journal of Computer Applications, 0, (): 0-0.
[2]	Yingjie GAO, Min LIN, Siriguleng, Bin LI, Shujun ZHANG. Prompt learning method for ancient text sentence segmentation and punctuation based on span-extracted prototypical network [J]. Journal of Computer Applications, 2024, 44(12): 3815-3822.
[3]	. Fault diagnosis method for train control on-board interface equipment of CTCS-3 based on temporal knowledge graph completion [J]. Journal of Computer Applications, 0, (): 0-0.
[4]	. Metaphor detection technique for improving representation in linguistic rules [J]. Journal of Computer Applications, 0, (): 0-0.
[5]	. Optimization method for sequence labeling combined with entity boundary offset [J]. Journal of Computer Applications, 0, (): 0-0.
[6]	. Chinese spelling correction model ReLM enhanced with deep semantic features [J]. Journal of Computer Applications, 0, (): 0-0.
[7]	. Nested named entity recognition combined with boundary generation by multi-objective learning [J]. Journal of Computer Applications, 0, (): 0-0.
[8]	. Survey of sequential pattern mining [J]. Journal of Computer Applications, 0, (): 0-0.
[9]	. Deep semi-supervised text clustering with intentional regularization [J]. Journal of Computer Applications, 0, (): 0-0.
[10]	. Combining preprocessing methods and adversarial learning for fair link prediction [J]. Journal of Computer Applications, 0, (): 0-0.
[11]	. Dependency type and distance enhanced aspect based sentiment analysis model [J]. Journal of Computer Applications, 0, (): 0-0.
[12]	. Nested entity recognition model for wind power equipment based on differential boundary enhancement [J]. Journal of Computer Applications, 0, (): 0-0.
[13]	. Visually guided word segmentation and part of speech [J]. Journal of Computer Applications, 0, (): 0-0.
[14]	. Aspect-based sentiment analysis method with code generation [J]. Journal of Computer Applications, 0, (): 0-0.
[15]	Yushan JIANG, Yangsen ZHANG. Large language model-driven stance-aware fact-checking [J]. Journal of Computer Applications, 2024, 44(10): 3067-3073.