Dialogue state tracking model based on slot correlation information extraction

doi:10.11772/j.issn.1001-9081.2022040508

Abstract

Abstract:

Dialogue State Tracking （DST） is an important module in task-oriented dialogue systems， but the existing open-vocabulary-based DST models do not make full use of the slot correlation information as well as the structural information of the dataset itself. To solve the above problems， a new DST model named SCEL-DST （SCE and LOW for Dialogue State Tracking） was proposed based on slot correlation information extraction. Firstly， a Slot Correlation Extractor （SCE） was constructed， and the attention mechanism was used to learn the correlation information between slots. Then the Learning Optimal sample Weights （LOW） strategy was applied in the training process to enhance the model's utilization of the dataset information without substantial increase in training time. Finally， the model details were optimized to build the complete SCEL-DST model. Experimental results show that SCE and LOW are critical to the performance improvement of SCEL-DST model， making SCEL-DST achieve higher joint goal accuracy on both experimental datasets. The SCEL-DST model has the joint goal accuracy improved by 1.6 percentage points on the MultiWOZ 2.3 （Wizard-of-OZ 2.3） dataset compared to TripPy （Triple coPy） under the same conditions， and by 2.0 percentage points on the WOZ 2.0 （Wizard-of-OZ 2.0） dataset compared to AG-DST （Amendable Generation for Dialogue State Tracking）.

Key words: Dialogue State Tracking (DST), attention mechanism, task-oriented dialogue, Curriculum Learning (CL), pre-trained model

摘要：

对话状态追踪（DST）是任务型对话系统中一个重要的模块，但现有的基于开放词表的DST模型没有充分利用槽位的相关信息以及数据集本身的结构信息。针对上述问题，提出基于槽位相关信息提取的DST模型SCEL-DST（SCE and LOW for Dialogue State Tracking）。首先，构建槽位相关信息提取器（SCE），利用注意力机制学习槽位之间的相关信息；然后，在训练过程中应用学习最优样本权重（LOW）策略，在未大幅增加训练时间的前提下，加强模型对数据集信息的利用；最后，优化模型细节，搭建完整的SCEL-DST模型。实验结果表明，SCE和LOW对SCEL-DST模型性能的提升至关重要，该模型在两个实验数据集上均取得了更高的联合目标准确率，其中在MultiWOZ 2.3 （Wizard-of-OZ 2.3）数据集上与相同条件下的TripPy（Triple coPy）相比提升了1.6个百分点，在WOZ 2.0 （Wizard-of-OZ 2.0）数据集上与AG-DST （Amendable Generation for Dialogue State Tracking）相比提升了2.0个百分点。

关键词: 对话状态追踪, 注意力机制, 任务型对话, 课程学习, 预训练模型

CLC Number:

TP183

Lifeng SHI, Zhengwei NI. Dialogue state tracking model based on slot correlation information extraction[J]. Journal of Computer Applications, 2023, 43(5): 1430-1437.

石利锋, 倪郑威. 基于槽位相关信息提取的对话状态追踪模型[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1430-1437.

Figures/Tables 10

References 23

1	陈红燕. 面向任务的对话状态追踪方法及应用［D］. 哈尔滨：哈尔滨工业大学， 2020：3-4.
	CHEN H Y. Task-oriented dialogue state tracking and application［D］. Harbin： Harbin Institute of Technology， 2020：3-4
2	黄伟. 任务型对话系统中对话状态追踪技术研究［D］. 兰州：兰州大学， 2021：6-7.
	HUANG W. Research on dialogue state tracking technology in task-based dialogue system［D］. Lanzhou： Lanzhou University， 2021：6-7.
3	GAO S Y， SETHI A， AGARWAL S， et al. Dialog state tracking： a neural reading comprehension approach［C］// Proceedings of the 20th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Stroudsburg， PA： ACL， 2019： 264-273. 10.18653/v1/w19-5932
4	GOEL R， PAUL S， HAKKANI-TÜR D. HyST： a hybrid approach for flexible and accurate dialogue state tracking［C］// Proceedings of the Interspeech 2019. ［S.l.］： International Speech Communication Association， 2019：1458-1462. 10.21437/interspeech.2019-1863
5	HECK M， van NIEKERK C， LUBIS N， et al. TripPy： a triple copy strategy for value independent neural dialog state tracking［C］// Proceedings of the 21st Annual Meeting of the Special Interest Group on Discourse and Dialogue. Stroudsburg， PA： ACL， 2020： 35-44.
6	SOVIANY P， IONESCU R T， ROTA P， et al. Curriculum learning： a survey［J］. International Journal of Computer Vision， 2022， 130（6）：1526-1565. 10.1007/s11263-022-01611-x
7	RASTOGI A， ZANG X X， SUNKARA S， et al. Towards scalable multi-domain conversational agents： the schema-guided dialogue dataset［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020：8689-8696. 10.1609/aaai.v34i05.6394
8	LEE H， LEE J， KIM T Y. SUMBT： slot-utterance matching for universal and scalable belief tracking［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2019： 5478-5483. 10.18653/v1/p19-1546
9	WANG Y， GUO Y， ZHU S. Slot attention with value normalization for multi-domain dialogue state tracking［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2020： 3019-3028. 10.18653/v1/2020.emnlp-main.243
10	XU P Y， HU Q. An end-to-end approach for handling unknown slot values in dialogue state tracking［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2018： 1448-1457. 10.18653/v1/p18-1134
11	VINYALS O， FORTUNATO M， JAITLY N. Pointer networks［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems — Volume 2. Cambridge： MIT Press， 2015：2692-2700.
12	WU C S， MADOTTO A， HOSSEINI-ASL E， et al. Transferable multi-domain state generator for task-oriented dialogue systems［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2019： 808-819. 10.18653/v1/p19-1078
13	ZHANG J G， HASHIMOTO K， WU C S， et al. Find or classify？ dual strategy for slot-value predictions on multi-domain dialog state tracking［C］// Proceedings of the 9th Joint Conference on Lexical and Computational Semantics. Stroudsburg， PA： ACL， 2020： 154-167. 10.21437/interspeech.2021-138
14	LE H， SOCHER R， HOI S C H. Non-autoregressive dialog state tracking［EB/OL］. （2020-02-19）［2021-08-15］.. 10.1145/3483845.3483880
15	CHEN L， LV B E， WANG C， et al. Schema-guided multi-domain dialogue state tracking with graph attention neural networks［C］// Proceedings of 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 7521-7528. 10.1609/aaai.v34i05.6250
16	AN J， CHO S， BANG J， et al. Domain-slot relationship modeling using a pre-trained language encoder for multi-domain dialogue state tracking［J］. IEEE/ACM Transactions on Audio， Speech and Language Processing， 2022， 30： 2091-2102. 10.1109/taslp.2022.3181350
17	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg， PA： ACL， 2019： 4171-4186. 10.18653/v1/n18-2
18	YE F H， MANOTUMRUKSA J， ZHANG Q， et al. Slot self-attentive dialogue state tracking［C］// Proceedings of the Web Conference 2021. New York： ACM， 2021： 1598-1608. 10.1145/3442381.3449939
19	DAI Y P， LI H Y， LI Y B， et al. Preview， attend and review： schema-aware curriculum learning for multi-domain dialog state tracking［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 2： Short Papers）. Stroudsburg， PA： ACL， 2021： 879-885. 10.18653/v1/2021.acl-short.111
20	SANTIAGO C， BARATA C， SASDELLI M， et al. LOW： training deep neural networks by learning optimal sample weights［J］. Pattern Recognition， 2021， 110： No.107585. 10.1016/j.patcog.2020.107585
21	HAN T， LIU X M， TAKANABU R， et al. MultiWOZ 2.3： a multi-domain task-oriented dialogue dataset enhanced with annotation corrections and co-reference annotation［C］// Proceedings of 2021 CCF International Conference on Natural Language Processing and Chinese Computing， LNCS 13029. Cham： Springer， 2021： 206-218.
22	WEN T H， VANDYKE D， MRKŠIĆ N， et al. A network-based end-to-end trainable task-oriented dialogue system［C］// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics： Volume 1： Long Papers. Stroudsburg， PA： ACL， 2019： 438-449.
23	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 6000-6010.

轮次（Turn）	域槽对（Domain-slot pair）	槽值（Value）	类型（Type）	共指（Coreference）
0	restaurant-pricerange	expensive	span
0	restaurant-area	south	span
1	restaurant-name	cambridge chop house	informed
1	restaurant-book_people	2	span
1	restaurant-book_time	14：15	span
1	restaurant-book_time	Sunday	span
2	hotel-stars	3 star	span
2	hotel-area	south	coreference	restaurant-area
2	hotel-pricerange	expensive	coreference	restaurant-pricerange
3	hotel-name	lensfield hotel	informed
3	hotel-book_people	two	span
3	hotel-book_stay	two nights	span
3	hotel-book_day	sunday	span

轮次（Turn）	域槽对（Domain-slot pair）	槽值（Value）	类型（Type）	共指（Coreference）
0	restaurant-pricerange	expensive	span
0	restaurant-area	south	span
1	restaurant-name	cambridge chop house	informed
1	restaurant-book_people	2	span
1	restaurant-book_time	14：15	span
1	restaurant-book_time	Sunday	span
2	hotel-stars	3 star	span
2	hotel-area	south	coreference	restaurant-area
2	hotel-pricerange	expensive	coreference	restaurant-pricerange
3	hotel-name	lensfield hotel	informed
3	hotel-book_people	two	span
3	hotel-book_stay	two nights	span
3	hotel-book_day	sunday	span

模型	联合目标准确率	模型	联合目标准确率
TRADE	49.2	SimpleTOD	51.3
SUMBT	52.9	SAVN	58.0
COMER	50.2	TripPy^*	61.6
SOM-DST	55.5	SCEL-DST	63.2

模型	联合目标准确率	模型	联合目标准确率
TRADE	49.2	SimpleTOD	51.3
SUMBT	52.9	SAVN	58.0
COMER	50.2	TripPy^*	61.6
SOM-DST	55.5	SCEL-DST	63.2

模型	联合目标准确率	模型	联合目标准确率
SUMBT	91.0	TripPy^*	90.9
GLAD	88.1	AG-DST	91.4
GCE	88.5	SCEL-DST	93.4