《计算机应用》唯一官方网站 ›› 2023, Vol. 43 ›› Issue (5): 1430-1437.DOI: 10.11772/j.issn.1001-9081.2022040508
• 人工智能 • 上一篇
收稿日期:
2022-04-11
修回日期:
2022-08-10
接受日期:
2022-08-16
发布日期:
2023-05-08
出版日期:
2023-05-10
通讯作者:
倪郑威
作者简介:
石利锋(1998—),男,浙江绍兴人,硕士研究生,CCF会员,主要研究方向:自然语言处理、机器学习基金资助:
Received:
2022-04-11
Revised:
2022-08-10
Accepted:
2022-08-16
Online:
2023-05-08
Published:
2023-05-10
Contact:
Zhengwei NI
About author:
SHI Lifeng, born in 1998, M. S. candidate. His research interests include natural language processing, machine learning.Supported by:
摘要:
对话状态追踪(DST)是任务型对话系统中一个重要的模块,但现有的基于开放词表的DST模型没有充分利用槽位的相关信息以及数据集本身的结构信息。针对上述问题,提出基于槽位相关信息提取的DST模型SCEL-DST(SCE and LOW for Dialogue State Tracking)。首先,构建槽位相关信息提取器(SCE),利用注意力机制学习槽位之间的相关信息;然后,在训练过程中应用学习最优样本权重(LOW)策略,在未大幅增加训练时间的前提下,加强模型对数据集信息的利用;最后,优化模型细节,搭建完整的SCEL-DST模型。实验结果表明,SCE和LOW对SCEL-DST模型性能的提升至关重要,该模型在两个实验数据集上均取得了更高的联合目标准确率,其中在MultiWOZ 2.3 (Wizard-of-OZ 2.3)数据集上与相同条件下的TripPy(Triple coPy)相比提升了1.6个百分点,在WOZ 2.0 (Wizard-of-OZ 2.0)数据集上与AG-DST (Amendable Generation for Dialogue State Tracking)相比提升了2.0个百分点。
中图分类号:
石利锋, 倪郑威. 基于槽位相关信息提取的对话状态追踪模型[J]. 计算机应用, 2023, 43(5): 1430-1437.
Lifeng SHI, Zhengwei NI. Dialogue state tracking model based on slot correlation information extraction[J]. Journal of Computer Applications, 2023, 43(5): 1430-1437.
轮次(Turn) | 域槽对(Domain-slot pair) | 槽值(Value) | 类型(Type) | 共指(Coreference) |
---|---|---|---|---|
0 | restaurant-pricerange | expensive | span | |
0 | restaurant-area | south | span | |
1 | restaurant-name | cambridge chop house | informed | |
1 | restaurant-book_people | 2 | span | |
1 | restaurant-book_time | 14:15 | span | |
1 | restaurant-book_time | Sunday | span | |
2 | hotel-stars | 3 star | span | |
2 | hotel-area | south | coreference | restaurant-area |
2 | hotel-pricerange | expensive | coreference | restaurant-pricerange |
3 | hotel-name | lensfield hotel | informed | |
3 | hotel-book_people | two | span | |
3 | hotel-book_stay | two nights | span | |
3 | hotel-book_day | sunday | span |
图1 MultiWOZ 2.3中的对话样例
Fig. 1 Example dialogues in MultiWOZ 2.3
轮次(Turn) | 域槽对(Domain-slot pair) | 槽值(Value) | 类型(Type) | 共指(Coreference) |
---|---|---|---|---|
0 | restaurant-pricerange | expensive | span | |
0 | restaurant-area | south | span | |
1 | restaurant-name | cambridge chop house | informed | |
1 | restaurant-book_people | 2 | span | |
1 | restaurant-book_time | 14:15 | span | |
1 | restaurant-book_time | Sunday | span | |
2 | hotel-stars | 3 star | span | |
2 | hotel-area | south | coreference | restaurant-area |
2 | hotel-pricerange | expensive | coreference | restaurant-pricerange |
3 | hotel-name | lensfield hotel | informed | |
3 | hotel-book_people | two | span | |
3 | hotel-book_stay | two nights | span | |
3 | hotel-book_day | sunday | span |
模型 | 联合目标准确率 | 模型 | 联合目标准确率 |
---|---|---|---|
TRADE | 49.2 | SimpleTOD | 51.3 |
SUMBT | 52.9 | SAVN | 58.0 |
COMER | 50.2 | TripPy* | 61.6 |
SOM-DST | 55.5 | SCEL-DST | 63.2 |
表1 MultiWOZ 2.3数据集上不同模型的联合目标准确率对比 ( %)
Tab. 1 Comparison of joint goal accuracies of different models on MultiWOZ 2.3 dataset
模型 | 联合目标准确率 | 模型 | 联合目标准确率 |
---|---|---|---|
TRADE | 49.2 | SimpleTOD | 51.3 |
SUMBT | 52.9 | SAVN | 58.0 |
COMER | 50.2 | TripPy* | 61.6 |
SOM-DST | 55.5 | SCEL-DST | 63.2 |
模型 | 联合目标准确率 | 模型 | 联合目标准确率 |
---|---|---|---|
SUMBT | 91.0 | TripPy* | 90.9 |
GLAD | 88.1 | AG-DST | 91.4 |
GCE | 88.5 | SCEL-DST | 93.4 |
表2 WOZ 2.0数据集上不同模型的联合目标准确率对比 ( %)
Tab. 2 Comparison of joint goal accuracies of different models on WOZ 2.0 dataset
模型 | 联合目标准确率 | 模型 | 联合目标准确率 |
---|---|---|---|
SUMBT | 91.0 | TripPy* | 90.9 |
GLAD | 88.1 | AG-DST | 91.4 |
GCE | 88.5 | SCEL-DST | 93.4 |
模型 | 联合目标准确率/% | |
---|---|---|
MultiWOZ 2.3 | WOZ 2.0 | |
TripPy* | 61.6 | 90.9 |
TripPy*+LOW | 62.0 | 92.7 |
TripPy*+SCE | 62.8 | 92.6 |
SCEL-DST | 63.2 | 93.4 |
表3 消融实验结果
Tab. 3 Results of ablation experiments
模型 | 联合目标准确率/% | |
---|---|---|
MultiWOZ 2.3 | WOZ 2.0 | |
TripPy* | 61.6 | 90.9 |
TripPy*+LOW | 62.0 | 92.7 |
TripPy*+SCE | 62.8 | 92.6 |
SCEL-DST | 63.2 | 93.4 |
图5 不同模型在train、attraction和taxi领域中每一个槽位的槽位门分类的准确率
Fig. 5 Slot gate accuracies of different versions of models in each slot in train, attraction and taxi domains
槽位 | TripPy* | SCE-DST |
---|---|---|
train-leaveAt | 0.941 076 136 | 0.940 766 010 |
hotel-type | 0.956 427 353 | 0.959 993 797 |
hotel-area | 0.959 373 546 | 0.965 886 184 |
attraction-type | 0.970 072 880 | 0.971 313 382 |
restaurant-name | 0.970 693 131 | 0.971 623 508 |
attraction-area | 0.972 243 759 | 0.972 553 884 |
restaurant-area | 0.971 778 570 | 0.973 639 324 |
train-arriveBy | 0.968 987 440 | 0.973 639 324 |
taxi-destination | 0.978 756 396 | 0.976 275 392 |
taxi-departure | 0.976 585 517 | 0.976 275 392 |
restaurant-pricerange | 0.977 981 082 | 0.977 205 768 |
hotel-pricerange | 0.972 864 010 | 0.977 360 831 |
train-departure | 0.976 430 454 | 0.978 446 271 |
train-book_people | 0.974 879 826 | 0.978 756 396 |
hotel-name | 0.980 772 213 | 0.979 841 836 |
restaurant-food | 0.983 253 218 | 0.980 151 962 |
attraction-name | 0.982 012 715 | 0.983 563 343 |
hotel-parking | 0.981 857 652 | 0.983 718 406 |
hotel-stars | 0.979 841 836 | 0.984 803 846 |
hotel-internet | 0.977 826 020 | 0.985 113 971 |
train-destination | 0.986 819 662 | 0.987 284 850 |
taxi-arriveBy | 0.991 781 672 | 0.991 471 546 |
hotel-book_people | 0.995 968 367 | 0.991 936 734 |
hotel-book_day | 0.993 487 362 | 0.993 022 174 |
taxi-leaveAt | 0.992 556 986 | 0.994 262 676 |
train-day | 0.994 262 676 | 0.994 262 676 |
restaurant-book_people | 0.993 642 425 | 0.995 503 179 |
restaurant-book_day | 0.996 588 618 | 0.995 813 304 |
hotel-book_stay | 0.996 433 556 | 0.996 743 681 |
restaurant-book_time | 0.996 743 681 | 0.996 898 744 |
表4 SCEL-DST与TripPy*在MultiWOZ 2.3测试集上每一个槽位的准确率
Tab. 4 Slot accuracies of SCEL-DST and TripPy* on MultiWOZ 2.3 test set
槽位 | TripPy* | SCE-DST |
---|---|---|
train-leaveAt | 0.941 076 136 | 0.940 766 010 |
hotel-type | 0.956 427 353 | 0.959 993 797 |
hotel-area | 0.959 373 546 | 0.965 886 184 |
attraction-type | 0.970 072 880 | 0.971 313 382 |
restaurant-name | 0.970 693 131 | 0.971 623 508 |
attraction-area | 0.972 243 759 | 0.972 553 884 |
restaurant-area | 0.971 778 570 | 0.973 639 324 |
train-arriveBy | 0.968 987 440 | 0.973 639 324 |
taxi-destination | 0.978 756 396 | 0.976 275 392 |
taxi-departure | 0.976 585 517 | 0.976 275 392 |
restaurant-pricerange | 0.977 981 082 | 0.977 205 768 |
hotel-pricerange | 0.972 864 010 | 0.977 360 831 |
train-departure | 0.976 430 454 | 0.978 446 271 |
train-book_people | 0.974 879 826 | 0.978 756 396 |
hotel-name | 0.980 772 213 | 0.979 841 836 |
restaurant-food | 0.983 253 218 | 0.980 151 962 |
attraction-name | 0.982 012 715 | 0.983 563 343 |
hotel-parking | 0.981 857 652 | 0.983 718 406 |
hotel-stars | 0.979 841 836 | 0.984 803 846 |
hotel-internet | 0.977 826 020 | 0.985 113 971 |
train-destination | 0.986 819 662 | 0.987 284 850 |
taxi-arriveBy | 0.991 781 672 | 0.991 471 546 |
hotel-book_people | 0.995 968 367 | 0.991 936 734 |
hotel-book_day | 0.993 487 362 | 0.993 022 174 |
taxi-leaveAt | 0.992 556 986 | 0.994 262 676 |
train-day | 0.994 262 676 | 0.994 262 676 |
restaurant-book_people | 0.993 642 425 | 0.995 503 179 |
restaurant-book_day | 0.996 588 618 | 0.995 813 304 |
hotel-book_stay | 0.996 433 556 | 0.996 743 681 |
restaurant-book_time | 0.996 743 681 | 0.996 898 744 |
1 | 陈红燕. 面向任务的对话状态追踪方法及应用[D]. 哈尔滨:哈尔滨工业大学, 2020:3-4. |
CHEN H Y. Task-oriented dialogue state tracking and application[D]. Harbin: Harbin Institute of Technology, 2020:3-4 | |
2 | 黄伟. 任务型对话系统中对话状态追踪技术研究[D]. 兰州:兰州大学, 2021:6-7. |
HUANG W. Research on dialogue state tracking technology in task-based dialogue system[D]. Lanzhou: Lanzhou University, 2021:6-7. | |
3 | GAO S Y, SETHI A, AGARWAL S, et al. Dialog state tracking: a neural reading comprehension approach[C]// Proceedings of the 20th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Stroudsburg, PA: ACL, 2019: 264-273. 10.18653/v1/w19-5932 |
4 | GOEL R, PAUL S, HAKKANI-TÜR D. HyST: a hybrid approach for flexible and accurate dialogue state tracking[C]// Proceedings of the Interspeech 2019. [S.l.]: International Speech Communication Association, 2019:1458-1462. 10.21437/interspeech.2019-1863 |
5 | HECK M, van NIEKERK C, LUBIS N, et al. TripPy: a triple copy strategy for value independent neural dialog state tracking[C]// Proceedings of the 21st Annual Meeting of the Special Interest Group on Discourse and Dialogue. Stroudsburg, PA: ACL, 2020: 35-44. |
6 | SOVIANY P, IONESCU R T, ROTA P, et al. Curriculum learning: a survey[J]. International Journal of Computer Vision, 2022, 130(6):1526-1565. 10.1007/s11263-022-01611-x |
7 | RASTOGI A, ZANG X X, SUNKARA S, et al. Towards scalable multi-domain conversational agents: the schema-guided dialogue dataset[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2020:8689-8696. 10.1609/aaai.v34i05.6394 |
8 | LEE H, LEE J, KIM T Y. SUMBT: slot-utterance matching for universal and scalable belief tracking[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 5478-5483. 10.18653/v1/p19-1546 |
9 | WANG Y, GUO Y, ZHU S. Slot attention with value normalization for multi-domain dialogue state tracking[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2020: 3019-3028. 10.18653/v1/2020.emnlp-main.243 |
10 | XU P Y, HU Q. An end-to-end approach for handling unknown slot values in dialogue state tracking[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: ACL, 2018: 1448-1457. 10.18653/v1/p18-1134 |
11 | VINYALS O, FORTUNATO M, JAITLY N. Pointer networks[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems — Volume 2. Cambridge: MIT Press, 2015:2692-2700. |
12 | WU C S, MADOTTO A, HOSSEINI-ASL E, et al. Transferable multi-domain state generator for task-oriented dialogue systems[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 808-819. 10.18653/v1/p19-1078 |
13 | ZHANG J G, HASHIMOTO K, WU C S, et al. Find or classify? dual strategy for slot-value predictions on multi-domain dialog state tracking[C]// Proceedings of the 9th Joint Conference on Lexical and Computational Semantics. Stroudsburg, PA: ACL, 2020: 154-167. 10.21437/interspeech.2021-138 |
14 | LE H, SOCHER R, HOI S C H. Non-autoregressive dialog state tracking[EB/OL]. (2020-02-19) [2021-08-15].. 10.1145/3483845.3483880 |
15 | CHEN L, LV B E, WANG C, et al. Schema-guided multi-domain dialogue state tracking with graph attention neural networks[C]// Proceedings of 34th AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2020: 7521-7528. 10.1609/aaai.v34i05.6250 |
16 | AN J, CHO S, BANG J, et al. Domain-slot relationship modeling using a pre-trained language encoder for multi-domain dialogue state tracking[J]. IEEE/ACM Transactions on Audio, Speech and Language Processing, 2022, 30: 2091-2102. 10.1109/taslp.2022.3181350 |
17 | DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Stroudsburg, PA: ACL, 2019: 4171-4186. 10.18653/v1/n18-2 |
18 | YE F H, MANOTUMRUKSA J, ZHANG Q, et al. Slot self-attentive dialogue state tracking[C]// Proceedings of the Web Conference 2021. New York: ACM, 2021: 1598-1608. 10.1145/3442381.3449939 |
19 | DAI Y P, LI H Y, LI Y B, et al. Preview, attend and review: schema-aware curriculum learning for multi-domain dialog state tracking[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Stroudsburg, PA: ACL, 2021: 879-885. 10.18653/v1/2021.acl-short.111 |
20 | SANTIAGO C, BARATA C, SASDELLI M, et al. LOW: training deep neural networks by learning optimal sample weights[J]. Pattern Recognition, 2021, 110: No.107585. 10.1016/j.patcog.2020.107585 |
21 | HAN T, LIU X M, TAKANABU R, et al. MultiWOZ 2.3: a multi-domain task-oriented dialogue dataset enhanced with annotation corrections and co-reference annotation[C]// Proceedings of 2021 CCF International Conference on Natural Language Processing and Chinese Computing, LNCS 13029. Cham: Springer, 2021: 206-218. |
22 | WEN T H, VANDYKE D, MRKŠIĆ N, et al. A network-based end-to-end trainable task-oriented dialogue system[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1: Long Papers. Stroudsburg, PA: ACL, 2019: 438-449. |
23 | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2017: 6000-6010. |
[1] | 张凯, 覃正楚, 刘月, 秦心怡. 多学习行为协同的知识追踪模型[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1422-1429. |
[2] | 祖佳贞, 周永霞, 陈乐. 结合注意力的双分支残差低光照图像增强[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1240-1247. |
[3] | 窦光义, 魏发南, 邱创一, 巢建树. 基于注意力自相关机制的跟踪外观特征[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1248-1254. |
[4] | 郝巨鸣, 杨景玉, 韩淑梅, 王阳萍. 引入Ghost模块和ECA的YOLOv4公路路面裂缝检测方法[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1284-1290. |
[5] | 袁泉, 徐雲鹏, 唐成亮. 基于路径标签的文档级关系抽取方法[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1029-1035. |
[6] | 孙浩, 曹健, 李海生, 毛典辉. 基于改进胶囊网络的会话型推荐模型[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1043-1049. |
[7] | 朱周华, 齐琦. 基于改进YOLOv5s电动车头盔的自动检测与识别[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1291-1296. |
[8] | 王惠茹, 李秀红, 李哲, 马春明, 任泽裕, 杨丹. 多模态预训练模型综述[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 991-1004. |
[9] | 陈路, 陈道喜, 陆一鸣, 陆卫忠. 基于注意力机制编码器‒解码器的手写数学公式识别模型[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1297-1302. |
[10] | 孙杰, 吴绍鑫, 王学军, 华璟. 基于Sophon SC5+芯片构架的行人搜索算法与优化[J]. 《计算机应用》唯一官方网站, 2023, 43(3): 744-751. |
[11] | 尹聪, 胡汉平. 基于时间注意力机制的时滞混沌系统参数辨识模型[J]. 《计算机应用》唯一官方网站, 2023, 43(3): 842-847. |
[12] | 何雪东, 宣士斌, 王款, 陈梦楠. 融合累积分布函数和通道注意力机制的DeepLabV3+图像分割算法[J]. 《计算机应用》唯一官方网站, 2023, 43(3): 936-942. |
[13] | 李佳东, 张丹普, 范亚琼, 杨剑锋. 基于改进YOLOv5的轻量级船舶目标检测算法[J]. 《计算机应用》唯一官方网站, 2023, 43(3): 923-929. |
[14] | 邵小萌, 张猛. 融合注意力机制的时间卷积知识追踪模型[J]. 《计算机应用》唯一官方网站, 2023, 43(2): 343-348. |
[15] | 王萍, 陈楠, 鲁磊. 基于场景先验及注意力引导的跌倒检测算法[J]. 《计算机应用》唯一官方网站, 2023, 43(2): 529-535. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||