Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (5): 1430-1437.DOI: 10.11772/j.issn.1001-9081.2022040508
• Artificial intelligence • Previous Articles
Received:
2022-04-11
Revised:
2022-08-10
Accepted:
2022-08-16
Online:
2023-05-08
Published:
2023-05-10
Contact:
Zhengwei NI
About author:
SHI Lifeng, born in 1998, M. S. candidate. His research interests include natural language processing, machine learning.Supported by:
通讯作者:
倪郑威
作者简介:
石利锋(1998—),男,浙江绍兴人,硕士研究生,CCF会员,主要研究方向:自然语言处理、机器学习基金资助:
CLC Number:
Lifeng SHI, Zhengwei NI. Dialogue state tracking model based on slot correlation information extraction[J]. Journal of Computer Applications, 2023, 43(5): 1430-1437.
石利锋, 倪郑威. 基于槽位相关信息提取的对话状态追踪模型[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1430-1437.
Add to citation manager EndNote|Ris|BibTeX
URL: http://www.joca.cn/EN/10.11772/j.issn.1001-9081.2022040508
轮次(Turn) | 域槽对(Domain-slot pair) | 槽值(Value) | 类型(Type) | 共指(Coreference) |
---|---|---|---|---|
0 | restaurant-pricerange | expensive | span | |
0 | restaurant-area | south | span | |
1 | restaurant-name | cambridge chop house | informed | |
1 | restaurant-book_people | 2 | span | |
1 | restaurant-book_time | 14:15 | span | |
1 | restaurant-book_time | Sunday | span | |
2 | hotel-stars | 3 star | span | |
2 | hotel-area | south | coreference | restaurant-area |
2 | hotel-pricerange | expensive | coreference | restaurant-pricerange |
3 | hotel-name | lensfield hotel | informed | |
3 | hotel-book_people | two | span | |
3 | hotel-book_stay | two nights | span | |
3 | hotel-book_day | sunday | span |
Fig. 1 Example dialogues in MultiWOZ 2.3
轮次(Turn) | 域槽对(Domain-slot pair) | 槽值(Value) | 类型(Type) | 共指(Coreference) |
---|---|---|---|---|
0 | restaurant-pricerange | expensive | span | |
0 | restaurant-area | south | span | |
1 | restaurant-name | cambridge chop house | informed | |
1 | restaurant-book_people | 2 | span | |
1 | restaurant-book_time | 14:15 | span | |
1 | restaurant-book_time | Sunday | span | |
2 | hotel-stars | 3 star | span | |
2 | hotel-area | south | coreference | restaurant-area |
2 | hotel-pricerange | expensive | coreference | restaurant-pricerange |
3 | hotel-name | lensfield hotel | informed | |
3 | hotel-book_people | two | span | |
3 | hotel-book_stay | two nights | span | |
3 | hotel-book_day | sunday | span |
模型 | 联合目标准确率 | 模型 | 联合目标准确率 |
---|---|---|---|
TRADE | 49.2 | SimpleTOD | 51.3 |
SUMBT | 52.9 | SAVN | 58.0 |
COMER | 50.2 | TripPy* | 61.6 |
SOM-DST | 55.5 | SCEL-DST | 63.2 |
Tab. 1 Comparison of joint goal accuracies of different models on MultiWOZ 2.3 dataset
模型 | 联合目标准确率 | 模型 | 联合目标准确率 |
---|---|---|---|
TRADE | 49.2 | SimpleTOD | 51.3 |
SUMBT | 52.9 | SAVN | 58.0 |
COMER | 50.2 | TripPy* | 61.6 |
SOM-DST | 55.5 | SCEL-DST | 63.2 |
模型 | 联合目标准确率 | 模型 | 联合目标准确率 |
---|---|---|---|
SUMBT | 91.0 | TripPy* | 90.9 |
GLAD | 88.1 | AG-DST | 91.4 |
GCE | 88.5 | SCEL-DST | 93.4 |
Tab. 2 Comparison of joint goal accuracies of different models on WOZ 2.0 dataset
模型 | 联合目标准确率 | 模型 | 联合目标准确率 |
---|---|---|---|
SUMBT | 91.0 | TripPy* | 90.9 |
GLAD | 88.1 | AG-DST | 91.4 |
GCE | 88.5 | SCEL-DST | 93.4 |
模型 | 联合目标准确率/% | |
---|---|---|
MultiWOZ 2.3 | WOZ 2.0 | |
TripPy* | 61.6 | 90.9 |
TripPy*+LOW | 62.0 | 92.7 |
TripPy*+SCE | 62.8 | 92.6 |
SCEL-DST | 63.2 | 93.4 |
Tab. 3 Results of ablation experiments
模型 | 联合目标准确率/% | |
---|---|---|
MultiWOZ 2.3 | WOZ 2.0 | |
TripPy* | 61.6 | 90.9 |
TripPy*+LOW | 62.0 | 92.7 |
TripPy*+SCE | 62.8 | 92.6 |
SCEL-DST | 63.2 | 93.4 |
槽位 | TripPy* | SCE-DST |
---|---|---|
train-leaveAt | 0.941 076 136 | 0.940 766 010 |
hotel-type | 0.956 427 353 | 0.959 993 797 |
hotel-area | 0.959 373 546 | 0.965 886 184 |
attraction-type | 0.970 072 880 | 0.971 313 382 |
restaurant-name | 0.970 693 131 | 0.971 623 508 |
attraction-area | 0.972 243 759 | 0.972 553 884 |
restaurant-area | 0.971 778 570 | 0.973 639 324 |
train-arriveBy | 0.968 987 440 | 0.973 639 324 |
taxi-destination | 0.978 756 396 | 0.976 275 392 |
taxi-departure | 0.976 585 517 | 0.976 275 392 |
restaurant-pricerange | 0.977 981 082 | 0.977 205 768 |
hotel-pricerange | 0.972 864 010 | 0.977 360 831 |
train-departure | 0.976 430 454 | 0.978 446 271 |
train-book_people | 0.974 879 826 | 0.978 756 396 |
hotel-name | 0.980 772 213 | 0.979 841 836 |
restaurant-food | 0.983 253 218 | 0.980 151 962 |
attraction-name | 0.982 012 715 | 0.983 563 343 |
hotel-parking | 0.981 857 652 | 0.983 718 406 |
hotel-stars | 0.979 841 836 | 0.984 803 846 |
hotel-internet | 0.977 826 020 | 0.985 113 971 |
train-destination | 0.986 819 662 | 0.987 284 850 |
taxi-arriveBy | 0.991 781 672 | 0.991 471 546 |
hotel-book_people | 0.995 968 367 | 0.991 936 734 |
hotel-book_day | 0.993 487 362 | 0.993 022 174 |
taxi-leaveAt | 0.992 556 986 | 0.994 262 676 |
train-day | 0.994 262 676 | 0.994 262 676 |
restaurant-book_people | 0.993 642 425 | 0.995 503 179 |
restaurant-book_day | 0.996 588 618 | 0.995 813 304 |
hotel-book_stay | 0.996 433 556 | 0.996 743 681 |
restaurant-book_time | 0.996 743 681 | 0.996 898 744 |
Tab. 4 Slot accuracies of SCEL-DST and TripPy* on MultiWOZ 2.3 test set
槽位 | TripPy* | SCE-DST |
---|---|---|
train-leaveAt | 0.941 076 136 | 0.940 766 010 |
hotel-type | 0.956 427 353 | 0.959 993 797 |
hotel-area | 0.959 373 546 | 0.965 886 184 |
attraction-type | 0.970 072 880 | 0.971 313 382 |
restaurant-name | 0.970 693 131 | 0.971 623 508 |
attraction-area | 0.972 243 759 | 0.972 553 884 |
restaurant-area | 0.971 778 570 | 0.973 639 324 |
train-arriveBy | 0.968 987 440 | 0.973 639 324 |
taxi-destination | 0.978 756 396 | 0.976 275 392 |
taxi-departure | 0.976 585 517 | 0.976 275 392 |
restaurant-pricerange | 0.977 981 082 | 0.977 205 768 |
hotel-pricerange | 0.972 864 010 | 0.977 360 831 |
train-departure | 0.976 430 454 | 0.978 446 271 |
train-book_people | 0.974 879 826 | 0.978 756 396 |
hotel-name | 0.980 772 213 | 0.979 841 836 |
restaurant-food | 0.983 253 218 | 0.980 151 962 |
attraction-name | 0.982 012 715 | 0.983 563 343 |
hotel-parking | 0.981 857 652 | 0.983 718 406 |
hotel-stars | 0.979 841 836 | 0.984 803 846 |
hotel-internet | 0.977 826 020 | 0.985 113 971 |
train-destination | 0.986 819 662 | 0.987 284 850 |
taxi-arriveBy | 0.991 781 672 | 0.991 471 546 |
hotel-book_people | 0.995 968 367 | 0.991 936 734 |
hotel-book_day | 0.993 487 362 | 0.993 022 174 |
taxi-leaveAt | 0.992 556 986 | 0.994 262 676 |
train-day | 0.994 262 676 | 0.994 262 676 |
restaurant-book_people | 0.993 642 425 | 0.995 503 179 |
restaurant-book_day | 0.996 588 618 | 0.995 813 304 |
hotel-book_stay | 0.996 433 556 | 0.996 743 681 |
restaurant-book_time | 0.996 743 681 | 0.996 898 744 |
1 | 陈红燕. 面向任务的对话状态追踪方法及应用[D]. 哈尔滨:哈尔滨工业大学, 2020:3-4. |
CHEN H Y. Task-oriented dialogue state tracking and application[D]. Harbin: Harbin Institute of Technology, 2020:3-4 | |
2 | 黄伟. 任务型对话系统中对话状态追踪技术研究[D]. 兰州:兰州大学, 2021:6-7. |
HUANG W. Research on dialogue state tracking technology in task-based dialogue system[D]. Lanzhou: Lanzhou University, 2021:6-7. | |
3 | GAO S Y, SETHI A, AGARWAL S, et al. Dialog state tracking: a neural reading comprehension approach[C]// Proceedings of the 20th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Stroudsburg, PA: ACL, 2019: 264-273. 10.18653/v1/w19-5932 |
4 | GOEL R, PAUL S, HAKKANI-TÜR D. HyST: a hybrid approach for flexible and accurate dialogue state tracking[C]// Proceedings of the Interspeech 2019. [S.l.]: International Speech Communication Association, 2019:1458-1462. 10.21437/interspeech.2019-1863 |
5 | HECK M, van NIEKERK C, LUBIS N, et al. TripPy: a triple copy strategy for value independent neural dialog state tracking[C]// Proceedings of the 21st Annual Meeting of the Special Interest Group on Discourse and Dialogue. Stroudsburg, PA: ACL, 2020: 35-44. |
6 | SOVIANY P, IONESCU R T, ROTA P, et al. Curriculum learning: a survey[J]. International Journal of Computer Vision, 2022, 130(6):1526-1565. 10.1007/s11263-022-01611-x |
7 | RASTOGI A, ZANG X X, SUNKARA S, et al. Towards scalable multi-domain conversational agents: the schema-guided dialogue dataset[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2020:8689-8696. 10.1609/aaai.v34i05.6394 |
8 | LEE H, LEE J, KIM T Y. SUMBT: slot-utterance matching for universal and scalable belief tracking[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 5478-5483. 10.18653/v1/p19-1546 |
9 | WANG Y, GUO Y, ZHU S. Slot attention with value normalization for multi-domain dialogue state tracking[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2020: 3019-3028. 10.18653/v1/2020.emnlp-main.243 |
10 | XU P Y, HU Q. An end-to-end approach for handling unknown slot values in dialogue state tracking[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: ACL, 2018: 1448-1457. 10.18653/v1/p18-1134 |
11 | VINYALS O, FORTUNATO M, JAITLY N. Pointer networks[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems — Volume 2. Cambridge: MIT Press, 2015:2692-2700. |
12 | WU C S, MADOTTO A, HOSSEINI-ASL E, et al. Transferable multi-domain state generator for task-oriented dialogue systems[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 808-819. 10.18653/v1/p19-1078 |
13 | ZHANG J G, HASHIMOTO K, WU C S, et al. Find or classify? dual strategy for slot-value predictions on multi-domain dialog state tracking[C]// Proceedings of the 9th Joint Conference on Lexical and Computational Semantics. Stroudsburg, PA: ACL, 2020: 154-167. 10.21437/interspeech.2021-138 |
14 | LE H, SOCHER R, HOI S C H. Non-autoregressive dialog state tracking[EB/OL]. (2020-02-19) [2021-08-15].. 10.1145/3483845.3483880 |
15 | CHEN L, LV B E, WANG C, et al. Schema-guided multi-domain dialogue state tracking with graph attention neural networks[C]// Proceedings of 34th AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2020: 7521-7528. 10.1609/aaai.v34i05.6250 |
16 | AN J, CHO S, BANG J, et al. Domain-slot relationship modeling using a pre-trained language encoder for multi-domain dialogue state tracking[J]. IEEE/ACM Transactions on Audio, Speech and Language Processing, 2022, 30: 2091-2102. 10.1109/taslp.2022.3181350 |
17 | DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Stroudsburg, PA: ACL, 2019: 4171-4186. 10.18653/v1/n18-2 |
18 | YE F H, MANOTUMRUKSA J, ZHANG Q, et al. Slot self-attentive dialogue state tracking[C]// Proceedings of the Web Conference 2021. New York: ACM, 2021: 1598-1608. 10.1145/3442381.3449939 |
19 | DAI Y P, LI H Y, LI Y B, et al. Preview, attend and review: schema-aware curriculum learning for multi-domain dialog state tracking[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Stroudsburg, PA: ACL, 2021: 879-885. 10.18653/v1/2021.acl-short.111 |
20 | SANTIAGO C, BARATA C, SASDELLI M, et al. LOW: training deep neural networks by learning optimal sample weights[J]. Pattern Recognition, 2021, 110: No.107585. 10.1016/j.patcog.2020.107585 |
21 | HAN T, LIU X M, TAKANABU R, et al. MultiWOZ 2.3: a multi-domain task-oriented dialogue dataset enhanced with annotation corrections and co-reference annotation[C]// Proceedings of 2021 CCF International Conference on Natural Language Processing and Chinese Computing, LNCS 13029. Cham: Springer, 2021: 206-218. |
22 | WEN T H, VANDYKE D, MRKŠIĆ N, et al. A network-based end-to-end trainable task-oriented dialogue system[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1: Long Papers. Stroudsburg, PA: ACL, 2019: 438-449. |
23 | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2017: 6000-6010. |
[1] | Kai ZHANG, Zhengchu QIN, Yue LIU, Xinyi QIN. Multi-learning behavior collaborated knowledge tracing model [J]. Journal of Computer Applications, 2023, 43(5): 1422-1429. |
[2] | Jiazhen ZU, Yongxia ZHOU, Le CHEN. Dual-branch residual low-light image enhancement combined with attention [J]. Journal of Computer Applications, 2023, 43(4): 1240-1247. |
[3] | Guangyi DOU, Fanan WEI, Chuangyi QIU, Jianshu CHAO. Tracking appearance features based on attention self-correlation mechanism [J]. Journal of Computer Applications, 2023, 43(4): 1248-1254. |
[4] | Juming HAO, Jingyu YANG, Shumei HAN, Yangping WANG. YOLOv4 highway pavement crack detection method using Ghost module and ECA [J]. Journal of Computer Applications, 2023, 43(4): 1284-1290. |
[5] | Quan YUAN, Yunpeng XU, Chengliang TANG. Document-level relation extraction method based on path labels [J]. Journal of Computer Applications, 2023, 43(4): 1029-1035. |
[6] | Hao SUN, Jian CAO, Haisheng LI, Dianhui MAO. Session-based recommendation model based on enhanced capsule network [J]. Journal of Computer Applications, 2023, 43(4): 1043-1049. |
[7] | Zhouhua ZHU, Qi QI. Automatic detection and recognition of electric vehicle helmet based on improved YOLOv5s [J]. Journal of Computer Applications, 2023, 43(4): 1291-1296. |
[8] | Lu CHEN, Daoxi CHEN, Yiming LU, Weizhong LU. Handwritten mathematical expression recognition model based on attention mechanism and encoder-decoder [J]. Journal of Computer Applications, 2023, 43(4): 1297-1302. |
[9] | Jie SUN, Shaoxin WU, Xuejun WANG, Jing HUA. Efficient person search algorithm and optimization with Sophon SC5+ chip architecture [J]. Journal of Computer Applications, 2023, 43(3): 744-751. |
[10] | Xuedong HE, Shibin XUAN, Kuan WANG, Mengnan CHEN. DeepLabV3+ image segmentation algorithm fusing cumulative distribution function and channel attention mechanism [J]. Journal of Computer Applications, 2023, 43(3): 936-942. |
[11] | Cong YIN, Hanping HU. Parameter identification model for time-delay chaotic systems based on temporal attention mechanism [J]. Journal of Computer Applications, 2023, 43(3): 842-847. |
[12] | Jiadong LI, Danpu ZHANG, Yaqiong FAN, Jianfeng YANG. Lightweight ship target detection algorithm based on improved YOLOv5 [J]. Journal of Computer Applications, 2023, 43(3): 923-929. |
[13] | Ping WANG, Nan CHEN, Lei LU. Fall detection algorithm based on scene prior and attention guidance [J]. Journal of Computer Applications, 2023, 43(2): 529-535. |
[14] | Xiaomeng SHAO, Meng ZHANG. Temporal convolutional knowledge tracing model with attention mechanism [J]. Journal of Computer Applications, 2023, 43(2): 343-348. |
[15] | Cong LIU, Genshun WAN, Jianqing GAO, Zhonghua FU. End-to-end speech recognition method based on prosodic features [J]. Journal of Computer Applications, 2023, 43(2): 380-384. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||