Joint intent-slot recognition method based on prompt-guided generation and tri-mutual attention

doi:10.11772/j.issn.1001-9081.2025091092

Journal of Computer Applications ›› 2026, Vol. 46 ›› Issue (10): 3107-3116.DOI: 10.11772/j.issn.1001-9081.2025091092

• Artificial intelligence • Previous Articles

Joint intent-slot recognition method based on prompt-guided generation and tri-mutual attention

Shangzhi TENG¹, Jin HUANG¹, Xueqiang LYU¹, Xindong YOU²()

^1.Beijing Key Laboratory of Internet Culture Digital Dissemination （Beijing Information Science and Technology University），Beijing 100101，China
^2.School of Information Engineering，Beijing Institute of Graphic Communication，Beijing 102600，China

Received:2025-09-19 Revised:2026-01-07 Accepted:2026-01-14 Online:2026-03-16 Published:2026-10-10
Contact: Xindong YOU
About author:TENG Shangzhi， born in 1989， Ph. D.， associate professor. His research interests include natural language processing.
HUANG Jin， born in 2000， M. S. candidate. His research interests include natural language processing.
LYU Xueqiang， born in 1970， Ph. D.， professor. His research interests include artificial intelligence， big data.
Supported by:
National Natural Science Foundation of China(62171043)

基于提示引导生成与三元交互注意力的意图槽位联合识别方法

滕尚志¹, 黄进¹, 吕学强¹, 游新冬²()

^1.网络文化与数字传播北京市重点实验室（北京信息科技大学），北京 100101
^2.北京印刷学院信息工程学院，北京 102600

通讯作者: 游新冬
作者简介:滕尚志（1989—），男，江苏沛县人，副教授，博士，主要研究方向：自然语言处理
黄进（2000—），男，河北唐山人，硕士研究生，主要研究方向：自然语言处理
吕学强（1970—），男，辽宁抚顺人，教授，博士，CCF会员，主要研究方向：人工智能、大数据
基金资助:
国家自然科学基金资助项目(62171043)

Abstract

Abstract:

To address the semantic ambiguity and difficulty in candidate selection caused by the diverse results generated by Large Language Models （LLMs） in intent detection and slot filling tasks， a joint intent-slot recognition method based on prompt-guided generation and tri-mutual attention mechanism was proposed， named PGT-Joint （Prompt-guided Generation and Tri-mutual attention for Joint intent-slot recognition）. First， the prompts were constructed based on domain knowledge to guide the LLM in generating multiple candidate intent-slot combinations. Second， a tri-mutual attention mechanism was designed to model the high-order semantic relationships among user questions， candidate intents， and slots explicitly. Finally， a contrastive learning ranking model was used to filter and optimize the candidate results. Experimental results on a Mini/Micro LED domain question-answering dataset show that compared with current mainstream models except Global-Locally Graph Interaction Network （GL-GIN）， such as BERT-Intent and BERT-Slot， PGT-Joint improves the intent detection accuracy by 0.4 to 3.8 percentage points and the slot filling F1 score by 4.1 to 6.6 percentage points. Notably， the most significant gain is in overall accuracy， with an improvement of 5.2 to 10.2 percentage points. In addition， ablation experimental results verify the key roles of prompt generation and tri-mutual attention mechanism in performance improvement， indicating that PGT-Joint can enhance semantic understanding capabilities in domain-specific dialogue systems effectively.

Key words: intent detection, slot filling, Large Language Model (LLM), contrastive learning, attention mechanism

摘要：

针对大语言模型（LLM）在意图识别与槽位填充任务中生成结果多样性带来的语义歧义和候选选择困难问题，提出一种基于提示引导生成与三元交互注意力机制的意图?槽位联合识别方法——PGT-Joint（Prompt-guided Generation and Tri-mutual attention for Joint intent-slot recognition）。首先，结合领域知识构建提示，引导LLM生成多组候选意图?槽位组合；其次，设计三元交互注意力机制，显式建模用户问题、候选意图和槽位之间的高阶语义关系；最后，通过对比学习排序模型对候选结果进行筛选与优化。实验结果表明，在Mini/Micro LED领域问答数据集上，与全局?局部图交互网络（GL-GIN）以外的当前主流模型如BERT-Intent、BERT-Slot相比，PGT-Joint在意图识别准确率上的提升幅度达到0.4~3.8个百分点，在槽位填充F1值上的提升幅度达到4.1~6.6个百分点，在意图与槽位完全匹配准确率上的提升最显著，提升幅度达到5.2~10.2个百分点。此外，消融实验结果验证了提示生成和三元交互注意力机制在性能提升中的关键作用，说明PGT-Joint能够在专业领域对话系统中有效增强语义理解能力。

关键词: 意图识别, 槽位填充, 大语言模型, 对比学习, 注意力机制

CLC Number:

TP391

Shangzhi TENG, Jin HUANG, Xueqiang LYU, Xindong YOU. Joint intent-slot recognition method based on prompt-guided generation and tri-mutual attention[J]. Journal of Computer Applications, 2026, 46(10): 3107-3116.

滕尚志, 黄进, 吕学强, 游新冬. 基于提示引导生成与三元交互注意力的意图槽位联合识别方法[J]. 《计算机应用》唯一官方网站, 2026, 46(10): 3107-3116.

Figures/Tables 15

Tab. 1 Examples of LLM output candidate results

序号	候选意图	候选槽位信息
1	查询产品	｛"产业链环节"： "中游"，"产业类别"： "芯片材料"，"产品类型"： "GaN"｝
2	查询产品	｛"产业链环节"： "中游"，"产业类别"： "芯片材料"，"产品类型"： "衬底"｝
3	查询产品	｛"产业类别"： "芯片材料"，"产品类型"： "GaN"｝
4	查询产品	｛"产业类别"： "芯片材料"，"产品类型"： "衬底"｝
5	查询产品	｛"产业链环节"： "中游"｝

Fig. 1 Overall framework of PGT-Joint

Fig. 2 Structure of tri-mutual attention mechanism

Fig. 3 Flow of question-intent mutual attention

Fig. 4 Mini/Micro LED domain knowledge graph

Tab. 2 Entity-slot distribution of dataset

实体类型	对应槽位	出现次数	占比/%
标准	标准名称、标准号	419	10.48
产品类型	衬底、驱动IC、固晶机、划片机、显示器件等	396	9.90
企业	企业名称、统一社会信用代码	384	9.60
产业类别	芯片工艺、芯片产品、芯片材料、封装设备、显示系统等	377	9.43
产业链环节	上游、中游、下游	352	8.80
省市	北京市、四川省、湖北省、广东省、天津市等	340	8.50
专利	专利名称、专利号	321	8.03
政策	政策名称	302	7.55
区域	京津冀、长三角、珠三角、川渝	254	6.35
产品	产品名称	246	6.15
时间约束	年份、日期	243	6.08
其他实体属性	标准类型、标准起草单位、专利技术参数、政策来源等	364	9.13

Tab. 3 Slot distribution of dataset

槽位名称	出现次数	占比/%	槽位名称	出现次数	占比/%
政策名称	302	7.55	专利名称	175	4.38
标准名称	297	7.43	专利号	146	3.65
政策区域	254	6.35	专利产品类型	145	3.63
产品类型	251	6.28	企业社会信用代码	141	3.53
产品名称	246	6.15	标准号	122	3.05
企业名称	243	6.08	政策日期	103	2.58
产品厂家	236	5.90	政策来源	97	2.43
企业省市	203	5.08	标准发布年份	84	2.10
专利产业类别	198	4.95	标准类型	75	1.88
产品产业链环节	187	4.68	其他	493	12.32

Tab. 4 Examples of Mini/Micro LED domain dataset

问题	候选意图	候选槽位信息	正/负例
列出川渝地区 2023年发布的 Mini LED产业政策	政策分析	｛"政策区域"： "川渝"， "政策日期"： "2023"｝	负例
	政策分析	｛"政策区域"： "川渝"， "政策日期"： "2023"｝	负例
	政策查询	｛"政策省份"： "四川省"， "政策日期"： "2023"｝	负例
	政策查询	｛"政策省份"： "重庆市"， "政策日期"： "2023"｝	负例
	政策查询	｛"政策区域"： "川渝"， "政策日期"： "2023"｝	正例
光刻机适用的标准有哪些	标准查询	｛"标准产业类别"： "芯片设备及系统"， "标准产品类型"： "光刻机"｝	负例
	标准查询	｛"标准产业链环节"： "上游"， "标准产品类型"： "光刻机"｝	负例
	标准查询	｛"标准分类"： "行业标准"， "标准产品类型"： "光刻机"｝	负例
	标准查询	｛"标准产业类别"： "芯片设备及系统"， "标准产业链环节"： "上游"， "标准产品类型"： "光刻机"｝	负例
	标准查询	｛"标准产品类型"： "光刻机"｝	正例

Tab. 5 Experimental environment and training parameters

参数类别	参数名称	参数值
实验环境	GPU	NVIDIA Tesla V100
	Torch	1.12.1
	Python	3.10.0
	Ubuntu	16.04.6
模型配置	预训练模型	BERT-base-Chinese
	隐藏层维度	768
	Co-Attention输出维度	768×3
Prompt策略	Few-shot示例条数	5
Prompt策略	领域知识注入条数	相关度前3条
训练设置	Optimizer	AdamW
	Learning Rate	$2 × 10 - 5$
	Batch Size	16
	Epochs	15
	Max_length	128
	Loss Function	Triplet Margin Loss

Tab. 5 Experimental environment and training parameters

参数类别	参数名称	参数值
实验环境	GPU	NVIDIA Tesla V100
	Torch	1.12.1
	Python	3.10.0
	Ubuntu	16.04.6
模型配置	预训练模型	BERT-base-Chinese
	隐藏层维度	768
	Co-Attention输出维度	768×3
Prompt策略	Few-shot示例条数	5
Prompt策略	领域知识注入条数	相关度前3条
训练设置	Optimizer	AdamW
	Learning Rate	$2 × 10 - 5$
	Batch Size	16
	Epochs	15
	Max_length	128
	Loss Function	Triplet Margin Loss

Tab. 6 Performance comparison of different methods

方法	Intent Accuracy	Slot F1 Score	Overall Accuracy
BERT-Intent	94.16	—	—
BERT-Slot	—	82.64	—
JointBERT	93.18	81.92	70.80
SF-ID	94.40	83.01	71.53
AGIF	95.62	83.64	72.23
GL-GIN	97.32	83.69	74.21
MISCA	96.59	84.19	75.67
GLM-4	94.93	82.27	71.04
GPT-4.5	95.13	83.67	72.40
Qwen3-Max	95.81	84.43	75.81
PGT-Joint	97.01	88.58	81.02

Tab. 7 Examples of recognition results in comparative experiments

问题	方法	意图识别结果	槽位填充结果
GB/T30855—2014与 GB/T30856—2014的适用范围的区别	JointBERT	标准对比	｛"标准名称"：［"GB/T30855—2014"，"GB/T30856—2014"］｝
	AGIF	标准对比	｛"标准名称"：［"GB/T30855—2014"，"GB/T30856—2014"］｝
	MISCA	标准对比	｛"标准代码"：［"GB/T30855—2014"，"GB/T30856—2014"］｝
	PGT-Joint	标准对比	｛"标准代码"：［"GB/T30855—2014"，"GB/T30856—2014"］，"查找属性"："适用范围"｝
长三角LED发展纲要的主要内容是什么	JointBERT	政策查询	｛"政策地区"："长三角"，"政策名称"："LED发展纲要"｝
	AGIF	政策分析	｛"政策地区"："长三角"，"政策名称"："LED发展纲要"｝
	MISCA	政策分析	｛"政策地区"："长三角"，"政策名称"："LED发展纲要"｝
	PGT-Joint	政策分析	｛"政策名称"："长三角LED发展纲要"｝
上游衬底产品最多的企业是哪家公司	JointBERT	企业查询	｛"产品产业链环节"："上游"，"产品类型"："衬底"｝
	AGIF	产品统计	｛"产品产业链环节"："上游"，"产品类型"："衬底"｝
	MISCA	产品统计	｛"产品产业链环节"："上游"，"产品类型"："衬底"｝
	PGT-Joint	产品统计	｛"产品产业链环节"："上游"，"产品类型"："衬底"，"统计维度"："生产企业"｝

Tab. 8 Ablation experimental results of attention mechanisms

方法	Intent Accuracy	Slot F1 Score	Overall Accuracy
w/o All Co-Attn	94.64	82.90	73.21
w/o I-S Attn	95.37	83.48	74.21
w/o Q-I Attn	95.86	84.05	75.66
w/o Q-S Attn	96.11	83.48	75.42
PGT-Joint	97.01	88.58	81.02

Tab. 9 Ablation experimental results of Prompt strategies

方法	Intent Accuracy	Slot F1 Score	Overall Accuracy
w/o Zero-shot	91.24	79.58	70.31
w/o Knowledge	93.43	81.97	72.26
PGT-Joint	97.01	88.58	81.02

Tab. 10 Ablation experimental results of contrastive ranking models

方法	Intent Accuracy	Slot F1 Score	Overall Accuracy
w/o Re-Rank	94.89	82.45	72.74
Random Rank	92.45	81.13	71.78
PGT-Joint	97.01	88.58	81.02

Fig. 5 Slot prediction results with different frequencies

References 31

[1]	叶铱雷，曹斌，范菁，等. 面向任务型多轮对话的粗粒度意图识别方法［J］. 小型微型计算机系统， 2020， 41（8）： 1620-1626.
	Ye Yilei， Cao Bin， Fan Jing， et al. Coarse-grained intent recognition method for task-oriented multi-turn dialogue［J］. Journal of Chinese Computer Systems， 2020， 41（8）： 1620-1626.
[2]	Goo C W， Gao G， Hsu Y K， et al. Slot-gated modeling for joint slot filling and intent prediction［C］// NAACL-HLT 2018， Volume 2 （Short Papers）. Stroudsburg： ACL， 2018： 753-757.
[3]	Zhang C， Li Y， Du N， et al. Joint slot filling and intent detection via capsule neural networks［C］// ACL 2019. Stroudsburg： ACL， 2019： 5259-5267.
[4]	王丽花，杨文忠，姚苗，等. 意图识别与语义槽填充的双向关联模型［J］. 计算机工程与应用， 2021， 57（3）： 196-202.
	Wang Lihua， Yang Wenzhong， Yao Miao， et al. Bidirectional association model for intent detection and slot filling［J］. Computer Engineering and Applications， 2021， 57（3）： 196-202.
[5]	Lafferty J D， McCallum A， Pereira F C N. Conditional random fields： probabilistic models for segmenting and labeling sequence data［C］// 18th International Conference on Machine Learning. San Francisco： Morgan Kaufmann Publishers Inc.， 2001： 282-289.
[6]	Xu P， Sarikaya R. Convolutional neural network based triangular CRF for joint intent detection and slot filling［C］// 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. Piscataway： IEEE， 2013： 78-83.
[7]	任元凯，谢振平. 大语言模型领域意图的精准性增强方法［J］. 计算机应用研究， 2024， 41（10）： 2893-2899.
	Ren Yuankai， Xie Zhenping. Intention recognition accuracy enhancing method for large language model［J］. Application Research of Computers， 2024， 41（10）： 2893-2899.
[8]	Baan J， Daheim N， Ilia E， et al. Uncertainty in natural language generation： from theory to applications［PP/OL］. arXiv （2023-07-28）［2025-06-09］..
[9]	Zhang X， Wang H. A joint model of intent determination and slot filling for spoken language understanding［C］// IJCAI 2016. Palo Alto： AAAI Press， 2016： 2993-2999.
[10]	Chen Q， Zhuo Z， Wang W. BERT for joint intent classification and slot filling［PP/OL］. arXiv （2019-02-28）［2025-06-10］..
[11]	OpenAI. GPT-4 technical report［PP/OL］. V6. arXiv （2024-03-04）［2025-06-10］..
[12]	Team GLM. ChatGLM： a family of large language models from GLM-130B to GLM-4 all tools［PP/OL］. V2. arXiv （2024-07-30）［2025-06-10］..
[13]	Gupta P， Jiao C， Yeh Y T， et al. InstructDial： improving zero and few-shot generalization in dialogue through instruction tuning［C］// EMNLP 2022. Stroudsburg： ACL， 2022： 505-525.
[14]	Mirza P， Sudhi V， Sahoo S R， et al. ILLUMINER： instruction-tuned large language models as few-shot intent classifier and slot filler［C］// LREC-COLING 2024. ［S.l.］： ELRA and ICCL， 2024： 8639-8651.
[15]	Liu J， Shen D， Zhang Y， et al. What makes good in-context examples for GPT-3？［C］// DeeLIO 2022： the 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures. Stroudsburg： ACL， 2022： 100-114.
[16]	Wang R， Tang D， Duan N， et al. K-Adapter： infusing knowledge into pre-trained models with adapters［C］// Findings of the Association for Computational Linguistics： ACL-IJCNLP 2021. Stroudsburg： ACL， 2021： 1405-1418.
[17]	Chen X， Zhang N， Xie X， et al. KnowPrompt： knowledge-aware prompt-tuning with synergistic optimization for relation extraction［C］// WWW 2022. New York： ACM， 2022： 2778-2788.
[18]	Baek J， Aji A F， Saffari A. Knowledge-augmented language model prompting for zero-shot knowledge graph question answering［C］// NLRSE@ACL 2023. Stroudsburg： ACL， 2023： 78-106.
[19]	Zhang J， Bui T， Yoon S， et al. Few-shot intent detection via contrastive pre-training and fine-tuning［C］// EMNLP 2021. Stroudsburg： ACL， 2021： 1906-1912.
[20]	Liu H， Zhang F， Zhang X， et al. An explicit-joint and supervised-contrastive learning framework for few-shot intent classification and slot filling［C］// Findings of the Association for Computational Linguistics： EMNLP 2021. Stroudsburg： ACL， 2021： 1945-1955.
[21]	Wang Z， Wang P， Huang L， et al. Incorporating hierarchy into text encoder： a contrastive learning approach for hierarchical text classification［C］// ACL 2022 （Volume 1： Long Papers）. Stroudsburg： ACL， 2022： 7109-7119.
[22]	Qin L， Xu X， Che W， et al. AGIF： an adaptive graph-interactive framework for joint multiple intent detection and slot filling［C］// Findings of the Association for Computational Linguistics： EMNLP 2020. Stroudsburg： ACL， 2020： 1807-1816.
[23]	Qin L， Wei F， Xie T， et al. GL-GIN： fast and accurate non-autoregressive model for joint multiple intent detection and slot filling［C］// ACL-IJCNLP 2021 （Volume 1： Long Papers）. Stroudsburg： ACL， 2021： 178-188.
[24]	Pham T， Tran C， Nguyen D Q. MISCA： a joint model for multiple intent detection and slot filling with intent-slot co-attention［C］// Findings of the Association for Computational Linguistics： EMNLP 2023. Stroudsburg： ACL， 2023： 12641-12650.
[25]	Wei J， Wang X， Schuurmans D， et al. Chain-of-thought prompting elicits reasoning in large language models［C］// NeurIPS 2022. Red Hook： Curran Associates Inc.， 2022： 24824-24837.
[26]	Brown T B， Mann B， Ryder N， et al. Language models are few-shot learners［C］// NeurIPS 2020. Red Hook： Curran Associates Inc.， 2020： 1877-1901.
[27]	Kang M， Baek J， Hwang S J. KALA： knowledge-augmented language model adaptation［C］// NAACL-HLT 2022. Stroudsburg： ACL， 2022： 5144-5167.
[28]	Vaswani A， Shazeer N， Parmar N， et al. Attention is all you need［C］// NeurIPS 2017. Red Hook： Curran Associates Inc.， 2017： 6000-6010.
[29]	Karpukhin V， Oğuz B， Min S， et al. Dense passage retrieval for open-domain question answering［C］// EMNLP 2020. Stroudsburg： ACL， 2020： 6769-6781.
[30]	Reimers N， Gurevych I. Sentence-BERT： sentence embeddings using Siamese BERT-networks［C］// EMNLP-IJCNLP 2019. Stroudsburg： ACL， 2019： 3982-3992.
[31]	E H， Niu P， Chen Z， et al. A novel bi-directional interrelated model for joint intent detection and slot filling［C］// ACL 2019. Stroudsburg： ACL， 2019： 5467-5471.

Joint intent-slot recognition method based on prompt-guided generation and tri-mutual attention

基于提示引导生成与三元交互注意力的意图槽位联合识别方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 31

Related Articles 15

Recommended Articles

Metrics

[1]	Changqing JI, Ziying ZHANG, Zumin WANG. Continuous non-invasive blood pressure prediction method based on multi-scale Transformer with enhanced graph neural network [J]. Journal of Computer Applications, 2026, 46(9): 3043-3053.
[2]	Yufeng OU, Chia-Hung WANG. Small-object defect detection method for printed circuit boards based on dual-branch feature extraction [J]. Journal of Computer Applications, 2026, 46(9): 2977-2986.
[3]	Mengjiao YU, Min JIN, Xinhua LIU. Agile enhancement development framework driven by human-multi-agent collaboration [J]. Journal of Computer Applications, 2026, 46(9): 2938-2947.
[4]	Yun BIAN, Haiquan WANG, Yi CHEN, Zhe CUI. Construction method of software defect prediction dataset with explainability [J]. Journal of Computer Applications, 2026, 46(9): 2948-2958.
[5]	Lixia XIE, Chaoyue CAO, Hongyu YANG. ADS-B attack detection method based on improved DeepLabV3+ [J]. Journal of Computer Applications, 2026, 46(9): 2867-2876.
[6]	Zhenhui GONG, Xiaoyu SHI, Yun LU, Yangcheng LIU, Mingsheng SHANG. Fair recommendation framework for large language models with sensitive attribute absence [J]. Journal of Computer Applications, 2026, 46(9): 2838-2846.
[7]	Yaoran HUO, Huizhi LI, Dahe ZENG, Zhenyu TANG, Chenmei LIU, Yan WU, Xiao ZHANG, Siqin CHEN, Houhui YANG. Contract review framework based on dynamic rule generation and neuro-symbolic reasoning using large language models [J]. Journal of Computer Applications, 2026, 46(9): 2820-2826.
[8]	Yifan WANG, Hu HAN, Dong LI, Yating FAN, Lin LI. Heterogeneous enhancement and multi-source knowledge fusion for aspect-based sentiment analysis [J]. Journal of Computer Applications, 2026, 46(9): 2752-2760.
[9]	Jielong BAI, Chenyun FANG, Zhiwei QIAO. Sparse CT artifact suppresion Transformer network based on multi-scale attention adaptive fusion [J]. Journal of Computer Applications, 2026, 46(8): 2660-2667.
[10]	Yan WANG, Xi CHEN, Zhongkai ZHAO, Huan ZHOU, Tao WU, Mengge AI, Xuesong XIAO. Similarity-aware link prediction model for ethnic culture knowledge graph [J]. Journal of Computer Applications, 2026, 46(8): 2447-2456.
[11]	Dong LI, Yiji ZHAO, Haiyan DING, Hao WU. Soft whitening inspired non-contrastive SSL framework for time-series forecasting [J]. Journal of Computer Applications, 2026, 46(8): 2457-2466.
[12]	Guanghui LI, Licheng QU. Dynamic dilated convolution and hierarchical local-global attention model based on improved Lite-Mono architecture [J]. Journal of Computer Applications, 2026, 46(8): 2567-2576.
[13]	Huihui LYU, Zhaoman ZHONG, Yu ZHANG, Jidong FAN. Syntactically enhanced aspect-based sentiment analysis via mutual information maximization and contrastive learning [J]. Journal of Computer Applications, 2026, 46(8): 2440-2446.
[14]	Yuanhang HUANG, Na RONG. Model-free photovoltaic hosting capacity assessment method using attention mechanism and deep reinforcement learning [J]. Journal of Computer Applications, 2026, 46(8): 2708-2715.
[15]	Xiaojin GUO, Xuyang SUI, Kenan ZHOU. Channel estimation of STAR-RIS-assisted hybrid-field communication system based on deep learning [J]. Journal of Computer Applications, 2026, 46(8): 2548-2554.