Journal of Computer Applications
Next Articles
Received:
Revised:
Accepted:
Online:
Published:
蒋玉茹1,2,那婷婷1,2*,李梦媛1,2,于燕超3
通讯作者:
基金资助:
Abstract: Since existing dialogue generation technologies in reminiscence therapy suffer from rigid interactions and poor guidance due to the lack of high-quality domain-specific corpora, a Scenario-driven Character Information Generation and Role-playing (SCIGR) approach was proposed to generate character information and construct multi-turn role-playing dialogue data. First, detailed image descriptions were produced to create reminiscence scenarios. Then, retrieval-augmented generation techniques and prompt engineering were integrated to develop character information that aligns with the life backgrounds of elderly individuals. Subsequently, photo-shooting stories were constructed to enrich the dialogue context. Finally, multi-turn dialogues were generated through role-playing based on the above information, resulting in the establishment of the first Chinese multi-turn dialogue dataset for image-based reminiscence therapy, named IRTDialogues. To verify the effectiveness of this dataset, a Multimodal Large Language Model (MLLM) was trained using Supervised Fine-Tuning (SFT), and a Multi-dimensional Multimodal Reminiscence Therapy benchmark (MMRT) covering five dimensions—including empathy and guidance—was developed for evaluation. Experimental results demonstrate that models train on the IRTDialogues dataset are capable of generating responses that are more contextually appropriate and emotionally supportive. This provides both data support and a new implementation pathway for developing intelligent elderly care technologies with greater humanistic concern.
Key words: reminiscence therapy, multi-turn dialogue generation, role-playing, intelligent elderly care, corpus construction, human-computer interaction
摘要: 针对现有对话生成技术在回忆疗法中因缺乏专用高质量语料导致交互生硬、引导效果不佳的问题,提出一种情景驱动的角色信息生成与角色扮演(SCIGR)多轮对话数据构建方法。首先,为图片生成详细描述以构建回忆情景;其次,融合检索增强生成技术与提示工程,创作贴近老年人生活背景的角色信息;再次,构建图片背后的拍摄故事以丰富对话上下文;最后,基于上述信息采用角色扮演技术生成多轮对话,构建首个中文图片回忆疗法多轮对话数据集IRTDialogues。为验证该数据集的有效性,采用多模态大语言模型(MLLM)进行监督微调(SFT),并构建包含共情性、引导性等5个维度的多维多模态回忆疗法评测基准(MMRT)进行评估。实验结果表明,基于IRTDialogues数据集训练的模型能生成更符合情境、更具情感支持性的回应,为构建更具人文关怀的智能养老技术提供了数据支持和新的实现路径。
关键词: 回忆疗法, 多轮对话生成, 角色扮演, 智能养老, 语料库构建, 人机交互
CLC Number:
TP391.1
蒋玉茹 那婷婷 李梦媛 于燕超. 情景驱动的多模态回忆疗法对话生成方法[J]. 《计算机应用》唯一官方网站, DOI: 10.11772/j.issn.1001-9081.2025101225.
0 / Recommend
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2025101225