关键词: 扩散模型, 序列扩散, 预训练模型, 文本生成, 可控生成, 细粒度情感

Abstract: Abstract: With the emergence of large-scale pre trained language models, text generation technology has made breakthrough progress. However, in the field of open text generation, the generated content lacks anthropomorphic emotional features, making it difficult for the generated text to resonate and connect emotionally. Controllable text generation is of great significance in compensating for the shortcomings of current text generation technologies. Firstly, the extension of theme and emotional attributes was completed on the basis of the ChnSensiCorp dataset. At the same time, in order to construct a multi variable controllable text generation model that can generate smooth text and rich emotions, a diffusion sequence based controllable text generation model DiffuSeq PT was proposed. This model is based on a diffusion model architecture, utilizing theme emotion attributes and text data to perform the diffusion process on the sequence under the guidance of no classifier. The encoding and decoding capabilities of the pre trained model ERNIE(Enhanced Representation through Knowledge Integration) are used to fit the noise and denoising process of the diffusion model, and ultimately, target text that matches the relevant theme and multi sentiment granularity can be generated. Compared with the benchmark model DiffuSeq, the proposed model achieved an improvement in BERTScore values of 0.014-0.128 on two publicly available real datasets (ChnSensiCorp and Debate datasets), while reducing confusion by 9.4-14.3.

Key words: diffusion model, Sequence diffusion, Pre-training model, Text generation, Controllable generation, Fine-grained emotion