Abstract:

Software defect prediction often lacks explainable information， such as defect localization， defect explanation， and repair suggestions， making the prediction results difficult to be applied in actual development. To address this problem， an explainable method for constructing a software defect prediction dataset was proposed on the basis of context engineering and Large Language Models （LLMs）， and HandPick， the first multi-programming-language dataset for software defect prediction， was released. First， the TriCogVuln-LLM （LLM-enhanced Triple Cognitive chain for multilingual code Vulnerability mining with common weakness enumeration knowledge） method was designed on the basis of software engineering principles and prior defect knowledge， so as to guide LLMs to complete function description generation， Common Weakness Enumeration （CWE） defect predictions， and repair suggestion generation step by step. Second， a consensus voting mechanism was designed to form an optimal generative model pool for defect prediction， thereby further improving the quality and diversity of the generated data. Finally， the HandPick dataset， covering four mainstream programming languages， was constructed through consensus-driven automated data generation. Downstream task validation results show that the Qwen2.5-14B-HandPick model， fine-tuned on the HandPick dataset， achieves significant improvements compared to baseline models on an independent and public test set， with gains of 19.29， 21.26， 24.11， and 18.30 percentage points in precision， recall， F1-score， and accuracy， respectively. This confirms that the HandPick dataset， generated by the TriCogVuln-LLM method， enhances model's defect identification and analysis capabilities significantly.

Key words: explainability, software defect prediction, context engineering, Large Language Model (LLM), Common Weakness Enumeration (CWE)

摘要：

针对软件缺陷预测缺乏缺陷定位、缺陷解释及修复建议等可解释性信息，导致预测结果难以在实际开发中应用的问题，提出一种基于上下文工程与大语言模型（LLM）具备可解释性的软件缺陷预测数据集构建方法，并发布首个配套多编程语言的软件缺陷预测数据集HandPick。首先，基于软件工程原则和缺陷先验知识，设计TriCogVuln-LLM（LLM-enhanced Triple Cognitive chain for multilingual code Vulnerability mining with common weakness enumeration knowledge）方法，引导LLM依次完成功能描述生成、通用缺陷枚举（CWE）缺陷预测和缺陷修复建议生成；其次，设计共识投票机制，以构建缺陷预测的最佳生成模型池，进一步提升生成数据的质量与多样性；最后，利用共识驱动的自动化数据生成，构建包含4种主流编程语言的软件缺陷预测数据集HandPick。下游任务验证结果表明，与基线模型相比，采用HandPick数据集微调后的Qwen2.5-14B-HandPick模型在独立公开测试集上的表现有显著提升，精确率、召回率、F1分数与准确率分别提升了19.29、21.26、24.11与18.30个百分点，验证了TriCogVuln-LLM方法构建的HandPick数据集能显著提升模型的缺陷识别和分析能力。

关键词: 可解释性, 软件缺陷预测, 上下文工程, 大语言模型, 通用缺陷枚举