基于组合分类算法的源代码注释质量评估方法

doi:10.11772/j.issn.1001-9081.2016.12.3448

计算机应用 ›› 2016, Vol. 36 ›› Issue (12): 3448-3453.DOI: 10.11772/j.issn.1001-9081.2016.12.3448

基于组合分类算法的源代码注释质量评估方法

余海^1,2,3, 李斌^2,3,4, 王培霞^2,3,4, 贾荻³, 王永吉^1,4

1. 中国科学院软件研究所互联网软件技术实验室, 北京 100190;
2. 中国科学院大学, 北京 100190;
3. 中国科学院软件研究所总体部, 北京 100190;
4. 中国科学院软件研究所基础软件国家工程研究中心, 北京 100190

收稿日期:2016-06-08 修回日期:2016-06-20 出版日期:2016-12-10 发布日期:2016-12-08
通讯作者: 王永吉
作者简介:余海(1989-),男,河南信阳人,硕士研究生,主要研究方向:操作系统、机器学习;李斌(1985-),男,甘肃天水人,工程师,博士研究生,主要研究方向:操作系统、代码分析;王培霞(1981-),女,山东潍坊人,高级工程师,博士研究生,主要研究方向:信息检索、自然语言处理;贾荻(1989-),女,北京人,助理工程师,硕士,主要研究方向:操作系统、数据处理;王永吉(1963-),男,辽宁营口人,研究员,博士,CCF高级会员,主要研究方向:虚拟化技术、隐蔽信道、实时系统、人工智能、数据挖掘、软件工程。
基金资助:
国家科技重大专项（2014ZX01029101-002）。

Source code comments quality assessment method based on aggregation of classification algorithms

YU Hai^1,2,3, LI Bin^2,3,4, WANG Peixia^2,3,4, JIA Di³, WANG Yongji^1,4

1. Laboratory for Internet Software Technologies, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China;
2. University of Chinese Academy of Sciences, Beijing 100190, China;
3. General Department, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China;
4. National Engineering Research Center of Fundamental Software, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

Received:2016-06-08 Revised:2016-06-20 Online:2016-12-10 Published:2016-12-08
Supported by:
This work is partially supported by the National Science and Technology Major Project (2014ZX01029101-002).

摘要/Abstract

摘要： 源代码注释是软件的重要组成部分，研究者往往需要利用人工或自动化的方法产生分析注释，注释的质量评估也往往是通过人工来完成，这无疑是低效不客观的。为此，首先从注释的格式、语言形式、内容以及与代码相关度4个方面出发构建注释评估准则；进而，基于这一准则提出了一种基于组合分类算法的注释质量评估方法。该方法将机器学习以及自然语言处理技术引入到注释质量评估中来，利用分类算法将注释分为不合格、合格、良好、优秀四个等级。通过对基本分类算法的组合使用，使得评估效果进一步提高。组合分类算法的准确率和F1值较单独使用某一种分类算法提高20个百分点左右，除宏平均F1值外，各项指标都达到了70%以上。实验结果表明，所提方法能够很好地应用于注释质量评估。

关键词: 源码注释, 质量评估, 文本分类, 组合算法, 自然语言处理

Abstract: Source code comments is an important part of the software, so researchers need to use manual or automated methods to generate comments. In the past, the quality assessment of this kind of comments is done manually, which is inefficient and not objective. In order to solve this problem, an assessment criterion was built in which four aspects of the comments including comment format, language form, content and code-related degree were considered. Then a code comments quality assessment method based on an aggregation of classification algorithms was proposed, in which machine learning and natural language processing technology were introduced into comments quality assessment, by using classification algorithms the comments were classified into four levels, including unqualified, qualified, good and excellent ones. The evaluation results were improved by the aggregation of the basic classification algorithms. The precision and F1 measure of the aggregated classification algorithm were improved about 20 percentage points compared with using a single classification algorithm, and all the indexes have reached more than 70% except the macro average F1 measure. The experimental results show that this method can be applied to assess the quality of comments effectively.

Key words: source code comments, quality assessment, text classification, aggregation algorithm, natural language processing

中图分类号:

TP311

余海, 李斌, 王培霞, 贾荻, 王永吉. 基于组合分类算法的源代码注释质量评估方法[J]. 计算机应用, 2016, 36(12): 3448-3453.

YU Hai, LI Bin, WANG Peixia, JIA Di, WANG Yongji. Source code comments quality assessment method based on aggregation of classification algorithms[J]. Journal of Computer Applications, 2016, 36(12): 3448-3453.

参考文献

[1] TAN S H, MARINOV D, TAN L, et al.@tComment:testing javadoc comments to detect comment-code inconsistencies[C]//Proceedings of the 5th IEEE International Conference on Software Testing, Verification and Validation. Washington, DC:IEEE Computer Society, 2012:260-269.
[2] DE SOUZA S C B, ANQUETIL N, DE OLIVEIRA K M. A study of the documentation essential to software maintenance[C]//Proceedings of the 23rd Annual International Conference on Design of Communication:Documenting & Designing for Pervasive Information. New York:ACM, 2005:68-75.
[3] KAJKO-MATTSSON M. A survey of documentation practice within corrective maintenance[J]. Empirical Software Engineering, 2005, 10(1):31-55.
[4] WONG E, YANG J, TAN L. AutoComment:mining question and answer sites for automatic comment generation[C]//Proceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering. Piscataway, NJ:IEEE, 2013:562-567.
[5] WONG E, LIU T, TAN L. CloCom:mining existing source code for automatic comment generation[C]//Proceedings of the 22nd IEEE International Conference on Software Analysis, Evolution, and Reengineering. Piscataway, NJ:IEEE, 2015:380-389.
[6] MORENO L, APONTE J, SRIDHARA G, et al. Automatic generation of natural language summaries for Java classes[C]//Proceedings of the 21st IEEE International Conference on Program Comprehension. Piscataway, NJ:IEEE, 2013:23-32.
[7] SRIDHARA G, POLLOCK L, VIJAY-SHANKER K. Generating parameter comments and integrating with method summaries[C]//Proceedings of the 19th IEEE International Conference on Program Comprehension. Piscataway, NJ:IEEE, 2011:71-80.
[8] SRIDHARA G, HILL E, MUPPANENI D, et al. Towards automatically generating summary comments for Java methods[C]//Proceedings of the IEEE/ACM International Conference on Automated Software Engineering. New York:ACM, 2010:43-52.
[9] TAN L, ZHOU Y, PADIOLEAU Y. aComment:mining annotations from comments and code to detect interrupt related concurrency bugs[C]//Proceedings of the 33rd International Conference on Software Engineering. New York:ACM, 2011:11-20.
[10] HIRATA Y, MIZUNO O. Do comments explain codes adequately?:investigation by text filtering[C]//Proceedings of the 8th Working Conference on Mining Software Repositories. New York:ACM, 2011:242-245.
[11] ARAFAT O, RIEHLE D. The commenting practice of open source[C]//Proceedings of the 24th ACM SIGPLAN Conference Companion on Object Oriented Programming Systems Languages and Applications. New York:ACM, 2009:857-864.
[12] STOREY M, RYALL J, BULL R I, et al. TODO or to bug:exploring how task annotations play a role in the work practices of software developers[C]//Proceedings of the 30th International Conference on Software Engineering. New York:ACM, 2008:251-260.
[13] KHAMIS N, WITTE R E, RILLING A J. Automatic quality assessment of source code comments:the JavadocMiner[C]//Proceedings of the 2010 Natural Language Processing and Information Systems, and 15th International Conference on Applications of Natural Language to Information Systems. Berlin:Springer-Verlag, 2010:68-79.
[14] FLURI B, WURSCH M, GALL H C. Do code and comments co-evolve? On the relation between source code and comment changes[C]//Proceedings of the 14th Working Conference on Reverse Engineering. Washington, DC:IEEE Computer Society, 2007:70-79.
[15] STEIDL D, HUMMEL B, JUERGENS E. Quality analysis of source code comments[C]//Proceedings of the 21st IEEE International Conference on Program Comprehension. Piscataway, NJ:IEEE, 2013:83-92.
[16] DIKLI S. An overview of automated scoring of essays[J]. The Journal of Technology, Learning, and Assessment, 2006, 5(1):1-36.
[17] XI Y, LIANG W. Automated computer-based CET4 essay scoring system[C]//Proceedings of the 3rd Pacific-Asia Conference on Circuits, Communications and System. Piscataway, NJ:IEEE, 2011:1-4.
[18] LI B, LU J, YAO J M, et al. Automated essay Scoring using the KNN algorithm[C]//Proceedings of the 2008 International Conference on Computer Science and Software Engineering. Piscataway, NJ:IEEE, 2008:735-738.
[19] ATTALI Y, BURSTEIN J. Automated essay scoring with e-rater? V. 2[J]. The Journal of Technology, Learning, and Assessment, 2006, 4(3):1-31.
[20] 黄志娥,谢佳莉,荀恩东.HSK自动作文评分的特征选取研究[J].计算机工程与应用,2014,50(6):118-122.(HUANG Z E, XIE J L, XUN E D. Study of feature selection in HSK automated essay scoring[J]. Computer Engineering and Applications, 2014, 50(6):118-122.)
[21] 彭星源, 柯登峰, 赵知,等.基于词汇评分的汉语作文自动评分[J].中文信息学报,2012,26(2):102-108.(PENG X Y, KE D F, ZHAO Z, et al. Automated Chinese essay scoring based on word scores[J]. Journal of Chinese Information Processing, 2012, 26(2):102-108.)
[22] 江进林.近五十年来自动评分研究综述——兼论中国学生英译汉机器评分系统的新探索[J].现代教育技术,2013,23(6):62-66.(JIANG J L. Rethinking 50 years of studies on automated scoring-explorations of computer scoring system for English-Chinese translations of Chinese learners[J]. Modern Educational Technology, 2013, 23(6):62-66.)
[23] POWERS D E, BURSTEIN J C, CHODOROW M, et al. Stumping e-rater:challenging the validity of automated essay scoring[J]. Computers in Human Behavior, 2002, 18(2):103-134.
[24] ZHANG Y, LO D, XIA X, et al. An empirical study of classifier combination for cross-project defect prediction[C]//Proceedings of the 39th IEEE Annual International Computers, Software & Applications Conference. Piscataway, NJ:IEEE, 2015:264-269.
[25] 王正群,孙兴华,杨静宇.多分类器组合研究[J].计算机工程与应用,2002,38(20):84-85.(WANG Z Q, SUN X H, YANG J Y. Study on multiple classifiers combination[J]. Computer Engineering and Applications, 2002, 38(20):84-85.)
[26] 付忠良.分类器线性组合的有效性和最佳组合问题的研究[J].计算机研究与发展,2009,46(7):1206-1216.(FU Z L. Effective property and best combination of classifier linear combination[J]. Journal of Computer Research and Development, 2009, 46(7):1206-1216.)
[27] PEDREGOSA F, VAROQUAUX G, GRAMFORT A, et al. Scikit-learn:machine learning in Python[J]. The Journal of Machine Learning Research, 2011, 12(10):2825-2830.
[28] 卢苇, 彭雅.几种常用文本分类算法性能比较与分析[J]. 湖南大学学报(自然科学版),2007,34(6):67-69.(LU W, PENG Y. Performance comparison and analysis of several general text classification algorithms[J]. Journal of Hunan University (Natural Sciences), 2007, 34(6):67-69.)

基于组合分类算法的源代码注释质量评估方法

Source code comments quality assessment method based on aggregation of classification algorithms

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[2]	刘雅璇, 钟勇. 基于头实体注意力的实体关系联合抽取方法[J]. 计算机应用, 2021, 41(9): 2517-2522.
[3]	周险兵, 樊小超, 任鸽, 杨勇. 基于多层次语义特征的英文作文自动评分方法[J]. 计算机应用, 2021, 41(8): 2205-2211.
[4]	张洋, 江铭虎. 基于注意力机制的文本作者识别[J]. 计算机应用, 2021, 41(7): 1897-1901.
[5]	李文惠, 曾上游, 王金金. 基于改进注意力机制的图像描述生成算法[J]. 计算机应用, 2021, 41(5): 1262-1267.
[6]	王朱君, 王石, 李雪晴, 朱俊武. 基于深度学习的事件因果关系抽取综述[J]. 计算机应用, 2021, 41(5): 1247-1255.
[7]	李雪晴, 王石, 王朱君, 朱俊武. 自然语言生成综述[J]. 计算机应用, 2021, 41(5): 1227-1235.
[8]	刘睿珩, 叶霞, 岳增营. 面向自然语言处理任务的预训练模型综述[J]. 计算机应用, 2021, 41(5): 1236-1246.
[9]	温超东, 曾诚, 任俊伟, 张. 结合ALBERT和双向门控循环单元的专利文本分类[J]. 计算机应用, 2021, 41(2): 407-412.
[10]	张阳, 王小宁. 基于Word2Vec词嵌入和高维生物基因选择遗传算法的文本特征选择方法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3151-3155.
[11]	杨璐, 何明祥. 基于门控机制和卷积神经网络的中文文本情感分析模型[J]. 计算机应用, 2021, 41(10): 2842-2848.
[12]	廖胜兰, 殷实, 陈小平, 张波, 欧阳昱, 张衡. 面向电力业务对话系统的意图识别数据集[J]. 计算机应用, 2020, 40(9): 2549-2554.
[13]	尹春勇, 何苗. 基于改进胶囊网络的文本分类[J]. 计算机应用, 2020, 40(9): 2525-2530.
[14]	陈佛计, 朱枫, 吴清潇, 郝颖明, 王恩德. 基于生成对抗网络的红外图像数据增强[J]. 计算机应用, 2020, 40(7): 2084-2088.
[15]	王敏蕊, 高曙, 袁自勇, 袁蕾. 基于动态路由序列生成模型的多标签文本分类方法[J]. 计算机应用, 2020, 40(7): 1884-1890.