Abstract:

In order to reduce the regression test set and improve the efficiency of regression test in the Continuous Integration （CI） environment， a regression test suite selection method for the CI environment was proposed. First， the commits were prioritized based on the historical failure rate and execution rate of each test suite related to each commit. Then， the machine learning method was used to predict the failure rates of the test suites involved in each commit， and the test suite with the higher failure rate were selected. In this method， the commit prioritization technology and the test suite selection technology were combined to ensure the increase of the failure detection rate and the reduction of the test cost. Experimental results on Google’s open-source dataset show that compared to the methods with the same commit prioritization method and test suite selection method， the proposed method has the highest improvement in the Average Percentage of Faults Detected per cost （APFDc） by 1% to 27%； At the same cost of test time， the TestRecall of this method increases by 33.33 to 38.16 percentage points， the ChangeRecall increases by 15.67 to 24.52 percentage points， and the test suite SelectionRate decreases by about 6 percentage points.

Key words: Continuous Integration (CI), regression test, commit prioritization, test suite selection, test suite failure rate prediction

摘要：

为在持续集成（CI）环境下减少回归测试集、提升回归测试的效率，提出一种适用于CI环境的回归测试套件选择方法。首先，根据每个提交的测试套件历史失败率和执行率信息，进行提交排序；然后，采用机器学习方法，对提交涉及的测试套件进行失败率预测，并选择具有较高失败率的测试套件。该方法综合使用提交排序技术和测试套件选择技术，从而保证既提高故障检测率又能在一定程度上降低测试成本。在Google的开源数据集上进行的实验结果表明：与同样采用提交排序的方法和采用测试套件选择的方法相比，所提方法的开销感知平均故障检测率APFDc提高了1%~27%；在相同的测试时间成本下，所提方法的测试召回提高了33.33~38.16个百分点，变更召回提高了15.67~24.52个百分点，测试套件选择率降低了约6个百分点。

关键词: 持续集成, 回归测试, 提交排序, 测试套件选择, 测试套件失败率预测