Journal of Computer Applications ›› 2017, Vol. 37 ›› Issue (12): 3569-3573.DOI: 10.11772/j.issn.1001-9081.2017.12.3569

Previous Articles     Next Articles

Evaluation model of mobile application crowdsourcing testers

LIU Ying, ZHANG Tao, LI Kun, LI Nan   

  1. School of Software and Microelectronics, Northwestern Polytechnical University, Xi'an Shaanxi 710072, China
  • Received:2016-05-20 Revised:2017-07-24 Online:2017-12-10 Published:2017-12-18
  • Supported by:
    This work is partially supported by the Industrial Science and Technology Research Project of Shaanxi Province (2016GY-100), the Aerospace Science and Technology Support Program (2014HTXGD), the Aerospace CAST-BISEE Fund (2015MC1001061).

移动应用众包测试人员评价模型

刘莹, 张涛, 李坤, 李楠   

  1. 西北工业大学 软件与微电子学院, 西安 710072
  • 通讯作者: 刘莹
  • 作者简介:刘莹(1996-),女,四川绵阳人,硕士研究生,主要研究方向:移动应用测试、众包测试;张涛(1976-),男,陕西宝鸡人,副教授,博士,主要研究方向:软件安全、移动应用测试;李坤(1993-),男,陕西西安人,硕士研究生,主要研究方向:移动应用测试;李楠(1996-),女,陕西商洛人,硕士研究生,主要研究方向:软件安全。
  • 基金资助:
    陕西省工业科技攻关项目(2016GY-100);航天科技支撑计划项目(2014HTXGD);航天CAST-BISEE基金资助项目(2015MC1001061)。

Abstract: Mobile application crowdsourcing testers are anonymous, non-contractual, which makes it difficult for task publishers to accurately evaluate the ability of crowdsourcing testers and quality of test results.To solve these problems, a new evaluation model of Analytic Hierarchy Process (AHP) for mobile application crowdsouring testers was proposed. The ability of crowdsourcing testers was evaluated comprehensively and hierarchically by using the multiple indexes, such as activity degree, test ability and integrity degree. The combination weight vector of each level index was calculated by constructing the judgment matrix and consistency test. Then, the proposed model was improved by introducing the requirement list and description list, which made testers and crowdsourcing tasks match better. The experimental results show that the proposed model can evaluate the ability of testers accurately, support the selection and recommendation of crowdsourcing testers based on the evaluation results, and improve the efficiency and quality of mobile application crowdsourcing testing.

Key words: mobile application crowdsourcing testing, personnel evaluation, Analytic Hierarchy Process (AHP), Pearson correlation coefficient, Spearman correlation coefficient

摘要: 移动应用众包测试人员具有匿名、非契约的特性,这使得任务发布者难以准确评估众包测试人员的能力与测试质量。针对该问题,提出了一种移动应用众包测试人员层次分析法(AHP)评价模型。该模型从活跃度、测试能力、诚信度等多指标分层综合评估众包测试人员能力,通过构造判断矩阵、一致性检验计算各层次指标的组合权重向量,并引入需求列表与描述列表改进本模型,使测试人员与众包任务更加匹配。实验结果表明,所提模型能够实现对测试人员能力的准确评估,支持基于评估结果的众包测试人员选择与推荐,提高了移动应用众包测试效率与质量。

关键词: 移动应用众包测试, 人员评价, 层次分析法, 皮尔逊相关性系数, 斯皮尔曼相关系数

CLC Number: