| 摘要: |
| 网络安全人才的培养和选拔,离不开一把衡量人才的“尺子”。以通用漏洞评分系统作为参考范例,一个具备可操作性的评价模型,不能只是一个抽象的思考模型,而是应当包含准则、权重、量化取值方法、计算公式、得分和评级6个要素,现有模型在这些要素上都有不同程度的缺失。因此,本文以多轮问卷调查的形式,综合运用了多种定性与定量评估方法,建立起了具备以上6个要素的渗透测试型人才能力评估的综合评价模型,取名为CEMoPT。首先,我们运用德尔菲法,通过文献阅读归纳形成了评价准则结构和准则项定义;然后,采用层次分析法、熵权法和组合赋权法,得到准则权重;并设计了基于隶属度矩阵标注任务的方法以获得准则量化取值;最后使用模糊综合评价法中相应的计算公式,得到人才的得分和评级。我们设计了在线问卷,招募了72名领域专家,对CEMoPT的6个要素依次开展评议,并严格遵循稳定性度量与共识性度量约束,经历了最长4轮迭代。具体来说,CEMoPT的评价准则包括5个基本度量组和18个准则项,权重是主观权重和客观权重的组合赋权结果,数学公式的核心元素是隶属度矩阵,综合评级分为新手、学徒、高手、专家和大师5级。本文通过设计对比实验,验证了CEMoPT的可靠性。模型建立过程严格遵循科学方法所要求的诸多度量和检验约束,保证了CEMoPT的有效性。 |
| 关键词: 渗透测试型人才 综合评价模型 德尔菲法 层次分析法 熵权法 组合赋权法 模糊综合评价法 |
| DOI：10.19363/J.cnki.cn10-1380/tn.2022.12.09 |
| 投稿时间：2020-08-22修订日期：2020-12-26 |
| 基金项目:本论文获得中国科学院网络测评技术重点实验室和网络安全防护技术北京市重点实验室资助。获得了北京市科委项目(No.Z191100007119010,No.Z181100002718002)课题资助。 |
|
| Establishing a Comprehensive Evaluation Model for the Competency Assessment of Pentesting Cybersecurity Talents |
| ZHANG Xiu,LIU Baoxu,GONG Xiaorui,YU Dongsong,ZHAO Beibei,LIU Yuan |
| Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China;School of Cyber Security, University of Chinese Academy of Sciences, Beijing 100049, China |
| Abstract: |
| The cultivation and selection of talents are indispensable for a “ruler” to measure them. Taking the CVSS as an example, an evaluation model with high operability can not just be an abstract model of thinking. Furthermore, there are six essential elements: criterion, weight, a method to map the criterion with a corresponding numerical value, computational formula, score, and rating. Prior models lack those elements to varying degrees. Therefore, in the form of multiple rounds of questionnaire surveys, this paper uses several qualitative and quantitative evaluation methods to establish a comprehensive evaluation model with the above six essential elements for the competency assessment of pentesting cybersecurity talents, named CEMoPT. First, we summarized the criterion structure and definition by combining literature review with the Delphi method. Then, we applied the analytic hierarchy process, the entropy weight method, and the combination weighting method to obtain the weight of the criteria. Next, we designed a method of labeling tasks based on the membership matrix to map the criterion with a corresponding numerical value. Finally, the score and rating were calculated by taking advantage of the computational formula in the fuzzy comprehensive evaluation method. We designed an online questionnaire, recruited 72 subject matter experts, conducted reviews on the six essential elements of the CEMoPT in turn, and strictly followed the constraints of stability measure and consensus measure, and experienced a maximum of 4 rounds of iterations. Specific to CEMoPT, the criteria make up of 5 basic metric groups and 18 criterion items. The weight is a combination weight, which is a compromise between the subject weight and the object weight. The membership matrix is the core of the mathematical formula. Based on the score, the rating is divided into 5 levels, i.e., novice, apprentice, journeyman, expert, and master. The reliability of CEMoPT was verified by conducting a comparative experiment. To ensure the validity of CEMoPT, the research process strictly followed many constraints required by scientific methods. |
| Key words: pentesting cybersecurity talents comprehensive evaluation model the Delphi method analytic hierarchy process entropy weight method combination weighting method fuzzy comprehensive evaluation |
