计算机应用 ›› 2019, Vol. 39 ›› Issue (5): 1336-1342.DOI: 10.11772/j.issn.1001-9081.2018102197

• 数据科学与技术 • 上一篇    下一篇

基于策略梯度算法的工作量证明中挖矿困境研究

王甜甜, 于双元, 徐保民   

  1. 北京交通大学 计算机与信息技术学院, 北京 100044
  • 收稿日期:2018-11-01 修回日期:2018-12-30 发布日期:2019-05-14 出版日期:2019-05-10
  • 通讯作者: 王甜甜
  • 作者简介:王甜甜(1994-),女,河南焦作人,硕士研究生,主要研究方向:文本挖掘、深度学习、强化学习;于双元(1965-),女,黑龙江佳木斯人,副教授,硕士,主要研究方向:大数据分析与挖掘、深度学习、并行与分布式计算、软件测试;徐保民(1966-),男,河南郑州人,教授,博士,主要研究方向:大数据处理、云计算。
  • 基金资助:
    国家自然科学基金资助项目(61572005);河北省高等教育科技研究重点项目(ZD2017304)。

Research on proof of work mining dilemma based on policy gradient algorithm

WANG Tiantian, YU Shuangyuan, XU Baomin   

  1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China
  • Received:2018-11-01 Revised:2018-12-30 Online:2019-05-14 Published:2019-05-10
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61572005), the Higher Education Science and Technology Research Key Project of Hebei Province (ZD2017304).

摘要: 针对区块链中工作量证明(PoW)共识机制下区块截留攻击导致的挖矿困境问题,将矿池间的博弈行为视作迭代的囚徒困境(IPD)模型,采用深度强化学习的策略梯度算法研究IPD的策略选择。利用该算法将每个矿池视为独立的智能体(Agent),将矿工的潜入率量化为强化学习中的行为分布,通过策略梯度算法中的策略网络对Agent的行为进行预测和优化,最大化矿工的人均收益,并通过模拟实验验证了策略梯度算法的有效性。实验发现,前期矿池处于相互攻击状态,平均收益小于1,出现了纳什均衡的问题;经过policy gradient算法的自我调整后,矿池由相互攻击转变为相互合作,每个矿池的潜入率趋于0,人均收益趋于1。实验结果表明,policy gradient算法可以解决挖矿困境的纳什均衡问题,最大化矿池人均收益。

关键词: 区块链, 工作量证明机制, 博弈论, 深度强化学习, 策略梯度算法

Abstract: In view of the mining dilemma problem caused by block withholding attack under Proof of Work (PoW) consensus mechanism in the blockchain, the game behavior between mining pools was regarded as an Iterative Prisoner's Dilemma (IPD) model and the policy gradient algorithm of deep reinforcement learning was used to study IPD's strategy choices. Each mining pool was considered as an independent Agent and the miner's infiltration rate was quantified as a behavior distribution in reinforcement learning. The policy network in the policy gradient was used to predict and optimize the Agent's behavior in order to maximize miners' average revenues. And the effectiveness of the policy gradient algorithm was validated through simulation experiments. Experimental results show that the mining pools attack each other at the beginning with miners' average revenue less than 1, which causes Nash equilibrium problem. After self-adjustment by the policy gradient algorithm, the relationship between the mining pools transforms from mutual attack to mutual cooperation with infiltration rate of each mining pool tending to zero and miners' average revenue tending to 1. The results show that the policy gradient algorithm can solve the Nash equilibrium problem of mining dilemma and maximize the miners' average revenue.

Key words: blockchain, Proof of Work (PoW), game, deep reinforcement learning, policy gradient algorithm

中图分类号: