计算机应用 ›› 2017, Vol. 37 ›› Issue (4): 1157-1163.DOI: 10.11772/j.issn.1001-9081.2017.04.1157

• 计算机软件技术 • 上一篇    下一篇

基于消息通信的分布式系统最终一致性平台

徐进1, 黄勃2, 冯炯1   

  1. 1. 上海你我贷互联网金融信息服务有限公司 技术中心, 上海 200120;
    2. 上海工程技术大学 电子电气工程学院, 上海 201620
  • 收稿日期:2016-08-30 修回日期:2016-12-28 出版日期:2017-04-10 发布日期:2017-04-19
  • 通讯作者: 徐进
  • 作者简介:徐进(1976-),男,安徽合肥人,高级工程师,硕士,CCF会员,主要研究方向:分布式系统、大数据;黄勃(1985-),男,湖北武汉人,讲师,博士,CCF会员,主要研究方向:人工智能、软件工程;冯炯(1974-),男,上海人,高级工程师,硕士,主要研究方向:自然语言处理。

Eventual consistency platform of distributed system based on message communication

XU Jin1, HUANG Bo2, FENG Jiong1   

  1. 1. Technology Center, Shanghai Niwodai Internet Financial Information Service Company Limited, Shanghai 200120, China;
    2. School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China
  • Received:2016-08-30 Revised:2016-12-28 Online:2017-04-10 Published:2017-04-19

摘要: 在分布式系统中为了满足高性能和吞吐量,一般采用异步消息通信方式,但消息通信没有解决分布式事务不一致问题。针对这个问题,提出建立一致性保障平台,通过这个平台实现最终一致性。首先,使系统满足幂等性以及业务数据与消息生产消费记录强一致性;其次,建立消息监控机制,根据监控规则和消费生产消费记录,判定消息正常还是需要补偿操作或者幂等操作,从而保证分布式系统基于消息通信的最终一致;最后,在整个设计实现过程中采用关注点分离和横向切分的思想与工程化的方法,实现一致性保障平台。通过实验和分析证明比较得出,与异步消息通信相比,分布式消息通信性能更优越;一致性保障平台能及时发现不一致并由系统及时处理,实现最终一致,即可以完全保障系统最终一致性;而且该平台通过平台化的实现方式在应用中可以快速复用到数十个业务系统。由此得出一致性保障平台可以解决分布式交易系统事务最终一致性问题,不仅性能优越而且经济。

关键词: 分布式, 消息通信, 消息中间件, 一致性, 幂等, 事务, 体系结构设计

Abstract: In order to meet the performance and throughput requirements of distributed systems, the asynchronous message communication is a common strategy. However, this strategy can not solve the consistency problem of the distributed system. In order to solve this problem, this paper proposed the establishment of consistency guarantee platform. Firstly, the system fulfilled idempotency and strong consistency between business data and message production/consumption records. Secondly, a message monitoring strategy was established. And it could be decided whether a message was correct or the compensation/idempotent operation was needed, according to the monitoring rules and production/consumption records, in order to realize the eventual consistency of the distributed system based on message communication. Lastly, the Separation of Concerns (SoC) and horizontal segmentation methods were adopted in design and realization of this platform. Experiments and analyses have shown the better performance of this distributed message communication, comparing to the asynchronous communication. This platform could timely check and handle the inconsistency and thus achieve the eventual consistency, i.e. the final eventual consistency of the whole system. Also the platform design could easily be adopted to multiply business systems, which means this platform is not only superior-performed but also economic.

Key words: distributed, message communication, message oriented middleware, consistency, idempotency, transaction, architecture design

中图分类号: