Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (2): 490-498.DOI: 10.11772/j.issn.1001-9081.2021122234

• Computer software technology • Previous Articles    

Improved instruction obfuscation framework based on obfuscator low level virtual machine

Yayi WANG, Chen LIU, Tianbo HUANG, Weiping WEN()   

  1. School of Software and Microelectronics,Peking University,Beijing 102600,China
  • Received:2022-01-06 Revised:2022-05-20 Accepted:2022-05-24 Online:2022-06-13 Published:2023-02-10
  • Contact: Weiping WEN
  • About author:WANG Yayi, born in 1998, M. S. candidate. Her research interests include code obfuscation, vulnerability mining.
    LIU Chen, born in 1999, M. S. candidate. His research interests include code obfuscation, vulnerability mining.
    HUANG Tianbo, born in 1997, M. S. candidate. His research interests include cyberspace security, malicious code detection, code obfuscation.
  • Supported by:
    Peking University Horizontal Research Project(2020001763)


王雅仪, 刘琛, 黄天波, 文伟平()   

  1. 北京大学 软件与微电子学院,北京 102600
  • 通讯作者: 文伟平
  • 作者简介:王雅仪(1998—),女,四川成都人,硕士研究生,主要研究方向:代码混淆、漏洞挖掘
  • 基金资助:


Focusing on the issue that only one instruction substitution with 5 operators and 13 substitution schemes is supported in Obfuscator Low Level Virtual Machine (OLLVM) at the instruction obfuscation level, an improved instruction obfuscation framework InsObf was proposed. InsObf, including junk code insertion and instruction substitution, was able to enhance the obfuscation effect at the instruction level based on OLLVM. For junk code insertion, firstly, the dependency of the instruction inside the basic block was analyzed, and then two kinds of junk code, multiple jump and bogus loop, were inserted to disrupt the structure of the basic block. For instruction substitution, based on OLLVM, it was expanded to 13 operators, with 52 instruction substitution schemes. The framework prototype was implemented on Low Level Virtual Machine (LLVM). Experimental results show that compared to OLLVM, InsObf has the cyclomatic complexity and resilience increased by almost four times, with a time cost of about 10 percentage points and a space cost of about 20 percentage points higher. Moreover, InsObf can provide higher code complexity compared to Armariris and Hikari, which are also improved on the basis of OLLVM, at the same order of magnitude of time and space costs. Therefore, InsObf can provide effective protection at the instruction level.

Key words: software protection, code obfuscation, instruction obfuscation, Obfuscator Low Level Virtual Machine (OLLVM), junk code insertion, instruction substitution



关键词: 软件保护, 代码混淆, 指令混淆, 底层虚拟机混淆器, 指令加花, 指令替换

CLC Number: