Binary-Level Reverse Engineering

Analyzing and transforming COTS binaries to improve their security is arguable the Holy Grail in software-security research. There is a vast number of programs whose source code is either unavailable or cannot be recompiled. Yet these programs are part of deployed systems to provide critical functionalities; as a result we must protect them from exploits. Therefore, binary-level techniques that analyze and transform stripped COTS binaries for security have been intensively studied.

However, despite decades of research, most state-of-the-art binary-level defense techniques are based on heuristics, make heavy assumptions about compilation toolchains, and do not handle code obfuscation well.

In this project, we focus on binary-level reverse engineering. Before we are able to perform analysis and transformation on a piece of binary, we must reverse engineer it to get its basic information, including its instructions, its control-flow graph, and basic dataflow information. Previous reverseengineering techniques are often ad hoc and do not have a formal basis. There is also no evaluation about what would be the best reverse-engineering algorithms in terms of precision and performance. We plan to construct a reverse-engineering tool that makes it easy for principled exploration of the design space of reverse-engineering algorithms.

Participants

• Monika Santra
• Bokai Zhang
• Sun Hyoung Kim
• Dongrui Zeng
• Gang Tan

Publications

• Zhang, B., Santra, M., Hussain, S. R., and Tan, G. (2026). BPA-X: An architecture-agnostic block-based points-to analysis for stripped binaries. In Workshop on Binary Analysis Research (BAR). [paper]
• Wang, P., Santra, M., Liu, M., Sun, C., Zeng, D., and Tan, G. (2025). Disa: Accurate learning-based static disassembly with attentions. In 31st ACM Conference on Computer and Communications Security (CCS), pages 843–857. [paper]
• Kim, S. H., Sun, C., Zeng, D., and Tan, G. (2022). Binpointer: Towards precise, sound, and scalable binary-level pointer analysis. In ACM SIGPLAN 2022 International Conference Compiler Construction (CC), page To appear. [paper]
• Kim, S. H., Sun, C., Zeng, D., and Tan, G. (2021). Refining indirect call targets at the binary level. In Network and Distributed System Security Symposium (NDSS). [paper]
• Zeng, D., Niu, B., and Tan, G. (2021). MazeRunner: Evaluating the attack surface of control-flow integrity policies. In 20th IEEE International Conference On Trust, Security And Privacy In Computing And Communications (TrustCom), page To appear
• Muntean, P., Fischer, M., Tan, G., Lin, Z., Grossklags, J., and Eckert, C. (2018). tauCFI: Type-assisted control flow integrity for x86-64 binaries. In International Symposium on Research in Attacks, Intrusions and Defenses (RAID), pages 423–444. [paper]
• Zeng, D. and Tan, G. (2018). From debugging-information based binary-level type inference to CFG generation. In 8th ACM Conference on Data and Application Security and Privacy (CODASPY), pages 366–376. Outstanding paper award. [paper]

Sponsors