Vol. 5 No. 1 (2019): Proceedings of Botconf 2019
Conference proceedings

YARA-Signator: Automated Generation of Code-based YARA Rules

PDF
  • Felix Bilstein,
  • Daniel Plohmann
Felix Bilstein
Fraunhofer FKIE
Daniel Plohmann
Fraunhofer FKIE

Published 2019-12-31

Copyright (c) 2019 Felix Bilstein, Daniel Plohmann (Author)

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Bilstein, F. ., & Plohmann, D. (2019). YARA-Signator: Automated Generation of Code-based YARA Rules. The Journal on Cybercrime and Digital Investigations, 5(1), 1-13. https://doi.org/10.18464/cybin.v5i1.24

Download Citation

Abstract

Effective detection and identification signatures are an important component in the toolkit for malware analysis. The creation of such signatures is still widely a manual task that requires notable experience and knowledge on the side of analysts. In this paper, we present YARA-Signator, an approach for the automated generation of code-based YARA rules. The method is based on the isolation of instruction n-grams that on the one hand appear frequently within a malware family and on the other hand are not found in any other family.
Applying YARA-Signator to the Malpedia data set, we show that in fact on average 51.85% of the instruction n-grams of length 4 and higher are only found in the respective family. The rules produced by the system using this data set achieve an overall F1 score of 0.983 and cause only very few false positives in a sanity check against a large goodware data set. YARASignator is made available as open source and a periodically updated reference rule set is provided for free through Malpedia.

 
PDF

References

  1. D. Bianco, “The pyramid of pain.” Blog post: http://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html.
  2. C. Blichmann, “Automatisierte Signaturgenerierung für Malware-Stämme,” 2008. Diploma Thesis.
  3. J. Zaddach and M. Graziano, “Bass - bass automated signature synthesizer,” 2017. Github repository: https://github.com/Cisco-Talos/BASS.
  4. D. Plohmann, M. Clauß, S. Enders, and E. Padilla, “Malpedia: a collaborative effort to inventorize the malware landscape,” Proceedings of Botconf, 2017.
  5. L. Gibelli, T. Edvin, T. Kojmnet, A. Wu, and N. Horne, “Clamav - open source anti virus engine,” 2004. Website: https://www.clamav.net/.
  6. V. M. Alvarez, “Yara - the pattern matching swiss knife for malware researchers,” 2014. Website: http://virustotal.github.io/yara/.
  7. I. Intel, “Intel-64 and ia-32 architectures software developer’s manual,” 2013.
  8. R. Edward, Z. Richard, R. Cox, J. Sylvester, P. Yacci, R. Ward, A. Tracy, M. McLean, and C. Nicholas, “An investigation of byte n-gram features for malware classification,” Journal of Computer Virology and Hacking Techniques, 2016.
  9. C. Blichmann, “vxsig - automatically generate av byte signatures from sets of similar binaries.,” 2019. Github repository: https://github.com/google/vxsig.
  10. T. Dullien, E. Ventura, S. Meyer-Eppler, T. Kornau, C. Blichmann, and J. Newger, “Zynamics,” 2004. Website: https://www.zynamics.com/software.html.
  11. S. H. Ding, B. C. Fung, and P. Charland, “Kam1n0: Mapreduce-based assembly clone search for reverse engineering,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, (New York, NY, USA), p. 461–470, Association for Computing Machinery, 2016.
  12. F. Roth, “yarGen,” 2013-12-18. Github Repository: "Github repository: https://github.com/Neo23x0/yarGen.
  13. C. Doman, “Yabin,” 2018. Github repository: https://github.com/AlienVault-OTX/yabin.
  14. H. Yi, “Hyara (ida plugin),” 2018. Github repository: https://github.com/hy00un/Hyara.
  15. KoreLogic Security, “Converting ida pat to yara signatures,” 2013. Blog post: https://blog.korelogic.com/blog/2013/11/15/pat2yara.
  16. W. Ballenthin, “Yara-fn,” 2019. Github repository: https://github.com/williballenthin/idawilli/tree/master/scripts/yara_fn.
  17. J. Martin, j0sm1, jovimon, and mmorenog, “Yara rules,” 2018. Github repository: https://github.com/Neo23x0/signaturebase/tree/master/yara.
  18. F. Roth, “Yara rules from signature base,” 2018. Github repository: https://github.com/Neo23x0/signature-base/tree/master/yara.
  19. M. Worth, “Open-source-yara-rules,” 2018. Github repository: https://github.com/mikesxrs/Open-Source-YARA-rules.
  20. R. Wesson and SupportIntelligence, “Project icewater,” 2018. Github repository: https://github.com/SupportIntelligence/Icewater.
  21. D. Plohmann, “SMDA - a minimalist recursive disassembler library for x86/64.,” 2018. Github repository: https://github.com/danielplohmann/smda.
  22. N. A. Quynh, “Capstone: Next-gen disassembly framework,” 2014. Website: http://www.capstone-engine.org/BHUSA2014-capstone.pdf.
  23. F. Bilstein, “Automatic generation of code-based yara-signatures,” 2018. Bachelor Thesis.
  24. C. Cohen and J. Havrilla, “Function Hashing for Malicious Code Analysis,” tech. rep., SEI, CMU, 2009.
  25. V. Chvatal, “A greedy heuristic for the set-covering problem,” Math. Oper. Res., vol. 4, p. 233–235, Aug. 1979.
  26. M. Stonebraker, “Postgresql,” 1989. Website: https://www.postgresql.org/.
  27. B. Levene and J. Grunzweig, “Sure, I’ll take that! New ComboJack Malware Alters Clipboards to Steal Cryptocurrency.” Blogpost: https://researchcenter.paloaltonetworks.com/2018/03/unit42-sure-ill-take-newcombojack-malware-alters-clipboardssteal-cryptocurrency/.
  28. T. Micro, “ShurL0ckr Ransomware as a Service Peddled on Dark Web, can Reportedly Bypass Cloud Applications.” Blogpost: https://www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digitalthreats/shurl0ckr-ransomware-as-aservice-peddled-on-dark-web-canreportedly-bypass-cloud-applications.
  29. J. Grunzweig and K. Wilhoit, “The Fractured Block Campaign: CARROTBAT Used to Deliver Malware Targeting Southeast Asia.” Blogpost: https://unit42.paloaltonetworks.com/unit42-the-fractured-block-campaigncarrotbat-malware-used-to-delivermalware-targeting-southeast-asia/.
  30. M. Talbi, “De-obfuscating Jump Chains with Binary Ninja.” Blogpost: https://thisissecurity.stormshield.com/2018/03/20/de-obfuscating-jump-chains-withbinary-ninja/.
  31. D. Plohmann, “Patchwork: Stitching against malware families with IDA Pro.” Presentation for SPRING2014: https://public.gdatasoftware.com/Web/Landingpages/DE/GISpring2014/slides/004_plohmann.pdf.
  32. D. Plohmann, “Empty msvc,” 2019. Github repository: https://github.com/danielplohmann/empty_msvc.
  33. E. Raff, W. Fleming, R. Zak, H. Anderson, B. Finlayson, C. Nicholas, and M. McLean, “Kilograms: Very large n-grams for malware classification,” 2019.
  34. gbrindisi, “Gozi ISFB Sourceccode.” Github Repository: https://github.com/gbrindisi/malware/tree/master/windows/gozi-isfb.
  35. A. Ivanov, “Scarabey Ransomware.” Blogpost: https://id-ransomware.blogspot.com/2017/12/scarabey-ransomware.html.