Wentao Gao
ABSTRACT
Fuzzing and particularly code coverage-guided greybox fuzzing is highly successful in automated vulnerability discovery, as evidenced by the multitude of vulnerabilities uncovered in real-world software systems. However, results on large benchmarks such as FuzzBench indicate that the state-of-the-art fuzzers often reach a plateau after a certain period, typically around 12 hours. With the aid of the newly introduced FuzzIntrospector platform, this study aims to analyze and categorize the fuzz blockers that impede the progress of fuzzers. Such insights can shed light on future fuzzing research, suggesting areas that require further attention. Our preliminary findings reveal that the majority of top fuzz blockers are not directly related to the program input, emphasizing the need for enhanced techniques in automated fuzz driver generation and modification.
- [n. d.]. American Fuzzy Lop. https://lcamtuf.coredump.cx/afl/
Google Scholar
- [n. d.]. Black Duck Open Hub. https://www.openhub.net/
Google Scholar
- [n. d.]. ClusterFuzz. https://google.github.io/clusterfuzz/
Google Scholar
- [n. d.]. Data Flow Sanitizer. https://clang.llvm.org/docs/DataFlowSanitizer.html
Google Scholar
- [n. d.]. Fuzz Introspector. https://github.com/ossf/fuzz-introspector
Google Scholar
- [n. d.]. Honggfuzz. https://github.com/google/honggfuzz
Google Scholar
- [n. d.]. Igraph The Network Analysis Package. https://igraph.org/
Google Scholar
- [n. d.]. libFuzzer-a library for coverage-guided fuzz testing. https://github. com/llvm-mirror/llvm/blob/master/docs/LibFuzzer.rst
Google Scholar
- [n. d.]. libpng home page. http://www.libpng.org/pub/png/libpng.html
Google Scholar
- [n. d.]. libpng manual. http://www.libpng.org/pub/png/libpng-manual.txt
Google Scholar
- [n. d.]. libprotobuf-mutator. https://github.com/google/libprotobuf-mutator/
Google Scholar
- [n. d.]. OpenSSL crytography and SSL/TLS toolket. https://www.openssl.org/
Google Scholar
- [n. d.]. OSS-Fuzz: Continuous Fuzzing for Open Source Software. https://github. com/google/oss-fuzz
Google Scholar
- [n. d.]. Successful case studies of FuzzIntrospector. https://github.com/ossf/fuzzintrospector/blob/main/doc/CaseStudies.md
Google Scholar
- Cornelius Aschermann, Tommaso Frassetto, Thorsten Holz, Patrick Jauernig, Ahmad-Reza Sadeghi, and Daniel Teuchert. 2019. NAUTILUS: Fishing for Deep Bugs with Grammars.. In NDSS.
Google Scholar
- Cornelius Aschermann, Sergej Schumilo, Ali Abbasi, and Thorsten Holz. 2020. Ijon: Exploring deep state spaces via fuzzing. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 1597-1612.
Google ScholarCross Ref
- Cornelius Aschermann, Sergej Schumilo, Tim Blazytko, Robert Gawlik, and Thorsten Holz. 2019. REDQUEEN: Fuzzing with Input-to-State Correspondence.. In NDSS, Vol. 19. 1-15.
Google ScholarCross Ref
- Jinsheng Ba, Marcel Böhme, Zahra Mirzamomen, and Abhik Roychoudhury. 2022. Stateful greybox fuzzing. In 31st USENIX Security Symposium (USENIX Security 22). 3255-3272.
Google Scholar
- Domagoj Babić, Stefan Bucur, Yaohui Chen, Franjo Ivančić, Tim King, Markus Kusano, Caroline Lemieux, László Szekeres, and Wei Wang. 2019. Fudge: fuzz driver generation at scale. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 975-985.
Google ScholarDigital Library
- Tim Blazytko, Cornelius Aschermann, Moritz Schlögel, Ali Abbasi, Sergej Schumilo, Simon Wörner, and Thorsten Holz. 2019. GRIMOIRE: Synthesizing Structure while Fuzzing.. In USENIX Security Symposium, Vol. 19.
Google Scholar
- Marcel Böhme, Cristian Cadar, and Abhik Roychoudhury. 2021. Fuzzing: Challenges and Reflections. IEEE Softw. 38, 3 ( 2021 ), 79-86.
Google Scholar
- Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. 2017. Directed greybox fuzzing. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2329-2344.
Google ScholarDigital Library
- Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. 2016. CoverageBased Greybox Fuzzing as Markov Chain. IEEE Transactions on Software Engineering 45 ( 2016 ), 489-506.
Google Scholar
- Gemma Catolino, Fabio Palomba, Andy Zaidman, and Filomena Ferrucci. 2019. Not all bugs are the same: Understanding, characterizing, and classifying bug types. Journal of Systems and Software 152 ( 2019 ), 165-181.
Google Scholar
- Jaeseung Choi, Joonun Jang, Choongwoo Han, and Sang Kil Cha. 2019. Grey-box concolic testing on binary code. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 736-747.
Google ScholarDigital Library
- Karine Even-Mendoza, Cristian Cadar, and Alastair F Donaldson. 2022. CsmithEdge: more efective compiler testing by handling undefined behaviour less conservatively. Empirical Software Engineering 27, 6 ( 2022 ), 129.
Google Scholar
- Andrea Fioraldi, Daniele Cono D'Elia, and Davide Balzarotti. 2021. The Use of Likely Invariants as Feedback for Fuzzers. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 2829-2846. https://www.usenix.org/ conference/usenixsecurity21/presentation/fioraldi
Google Scholar
- Andrea Fioraldi, Dominik Maier, Heiko Eißfeldt, and Marc Heuse. 2020. AFL++: Combining Incremental Steps of Fuzzing Research. In 14th USENIX Workshop on Ofensive Technologies (WOOT 20). USENIX Association.
Google Scholar
- Shuitao Gan, Chao Zhang, Peng Chen, Bodong Zhao, Xiaojun Qin, Dong Wu, and Zuoning Chen. 2020. GREYONE: Data Flow Sensitive Fuzzing.. In USENIX Security Symposium. 2577-2594.
Google Scholar
- Adrian Herrera, Hendra Gunadi, Shane Magrath, Michael Norrish, Mathias Payer, and Antony L Hosking. 2021. Seed selection for successful fuzzing. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis. 230-243.
Google ScholarDigital Library
- Renáta Hodován, Ákos Kiss, and Tibor Gyimóthy. 2018. Grammarinator: a grammar-based open source fuzzer. In Proceedings of the 9th ACM SIGSOFT international workshop on automating TEST case design, selection, and evaluation. 45-48.
Google ScholarDigital Library
- Kyriakos K Ispoglou, Daniel Austin, Vishwath Mohan, and Mathias Payer. 2020. Fuzzgen: Automatic fuzzer generation. In Proceedings of the 29th USENIX Conference on Security Symposium. 2271-2287.
Google Scholar
- Jie Liang, Yu Jiang, Yuanliang Chen, Mingzhe Wang, Chijin Zhou, and Jiaguang Sun. 2018. Pafl: extend fuzzing optimizations of single mode to industrial parallel mode. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 809-814.
Google ScholarDigital Library
- Jie Liang, Mingzhe Wang, Yuanliang Chen, Yu Jiang, and Renwei Zhang. 2018. Fuzz testing in practice: Obstacles and solutions. In 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 562-566.
Google ScholarCross Ref
- Dongge Liu, Jonathan Metzman, Marcel Böhme, Oliver Chang, and Abhishek Arya. 2023. SBFT Tool Competition 2023-Fuzzing Track. arXiv preprint arXiv:2304.10070 ( 2023 ).
Google Scholar
- Valentin JM Manès, HyungSeok Han, Choongwoo Han, Sang Kil Cha, Manuel Egele, Edward J Schwartz, and Maverick Woo. 2019. The art, science, and engineering of fuzzing: A survey. IEEE Transactions on Software Engineering 47, 11 ( 2019 ), 2312-2331.
Google Scholar
- Jonathan Metzman, László Szekeres, Laurent Simon, Read Sprabery, and Abhishek Arya. 2021. Fuzzbench: an open fuzzer benchmarking platform and service. In Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering. 1393-1403.
Google ScholarDigital Library
- Sebastian Österlund, Elia Geretto, Andrea Jemmett, Emre Güler, Philipp Görz, Thorsten Holz, Cristiano Giufrida, and Herbert Bos. 2021. Collabfuzz: A framework for collaborative fuzzing. In Proceedings of the 14th European Workshop on Systems Security. 1-7.
Google ScholarDigital Library
- Lianglu Pan, Shaanan Cohney, Toby Murray, and Van-Thuan Pham. 2023. Detecting Excessive Data Exposures in Web Server Responses with Metamorphic Fuzzing. arXiv preprint arXiv:2301.09258 ( 2023 ).
Google Scholar
- Jiwon Park, Dominik Winterer, Chengyu Zhang, and Zhendong Su. 2021. Generative type-aware mutation for testing SMT solvers. Proceedings of the ACM on Programming Languages 5, OOPSLA ( 2021 ), 1-19.
Google ScholarDigital Library
- Hui Peng and Mathias Payer. 2020. Usbfuzz: A framework for fuzzing USB drivers by device emulation. In Proceedings of the 29th USENIX Conference on Security Symposium. 2559-2575.
Google Scholar
- Van-Thuan Pham, Manh-Dung Nguyen, Quang-Trung Ta, Toby Murray, and Benjamin I.P. Rubinstein. 2021. Towards Systematic and Dynamic Task Allocation for Collaborative Parallel Fuzzing. In Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering : NIER Track.
Google Scholar
- Van-Thuan Pham, Marcel Böhme, and Abhik Roychoudhury. 2020. AFLNet: a greybox fuzzer for network protocols. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). IEEE, 460-465.
Google ScholarCross Ref
- Van-Thuan Pham, Marcel Böhme, Andrew E Santosa, Alexandru Răzvan Căciulescu, and Abhik Roychoudhury. 2019. Smart greybox fuzzing. IEEE Transactions on Software Engineering 47, 9 ( 2019 ), 1980-1997.
Google Scholar
- Manuel Rigger and Zhendong Su. 2020. Testing Database Engines via Pivoted Query Synthesis.. In OSDI, Vol. 20. 667-682.
Google Scholar
- Yang Song and Oscar Chaparro. 2020. BEE: a tool for structuring and analyzing bug reports. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1551-1555.
Google ScholarDigital Library
- Junjie Wang, Bihuan Chen, Lei Wei, and Yang Liu. 2019. Superion: Grammaraware greybox fuzzing. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 724-735.
Google ScholarDigital Library
- Qian Zhang, Jiyuan Wang, and Miryung Kim. 2021. Heterofuzz: Fuzz testing to detect platform dependent divergence for heterogeneous applications. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 242-254.
Google ScholarDigital Library
- Zenong Zhang, George Klees, Eric Wang, Michael Hicks, and Shiyi Wei. 2023. Fuzzing Configurations of Program Options. ACM Transactions on Software Engineering and Methodology 32, 2 ( 2023 ), 1-21.
Google Scholar
- Rui Zhong, Yongheng Chen, Hong Hu, Hangfan Zhang, Wenke Lee, and Dinghao Wu. 2020. Squirrel: Testing database management systems with language validity and coverage feedback. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security. 955-970.
Google ScholarDigital Library
Index Terms
- Beyond the Coverage Plateau: A Comprehensive Study of Fuzz Blockers (Registered Report)
Recommendations
InFuzz: An Interactive Tool for Enhancing Efficiency in Fuzzing through Visual Bottleneck Analysis (Registered Report)
FUZZING 2023: Proceedings of the 2nd International Fuzzing WorkshopDespite the effectiveness of current fuzzing methods, fully automated fuzzing techniques still face an important challenge in overcoming complex code constraints to achieve high coverage and find new vulnerabilities. As a result, experts in practice ...
Network protocol fuzz testing for information systems and applications: a survey and taxonomy
Fuzzing or fuzz testing has been introduced as a software testing technique to reduce vulnerabilities in software systems or given targets. To achieve a maximum benefit-to-cost ratio and without complication, we use fuzz testing [11]. In addition, ...
FUDGE: fuzz driver generation at scale
ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringAt Google we have found tens of thousands of security and robustness bugs by fuzzing C and C++ libraries. To fuzz a library, a fuzzer requires a fuzz driver—which exercises some library code—to which it can pass inputs. Unfortunately, writing fuzz ...
Comments