Is it possible to support Catch2 test framework? #2219
-
|
Working on a project with a bunch of tests built with Catch2 framework, I ran on a link issue which really seems to come from an incompatibility between AFL++ and Catch2. At least, Catch::Matchers::StdString::StartsWithMatcher produces a large bunch of error messages at link time. Of course, I may work around temporarily this issue not building the tests, but that's not really practical as I expected to benchmark several fuzzing tools using the same test case. Any help would be thankfully welcome. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
I have no clue what Catch2 is. but instrumenting tests does not make sense, or are you trying to create harness functions that are based on Catch2 (whatever that is)? |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for the time passing by. I was sort of caught by other subjects and had not much time to dedicate to fuzzing. As I can't imagine how to fuzz the fully fledged binaries (far too big), I focussed on tests (which are mainly intermediate tests - not really at "unit" level nor at "integration" level). I understand quite well that my original question lacks of context and precise error. I don't really get your point
As it seems to me absolutely impossible to instrument (and especially to interpret and exploit the results of this instrumentation) for a large scale multi-process client-server software solution, built from millions of lines of C++ code, instrumenting tests does make sens for me. Of course, I may be wrong regarding this focus/scope notion and the ability for AFL++ to synthesize the results even on a very big application. |
Beta Was this translation helpful? Give feedback.
-
|
If you just instrument tests then you do not need to fuzz with coverage. I understand that instrumenting the whole program will slow the fuzzing down. you could create a CFG and just instrument the functions a test is calling. If that is not an option, sure base your tests as fuzzing harnesses, but you will have to instrument the whole base code (you can skip or just instrument specific extra libraries though) |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for this hint. |
Beta Was this translation helpful? Give feedback.
If you just instrument tests then you do not need to fuzz with coverage. I understand that instrumenting the whole program will slow the fuzzing down. you could create a CFG and just instrument the functions a test is calling. If that is not an option, sure base your tests as fuzzing harnesses, but you will have to instrument the whole base code (you can skip or just instrument specific extra libraries though)