Abstract

Effective debugging is crucial in fuzzing engine development. Compilation and fuzzing phases introduce recurring errors that can be systematically addressed. Identifying execution flow issues and tracking key metrics ensures robust fuzzing performance.

본문

Developing a fuzzing engine requires effective debugging strategies to ensure stability. Both the compilation phase and the fuzzing phase may introduce fatal errors. They mostly have a recurring pattern, so when getting used to it once, it becomes not hard to catch-and-patch the errors.

Fatal errors during compilation are split into two categories: during the initial configuration, and during the actual compilation. AFL uses clang compiler and loads the custom instrumentation maker as a plugin. If the code for custom instrumentation maker fails to be compiled, the specific information is logged in the config.log file, created inside the folder of the target benchmark code. On the other hand, errors during the actual compilation mostly comes from wrong handling of the target-related information files. Checking if the whole file is read as intended mainly solves all issues.

Fatal errors during the fuzzing phase have another way of being debugged. AFL allows debugging with gdb, and setting the AFL_DEBUG environment variable enables detailed execution logging at each step. Hence, finding out the error stack trace and the problematic variable values is easy. These methods helped a lot recently when integrating our fuzzing technique into AFL++. The hard part lies in understanding the wrong point of the mechanism that is overlooked in the development. From personal experience, the error lies mostly on the wrong order of multiple initialization functions. Hence, checking which initialization function is handled at what time is important. Since initialization happens only once, it is not costly to log and print out the execution of each steps. The rest of the errors lies mostly when the fuzzer changes its mode of working. Like all two-stage (exploration + exploitation) fuzzers, our one also switches between multiple modes, a potential source of coding mistake. For debugging, shortening the time interval between mode switches is helpful.

In summary, there are stereotypical error types encountered during the development of a fuzzing engine. Experience helps shortening the debugging process too. However, there are still tricky errors, that are not fatal hence just undergoes like if the fuzzer has no bugs. To discover such potential errors, tracking execution speed, seed count, and replaying the seed queue between similar fuzzers to check the quality of seeds is crucial. The comparative approach ensures the reliability of new fuzzing implementations.