This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
View analytic
Monday, March 27 • 11:45 - 12:25
Adventures in Fuzzing Instruction Selection

Sign up or log in to save this to your schedule and see who's attending!

Recently there has been a lot of work on GlobalISel, which aims to entirely replace the existing instruction selectors for LLVM. In order to approach such a transition, we need an effective way to test instruction selection and evaluate the new selector compared to the older ones.

This talk will focus on our experiments and results in using fuzzing and input generation to test instruction selection. We'll discuss the tradeoffs in how to find valuable test inputs as well as the approach to validating the generated code. This will essentially consist of three parts:

- Generating useful inputs to test instruction selection
- Evaluating the output of instruction selection effectively
- Results and lessons learned

Generating Inputs

We will discuss the tradeoffs between types of input generation and look at the options in terms of the level of abstraction of those inputs. Here we talk about how we improved on the input generation of the llvm-stress tool by leveraging libfuzzer and embracing coverage guided testing and input mutation. We also go into the relative effectiveness of generating LLVM IR versus generating machine-level IR directly in terms of finding valuable test cases.

Evaluating Outputs

Given that we're feeding instruction selection arbitrary inputs, we need to come up with ways to evaluate whether the results are sane. Here we'll discuss the kinds of bugs that were found simply by looking for crashes and error paths versus those found by comparing against the older instruction selectors. We also explain the complexity of trying to compare instruction selectors and evaluate whether or not differences are functionally relevant.


Finally, we'll talk about the effectiveness of these experiments and the adaptibility of these methods to other problem spaces.


Monday March 27, 2017 11:45 - 12:25
E2 2 (Günter Hotz Hall)

Attendees (17)