encounter
/
dawn-cmake
mirror of https://github.com/encounter/dawn-cmake.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
dawn-cmake/docs/fuzzing.md

21 lines
1.5 KiB
Markdown
Raw Normal View History

Add script to generate fuzzer seed corpus from tests This adds a script which runs the end2end_tests, captures a wire trace, and then minimizes the corpus with the fuzzer. Minimizing the corpus requires libfuzzer, so this only works in a Chromium checkout. Unseeded, the fuzzer starts with coverage of about 600 features. Using a seed corpus captured from the tests, the fuzzer quickly increases coverage to about 10,000 features. Change-Id: I8d0db5121745bd5ee4a350cf46fb37cfa434e3dc Bug: dawn:295 Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14242 Commit-Queue: Austin Eng <enga@chromium.org> Reviewed-by: Kai Ninomiya <kainino@chromium.org>
2019-12-13 01:27:31 +00:00
# Fuzzing Dawn
## `dawn_wire_server_and_frontend_fuzzer`
The `dawn_wire_server_and_frontend_fuzzer` sets up Dawn using the Null backend, and passes inputs to the wire server. This fuzzes the `dawn_wire` deserialization, as well as Dawn's frontend validation.
## Updating the Seed Corpus
Using a seed corpus significantly improves the efficiency of fuzzing. Dawn's fuzzers use interesting testcases discovered in previous fuzzing runs to seed future runs. Fuzzing can be further improved by using Dawn tests as a example of API usage which allows the fuzzer to quickly discover and use new API entrypoints and usage patterns.
Revert "fuzzing: Add supportsErrorInjection option to DawnWireServerFuzzer" This reverts commit f58f69f66bacefb0aaf78e91b462b5db97cea391. Reason for revert: This breaks the MSVC build because the whitebox end2end tests try to both import and export the error injector symbols from libdawn_native Original change's description: > fuzzing: Add supportsErrorInjection option to DawnWireServerFuzzer > > This option will be used by backends that support error injection so > that errors can be injected into a "clean" corpus to generate a seed > corpus with good examples of injected error conditions. > > Bug: dawn:295 > Change-Id: I837acdde6dd4274adb56edf8e4307427f8d6333b > Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14681 > Reviewed-by: Corentin Wallez <cwallez@chromium.org> > Commit-Queue: Austin Eng <enga@chromium.org> TBR=cwallez@chromium.org,kainino@chromium.org,enga@chromium.org Change-Id: I14a15fcd094d431cbb8a29d5642a4a7fe6a11f4c No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: dawn:295 Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14741 Reviewed-by: Austin Eng <enga@chromium.org> Commit-Queue: Austin Eng <enga@chromium.org>
2019-12-20 23:07:40 +00:00
The script [update_fuzzer_seed_corpus.sh](../scripts/update_fuzzer_seed_corpus.sh) can be used to capture a trace while running Dawn tests, and upload it to the existing fuzzzer seed corpus.
Add script to generate fuzzer seed corpus from tests This adds a script which runs the end2end_tests, captures a wire trace, and then minimizes the corpus with the fuzzer. Minimizing the corpus requires libfuzzer, so this only works in a Chromium checkout. Unseeded, the fuzzer starts with coverage of about 600 features. Using a seed corpus captured from the tests, the fuzzer quickly increases coverage to about 10,000 features. Change-Id: I8d0db5121745bd5ee4a350cf46fb37cfa434e3dc Bug: dawn:295 Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14242 Commit-Queue: Austin Eng <enga@chromium.org> Reviewed-by: Kai Ninomiya <kainino@chromium.org>
2019-12-13 01:27:31 +00:00
To run the script:
Revert "fuzzing: Add supportsErrorInjection option to DawnWireServerFuzzer" This reverts commit f58f69f66bacefb0aaf78e91b462b5db97cea391. Reason for revert: This breaks the MSVC build because the whitebox end2end tests try to both import and export the error injector symbols from libdawn_native Original change's description: > fuzzing: Add supportsErrorInjection option to DawnWireServerFuzzer > > This option will be used by backends that support error injection so > that errors can be injected into a "clean" corpus to generate a seed > corpus with good examples of injected error conditions. > > Bug: dawn:295 > Change-Id: I837acdde6dd4274adb56edf8e4307427f8d6333b > Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14681 > Reviewed-by: Corentin Wallez <cwallez@chromium.org> > Commit-Queue: Austin Eng <enga@chromium.org> TBR=cwallez@chromium.org,kainino@chromium.org,enga@chromium.org Change-Id: I14a15fcd094d431cbb8a29d5642a4a7fe6a11f4c No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: dawn:295 Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14741 Reviewed-by: Austin Eng <enga@chromium.org> Commit-Queue: Austin Eng <enga@chromium.org>
2019-12-20 23:07:40 +00:00
1. Make sure gcloud is installed: https://g3doc.corp.google.com/cloud/sdk/g3doc/index.md?cl=head
2. Login with @google.com credentials: `gcloud auth login`
3. You must be in a Chromium checkout using the GN arg `use_libfuzzer=true`
4. Run `./third_party/dawn/scripts/update_fuzzer_seed_corpus.sh <out_dir> <fuzzer> <test>`.
Add script to generate fuzzer seed corpus from tests This adds a script which runs the end2end_tests, captures a wire trace, and then minimizes the corpus with the fuzzer. Minimizing the corpus requires libfuzzer, so this only works in a Chromium checkout. Unseeded, the fuzzer starts with coverage of about 600 features. Using a seed corpus captured from the tests, the fuzzer quickly increases coverage to about 10,000 features. Change-Id: I8d0db5121745bd5ee4a350cf46fb37cfa434e3dc Bug: dawn:295 Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14242 Commit-Queue: Austin Eng <enga@chromium.org> Reviewed-by: Kai Ninomiya <kainino@chromium.org>
2019-12-13 01:27:31 +00:00
Revert "fuzzing: Add supportsErrorInjection option to DawnWireServerFuzzer" This reverts commit f58f69f66bacefb0aaf78e91b462b5db97cea391. Reason for revert: This breaks the MSVC build because the whitebox end2end tests try to both import and export the error injector symbols from libdawn_native Original change's description: > fuzzing: Add supportsErrorInjection option to DawnWireServerFuzzer > > This option will be used by backends that support error injection so > that errors can be injected into a "clean" corpus to generate a seed > corpus with good examples of injected error conditions. > > Bug: dawn:295 > Change-Id: I837acdde6dd4274adb56edf8e4307427f8d6333b > Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14681 > Reviewed-by: Corentin Wallez <cwallez@chromium.org> > Commit-Queue: Austin Eng <enga@chromium.org> TBR=cwallez@chromium.org,kainino@chromium.org,enga@chromium.org Change-Id: I14a15fcd094d431cbb8a29d5642a4a7fe6a11f4c No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: dawn:295 Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/14741 Reviewed-by: Austin Eng <enga@chromium.org> Commit-Queue: Austin Eng <enga@chromium.org>
2019-12-20 23:07:40 +00:00
Example: `./third_party/dawn/scripts/update_fuzzer_seed_corpus.sh out/fuzz dawn_wire_server_and_frontend_fuzzer dawn_end2end_tests`
5. The script will print instructions for testing, and then uploading new inputs. Please, only upload inputs after testing the fuzzer with new inputs, and verifying there is a meaningful change in coverage.