`-o` will emit the files to the given root output directory
`--verbose` will print what's going on, to help with debugging
Omitting these flags will behave as before.
Also consolidate the utils package into fileutils. These were two packages with near identical functionality.
Change-Id: I855dd4b57807fb9239a52e7f357842d4ba2517ee
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/107687
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
• Add `cts validate` command used to check for expectation collisions.
Can be used as a presubmit check.
This is more tightly checked than the previous logic, as this works on just
the expectations, instead of results.
• Fix an issue where the test result reduction could introduce collisions with
'Skip' expectations.
To fix this, the update process first adds 'consumed' results for the skipped
tests, preventing test tree reduction for that part of the tree.
• Fix a bug in the generation of 'New failures' and 'New flakes' which produced
more expectations than was necessary.
The issue here was that the tree roots could contain overlaps, and roots could
be processed before sub-trees, resulting in inefficient expectations.
• Fix collisions in the expectations file, and update with results from
the most recent roll.
Change-Id: I7b64553408998fb4416458ce564fc49c8f6d4d07
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/101860
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
may_exonerate indicates that a test failed for a known issue that
we could exonerate. Merging of test results now removes results
with may_exonerate unless all of them were tagged as such. So, if
for example, a test fails for a known timeout issue, but has a
subsequent pass, the timeout will be ignored.
This serves to reduce the impact of known, hard-to-fix issues and
allow the CTS roller to make progress with less noise.
Change-Id: I5103a666496398a17b3aa6ccf3f267421e40ba97
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/101804
Reviewed-by: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
• Add the included trybots in the CL description. All of these trybots are tested by the roll, but the final CQ-submit wouldn't necessarily test all of the variants before landing. This would mean that the 'cts export' could miss some results, as it takes the last PS with any results.
• Add --force flag to cts roll to force a roll. Useful for testing.
• Emit timing diagnostics for tests labelled 'Slow' instead of unhelpfully stating they pass.
• Enable the --cl and --ps flags for cts export
• Export with the most recent data to the top of the spreadsheet
Bug: dawn:1401
Change-Id: Id926367ab805bfb9f3032fce9cce7f00daf7a5d4
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88661
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
MinimalVariantTags accepts a list of tag-sets (e.g GPU tags, OS tags, etc),
and returns an optimized list of variants, folding together variants that
have identical result query-to-status mappings, and removing redundant tags.
Bug: dawn:1342
Change-Id: I759c82e9a0631a9d321d376656e5a2dbbf5f5507
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87643
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Add result.List.StatusTree() for building a query.Tree[Status].
Add helpers for serializing results.
Add helpers for merging and de-duplicating results.
Change the interface of result.List.ReplaceDuplicates() so that the
merging function takes a status set instead of a list of results.
Bug: dawn:1342
Change-Id: I77580ec5fd4c8f12109fb6e9e83afea8b740260c
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87240
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
