Commit Graph

278 Commits

Author SHA1 Message Date
Ben Clayton f8e0aac2a6 tools: Port 'cov' package from SwiftShader
A library for dealing with coverage information, and compressing CTS test information down into something that can be easily downloaded and visualized in a web page

Change-Id: If025e1555665609f7c963bad14c9c91d8324a390
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/113642
Reviewed-by: Austin Eng <enga@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-12-12 21:49:02 +00:00
Ben Clayton f927bcb5a6 tools: Port turbo-cov from SwiftShader
A tiny utility that emits just the per segment coverage in a binary stream. This avoids the overhead of encoding to JSON, which provides substantial performance improvements.

Change-Id: I36a588069d69c5c800d31bca8dd5c542bcdbe313
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/113641
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-12-12 21:21:54 +00:00
Ben Clayton 5f55990471 tools/cts: Fix expectations update failure
appendConsumedResultsForSkippedTests() adds fake results for expectations that are 'Skip'ed, so that these sub-trees aren't collapsed as all passing.

However, this code was not handling the fact that there might actually be results for the cases.
This happened because there was an expectation collision in the expectations.txt file, and tests were not being skipped even though there was an expectation with a Skip (a collision expectation was used instead).

Change-Id: I2a1543d231db44dc8aa6683d051f884f4cb96853
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/113520
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Ben Clayton <bclayton@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
2022-12-08 20:59:01 +00:00
Ben Clayton f2b8d2b1ab tools: Parallelize ./tools/format
10x speed improvement on my 3990x machine

Change-Id: Ie4b2377719e9a25860ec05bb7649aa652d26ccf0
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/113022
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Ben Clayton <bclayton@google.com>
2022-12-06 18:29:08 +00:00
Ben Clayton d5d207ba9f tint: Add builtin type aliases (vec3f, etc)
Fixed: tint:1772
Change-Id: I4bed36ded91ca5288875ed6ea819ff4bbb432186
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/112340
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: James Price <jrprice@google.com>
2022-12-01 13:41:56 +00:00
Ben Clayton d6800098e7 Add tools/src/cmd/git-stats
Can be run with:
./tools/run git-stats

Change-Id: Ie46be01ff3318ff50acdb13eb809d5304f867f04
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/111180
Auto-Submit: Ben Clayton <bclayton@google.com>
Commit-Queue: David Neto <dneto@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: David Neto <dneto@google.com>
2022-11-23 21:08:52 +00:00
Ben Clayton 2f60bbdf9c tools: Add tint-bench
A simple tool to benchmark tint based on a template file.

Bug: tint:1122
Change-Id: I34ee0fb98e6d2187c145cc38ec3cb48606242cb4
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/110820
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-11-21 19:14:32 +00:00
Ben Clayton d114055e4e tools/src: Move template logic of 'gen' to package
And add tests.

This is useful for other tooling.

Change-Id: Ia399071baf6d4bb617f3c73e4ccd4ed72d522c2e
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/111020
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-11-21 17:43:11 +00:00
Ben Clayton 19f6cf9ec3 tools cts/roll: Generate cache file list
Will be used by GN to know the outputs of the generate cache step.

Change-Id: Iae09477fb355eac41ff9b2204605f46ed78e08ec
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/110620
Reviewed-by: Austin Eng <enga@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-11-17 22:38:04 +00:00
Ben Clayton bca96d9bc8 PRESUBMIT: Check that generated files aren't stale
Fails presubmit if you need to run `./tools/run gen`

Change-Id: I05311cd668c5a1f4f484b25cc1367f680a9d24eb
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/110140
Commit-Queue: Ben Clayton <bclayton@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Ben Clayton <bclayton@google.com>
2022-11-16 21:34:14 +00:00
Ben Clayton 6be02b607f tint: Fix stability of intrinsic_table.inl
Use a StableSort() to ensure the output is the same between different
versions of golang.

Change-Id: Ic81688d189ce62816ba0aefddcef607b76415a5d
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/110361
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Ben Clayton <bclayton@google.com>
2022-11-16 21:30:13 +00:00
Ben Clayton 1ef277db0e tint: Skip e2e test validation for known-good
Cache the hash of the test output, and only re-validate if the output changes.
This speeds up incremental end-to-end testing.

Change-Id: I44bc1eff62320df1e80524e8481d1c012fb67f42
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/109761
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: James Price <jrprice@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-11-14 13:58:36 +00:00
Ben Clayton 5e4fc69a69 tools/intrinsic: Handle CRLF
Fixes cryptic error message:

error calling Sem: src/tint/intrinsics.def:14:1: unexpected '

Change-Id: Ic343e77f6a1976ebdc4952eb590751c34b014be8
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/108143
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-11-01 22:45:35 +00:00
Ben Clayton 68ed8d92d3 tools/gen: Add flags for explicit output
`-o` will emit the files to the given root output directory
`--verbose` will print what's going on, to help with debugging

Omitting these flags will behave as before.

Also consolidate the utils package into fileutils. These were two packages with near identical functionality.

Change-Id: I855dd4b57807fb9239a52e7f357842d4ba2517ee
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/107687
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-10-31 18:28:22 +00:00
dan sinclair c3cbc35650 Add negative number parsing into `@test_value`
This CL updates the intrinsics lexer to allow negative values for int
and float numerics. This allows doing `@test_value(-2)` in the def file.

Change-Id: I2cad9b25a2932057ce9bc51dec6c32231e06f0a0
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/107440
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
Reviewed-by: Ben Clayton <bclayton@google.com>
Auto-Submit: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-10-27 18:44:50 +00:00
Ben Clayton 95dd428c75 tint/intrinsics.def: Add @test_value() annotation
Specifies the value to use for argument values when generating end-to-end tests.

Use this to provide a legal value for atanh().

Change-Id: I008050c856f9d687ab918c68e90678c4e74f3a1d
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/106887
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
2022-10-25 13:08:45 +00:00
dan sinclair 6e77b47ed9 Update internals to initializer instead of constructor.
This CL catches up the internals (along with a few error messages) to
say `initializer` instead of `constructor.

Bug: tint:1600
Change-Id: I8e56572c310d77da1130380bdd32b334f27c8e46
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/106462
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Auto-Submit: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Antonio Maiorano <amaiorano@google.com>
2022-10-20 13:38:28 +00:00
Ben Clayton 138ddcc08c tools/setup-build: Support x86
Change-Id: I03cc04ac3f8f511be530ccc57055b31574ed3d94
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/105560
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-10-13 12:55:42 +00:00
Ben Clayton db368f1b9c tools/tint/intrinsic: Sort enum values lexicographically
Reorders the output of the CanonicalizeEntryPointIO transform, but
otherwise is a no-op.

This will help with diagnostics that print the enum values.

Bug: tint:1629
Change-Id: Iff517d88836d2a8cd42ce3cfde6363c1973df0dd
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/105321
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-10-11 18:26:18 +00:00
Ben Clayton 5dac4f9644 tools/remote-compile: Handle socket disconnects
If the socket disconnected mid-communication, the server could spin, waiting for new data.

Actually handle recv() errors, preventing the server spinning itself to death.

Also fix code style to be more tint-like (snake_case variables, PascalCase functions)

Change-Id: I9fcbfde303a8624e7e1ff87abd33581589f4da42
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/105142
Auto-Submit: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-10-10 15:16:23 +00:00
Ben Clayton 467e6e3fc6 tools/remote-compile: clang-format
This was using the old tint code style

Change-Id: I1aff541eb4cc0d7ec0e045b555710aa605c4da28
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/105141
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-10-10 12:17:03 +00:00
dan sinclair ff7cf21021 Rename StorageClass to AddressSpace.
This CL updates the internals to use AddressSpace instead of the old
StorageClass name.

Bug: tint:1404
Change-Id: Iecc208e839453437f4d630f65e0152206a52db7e
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/104420
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
Auto-Submit: Dan Sinclair <dsinclair@chromium.org>
2022-10-03 14:05:23 +00:00
Ben Clayton 62f8e733b1 tools/cts roll: Add flag to run CI only tests.
Fixed: dawn:1554
Change-Id: I1acd5e5d6502b56f26bd4a0257ac15e1baa10fcf
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/104044
Commit-Queue: Ben Clayton <bclayton@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
Reviewed-by: Austin Eng <enga@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-09-27 19:29:56 +00:00
Ben Clayton aad2e9c0b5 tools/cts: Add `cts validate`, improvements & fixes
• Add `cts validate` command used to check for expectation collisions.
  Can be used as a presubmit check.
  This is more tightly checked than the previous logic, as this works on just
  the expectations, instead of results.

• Fix an issue where the test result reduction could introduce collisions with
  'Skip' expectations.
  To fix this, the update process first adds 'consumed' results for the skipped
  tests, preventing test tree reduction for that part of the tree.

• Fix a bug in the generation of 'New failures' and 'New flakes' which produced
  more expectations than was necessary.
  The issue here was that the tree roots could contain overlaps, and roots could
  be processed before sub-trees, resulting in inefficient expectations.

• Fix collisions in the expectations file, and update with results from
  the most recent roll.

Change-Id: I7b64553408998fb4416458ce564fc49c8f6d4d07
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/101860
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
2022-09-17 19:30:29 +00:00
Austin Eng a48e46f3a4 CTS: Handle may_exonerate tag
may_exonerate indicates that a test failed for a known issue that
we could exonerate. Merging of test results now removes results
with may_exonerate unless all of them were tagged as such. So, if
for example, a test fails for a known timeout issue, but has a
subsequent pass, the timeout will be ignored.

This serves to reduce the impact of known, hard-to-fix issues and
allow the CTS roller to make progress with less noise.

Change-Id: I5103a666496398a17b3aa6ccf3f267421e40ba97
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/101804
Reviewed-by: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
2022-09-14 14:29:07 +00:00
dan sinclair e214a130dd Run substitute_overrides for all tests.
This CL updates all of the integration tests to run the substitute_override
transform where needed. The test runner is updated to match comments in
the spvasm files as well as the wgsl files.

Bug: tint:1155
Change-Id: I8187d426970d056d744e530ed6447e4ec69db5f4
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/101661
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-09-09 16:04:39 +00:00
Austin Eng 185fd3deca Add go tests to presubmit
Change-Id: I2d56a463cecf0b1bcb96c51ae134c6fa07d87b21
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/100560
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
2022-09-06 18:20:33 +00:00
Loko Kung 4681f5bddd Moves sorting into `format` and out of `parse`.
- Reverts test updates in
  https://dawn-review.googlesource.com/c/dawn/+/98761 that were
  introduced because of the sorting in `parse`.

Change-Id: Ia27386e9aa2f16b9a4ba26e77207e5a4d6f4df97
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/99020
Auto-Submit: Loko Kung <lokokung@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-08-12 22:05:13 +00:00
Loko Kung 0d3a7bbd7e Fixes go test failures and issues from dawn/98241.
Change-Id: Idf4f47eba595dcf3f2c44b5f73179b2773a68aef
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/98761
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-08-11 07:50:21 +00:00
Loko Kung eaca2ebd4c Update cts expectations to be sorted by bug first.
- Run format on the current expectations file to make it up to date.
- It's a lot easier to read the failures when they are grouped by the bug.

Change-Id: I333a0b237ccd405d204e47809eb7ab723824fddd
No-Try: True
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/98241
Auto-Submit: Loko Kung <lokokung@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Loko Kung <lokokung@google.com>
2022-08-09 16:40:32 +00:00
Zhaoming Jiang 9443cebd53 tint: End-to-end tests for f16 built-in
This patch add Tint end-to-end tests for built-ins using f16 types.

Bug: tint:1473, tint:1502
Change-Id: I09db6e0b7e90541fb246e11d475e27be96a6d07e
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97340
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Zhaoming Jiang <zhaoming.jiang@intel.com>
2022-08-05 15:15:17 +00:00
Ben Clayton 92f2cb79b5 tools/gen: clang-format generated .cc, .h and .inl files
Change-Id: I5a79cc0b5da1967632d9df02e058a8e3e5073d2d
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97441
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-08-01 19:29:14 +00:00
Zhaoming Jiang 6fe1f515d4 tint: Add f16 support for parts of float built-in, part 1
This patch add f16 support for a major part of numeric built-in, and
implement corresponding unittests for resolver and backends. This patch
also enable f16 constant evaluation for unary minus operator, `atan2`
and `clamp`.

The following numeric built-ins are not supported yet:
* frexp
* modf

The end-to-end tests for f16 built-in are not added yet.

Bug: tint:1473, tint:1502
Change-Id: If807185617b21c510a1a9c371179a60800c4f875
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96722
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Zhaoming Jiang <zhaoming.jiang@intel.com>
2022-07-29 11:41:51 +00:00
Ben Clayton a253a5f7e7 tools: Expand / clean-up the template WGSL type helpers
Fix `IsAbstract()` so that it doesn't consider most-nested element types.
Add `ElementType()` and `DeepestElementType()` helpers.
Add `OverloadUsesF16` as a helper for https://dawn-review.googlesource.com/c/dawn/+/96722.

Simplifies template code.

Change-Id: Iff5a9a7258caea06e00ee37c29e5298d9c35b799
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97361
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
2022-07-27 19:36:49 +00:00
Zhaoming Jiang f8fb94d9e8 tint: Emitting enable directive for DP4A End-to-End test
This patch modify the test/tint/builtins/gen/gen.wgsl.tmpl to emit
enable directive for dot4I8Packed and dot4U8Packed built-in function.
The expectaion files are added.

Bug: tint:1497
Change-Id: I53331695fe2e6609858e94bc261383ba3028d77c
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96640
Reviewed-by: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Zhaoming Jiang <zhaoming.jiang@intel.com>
2022-07-27 13:27:56 +00:00
Ben Clayton 695a6d82ae tools/cmd/gen: Add more utilities to the templates
* Ensure that copyright years are preserved when regenerating.
* Add 'Split' function which maps to strings.Split.
* Add 'Scramble' function with messes with a string.
* Add 'Import' function which allows templates to be imported, allowing
  for reusable macros.

Change-Id: Ib77f59a989cf55addcced3e337c9031062a83470
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97149
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-07-27 01:10:15 +00:00
Ben Clayton c768e640f4 tint/intrinsics.def: Change language for enums
Previously enum members were added to the global namespace, so that
overload template parameters could be constrained to a single
enum-entry without the need to declare a matcher. While this was a minor
convenience feature, it means that you cannot declare an enum with
members that share the same name as a type. This will be very common for
extensions, like 'f16' where 'f16' is the name of an extension and a
type name.

Change scoping so that enum members need to be fully qualified. Also
change the intrinsic syntax so that enums always need to use a matcher
for enums.

Change-Id: Ided91130e9df537d38dc8ecb41325c0992dea14b
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97146
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-07-26 17:49:54 +00:00
Ben Clayton cde5009be3 tools: Shuffle 'intrinsic-gen' tooling
Rename to 'gen', so that more templating can be added without having a confusing name.

Can now be run with './tools/run gen'

Move the bulk of the intrinsic-gen logic to `tools/src/tint/intrinsic`

Change-Id: I750989a5aa86272c10c2ad37adffe7def11c61f2
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97141
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
2022-07-26 15:46:44 +00:00
Ben Clayton ee36e39296 test/tint/builtins: Generate abstract numeric tests
Change-Id: I2da181af0694f73ec4be07ed40e4e70d49b53583
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97140
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-07-26 14:27:25 +00:00
Ben Clayton 1308395055 Add tools/cfi/ignores.txt
Empty file. Allows you to build with Control Flow Integrity enabled.

Bug: dawn:1407
Change-Id: I8a67382c9c50cfdf005dba6d46cd148fdaba0fc3
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97005
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-07-25 15:56:38 +00:00
Corentin Wallez 3b7e08857c tint test-runner: exclude .fxc/dxc.hlsl from glob
Bug: tint:1620
Change-Id: I5d5182025c140f71b69d3a9e540ae9ef8aec04e3
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97003
Commit-Queue: Ben Clayton <bclayton@google.com>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Auto-Submit: Corentin Wallez <cwallez@chromium.org>
2022-07-25 08:47:38 +00:00
Ben Clayton 98f18d2789 tools/cts: Generate test list as part of roll
Also parallelize some of the more lengthy file generation steps

Bug: dawn:1479
Change-Id: I7674fca4958e4d9948e287008916c4b0d33e1ca1
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97022
Reviewed-by: Austin Eng <enga@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-07-22 19:52:36 +00:00
Ben Clayton 7d34de88f1 tint/test-runner: Split expectations for FXC and DXC
Change tint's `--fxc` flag to take the path of the FXC compiler DLL.
Have tint attempt to validate with both FXC and DXC if `--validate` is
passed.

Fix the 'dirsWithNoPassExpectations' logic which looks like it got
broken with the tint -> dawn merge. It also incorrectly applied
filepath.FromSlash() on windows.

Change-Id: I0f46aa5c21bc48a2abc48402c41f846aff4a8633
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96800
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-07-22 17:43:27 +00:00
Zhaoming Jiang 1e98a9ba4a tint: Normalizing type matcher's name in intrinsics.def
This patch normalize the abstract interger and abstract float type name
to `ia` and `fa` and the type matcher name to `fia_fiu32_f16` format in
intrinsics.def, and also rename `aiu32Dispatch` and `afi32f16Dispatch`
in resolver/const_eval.cc to `Dispatch_ia_iu32` and
`Dispatch_fia_fi32_f16`.

Bug: tint:1581
Change-Id: Ia48e150d6028e4a7b999f72e57508df369b3e2d7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96721
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Antonio Maiorano <amaiorano@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-07-21 19:55:35 +00:00
Ben Clayton 056f97a9e5 tint: Add a benchmark for atan2 const eval
And increase the test runner timeout from 30s to 2min.
FXC really doesn't like this shader, however I expect this to be
made much faster once constant evaluation for atan2 is implemented.

Change-Id: Id8a8ba97b5a99a2f94633a0732300a35ba6dc1c3
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96401
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-07-19 14:50:33 +00:00
Ben Clayton 6a80ce6c66 tint/resolver: Use the intrinsic table for type const dispatch
Use the @const annotations to more efficently dispatch to the right
method of the ConstEval class.

Reduces a whole lot of dynamic casting logic.

Change-Id: I6791aac51b935b46d63af29abd1e577b3306a0ff
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/95950
Commit-Queue: Antonio Maiorano <amaiorano@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-07-15 22:27:50 +00:00
Ben Clayton 8f488524b8 perfmon: Use backticks for analysis table
Using whitespace for code blocks no longer works with gerrit

Change-Id: I614aff78297ab96a3663888d59c10d8ad295c076
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96280
Commit-Queue: Antonio Maiorano <amaiorano@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-07-15 16:23:19 +00:00
Ben Clayton 2777cbc441 tools/cts: Fix go unit tests
Got broken with https://dawn-review.googlesource.com/c/dawn/+/95841.

Change-Id: I75e0996dfb453ffa66dd580252a7f6fbcf8e46a7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/95845
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-07-08 14:23:15 +00:00
Ben Clayton be88e62c0f tools/cts: Treat 'Skip' results as 'Pass'
Change-Id: I2548c2449e535cff1f6907a9f2e892037038aa02
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/95841
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-07-08 12:33:46 +00:00
Ben Clayton 808f616762 gn: Add empty ubsan ignore list
Required to build Dawn with GN flag `is_ubsan = true`

Change-Id: I80b49b81377f0a9c8bd2c1310e12786427947c38
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/95760
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
2022-07-07 16:34:01 +00:00
Austin Eng 65d35f8e7d Report javascript duration in CTS tooling
Always report the Javascript duration from the page instead of
the total duration from ResultDB. Reporting of the total duration
could be added back in the future, but usages of the tooling are
currently interested in the Javascript duration.

Bug: chromium:1336333
Change-Id: I74e91e7a9c29f3a4c45ed67898cc6f2de62e19a4
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/94762
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
2022-06-28 00:15:05 +00:00
dan sinclair 0a4e2a1373 Remove NOLINT lines from c++17 headers
The depot_tools cpplint.py has been updated to accept the c++17 headers
so we no-longer need to mark with NOLINT.

Bug: dawn:1379
Change-Id: I938a4cac5ca881cd1b556f4b8d58741b6a1e6af1
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/94240
Auto-Submit: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
2022-06-18 14:34:00 +00:00
Austin Eng faf98b1bbc Fix roll_test.go
Change-Id: Ife55a04d5dc2fb4f2f1775dc0d1f0a1169cecc70
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/93306
Reviewed-by: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
2022-06-10 18:33:23 +00:00
Austin Eng 737ff5b482 times.go: Add test query filter
Usage: `./tools/run cts time -query QUERY ...`

Change-Id: Ifac16ee2405d1b5d03d246bf1556245e2dafe8c1
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/93304
Commit-Queue: Austin Eng <enga@chromium.org>
Reviewed-by: Ben Clayton <bclayton@google.com>
2022-06-10 17:24:23 +00:00
Austin Eng 9d14b164bf Update CTS roller to generate reftest/idl test sources
These sources need to be generated in Dawn's source tree
so that Chromium's //third_party/webgpu-cts DEP can be
removed.

Bug: chromium:1333969
Change-Id: I03c8cba691bcbfac00839f0cdd40fab6198ec83f
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/91060
Commit-Queue: Austin Eng <enga@chromium.org>
Reviewed-by: Ben Clayton <bclayton@google.com>
2022-06-10 16:25:43 +00:00
Austin Eng 6cc677761d Report test-based aggregation of CTS times
Adds an "-aggregate" flags to ./tools/run/cts time to report results
aggregated by test

Change-Id: Id8d15188846b3bff79ac4c65a09e5d6740acde73
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/93300
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
2022-06-10 16:13:13 +00:00
Ben Clayton 37b1f0fb5a tools/cts: Rekick cancelled or infra-failed builds
For commands like `cts update`, `cts export`:
If a build completed with `StatusInfraFailure` or `StatusCanceled` and we're trying to obtain the results, re-kick the build.

Bug: dawn:1342
Change-Id: I84565b06eb70e92f3b75a0ab2befd7ed83315140
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92940
Auto-Submit: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
2022-06-07 16:51:54 +00:00
dan sinclair d574be5750 Convert @stage to short form
This CL converts remaining @stage instances in the dawn tree to use
the equivalent shorter variant.

Bug: tint:1503
Change-Id: I74594cd68544fbd692f77d4646991d9c27e218f8
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92484
Reviewed-by: Ryan Harrison <rharrison@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
2022-06-07 15:57:35 +00:00
Ben Clayton c0af5c5c9c tint: Add builtin tests for arguments passed by var
Change-Id: I81b69d23e40675d7f525e6369afec9aa0659d043
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92321
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-06-02 14:36:10 +00:00
Ben Clayton 451eee0fed tint: Stub intrinsic @const evaluation
Add support for @const to builtins in intrinsics.def.
Propagate this flag through to the intrinsic table.
Handle builtins that are @const annotated in the resolver.

Currently no intrinsics are decorated with @const, so there's nothing to
test (yet).

Bug: tint:1504
Change-Id: I172483688617782bd7c58b70e3f38d0222a5d1af
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92323
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-06-01 23:57:20 +00:00
Ben Clayton e3e91c0d75 tools: intrinsic-gen - [[decoration]] -> @attribute
More closely resembles WGSL.

Change-Id: Ia1b93bb443b4e3bf9329f8026d1924f12648cb7b
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92245
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-06-01 20:44:50 +00:00
Ben Clayton 8cbfad7e5d cts: Remove use of intel-0x5912 and intel-0x3e9b tags
The CTS runner can randomly pick between these two GPUs, leading to unstable expectations.

Just use 'intel' for now. We might get a more targetted tag in the future, but this will do for today.

Bug: dawn:1387
Bug: dawn:1444
Change-Id: I2c77ab0060976c40e34b1597b5594894867b7edd
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92242
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-06-01 12:15:49 +00:00
Ben Clayton 53edbc7635 tools: Add colored icons to the CTS roll msgs
Should help quickly identify errors vs warnings vs notes

Bug: dawn:1342
Change-Id: I8063be0d7cff06428be998a6fa0ad76048ccbc29
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/91960
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-05-30 16:36:18 +00:00
Ben Clayton 2cefd07454 tools: Automatically generate resource_files.txt
When running `tools/run cts roll`

Bug: dawn:1342
Change-Id: I6af9a34fd4236c804ece32e0777f58ec98504f7b
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/91849
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
Commit-Queue: Austin Eng <enga@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-05-27 18:58:30 +00:00
Ben Clayton 3e6bf1ae73 tools/setup-build: Properly emit release GN args
is_debug defaults to true, not false. Be explicit to fix the use of:

  ./tools/setup-build gn release

Change-Id: I225f0009f99901e6afeb0c504c98b2be961b7f76
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/91423
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-05-25 22:47:23 +00:00
Ben Clayton 6ac00ed0c0 tint: IntrinsicTable: Add abstract numeric types
These aren't used by anything (yet). Baby steps.

Bug: tint:1504
Change-Id: Icf0261ec9c6802f004d9f1bc4780a6376ebb8dfb
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90530
Reviewed-by: David Neto <dneto@google.com>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-05-19 18:30:39 +00:00
Ben Clayton 0a63c6d2fb tint: intrinsic-gen: Fix parsing of multiple decorations
Change-Id: I61365dea17ab9174cfcb2d513cf1f7ee28ce0647
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90528
Reviewed-by: David Neto <dneto@google.com>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-05-19 18:05:09 +00:00
Ben Clayton c670018aea tint: intrinsics.def Support [[precedence]] decoration
Add support for a [[precedence(N)]] decoration on intrinsic table type
declarations. This will be used to ensure the type with the lowest
conversion rank is matched when a matcher could match multiple types
for a given abstract numeric argument type.

Bug: tint:1504
Change-Id: I96475b000c0917bbfa4e2873b1731ce048b96a7d
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90664
Reviewed-by: David Neto <dneto@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-18 18:56:58 +00:00
Ben Clayton 4c9ed74b5e tint: IntrinsicTable: Rename open/closed -> template
The concept of 'closing' an open type or number made sense when these
were immutable once the first type/number had been matched during
overload resolution.
In order to support abstract numerics, these template parameters need to
be constrained as the arguments are evaluated, so there's no longer a
binary open / closed state.

Give up on this concept, and rename everything to 'template type' and
'template number'. This is likely easier for people to understand
anyway.

Also fix a small typo in the ICE message printed when there's an
ambiguous overload resolution (should never happen with the current
entries in the table).

Bug: tint:1504
Change-Id: I2bf043c71e5afa757259968eae4af830c50f38e0
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90662
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: David Neto <dneto@google.com>
2022-05-17 22:42:32 +00:00
Corentin Wallez 8520f3bb93 CTS roller: Give a better error message when TSC is not found.
Bug: None
Change-Id: Ic9355346cf834bd4c2989e473f9e98e0be6f1f22
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90503
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Auto-Submit: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: Ben Clayton <bclayton@google.com>
2022-05-16 17:00:51 +00:00
Ben Clayton 5a69597698 tint: Add type constructors and converters support to intrinsic-gen
These are currently not used, but the first step towards moving type
constructors and converters over to using the intrinisc table.

This will simplify maintenance of type functions, and will greatly
simplify the [AbstractInt -> i32|u32] [AbstractFloat -> f32|f16] logic.

Bug: tint:1504
Change-Id: I15526670a6ff801e66551ab5adc37b1570ac49de
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90242
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-16 09:48:41 +00:00
Ben Clayton 59e23943f3 tint: Minor IntrinsicTable cleanup
Fix the namespace - this should have been tint::resolver.
Use a single u32 bitset for overload flags instead of multiple fields.

Bug: tint:1504
Change-Id: I633b21ce14e20fc9aeeed5221886c1d22e327bdc
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90241
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-05-13 16:14:17 +00:00
Ben Clayton 77473b4699 tint: Split tables for unary and binary operators
Do the partitioning of unary and binary operators in the intrinsic table
generators, instead of searching all operators at runtime.

Will allow code to be simplified.

Bug: tint:1504
Change-Id: I67246b954e530e0542b1b67c99fb34a756cf532a
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90240
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-05-13 14:35:37 +00:00
Ben Clayton 08f94e9004 tools/CTS: merge [Slow] + [Pass] -> [Slow]
Instead of Failure.

If we have duplicate results (same query, same tags), but with different results, then we have to merge them. Previously we'd merge Pass and Slow as Failure, which was just odd. Fix this so the combined result is Slow.

Bug: dawn:1342
Change-Id: If9b035bb3c1e59b7a93a5c8b194f854405e656d6
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/89960
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-05-12 10:08:46 +00:00
Ben Clayton b61e0452f8 tint: Add unary-ops to the intrinsic table
• Declare all the unary ops in the intrinsics.def file.
• Reimplement the bulk of Resolver::UnaryOp() with the IntrinsicTable.

This will simplify maintenance of the operators, and will greatly
simplify the [AbstractInt -> i32|u32] [AbstractFloat -> f32|f16] logic.

Bug: tint:1504
Change-Id: Ifc646d086fc93cfbe3f3f861b8c447178664c1f7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/89028
Reviewed-by: James Price <jrprice@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-05-09 21:22:24 +00:00
Ben Clayton e6e96def66 tint: Add operator support to intrinsic-gen
Adapt the builtin parsing and resolving to also support operators.
Will be used to generate intrinsic table entries for operators.

This will simplify maintenance of the operators, and will greatly
simplify the [AbstractInt -> i32|u32] [AbstractFloat -> f32|f16] logic.

Bug: tint:1504
Change-Id: Id75735ea24e501877418812185796f3fba88a521
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/89026
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
2022-05-09 18:08:23 +00:00
Ben Clayton e132516e6f tint: Rename and move builtin_table and builtin-gen
• Rename 'builtin-gen' back to 'intrinsic-gen', as 'intrinsics' now
  include both builtins and operators.
• Move the intrinsic definitions, and IntrinsicTable to the resolver
  package, where it belongs.

Bug: tint:1504
Change-Id: I5ad5c285c1e360a224ee1235c293ccd55eb2693d
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/89025
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-05-06 15:13:01 +00:00
Ben Clayton 8495affacf tools: More CTS tooling improvements
• Add the included trybots in the CL description. All of these trybots are tested by the roll, but the final CQ-submit wouldn't necessarily test all of the variants before landing. This would mean that the 'cts export' could miss some results, as it takes the last PS with any results.
• Add --force flag to cts roll to force a roll. Useful for testing.
• Emit timing diagnostics for tests labelled 'Slow' instead of unhelpfully stating they pass.
• Enable the --cl and --ps flags for cts export
• Export with the most recent data to the top of the spreadsheet

Bug: dawn:1401
Change-Id: Id926367ab805bfb9f3032fce9cce7f00daf7a5d4
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88661
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-03 16:58:43 +00:00
Ben Clayton 174c508c8d tools/cts: Bunch of fixes for 'cts roll'
• Strip 'release-x64' tags.
  These are always paired with 'release', and all
  other tests just have the 'release' tag. This
  means that results with the 'release' +
  'release-x64' tags are a subset of just the
  'release' tags, leading to many broken
  assumptions in the expectation update.

• Don't consider tests with the error reason
  'asyncio.exceptions.TimeoutError' as slow, but
  as a failure. The CTS expectation updater will
  already mark tests exceeding the config
  threshold as slow. This reduces the amount of
  data requested from resultdb, and prevents
  failed rolls when results are actually
  deadlocking.

• Fix inferring of patchset in GetResults().
  LatestPatchest() was called but was assigned to
  a nested scope.

Bug: dawn:1401
Change-Id: I57b2c9a029b31430d63a056f7b13c4bf1bc5b437
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88450
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
2022-05-02 20:44:21 +00:00
Ben Clayton bf464ddbdf tools: Add the 'cts roll' sub-command
Performs a CTS roll, updating ts_sources.txt and expectations.txt.

Bug: dawn:1342
Change-Id: I18cc644e9eef1db6034fc07fd94f21f070561265
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88444
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-02 20:44:21 +00:00
Ben Clayton d9f2a61c6f tools: Add the 'cts export' sub-command
Updates a Google spreadsheet with latest CTS statistics.

Bug: dawn:1342
Change-Id: I01a9c99032c755d6a78af674675d8c9cbfc79019
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88443
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-02 20:44:21 +00:00
Ben Clayton 446166c81b tools/setup-build: Into the 'active' symlink directory
Previously we'd generate the build files into the symlink target directory.
CMake gets grumpy if the cache is invalidated, and you're building in the 'active' directory, which does not match the directory path it remembered.

Also: Enable ccache if found on PATH

Change-Id: I7e0dc93516b2c59d6bf346fc943acea3d0021087
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88311
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-04-29 20:18:45 +00:00
Ben Clayton 665c07ac6a tools: Add the 'cts update' sub-command
Updates the expectations file with new results.

Bug: dawn:1342
Change-Id: I05cab48eb1cf0f9ab7e652a1b7c3da361c45b0b1
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88442
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
2022-04-29 19:09:17 +00:00
Ben Clayton 1ed858c20a tools: Add the 'cts merge' sub-command
Merges results files into one.

Bug: dawn:1342
Change-Id: Ia241afc982c88c840a102abba211053611f77f5b
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88441
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-04-29 19:09:17 +00:00
Ben Clayton 09eafe71b5 tools: Add the 'cts results' sub-command
Pulls the CTS results from a patchset

Bug: dawn:1342
Change-Id: I879605b71517004d3a8e81a1b4481bbc0105a22c
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88440
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-04-29 19:09:17 +00:00
Ben Clayton 40bed821f8 tools: Add the 'cts time' sub-command
Add the common utilities for starting and fetching build results.

'cts time' provides views on the longest running tests, along with
a histogram view of all test times.

Bug: dawn:1342
Change-Id: Ia3707f7f062ea26a2406e3163a26e1cc7e30e3b2
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88319
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-04-29 19:09:17 +00:00
Ben Clayton 251e464af1 tools: Add the 'cts format' sub-command
Formats an expectation file

Bug: dawn:1342
Change-Id: Idc7728ad5b955fcbdb4062e511aff6665a711718
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88318
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
2022-04-29 19:09:17 +00:00
Ben Clayton 846e3149b7 tools: Begin adding the CTS command line tool
This CL introduces the Config structure, and config file.
Used by all the sub-commands.

Bug: dawn:1342
Change-Id: Iaad5a02bc2ceba2c63265cf0d1086dc7901e7ed7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88317
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-04-29 19:09:17 +00:00
Ben Clayton e822f5e016 tools: Add 'Duration' field to cts.Result.
Will help identify slow tests.

Bug: dawn:1342
Change-Id: I422d345361785addcc2faa6281e0608de02629b7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88316
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
2022-04-29 19:09:17 +00:00
Ben Clayton 15a85dec7e tools: Add 'subcmd' package
Helper for writing command line tools that have sub-commands.
Will be used by the CTS tool (roll, results, export, etc).

Bug: dawn:1342
Change-Id: Id22beecbc02b73af23064adc67c590f825b37789
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88315
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-04-29 16:30:24 +00:00
Ben Clayton 526087bde7 tools: Add more functionality to gerrit package
Add Patchset.RefsChanges(). Returns the 'refs/changes/X/Y/Z' string
for the given patchset.

Add []FileComment parameter to Gerrit.Comment(), allowing the posting
of comments on particular lines of the change.

Bug: dawn:1342
Change-Id: Ia4113a158b285c606b2c6c520cef6c9e8030fae7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88314
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
2022-04-29 15:15:43 +00:00
Ben Clayton 2fb28e008d tools: Extract getDawnRoot() from run-cts to utils
This is needed elsewhere.
Rename location.go to paths.go, as this is really dealing with file
paths.

Bug: dawn:1342
Change-Id: Ifdbfafc0fd763ac7ec2b8f692efc35772c4793da
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87704
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-04-29 11:18:43 +00:00
Ben Clayton 27f480b7e6 tools: Add src/cts/expectations Update()
Implement the logic to update an existing expectations file with new
results.

Bug: dawn:1342
Change-Id: Idcbad57946712539cc5f0d238f89a2adf42f5aa0
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87702
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-04-29 11:06:34 +00:00
Ben Clayton 2c1154c36f tools: Add src/cts/results.List.MinimalVariantTags
MinimalVariantTags accepts a list of tag-sets (e.g GPU tags, OS tags, etc),
and returns an optimized list of variants, folding together variants that
have identical result query-to-status mappings, and removing redundant tags.

Bug: dawn:1342
Change-Id: I759c82e9a0631a9d321d376656e5a2dbbf5f5507
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87643
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-04-29 08:13:19 +00:00
Ben Clayton 68e039c456 tools: Add src/cts/expectations
Implement an expectation parser, data structures and writer.

Bug: dawn:1342
Change-Id: I53587a9b55346ccf1543e15c9cec5ff68c6849ad
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87641
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-04-28 22:32:23 +00:00
Ben Clayton bd18618bf6 tools: Move go.mod & go.sum files to root
Go tooling does not work properly when modifying go files in subdirectories from the root of the IDE.

Consolidate the two go modules in the source tree.
We'll eventually want to move `src/dawn/node/tools` to `tools/src`.

Allows us to type `go test ./...` at the root, and test all go code in
the entire repo.

Bug: dawn:1342
Change-Id: Ie3dd2c89a769cd25c2b68884970ef0f2ba8c7560
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/86532
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
2022-04-28 09:51:03 +00:00
Ben Clayton 1c3f88eb0c tools: Fix LoadCredentials()
If the URL ends in a slash, this would fail to match the gitcookies file

Change-Id: I987ec637b2f6db172d4173938960060321722bfe
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87920
Auto-Submit: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Antonio Maiorano <amaiorano@google.com>
2022-04-26 14:53:42 +00:00
Ben Clayton 368b3eaae4 tools/src/cmd/perfmon: Fixes / improvements
• Fix up various references of 'tint' to 'dawn' post-migration.
• Round the CPU speed to 100MHz. This appears to fluctuate on reboots
  for some machines.
• Fix git.Log() - the Count was trimming from the wrong end, leading
  to perfmon spamming the github repo with repeat results.
• Instead of using google-benchmark's `--benchmark_repetitions` flag
  for repeating benchmarks, calculate averages by re-running the
  benchmark executable. Use '--benchmark_enable_random_interleaving'
  to randomize the order in which benchmarks are run, which greatly
  helps reduce noise in the averaged results.
• If the host machine supports CPU temperature sensors, wait for
  thermals to stabilise before running the benchmarks. Further helps
  reduce result noise.
• Breakout of the historic benchmarking loop every 15 minutes to check
  for new Gerrit changes to benchmark.
• When idle, attempt to re-benchmark historic results that are 'spiky'
  to reduce noise in the graphs.
• Specify more CMake flags to avoid building non-benchmark executables.
• Update the default base change for historic results. Attempting to
  use the old hash, which was prior to the tint -> dawn  merge makes,
  the git log go back to T0 on the dawn branch. We don't want to
  benchmark nearly 3k changes.

Change-Id: I8e59c7838720eb8bd11f217e9bd3104ba1eb51c3
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87642
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-04-25 21:06:21 +00:00
Ben Clayton 2363ad16ea tools/src/cts/result: Add more helpers
Add result.List.StatusTree() for building a query.Tree[Status].
Add helpers for serializing results.
Add helpers for merging and de-duplicating results.

Change the interface of result.List.ReplaceDuplicates() so that the
merging function takes a status set instead of a list of results.

Bug: dawn:1342
Change-Id: I77580ec5fd4c8f12109fb6e9e83afea8b740260c
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87240
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-04-20 19:10:42 +00:00
Ben Clayton 1a10b73552 tools/src/cts/query: Add Tree
A tree of query to data. Has utilities for reducing the tree based on a
custom merger function.

Bug: dawn:1342
Change-Id: If1c0503be05ee04bcf55dd5bdc9aa3caf6fb56ee
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/87222
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-04-20 19:10:42 +00:00