We don't have CTS results for the associated CI bot.
Change-Id: I3cfc62bd299e2c7bdb30fd0389f20159ab2f154e
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/132482
Kokoro: Austin Eng <enga@chromium.org>
Commit-Queue: Austin Eng <enga@chromium.org>
Reviewed-by: Ryan Harrison <rharrison@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
* Bump timeouts
* Include changes description in results
* Use median times instead of means
* Use CPU time instead of real time
* Bump prority of the benchmark process
Change-Id: I40cb5635016fab4d6a1d1ee2434e94c120f7f6f5
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/121700
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Allow perfmon to benchmark shaders that are defined outside of the repo
Change-Id: Iee53b64cf357facdaa9941602d8d571611680097
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/128040
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Use the new `cmdline.ts --coverage` flag in https://github.com/gpuweb/cts/pull/2206 to write out the per-test coverage to a unique file.
Change-Id: Iea273217eede2fa615b78cd6f69f036976dcfd35
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/117883
Kokoro: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
It appears that FXC and DXC do some validation post dead-code-elimination.
These tests have been updated so that the return value is assigned to a storage buffer, ensuring that all validation is performed.
Many DXC tests are affected by https://github.com/microsoft/DirectXShaderCompiler/issues/5082, which have been SKIP'ed.
Fixed: tint:1859
Change-Id: I0001a9a9821846cd0855c3d8ce2bec79ab8e64ef
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/122662
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: James Price <jrprice@google.com>
Commit-Queue: James Price <jrprice@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
The default behaviour is a 1 minute timeout, which depending on
network load and other factors, may not be enough time to clone the
CTS repo when running the tool.
Increasing the timeout for this step to 10 minutes, since the tool is
expected to take a significant amount of time to run, so a bit of
extra time on setup isn't noticable.
Change-Id: If51ed154a8b2373bb9b246f29467fb17c7d7e345
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/119300
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Auto-Submit: Ryan Harrison <rharrison@chromium.org>
* Use common_compile_options(turbo-cov) to fix RTTI linker errors on macOS + debug builds.
* Delete the .profraw files after they've been consumed.
* Include the test-case parameters in the coverage information
Change-Id: I02c6f408ed6e65c6f7e9eb1b4cd8443e4cfa7939
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/117882
Kokoro: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
We weren't actually enabling WGSL validating in Tint in the E2E test
runner.
Mark a few tests as SKIP for cases that do not actually validate.
Change-Id: I62a55ce701f704d744c32f2cd5cb97683e270e96
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/117960
Commit-Queue: James Price <jrprice@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Kokoro: James Price <jrprice@google.com>
Kokoro: Ben Clayton <bclayton@google.com>
Default to no-validation.
Change-Id: I4b3506e2addf580256175b13efbeb47f8ef96aaf
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/117881
Kokoro: Ben Clayton <bclayton@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Common:
* The turbocov build target (somehow) never got hooked up to the root CMakeLists.txt file. This is now fixed.
macOS:
* Emit coverage for 'AppleClang' compiler.
* Have run-cts find the tucked-away `llvm-profdata` executable.
Windows:
* Various fixes for building with clang
* Fix turbocov stdout CRLF corruption
* Fix bad JSON with backslashes
Change-Id: I481cceafe2e72b544e13168172fc1456e5df2005
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/117880
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Kokoro: Ben Clayton <bclayton@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Can dramatically reduce latency for small test runs.
Requires: https://github.com/gpuweb/cts/pull/2167
Change-Id: Ib5ff5d0fbcaf151ffe67bf30e97c9ccdf34443e0
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/117200
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Ben Clayton <bclayton@google.com>
Enable coverage collection when using the test server, which is substantially faster than running in separate, isolated processes.
Use clang's `__llvm_profile_*` APIs to reset the counters between each test case run.
Change-Id: I01f8d0c1b3f215f66cfa50ef0fd51f2522c2ea57
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/113880
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Add the "--adapter" flag to run-cts as a helper for setting this.
Make "--verbose" print the picked adapter.
Rename "dawn-backend" flag to just "backend" - this is already specific to the "gpu provider" (dawn).
Change-Id: Idc8d0eb3ccf5fa23325c06f0f9520aa9b528d9dd
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/116295
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
Kokoro: Ben Clayton <bclayton@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
With the CTS global timeout extended, and tweaks to how const-eval loops are emitted, we now have a whole lot of passing tests.
Adjust the 'Slow' threshold to account for the new global timeout.
Cq-Include-Trybots: luci.chromium.try:dawn-try-win10-x86-rel,linux-dawn-rel,mac-dawn-rel,win-dawn-rel
Change-Id: I9a4d3278048e72e6560552164522395423dedf99
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/116296
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
These configurations are no longer used and replaced with newer
machines.
Bug: dawn:1387, dawn:1444
Change-Id: I13fcaf8fc5a42329cd74e9fd9a8ec9d10de63f68
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/116285
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Auto-Submit: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Displays a per-test coverage viewer in your browser
Change-Id: I0b808bfadf01dab0540143760580cd7ca680e93b
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/113644
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
This is the standard place for tooling.
Update src/dawn/node/README.md with the new paths, and drop inferred
arguments from the examples.
Change-Id: Ib944ca045366b81b8897d9548112a8889e097769
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/113643
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
A tiny utility that emits just the per segment coverage in a binary stream. This avoids the overhead of encoding to JSON, which provides substantial performance improvements.
Change-Id: I36a588069d69c5c800d31bca8dd5c542bcdbe313
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/113641
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
10x speed improvement on my 3990x machine
Change-Id: Ie4b2377719e9a25860ec05bb7649aa652d26ccf0
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/113022
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Ben Clayton <bclayton@google.com>
Can be run with:
./tools/run git-stats
Change-Id: Ie46be01ff3318ff50acdb13eb809d5304f867f04
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/111180
Auto-Submit: Ben Clayton <bclayton@google.com>
Commit-Queue: David Neto <dneto@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: David Neto <dneto@google.com>
A simple tool to benchmark tint based on a template file.
Bug: tint:1122
Change-Id: I34ee0fb98e6d2187c145cc38ec3cb48606242cb4
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/110820
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
And add tests.
This is useful for other tooling.
Change-Id: Ia399071baf6d4bb617f3c73e4ccd4ed72d522c2e
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/111020
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Will be used by GN to know the outputs of the generate cache step.
Change-Id: Iae09477fb355eac41ff9b2204605f46ed78e08ec
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/110620
Reviewed-by: Austin Eng <enga@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Fails presubmit if you need to run `./tools/run gen`
Change-Id: I05311cd668c5a1f4f484b25cc1367f680a9d24eb
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/110140
Commit-Queue: Ben Clayton <bclayton@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Ben Clayton <bclayton@google.com>
Cache the hash of the test output, and only re-validate if the output changes.
This speeds up incremental end-to-end testing.
Change-Id: I44bc1eff62320df1e80524e8481d1c012fb67f42
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/109761
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: James Price <jrprice@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
`-o` will emit the files to the given root output directory
`--verbose` will print what's going on, to help with debugging
Omitting these flags will behave as before.
Also consolidate the utils package into fileutils. These were two packages with near identical functionality.
Change-Id: I855dd4b57807fb9239a52e7f357842d4ba2517ee
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/107687
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
This CL updates the intrinsics lexer to allow negative values for int
and float numerics. This allows doing `@test_value(-2)` in the def file.
Change-Id: I2cad9b25a2932057ce9bc51dec6c32231e06f0a0
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/107440
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
Reviewed-by: Ben Clayton <bclayton@google.com>
Auto-Submit: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Specifies the value to use for argument values when generating end-to-end tests.
Use this to provide a legal value for atanh().
Change-Id: I008050c856f9d687ab918c68e90678c4e74f3a1d
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/106887
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
If the socket disconnected mid-communication, the server could spin, waiting for new data.
Actually handle recv() errors, preventing the server spinning itself to death.
Also fix code style to be more tint-like (snake_case variables, PascalCase functions)
Change-Id: I9fcbfde303a8624e7e1ff87abd33581589f4da42
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/105142
Auto-Submit: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
This was using the old tint code style
Change-Id: I1aff541eb4cc0d7ec0e045b555710aa605c4da28
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/105141
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
• Add `cts validate` command used to check for expectation collisions.
Can be used as a presubmit check.
This is more tightly checked than the previous logic, as this works on just
the expectations, instead of results.
• Fix an issue where the test result reduction could introduce collisions with
'Skip' expectations.
To fix this, the update process first adds 'consumed' results for the skipped
tests, preventing test tree reduction for that part of the tree.
• Fix a bug in the generation of 'New failures' and 'New flakes' which produced
more expectations than was necessary.
The issue here was that the tree roots could contain overlaps, and roots could
be processed before sub-trees, resulting in inefficient expectations.
• Fix collisions in the expectations file, and update with results from
the most recent roll.
Change-Id: I7b64553408998fb4416458ce564fc49c8f6d4d07
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/101860
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
may_exonerate indicates that a test failed for a known issue that
we could exonerate. Merging of test results now removes results
with may_exonerate unless all of them were tagged as such. So, if
for example, a test fails for a known timeout issue, but has a
subsequent pass, the timeout will be ignored.
This serves to reduce the impact of known, hard-to-fix issues and
allow the CTS roller to make progress with less noise.
Change-Id: I5103a666496398a17b3aa6ccf3f267421e40ba97
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/101804
Reviewed-by: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
This CL updates all of the integration tests to run the substitute_override
transform where needed. The test runner is updated to match comments in
the spvasm files as well as the wgsl files.
Bug: tint:1155
Change-Id: I8187d426970d056d744e530ed6447e4ec69db5f4
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/101661
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Fix `IsAbstract()` so that it doesn't consider most-nested element types.
Add `ElementType()` and `DeepestElementType()` helpers.
Add `OverloadUsesF16` as a helper for https://dawn-review.googlesource.com/c/dawn/+/96722.
Simplifies template code.
Change-Id: Iff5a9a7258caea06e00ee37c29e5298d9c35b799
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97361
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
* Ensure that copyright years are preserved when regenerating.
* Add 'Split' function which maps to strings.Split.
* Add 'Scramble' function with messes with a string.
* Add 'Import' function which allows templates to be imported, allowing
for reusable macros.
Change-Id: Ib77f59a989cf55addcced3e337c9031062a83470
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97149
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Rename to 'gen', so that more templating can be added without having a confusing name.
Can now be run with './tools/run gen'
Move the bulk of the intrinsic-gen logic to `tools/src/tint/intrinsic`
Change-Id: I750989a5aa86272c10c2ad37adffe7def11c61f2
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97141
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Also parallelize some of the more lengthy file generation steps
Bug: dawn:1479
Change-Id: I7674fca4958e4d9948e287008916c4b0d33e1ca1
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97022
Reviewed-by: Austin Eng <enga@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Change tint's `--fxc` flag to take the path of the FXC compiler DLL.
Have tint attempt to validate with both FXC and DXC if `--validate` is
passed.
Fix the 'dirsWithNoPassExpectations' logic which looks like it got
broken with the tint -> dawn merge. It also incorrectly applied
filepath.FromSlash() on windows.
Change-Id: I0f46aa5c21bc48a2abc48402c41f846aff4a8633
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96800
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Austin Eng
Loko Kung
Ben Clayton
Yuly Novikov
Ben Clayton
Antonio Maiorano
Ryan Harrison
James Price
Corentin Wallez
dan sinclair