Commit Graph

188 Commits

Author SHA1 Message Date
Corentin Wallez 3b7e08857c tint test-runner: exclude .fxc/dxc.hlsl from glob
Bug: tint:1620
Change-Id: I5d5182025c140f71b69d3a9e540ae9ef8aec04e3
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97003
Commit-Queue: Ben Clayton <bclayton@google.com>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Auto-Submit: Corentin Wallez <cwallez@chromium.org>
2022-07-25 08:47:38 +00:00
Ben Clayton 98f18d2789 tools/cts: Generate test list as part of roll
Also parallelize some of the more lengthy file generation steps

Bug: dawn:1479
Change-Id: I7674fca4958e4d9948e287008916c4b0d33e1ca1
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/97022
Reviewed-by: Austin Eng <enga@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-07-22 19:52:36 +00:00
Ben Clayton 7d34de88f1 tint/test-runner: Split expectations for FXC and DXC
Change tint's `--fxc` flag to take the path of the FXC compiler DLL.
Have tint attempt to validate with both FXC and DXC if `--validate` is
passed.

Fix the 'dirsWithNoPassExpectations' logic which looks like it got
broken with the tint -> dawn merge. It also incorrectly applied
filepath.FromSlash() on windows.

Change-Id: I0f46aa5c21bc48a2abc48402c41f846aff4a8633
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96800
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-07-22 17:43:27 +00:00
Zhaoming Jiang 1e98a9ba4a tint: Normalizing type matcher's name in intrinsics.def
This patch normalize the abstract interger and abstract float type name
to `ia` and `fa` and the type matcher name to `fia_fiu32_f16` format in
intrinsics.def, and also rename `aiu32Dispatch` and `afi32f16Dispatch`
in resolver/const_eval.cc to `Dispatch_ia_iu32` and
`Dispatch_fia_fi32_f16`.

Bug: tint:1581
Change-Id: Ia48e150d6028e4a7b999f72e57508df369b3e2d7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96721
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Antonio Maiorano <amaiorano@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-07-21 19:55:35 +00:00
Ben Clayton 056f97a9e5 tint: Add a benchmark for atan2 const eval
And increase the test runner timeout from 30s to 2min.
FXC really doesn't like this shader, however I expect this to be
made much faster once constant evaluation for atan2 is implemented.

Change-Id: Id8a8ba97b5a99a2f94633a0732300a35ba6dc1c3
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96401
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-07-19 14:50:33 +00:00
Ben Clayton 6a80ce6c66 tint/resolver: Use the intrinsic table for type const dispatch
Use the @const annotations to more efficently dispatch to the right
method of the ConstEval class.

Reduces a whole lot of dynamic casting logic.

Change-Id: I6791aac51b935b46d63af29abd1e577b3306a0ff
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/95950
Commit-Queue: Antonio Maiorano <amaiorano@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-07-15 22:27:50 +00:00
Ben Clayton 8f488524b8 perfmon: Use backticks for analysis table
Using whitespace for code blocks no longer works with gerrit

Change-Id: I614aff78297ab96a3663888d59c10d8ad295c076
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/96280
Commit-Queue: Antonio Maiorano <amaiorano@google.com>
Auto-Submit: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-07-15 16:23:19 +00:00
Ben Clayton 2777cbc441 tools/cts: Fix go unit tests
Got broken with https://dawn-review.googlesource.com/c/dawn/+/95841.

Change-Id: I75e0996dfb453ffa66dd580252a7f6fbcf8e46a7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/95845
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-07-08 14:23:15 +00:00
Ben Clayton be88e62c0f tools/cts: Treat 'Skip' results as 'Pass'
Change-Id: I2548c2449e535cff1f6907a9f2e892037038aa02
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/95841
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-07-08 12:33:46 +00:00
Ben Clayton 808f616762 gn: Add empty ubsan ignore list
Required to build Dawn with GN flag `is_ubsan = true`

Change-Id: I80b49b81377f0a9c8bd2c1310e12786427947c38
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/95760
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
2022-07-07 16:34:01 +00:00
Austin Eng 65d35f8e7d Report javascript duration in CTS tooling
Always report the Javascript duration from the page instead of
the total duration from ResultDB. Reporting of the total duration
could be added back in the future, but usages of the tooling are
currently interested in the Javascript duration.

Bug: chromium:1336333
Change-Id: I74e91e7a9c29f3a4c45ed67898cc6f2de62e19a4
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/94762
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
2022-06-28 00:15:05 +00:00
dan sinclair 0a4e2a1373 Remove NOLINT lines from c++17 headers
The depot_tools cpplint.py has been updated to accept the c++17 headers
so we no-longer need to mark with NOLINT.

Bug: dawn:1379
Change-Id: I938a4cac5ca881cd1b556f4b8d58741b6a1e6af1
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/94240
Auto-Submit: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
2022-06-18 14:34:00 +00:00
Austin Eng faf98b1bbc Fix roll_test.go
Change-Id: Ife55a04d5dc2fb4f2f1775dc0d1f0a1169cecc70
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/93306
Reviewed-by: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
2022-06-10 18:33:23 +00:00
Austin Eng 737ff5b482 times.go: Add test query filter
Usage: `./tools/run cts time -query QUERY ...`

Change-Id: Ifac16ee2405d1b5d03d246bf1556245e2dafe8c1
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/93304
Commit-Queue: Austin Eng <enga@chromium.org>
Reviewed-by: Ben Clayton <bclayton@google.com>
2022-06-10 17:24:23 +00:00
Austin Eng 9d14b164bf Update CTS roller to generate reftest/idl test sources
These sources need to be generated in Dawn's source tree
so that Chromium's //third_party/webgpu-cts DEP can be
removed.

Bug: chromium:1333969
Change-Id: I03c8cba691bcbfac00839f0cdd40fab6198ec83f
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/91060
Commit-Queue: Austin Eng <enga@chromium.org>
Reviewed-by: Ben Clayton <bclayton@google.com>
2022-06-10 16:25:43 +00:00
Austin Eng 6cc677761d Report test-based aggregation of CTS times
Adds an "-aggregate" flags to ./tools/run/cts time to report results
aggregated by test

Change-Id: Id8d15188846b3bff79ac4c65a09e5d6740acde73
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/93300
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Austin Eng <enga@chromium.org>
2022-06-10 16:13:13 +00:00
Ben Clayton 37b1f0fb5a tools/cts: Rekick cancelled or infra-failed builds
For commands like `cts update`, `cts export`:
If a build completed with `StatusInfraFailure` or `StatusCanceled` and we're trying to obtain the results, re-kick the build.

Bug: dawn:1342
Change-Id: I84565b06eb70e92f3b75a0ab2befd7ed83315140
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92940
Auto-Submit: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
2022-06-07 16:51:54 +00:00
dan sinclair d574be5750 Convert @stage to short form
This CL converts remaining @stage instances in the dawn tree to use
the equivalent shorter variant.

Bug: tint:1503
Change-Id: I74594cd68544fbd692f77d4646991d9c27e218f8
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92484
Reviewed-by: Ryan Harrison <rharrison@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Dan Sinclair <dsinclair@chromium.org>
2022-06-07 15:57:35 +00:00
Ben Clayton c0af5c5c9c tint: Add builtin tests for arguments passed by var
Change-Id: I81b69d23e40675d7f525e6369afec9aa0659d043
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92321
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-06-02 14:36:10 +00:00
Ben Clayton 451eee0fed tint: Stub intrinsic @const evaluation
Add support for @const to builtins in intrinsics.def.
Propagate this flag through to the intrinsic table.
Handle builtins that are @const annotated in the resolver.

Currently no intrinsics are decorated with @const, so there's nothing to
test (yet).

Bug: tint:1504
Change-Id: I172483688617782bd7c58b70e3f38d0222a5d1af
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92323
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-06-01 23:57:20 +00:00
Ben Clayton e3e91c0d75 tools: intrinsic-gen - [[decoration]] -> @attribute
More closely resembles WGSL.

Change-Id: Ia1b93bb443b4e3bf9329f8026d1924f12648cb7b
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92245
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-06-01 20:44:50 +00:00
Ben Clayton 8cbfad7e5d cts: Remove use of intel-0x5912 and intel-0x3e9b tags
The CTS runner can randomly pick between these two GPUs, leading to unstable expectations.

Just use 'intel' for now. We might get a more targetted tag in the future, but this will do for today.

Bug: dawn:1387
Bug: dawn:1444
Change-Id: I2c77ab0060976c40e34b1597b5594894867b7edd
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/92242
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-06-01 12:15:49 +00:00
Ben Clayton 53edbc7635 tools: Add colored icons to the CTS roll msgs
Should help quickly identify errors vs warnings vs notes

Bug: dawn:1342
Change-Id: I8063be0d7cff06428be998a6fa0ad76048ccbc29
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/91960
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-05-30 16:36:18 +00:00
Ben Clayton 2cefd07454 tools: Automatically generate resource_files.txt
When running `tools/run cts roll`

Bug: dawn:1342
Change-Id: I6af9a34fd4236c804ece32e0777f58ec98504f7b
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/91849
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: Austin Eng <enga@chromium.org>
Commit-Queue: Austin Eng <enga@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-05-27 18:58:30 +00:00
Ben Clayton 3e6bf1ae73 tools/setup-build: Properly emit release GN args
is_debug defaults to true, not false. Be explicit to fix the use of:

  ./tools/setup-build gn release

Change-Id: I225f0009f99901e6afeb0c504c98b2be961b7f76
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/91423
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-05-25 22:47:23 +00:00
Ben Clayton 6ac00ed0c0 tint: IntrinsicTable: Add abstract numeric types
These aren't used by anything (yet). Baby steps.

Bug: tint:1504
Change-Id: Icf0261ec9c6802f004d9f1bc4780a6376ebb8dfb
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90530
Reviewed-by: David Neto <dneto@google.com>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-05-19 18:30:39 +00:00
Ben Clayton 0a63c6d2fb tint: intrinsic-gen: Fix parsing of multiple decorations
Change-Id: I61365dea17ab9174cfcb2d513cf1f7ee28ce0647
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90528
Reviewed-by: David Neto <dneto@google.com>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-05-19 18:05:09 +00:00
Ben Clayton c670018aea tint: intrinsics.def Support [[precedence]] decoration
Add support for a [[precedence(N)]] decoration on intrinsic table type
declarations. This will be used to ensure the type with the lowest
conversion rank is matched when a matcher could match multiple types
for a given abstract numeric argument type.

Bug: tint:1504
Change-Id: I96475b000c0917bbfa4e2873b1731ce048b96a7d
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90664
Reviewed-by: David Neto <dneto@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-18 18:56:58 +00:00
Ben Clayton 4c9ed74b5e tint: IntrinsicTable: Rename open/closed -> template
The concept of 'closing' an open type or number made sense when these
were immutable once the first type/number had been matched during
overload resolution.
In order to support abstract numerics, these template parameters need to
be constrained as the arguments are evaluated, so there's no longer a
binary open / closed state.

Give up on this concept, and rename everything to 'template type' and
'template number'. This is likely easier for people to understand
anyway.

Also fix a small typo in the ICE message printed when there's an
ambiguous overload resolution (should never happen with the current
entries in the table).

Bug: tint:1504
Change-Id: I2bf043c71e5afa757259968eae4af830c50f38e0
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90662
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Reviewed-by: David Neto <dneto@google.com>
2022-05-17 22:42:32 +00:00
Corentin Wallez 8520f3bb93 CTS roller: Give a better error message when TSC is not found.
Bug: None
Change-Id: Ic9355346cf834bd4c2989e473f9e98e0be6f1f22
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90503
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Auto-Submit: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: Ben Clayton <bclayton@google.com>
2022-05-16 17:00:51 +00:00
Ben Clayton 5a69597698 tint: Add type constructors and converters support to intrinsic-gen
These are currently not used, but the first step towards moving type
constructors and converters over to using the intrinisc table.

This will simplify maintenance of type functions, and will greatly
simplify the [AbstractInt -> i32|u32] [AbstractFloat -> f32|f16] logic.

Bug: tint:1504
Change-Id: I15526670a6ff801e66551ab5adc37b1570ac49de
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90242
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-16 09:48:41 +00:00
Ben Clayton 59e23943f3 tint: Minor IntrinsicTable cleanup
Fix the namespace - this should have been tint::resolver.
Use a single u32 bitset for overload flags instead of multiple fields.

Bug: tint:1504
Change-Id: I633b21ce14e20fc9aeeed5221886c1d22e327bdc
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90241
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-05-13 16:14:17 +00:00
Ben Clayton 77473b4699 tint: Split tables for unary and binary operators
Do the partitioning of unary and binary operators in the intrinsic table
generators, instead of searching all operators at runtime.

Will allow code to be simplified.

Bug: tint:1504
Change-Id: I67246b954e530e0542b1b67c99fb34a756cf532a
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/90240
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-05-13 14:35:37 +00:00
Ben Clayton 08f94e9004 tools/CTS: merge [Slow] + [Pass] -> [Slow]
Instead of Failure.

If we have duplicate results (same query, same tags), but with different results, then we have to merge them. Previously we'd merge Pass and Slow as Failure, which was just odd. Fix this so the combined result is Slow.

Bug: dawn:1342
Change-Id: If9b035bb3c1e59b7a93a5c8b194f854405e656d6
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/89960
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Auto-Submit: Ben Clayton <bclayton@google.com>
2022-05-12 10:08:46 +00:00
Ben Clayton b61e0452f8 tint: Add unary-ops to the intrinsic table
• Declare all the unary ops in the intrinsics.def file.
• Reimplement the bulk of Resolver::UnaryOp() with the IntrinsicTable.

This will simplify maintenance of the operators, and will greatly
simplify the [AbstractInt -> i32|u32] [AbstractFloat -> f32|f16] logic.

Bug: tint:1504
Change-Id: Ifc646d086fc93cfbe3f3f861b8c447178664c1f7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/89028
Reviewed-by: James Price <jrprice@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
Commit-Queue: Ben Clayton <bclayton@chromium.org>
2022-05-09 21:22:24 +00:00
Ben Clayton e6e96def66 tint: Add operator support to intrinsic-gen
Adapt the builtin parsing and resolving to also support operators.
Will be used to generate intrinsic table entries for operators.

This will simplify maintenance of the operators, and will greatly
simplify the [AbstractInt -> i32|u32] [AbstractFloat -> f32|f16] logic.

Bug: tint:1504
Change-Id: Id75735ea24e501877418812185796f3fba88a521
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/89026
Commit-Queue: Ben Clayton <bclayton@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
2022-05-09 18:08:23 +00:00
Ben Clayton e132516e6f tint: Rename and move builtin_table and builtin-gen
• Rename 'builtin-gen' back to 'intrinsic-gen', as 'intrinsics' now
  include both builtins and operators.
• Move the intrinsic definitions, and IntrinsicTable to the resolver
  package, where it belongs.

Bug: tint:1504
Change-Id: I5ad5c285c1e360a224ee1235c293ccd55eb2693d
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/89025
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-05-06 15:13:01 +00:00
Ben Clayton 8495affacf tools: More CTS tooling improvements
• Add the included trybots in the CL description. All of these trybots are tested by the roll, but the final CQ-submit wouldn't necessarily test all of the variants before landing. This would mean that the 'cts export' could miss some results, as it takes the last PS with any results.
• Add --force flag to cts roll to force a roll. Useful for testing.
• Emit timing diagnostics for tests labelled 'Slow' instead of unhelpfully stating they pass.
• Enable the --cl and --ps flags for cts export
• Export with the most recent data to the top of the spreadsheet

Bug: dawn:1401
Change-Id: Id926367ab805bfb9f3032fce9cce7f00daf7a5d4
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88661
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-03 16:58:43 +00:00
Ben Clayton 174c508c8d tools/cts: Bunch of fixes for 'cts roll'
• Strip 'release-x64' tags.
  These are always paired with 'release', and all
  other tests just have the 'release' tag. This
  means that results with the 'release' +
  'release-x64' tags are a subset of just the
  'release' tags, leading to many broken
  assumptions in the expectation update.

• Don't consider tests with the error reason
  'asyncio.exceptions.TimeoutError' as slow, but
  as a failure. The CTS expectation updater will
  already mark tests exceeding the config
  threshold as slow. This reduces the amount of
  data requested from resultdb, and prevents
  failed rolls when results are actually
  deadlocking.

• Fix inferring of patchset in GetResults().
  LatestPatchest() was called but was assigned to
  a nested scope.

Bug: dawn:1401
Change-Id: I57b2c9a029b31430d63a056f7b13c4bf1bc5b437
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88450
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
2022-05-02 20:44:21 +00:00
Ben Clayton bf464ddbdf tools: Add the 'cts roll' sub-command
Performs a CTS roll, updating ts_sources.txt and expectations.txt.

Bug: dawn:1342
Change-Id: I18cc644e9eef1db6034fc07fd94f21f070561265
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88444
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-02 20:44:21 +00:00
Ben Clayton d9f2a61c6f tools: Add the 'cts export' sub-command
Updates a Google spreadsheet with latest CTS statistics.

Bug: dawn:1342
Change-Id: I01a9c99032c755d6a78af674675d8c9cbfc79019
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88443
Reviewed-by: Dan Sinclair <dsinclair@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-05-02 20:44:21 +00:00
Ben Clayton 446166c81b tools/setup-build: Into the 'active' symlink directory
Previously we'd generate the build files into the symlink target directory.
CMake gets grumpy if the cache is invalidated, and you're building in the 'active' directory, which does not match the directory path it remembered.

Also: Enable ccache if found on PATH

Change-Id: I7e0dc93516b2c59d6bf346fc943acea3d0021087
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88311
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
Commit-Queue: Ben Clayton <bclayton@google.com>
2022-04-29 20:18:45 +00:00
Ben Clayton 665c07ac6a tools: Add the 'cts update' sub-command
Updates the expectations file with new results.

Bug: dawn:1342
Change-Id: I05cab48eb1cf0f9ab7e652a1b7c3da361c45b0b1
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88442
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
2022-04-29 19:09:17 +00:00
Ben Clayton 1ed858c20a tools: Add the 'cts merge' sub-command
Merges results files into one.

Bug: dawn:1342
Change-Id: Ia241afc982c88c840a102abba211053611f77f5b
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88441
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-04-29 19:09:17 +00:00
Ben Clayton 09eafe71b5 tools: Add the 'cts results' sub-command
Pulls the CTS results from a patchset

Bug: dawn:1342
Change-Id: I879605b71517004d3a8e81a1b4481bbc0105a22c
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88440
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-04-29 19:09:17 +00:00
Ben Clayton 40bed821f8 tools: Add the 'cts time' sub-command
Add the common utilities for starting and fetching build results.

'cts time' provides views on the longest running tests, along with
a histogram view of all test times.

Bug: dawn:1342
Change-Id: Ia3707f7f062ea26a2406e3163a26e1cc7e30e3b2
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88319
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-04-29 19:09:17 +00:00
Ben Clayton 251e464af1 tools: Add the 'cts format' sub-command
Formats an expectation file

Bug: dawn:1342
Change-Id: Idc7728ad5b955fcbdb4062e511aff6665a711718
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88318
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
2022-04-29 19:09:17 +00:00
Ben Clayton 846e3149b7 tools: Begin adding the CTS command line tool
This CL introduces the Config structure, and config file.
Used by all the sub-commands.

Bug: dawn:1342
Change-Id: Iaad5a02bc2ceba2c63265cf0d1086dc7901e7ed7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88317
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Antonio Maiorano <amaiorano@google.com>
2022-04-29 19:09:17 +00:00
Ben Clayton e822f5e016 tools: Add 'Duration' field to cts.Result.
Will help identify slow tests.

Bug: dawn:1342
Change-Id: I422d345361785addcc2faa6281e0608de02629b7
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88316
Kokoro: Kokoro <noreply+kokoro@google.com>
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
2022-04-29 19:09:17 +00:00
Ben Clayton 15a85dec7e tools: Add 'subcmd' package
Helper for writing command line tools that have sub-commands.
Will be used by the CTS tool (roll, results, export, etc).

Bug: dawn:1342
Change-Id: Id22beecbc02b73af23064adc67c590f825b37789
Reviewed-on: https://dawn-review.googlesource.com/c/dawn/+/88315
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Ben Clayton <bclayton@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>
2022-04-29 16:30:24 +00:00