This patch implements the creation of 2D array textures and copying
between a buffer and a layer of a 2D array texture on D3D12, Metal
and OpenGL back-ends.
TEST=dawn_end2end_tests
Linking against their .lib makes loading Dawn fail on systems that don't
have the DLLs. This happens for example on Windows7 that doesn't have
d3d12.dll. Instead we dynamically load functions pointers from these
DLLs at d3d12::Device startup.
Change-Id: I4d01a12d0f91bec45bf125450d2c08aaa9ff9fac
This patch implements a micro DAWN_SKIP_TEST_IF in DawnTest.h which
can be used to skip running a test in dawn_end2end_tests when the
given condition is satisfied.
Implement copying between a buffer and a texture 2D array on Vulkan
This patch implements the creation of a 2D array texture and data
copying between a buffer and a layer of a 2D array texture on
Vulkan back-ends.
TEST=dawn_end2end_tests
This patch introduces texture descriptor for texture creation instead
of texture builders.
This patch also adds "arrayLayer" to texture descriptor and removes
mDevice in TextureD3D12.
WIN32 is defined by the SDK, but _WIN32 is defined by the compiler.
Since Dawn doesn't necessarily include <windows.h> everywhere dawn.h is
included, we should use _WIN32 (which will always be defined).
In the DEPS file we are going to use Chromium's mirror of
github.com/KhronosGroup/SPIRV-Cross so we need to adjust .gitmodules to
point to it instead of Kangz/SPIRV-Cross. Also take that opportunity to
roll SPIRV-Cross.
Change-Id: I4f53a4fc54f5b1b9a754ac55e976c81e5eeabeb2
On 32bit platforms, even if it contains a uint64_t the handle wrapper
will align to 4 bytes because the assembly load operations can't take
advantage of 64bit aligned loads.
Fix this by forcing the alignment to the alignment of what would be the
native Vulkan handle type.
Also use macros with defined()
Change-Id: I0de9b4a77e648c416b04311b854c956762248868
We still keep a dummy BufferBuilder object around so that it can be used
for the builder error callback tests of the wire.
Change-Id: If0c502bb8b62ee3ed61815e34e9b6ee6c03a65ef
Resources used to have both a current and an allowed usage but the
concept of current usage has been removed so we can rename "allowed
usage" to "usage" to make the name match WebGPU's
WebGPUBufferDescriptor::usage and WebGPUTextureDescriptor::usage
Change-Id: I5190950bf7f7f5b86c92247ef0240fead9886268
BindGroup usage isn't something that's part of WebGPU's sketch.idl and
it might never exist. Remove it to simplify the migration of bindgroup
to descriptor.
Change-Id: I21e0a98eb60434d4009e748cd9afcbf89edd7e6a
This includes a bunch of fixes for clang warnings in Windows specific
code that was only compiled by MSVC previously. This also tidies up some
BUILD.gn issues on Windows.
This required adding some missing dependencies, splitting public headers
of libdawn_[native|wire] so they aren't hidden in the
libdawn_[native|wire]_sources targets, and making unittests depend on
sources directly instead of static libraries (which is almost equivalent).
As a byproduct, Empty.cpp is no longer needed and is removed.
This required putting Queue::Submit on QueueBase which is something we
would want to do anyway, and removes the need for Queue::ValidateSubmit
being called in the ProcTable.
This removes the need for all the "GeneratedCodeIncludes" files and
leads to a bunch of simplifications in BindGroup as well as the
dawn_native CMakeLists.txt.
Finally this was done in order to simplify the writing of BUILD.gn
files.
Also moves the TerribleCommandBuffer to utils:: because it isn't part of
the implementation of the wire, renames dawn::wire to dawn_wire, moves
src/wire to src/dawn_wire and puts the interface of dawn_wire in
src/include/dawn_wire.
The interface exposed by libdawn_native is declared in the new headers
living in src/include/dawn_native so that they both the users and the
libraries use the DAWN_NATIVE_EXPORT macros.
The dawn.h and dawncpp.h structure definitions references dawnFoo or
dawn::Foo respectively when it should reference dawn_native::FooBase* in
dawn_native. Autogenerate files to declare the dawn_native version of
the structs and change the ProcTable generation to use it instead.
This is important to make libdawn_native a shared library because
currently it was depending on dawncpp's definition of .Get().
This extends our Vulkan handle wrapper to have conversions from uint64_t
as well as the native Vulkan types:
- The dawn_wsi interface uses the uint64_t version.
- The backend interface uses the native Vulkan version
libdawn will be one of the libraries produced but other libraries like
libdawn_native don't need to link against it. However they do need the
Dawn headers so we generate them separately.
This also makes all internal targets depend on the header generation and
have the include directories necessary for those headers.
Also has a small fix for setting compile flags only for C++ files.
This adds DeviceBase::ConsumedError which is a helper method intended to
be the main place where Error.h errors are consumed.
To test this mechanism, object creation using descriptors is changed to
use ConsumedError, which finally gets their validation errors go
somewhere. This mechanism isn't final though because we have yet to
implement WebGPU error handling.
windows.h adds macros such as `#define GetMessage GetMessageA` which
cause conflicts in naming when for example a class definition is before
windows.h and a use is after windows.h
Add a header that can be used both to include windows.h and remove
defines, and to only remove defines (when windows.h is included by)
external dependencies.
This removes the following for both Buffer and Texture:
- The builder's SetInitialUsage
- The object's FreezeUsage and TransitionUsage methods
- The CommandBuffer Transition<Object>Usage methods
All samples and tests are simplified as a result. This also obsoletes
the UsageValidationTest which is removed.
Some validation was dependent on "current usage" and hasn't been
reintroduced for implicit transitions yet:
- Buffers can be used while mapped
- Swapchain textures can be used after they have been presented.
Validation for these will involve collecting all the resources used by a
command buffer and will be done in a follow-up patch.
With this the backend ignores explicit usage transition hints from the
frontent and generates all the barriers automatically based on resource
usage.
The current implementation is very naive and encodes a barrier
immediately just before a resource is used in a new state.
With this commit the Vulkan backend completely ignores the explicit
barrier commands passed from the frontend, and generates its own
pipeline barriers.
Right now it encodes each barrier just before the resources are used,
which is quite bad but will be optimized later.
This commit also makes the frontend command buffer validation perform
the checks necessary for implicit barriers (although they are redundant
with checks for explicit barriers) because the tracking can pre-compute
pass usage information that's useful for the Vulkan backend.
Tests for usage validation inside passes will be added once the concept
of transition is removed from the API.
These should hopefully subsume the ComputeBoids demo for testing purposes. Not all of these tests pass currently:
* ComputeCopyStorageBufferTests.StructTest/D3D12 fails due to "Reading structs from ByteAddressBuffer not yet supported."
* The disabled tests fail for various reasons on various backends.
Previously, the renumbering loop would sometimes iterate in the wrong order. To fix this, instead use the binding info already correctly extracted by `ExtractSpirvInfo`.
Fixes ComputeCopyStorageBufferTests.BasicTest/D3D12.
* Use a descriptor for BindGroupLayout
* Fix MatchesLambda
* Add WireTests.StructureOfStructureArrayArgument
* Add BindGroupValidationTests.BindGroupLayoutCache
This introduces a small amount of code duplication in the code that
handles push constants. On the plus side it removes the need for all the
asserts around which Metal encoder is active.
This changes the validation to have one iteration loop for the main
buffer that calls to pass-specific iteration loops for compute passes
and render passes. This is done to simplify code a bit and will help
implement implicit transitions that are "precomputed" per pass in the
command buffer validation and re-used in the backends.
A number of code simplifications were made to CommandBufferStateTracker
since it doesn't need to know if we are in a pass anymore. Further
simplifications will happen when implicit barriers are implemented.
Instead of initializing all of the bind group layouts in the pipeline
layout to default values, let them be nullptr and allow this elsewhere.
Follow-up on some changes in #206
We are changing all object creation to use descriptors but there is no
creation argument to pass for queue, so instead Device::CreateQueue
takes no argument.
The flow of commands is a bit more involved than for MapReadAsync and
goes like this:
- C->S MapAsync isWrite = true
- S: Call MapWriteAsync
- S: MapWriteAsync callback fired
- S->C: MapWriteAsyncCallback (no data compared to the read case)
- C: Call the MapWriteAsync callback with a zeroed out buffer
- C: Application calls unmap.
- C->S: UpdateMappedData with the content of the mapped pointer
- S: Copy the data in the mapped pointer
- C->S: Regular unmap command
- S: Call unmap
Makes nxt_end2end_tests -w pass all tests.
Also duplicates the MapRead wire tests for the write cases
This will help with follow-up changes that add support for a more
complete grammer of types, including structures containing pointers
to objects or other structures.
Instead of having the wire::Client and wire::Server directly act on
buffer memory, a couple interfaces are introduced so that WireCmd can do
things like get the object<->id mapping and temporary allocations.
While the serialization and deserialization of most commands was moved
into WireCmd, the commands that don't directly correspond to NXT methods
have their logic moved inside Client and Server and will be made to
expose the new interface in a follow-up commit.
This will be useful to provide additional coverage of the wire in
addition to the very focused WireTests unittests. All tests pass except
the MapWriteAsync ones that aren't implemented in the wire yet. They can
be skipped with --gtest_filter=-*MapWrite*
These types are meant to be used for computations that might but are not
expected to fail in backend/, such that the error case can be much
slower than the success case.
The NXT_TRY and NXT_TRY_RESULT macros are added to help write more
concise code that uses Error and ResultOrError.
The sampler builder will be removed as the first builder being converted
to a descriptor. Use CommandBufferBuilder instead as it is not expected
to change soon.
This as an API change to get closer to the direction in which WebGPU is
headed. The API change in next.json caused a ton of files to be changed
in the same commit to keep things compiling.
API: the Framebuffer and RenderPass objects are now merged in a single
RenderPassInfo that contains the attachments, loadOps and clear values
for a BeginRenderPass command. The concept of subpass is removed.
The RenderPass creation argument to RenderPipelines is replaced by
explicitly setting the format of attachments for RenderPipeline.
Validation: SetPipeline checks are changed to check that the attachments
info set on a RenderPipeline matches the attachments of the render pass.
Backends: Most changes are simplifications of the backends that no
longer require and indirection to query the current subpass out of the
render pass in BeginSubpass, and don't need to get the attachment info
from a RenderPass when creating RenderPipelines. In the Vulkan backend,
a VkRenderPass cache is added to reuse VkRenderPasses between
RenderPassInfos and RenderPipelines.
Tests and examples: they are updated with the simplified API. Tests
specific to the Framebuffer and RenderPass objects were removed and
validation tests for RenderPassInfo were added.
Tested by running CppHelloTriangle on all backends, end2end tests on all
platforms and all examples on the GL backend.
HashCombine will be used in more than just BindGroupLayout caching so we
extract it to a separate header. Add a better mixing constant for 64bit
systems.
Delete the TextureBuilder created in SwapChainBase::GetNextTexture().
Delete the InputStateBuilder and DepthStencilStateBuilder created in
RenderPipelineBuilder::GetResultImpl().
PipelineLayout was re-entering in NXT to create the default empty BGLs,
but forgot to remove the initial external refcount for them.
Fixing this showed an issue where it was impossible to externally
reference a RefCounted that had only internal references, fix this as
well and add a test.
When a BGL was first created in the cache, it started with an external
refcount of 1 and we then referenced again in
BindGroupLayoutBuilder::GetResultImpl, causing a leak.
Fix this by referencing in the cache lookup only if the BGL already
exists.
Don't assert on color formats which can be cleared with a
floating-point clear value.
So far, these are { unorm } x { r, rg, rgba, bgra }.
Fix linter error when no files to check.
The indexOffset of the draw was not being used. It must be
included in the indexBufferOffset.
Renamed indexBufferOffset -> indexBufferBaseOffset.
Add a DrawElements test which exercises zero and non-zero index offsets.
Backend support implemented in GL, Metal and D3D12.
Support for unorm values in the GL backend requires a
utility function to indicate if the format's components
are normalized.
Note that unorm_r8 is only supported on more recent MacOS SDKs (10.13),
so it's omitted for now.
When calling unmap on a mapped buffer for which the callback hasn't
fired yet, the callback should be called with UNKNOWN. The code marked
the callback as called only after calling it, causing problems with
re-entrancy where the callback would be called twice.
This could also get triggered by destroying the buffer inside the
callback.
Fix this in backend::Buffer and the WireClient and add test for both.
Previously we would remove the reference to pipelines in the destructor
of the d3d12::*Pipeline objects which could cause the D3D12 pipeline
state to be destroyed while still used by in-flight commands. Add a
global queue of ComPtrs to keep alive in the d3d12::Device to fix this.
We want all of NXT to have the origin in the top left. OpenGL's origin
is the bottom left instead. We are doing a blit already to decouple
the NXT render targets from the OpenGL default framebuffer. Take
advantage of this and invert the blit to make the OpenGL's backend
origin be top left.
PRESENT is an exclusive flag in NXT and can't be combined with other flags. The existing implementation treats the D3D12_RESOURCE_STATE_PRESENT as flag which is confusing due to being 0.
The Swapchain::Configure was changed to not require an initial usage
anymore. Previously the code was doing a transition to this usage
causing the code to now transition to <uninitialized data> usage.
Fix this by deleting code. Also make TextureD3D12 responsible for the
transition to PRESENT.
Making all textures represented by pointers is a problem for Vulkan
where VkImage is a 64bit type and wouldn't fit in a pointer on 32bit
builds. Make texture contain on of each useful type so that each backend
can choose which one it wants to receive.
This file changes the non-dispatchable Vulkan handle types. We want to
use some of these handles in utils/VulkanSwapChain.cpp so it needs to
have access to it. The file could include
backend/vulkan/vulkan_platform.h but it seems a bit cleaner to move the
header in common/ instead with a warning if the Vulkan backend isn't
enabled.
For the Vulkan backend it will make sense to have the SwapChain be
implemented inside the backend as it will need to interact with a lot of
things there. It will need SwapChainImpl and cannot see utils/ so
SwapChainImpl has to be moved in common/
Also makes SwapChainImpl a function called CreateSwapChainImplementation
as the inheritance was only used to share a static method.
In Vulkan images are created in no particular layout and must be
transitioned before they can be used. This didn't appear before because
the test were creating the textures with not initial usage and then
transitioned them. This isn't the case with InputStateTest, which is
what uncovered this issue.
There was a lot of missing around with viewports and flip the Y
coordinate in vertex shaders before. Turns out things are simpler than
we thought: *all* APIs have gl_Position(-1, -1) map to texel (0, 0). It
is just the present coordinate system that changes.
Remove some of the hacks we had to work around non-existent viewport
issues and fix tests.
Sometimes NXT provides default objects for parts of the pipelines, for
example a default pipeline layout. This objects were create with code
like: device->CreateFooBuilder()->GetResult(); and stored in a Ref<>.
This caused the object to have on external reference and two internal
references and not get destroyed when the Ref<> goes out. Call Release
on these objects to remove the external reference and fix the leak.
Was found via the Vulkan validation layers that were complaining that a
VkPipelineLayout was leaked.
This as this is the first command handled by the Vulkan backend, this
commit also introduces the b:✌️:CommandBUffer class and implements
b:✌️:Queue::Submit.
Also enables the BufferSetSubData tests that are now passing on Vulkan
even though the buffer transitions are unimplemented.
This introduce a new FencedDeleter service as part of the Device
objects that tracks when resources are no longer used and deletes them.
BufferVk takes advantage of this to defer the deletion of its handle
that was previously incorrectly delete directly in ~BufferVk.
On 32 bit Vulkan typedefs these handles to uint64_t which breaks
function overload resolution. Replace the
VK_DEFINE_NON_DISPATCHABLE_HANDLE and VK_NULL_HANDLE defines to be
opaque C++ types with the same capabilities.
Previously we would modify the GL state as soon as we saw
SetIndex/VertexBuffers. This GL state is owned by the VAOs in the
InputState and was disappearing on a Pipeline change. Fix this by
applying the index / vertex buffers lazily.
This was because the clip space of OpenGL has an inverted Y compared to
other APIs. This commit:
- Updates SPIRV-Cross to the latest version
- Uses the new vertex.flip_vert_y option in ShaderModuleGL
- Enables tests that are now passing in GL.
Explicitly configure swap chain usage in bindings and examples
Fix missing case in switch
Make swap chain Present usage implicit
Author: Austin Eng <enga@google.com>
This is to avoid a potential future problem if SetPipelineCommon were to
ever return false (which is currently not possible), some state might
have been modified incorrectly.
Previously didn't tell SPIRV-Cross at which MSL resource indices the
different SPIRV binding should be, and were lucky that it's giving
indices in increasing order matched the PipelineLayout in all our
samples.
Fix this by making SPIRV->MSL compilation depend on the PipelineLayout
so we can tell SPIRV-Cross which binding goes where.
We should do the same for vertex attributes eventually as they are
hardcoded to start at kMaxBindingsPerGroup currently.
Also a couple unrelated cleanups (unused function, usage of
IterateBitSet).
* clang/gcc: enable -pedantic warnings
* suppress a GCC-specific warning in stb_image
* And some clang-specific warnings
* -Wconversion (clang) -Wold-style-cast (clang+gcc)
and fix a few warnings that show up with these (and a few more with
-Wconversion on gcc, even though that's not enabled by default)
* bunch more warnings
* fixes
* remove merge error
Adds the swap chain interfaces to the API without changing the behavior
of anything else. This includes the C APIs for applications to provide
swap chain implementations. Also adds stub implementations on every
backend.
The test doing many small SetSubData freezes the Metal driver on a
MTLBuffer allocation. This is because we don't use a ringbuffer for
SetSubData and instead create one small upload buffer per call.
The same test is skipped on D3D12 as it is bound to have the same issue.
Also adds comments that were forgotten before every MapReadAsync test.
RenderDoc uses presents as the points to start and end a capture, so to
debug test we have to do the following:
TEST_F(...)
while(true) {
// All the test code
SwapBuffers();
}
}
Implements BeginRenderSubpass on the D3D12 backend. Descriptors for render target and depth stencil views are recorded in a descriptor heap for each framebuffer. For now, we still have the hack where no attachment renders to the backbuffer, so the CommandBuffer records those when necessary when it is submitted.
This PR also enables input states for D3D12 which are mostly working. One failure seems to be happening because our texture copies are not yet correct.
Probably not the most efficient implementation, but works.
Issues:
* Doesn't seem to render until the second frame. No clue why, yet.
* Hardcoded 640x480 in more places.
* Creates new FBOs for every subpass every frame. Should be done at Framebuffer build or CommandBuffer build time.
This helps Visual Studio users have less clutter in their solution
explorer. This also updates spirv-tools to a newer version that folders
itself. This also updates spirv-headers so that spirv-tools compiles.
This macro has some advantages over the standard library one:
- It prints the place where the macro was triggered
- It "references" the condition even in Release to avoid warnings
- In release, if possible, it gives compiler hints
It is basically is stripped down version of the ASSERT macros I wrote
for the Daemon engine in src/common/Assert.h
This commit also removes the stray "backend" namespaces for common/
code.
This directory used to contain both the state tracking code for the
backends, and the common utilities that could be used both by the
backends and the rest of the code. Things are now:
- src/common is utility code for the whole repo
- src/backend contains libNXT's code
- src/utils is utility code that we don't want in libNXT
This commit also changes all includes to use global paths from src/
bacause it had to touch a bunch of #include statements anyway.
Buffers with MapRead allowed are created on the READBACK heap and always
add the D3D12_RESOURCE_STATE_COPY_DEST state (required by D3D12).
Likewise MapWrite adds the D3D12_RESOURCE_STATE_GENERIC_READ state and
places resources on the UPLOAD heap. Because these states are
required, transitions for mapped buffers do nothing.
This makes rendering of the samples have the wrong colors on the Metal
backend, but using BGRA made end2end tests fail. The rendering color
will be fixed when the WSI is introduced.
Extract descriptor offset computation and CPU descriptor recording to
BindGroupLayout and BindGroup. Refactor descriptor heap allocation to
copy from a large CPU heap to a GPU heap.
Previously the Metal backend used a manual mutex system to make sure the
BufferSetSubData didn't have data races with reads from the GPU. Replace
this with a non-hacky version
- Make the Buffer objects allocated on the GPU
- Make SetSubData use a ResourceUploader that allocates a CPU buffer
and schedules a CPU->GPU copy.
- Have a list of pending commands and a finished command serial to
order operations and track when resource become unused.
No functional changes intended, but there are a couple additional
cleanups:
- Use anonymous namespaces instead of static functions
- Don't store an extra Device pointer in objects
Unfortunately you can't template on const-ness in C++ so we have to
duplicate all the iterator code for SerialQueue (that, or introduce very
heavy templating).
This also expands the Buffer validation tests to cover more creation
code paths and SetSubData. It also introduces a mechanism for
ValidationTests to check for device errors.
We want to test BufferMapRead validation using the null backend. To get
closer to conditions on a real backend, we call the callback only on the
next Queue::Submit. This is because on real backends, we would have to
wait for the GPU to be finished with the buffer, to be sure the correct
data is read.
- defaults to depth and stencil tests off
- whether or not depth and stencil tests are enabled is inferred from the comparison functions and stencil operations
- only one stencil reference. D3D12 does not support separate references
- change SetDepthWriteMode to SetDepthWriteEnabled and use a bool instead of enum
- Create PersistentPipelineState class for OpenGL backend with simple state tracking
- Add validation so DepthStencilState properties are only set once
- Update API usage in HelloDepthStencil
- refactor tracking of the DepthStencilState in the Metal backend
- validate that compute pipeline does not have a depth stencil state
Add depth and stencil tests. This is currently only implemented for the
OpenGL API. HelloDepthStencil is a test using the depth and stencil
buffers to do reflections. Currently clearing / stencil clearing is not
working properly.
Previously WireCmd.h/cpp that is used in wire_autogen wasn't included in
the sources, causing a link error on some platforms.
With WireCmd.* moved in the EXTRA_SOURCES, the nxt_wire target didn't
contain any non-header file and caused a link error on OSX. Fix it by
properly splitting the declaration and implementation of
TerribleCommandBuffer in a .h and .cpp file.
This makes sure it works correctly with the ScanForward Math type.
However this isn't a very good fix, the right solution would be to
detect whether we are compiling in 32bit or 64bit and use one of
ScanForward32 or ScanForward64.
For shared library to work on Windows to work, we need to add
declspec(export) and declspec(import) annotations to the symbols to
export. This fixes the problem by making all libraries static on
Windows, but we'll need to revisit and do proper symbol exports.
Validation tests are tests of the backend state-tracking and validation
code that don't require a GPU as they are running on the null backend.
This commit adds a very simple (and almost useless) BufferValidationTest
as an example of a validation test.
* First API design (many features missing, including input attachments)
* Metal implementation (no OpenGL yet)
* Render-to-texture demo (a little broken until we have depth buffers)
* Update examples to use render passes
- Add test for passing value arguments as well as aarrays of them
- Add test for passing C string
- Rework Object array argument test to pass multiple objects
This adds support for "natively defined" API types like callbacks that
will have to be implemented manually for each target language. Also this
splits the concept of "native method" into a set of native methods per
language.
Removes the "Synchronous error" concept that was used to make builders
work in the maybe Monad, this will have to be reinroduced with builder
callbacks.
Corentin Wallez
Li Hao
Jiawei Shao
Stephen White
Kai Ninomiya
Kai Ninomiya
Stephen White
Corentin Wallez
msiglreith
Austin Eng
Ben Constable
Austin Eng