Tolerate some errors while building SPIR-V corpus

When preparing a corpus of SPIR-V shaders for fuzzing, spirv-as is
invoked repeatedly. It could be that a bug in spirv-as leads to
conversion failing for some of the shaders. This should not prevent the
overall corpus from being generated, as long as the number of overall
failures is reasonably small. This change adds some tolerance for such
failures.

Change-Id: I77750fdeab15a252201bff33e952e1bd44c42331
Reviewed-on: https://dawn-review.googlesource.com/c/tint/+/64543
Auto-Submit: Alastair Donaldson <afdx@google.com>
Reviewed-by: Ben Clayton <bclayton@google.com>
Commit-Queue: Alastair Donaldson <afdx@google.com>
Kokoro: Kokoro <noreply+kokoro@google.com>

fuzzers/generate_spirv_corpus.py

@@ -52,6 +52,16 @@ def main():
     if os.path.exists(corpus_dir):
         shutil.rmtree(corpus_dir)
     os.makedirs(corpus_dir)
+
+    # It might be that some of the attempts to convert SPIR-V assembly shaders
+    # into SPIR-V binaries go wrong. It is sensible to tolerate a small number
+    # of such errors, to avoid fuzzer preparation failing due to bugs in
+    # spirv-as. But it is important to know when a large number of failures
+    # occur, in case something is more deeply wrong.
+    num_errors = 0
+    max_tolerated_errors = 10
+    logged_errors = ""
+
     for in_file in list_spvasm_files(input_dir):
         if in_file.endswith(".expected.spvasm"):
             continue
@@ -69,8 +79,13 @@ def main():
                                 stderr=subprocess.PIPE)
         stdout, stderr = proc.communicate()
         if proc.returncode != 0:
-            print("Error running " + " ".join(cmd) + ": " + stdout, stderr)
+            num_errors += 1
-            return 1
+            logged_errors += "Error running " + " ".join(cmd) + ": " + stdout.decode('utf-8') + stderr.decode('utf-8')
+
+    if num_errors > max_tolerated_errors:
+        print("Too many (" + str(num_errors) + ") errors occured while generating the SPIR-V corpus.")
+        print(logged_errors)
+        return 1
 
 
 if __name__ == "__main__":