Repository navigation
Go verifier: Python's check names, schema copies checked, v2 needs -trust; E21 scores the blind pools - #505
Merged
Merged
Conversation
…a copies, ask a v2 bundle for -trust (#501) - A failure names the reference verifier's check it fails (signatures, run chain, inclusion, chain intact, ...), and the warnings Go already computes carry Python's names (keys, trust root, run completeness, run boundaries, run tail witnessed). The v2 verifier stops at its first failure, so the parity gate requires its one failed check to be among Python's. - KNOWN_GAPS keeps only the checks Go doesn't perform: the gate compares verdicts only there, and the README lists them and says why. - TestEmbeddedSchemasAreTheRepositorys fails when an embedded event schema differs from tracekit/schema/. - The CLI reads the format from the manifest: a v2 bundle without -trust is told it needs one, not that its v1 manifest is invalid.
eval/e21_blind/*.jsonl (#357, #401) are scored in e21_policy_robustness.py, together and pool by pool, recorded in eval/results/ and reported in docs/evaluation.md. The output says these aren't blind scores: rules written after each pool was published came from its misses. The blind score at each pool's publication is computed from git history (each merge commit's own scorer and packs) and matches what the pull requests reported. The blind false-positive rates join the no-regression gates; the recorded latency block is kept as it was.
- The parity gate exempts each verdict-only check only with the status Python gives it: Go's failures of head matches a witness checkpoint are compared. - The text report prints a reason no failed check line gives, such as v1 findings reached after a failed check. - A v1 manifest whose file list is not an object fails bundle structure, as in Python. - The Go README says which unsupported evidence is exit 2 and which fails keys (exit 1), and how an invalid signer.pub differs from Python. - E21 scores each blind pool once; the latency text names the sets it times.
…fest; wrap an E21 docstring line (#501) As in the Python verifier, a missing file list lists no file, so every file is unlisted; only a list that is not an object (null included) is bundle structure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Fixes #501, from the 2 November review of merged outside contributions.
TestEmbeddedSchemasAreTheRepositoryscompares both embedded copies (v2, v1) byte for byte withtracekit/schema/. Changing one byte fails it.KNOWN_GAPSkeeps only the checks Go doesn't perform at all. The script and the README call these verdict-only and say why.-trust, the Go CLI said "invalid v1 manifest" (UNUSABLE, exit 2). It now reads the format from the manifest and says a v2 bundle needs-trust(UNVERIFIABLE, exit 2).blindset, ineval/results/anddocs/evaluation.md. The output says these aren't blind scores any more: rules were written after the pools were published.How I ran it as a user
demo-run.tkb, no flagstrust rootwarning with what to passdemo-run-tampered.tkb[FAIL] record seq 6: event hash mismatch[FAIL] chain intact — record seq 6: event hash mismatch-trustinvalid v1 manifest, UNUSABLEa v2 bundle needs -trust …, UNVERIFIABLEChecklist
make test: 4,034 passed, 0 failed. Go vet is clean; Go tests: 370 passed, 0 skipped.make test-tspasses withTRACEKIT_PYTHONset to the venv's Python.Security invariants
Evidence format