test: Add §3.4 range-cap and §7.3.1 Step.Name job conformance fixtures - #155
Conversation
| # Template Schemas 3.4: a task parameter's range may take on at most 1024 | ||
| # values. A user-SUPPLIED RANGE_EXPR value is only expanded at job creation, | ||
| # so the cap must be enforced there: "1-1025" expands to 1025 values and job | ||
| # creation must fail, even though the template itself is valid. |
There was a problem hiding this comment.
This sounds wrong, range expressions are not limited the same as range values. I remember we made sure the range expression wording didn't include that limit, seems like we need to amend that wording to be clear that the range limit does not apply.
There was a problem hiding this comment.
Quorum verdict: NOT-GOOD (5/5) — spec: §3.4.1.1.1 <IntRangeExpr>.
Upstream commit 3069673 added normative text: the number of values an <IntRangeExpr> expands to "is not constrained by this specification", and the §3.4.1.1 1024-element cap "does not apply to this form" — a >1024-value expansion is the motivating use-case (see the new mainline fixture 3.4--wide-int-range-expression.yaml, which accepts 1-5000). This fixture pins rejection of a supplied RANGE_EXPR "1-1025", i.e. the opposite of current spec. A conformant implementation exits 0 and fails the test; a non-conformant one passes for any rejection reason. Recommend: drop, or invert to an acceptance fixture.
|
Quorum review (5 independent agents: spec-literalist, adversarial, test-craft, service-compat, coverage). Per-fixture verdicts posted as file comments. Net: 1 NOT-GOOD, 2 GOOD-WITH-NITS, 1 GOOD. Blocking item: |
c745815 to
4348402
Compare
|
Quorum-review fixes applied and pushed (rebased onto mainline 3069673):
Verified: all 4 fixtures pass against openjd-rs built from upstream/main |
4348402 to
acb7a01
Compare
Four fixtures pinning create-time and run-time behavior that no existing
fixture covered, updated for the §3.4.1.1.1 spec fix (IntRangeExpr
expansion is uncapped; only the <IntRangeList> form carries the 1024 cap):
- 3.4--range-from-list-param-at-limit.test: exactly 1024 list-resolved
range elements must be accepted (list-form cap, accept side).
- 3.4--range-from-list-param-too-long.invalid.test: a supplied LIST[INT]
of 1025 items through a typed whole-field range "{{Param.Values}}" is
decode-clean but must fail at job creation (list-form cap, reject side).
- 3.4--range-expr-supplied-value-beyond-list-cap.test: a supplied
RANGE_EXPR "1-1025" must be ACCEPTED and run 1025 tasks — IntRangeExpr
expansion is explicitly unconstrained by the spec (3.4.1.1.1). Replaces
the earlier .invalid fixture, which pre-dated the spec fix and asserted
the opposite.
- 7.3.1--step-name-in-step-environment.test: Step.Name resolves in a
step environment's onEnter/onExit, directly and via step-level let.
Review: quorum-review fixes — corrected RFC citations (whole-field
resolution is RFC 0005 / Expression Language 1.3.2, not RFC 0006;
Step.Name is RFC 0005, not RFC 0007) and scoped all cap language to the
list form.
Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
acb7a01 to
d771412
Compare
2 participants
What was the problem/requirement? (What/Why)
Review of the RFC 0007/0008 implementation stack (openjd-model-for-python #318/#313, openjd-sessions-for-python #333, openjd-cli #230) found two spec-observable behaviors that no conformance fixture pinned — both had silently diverged between the Python and Rust implementations:
"{{Param.Values}}"over aLIST[*]parameter, or a suppliedRANGE_EXPR— are only expanded at job creation, where openjd-rs enforced the cap but the Python model did not.Step.Nameresolves inside a step environment's actions — the Rust CLI passed this while the Python CLI failed it.What was the solution? (How)
Four fixtures in
conformance-tests/2023-09/EXPR/jobs/:3.4--range-from-list-param-too-long.invalid.test.yaml— suppliedLIST[INT]of 1025 items via a typed whole-field range; decode-clean, must fail at job creation.3.4--range-expr-supplied-value-too-long.invalid.test.yaml— suppliedRANGE_EXPR"1-1025"; decode-clean, must fail at job creation.3.4--range-from-list-param-at-limit.test.yaml— exactly 1024 values accepted (guards the off-by-one an at-the-cap rejection bug would introduce). Note: runs 1024 trivial tasks — seconds on the Rust CLI, ~3 minutes on the Python CLI; the fixture header documents this.7.3.1--step-name-in-step-environment.test.yaml—Step.Nameresolves in a step environment's onEnter/onExit at run time, directly and through a step-levelletbinding. (Negative polarity — Step.Name rejected in job environments — already exists as decode-time.invalidfixtures.)How was this change tested?
All four fixtures validated against both implementations via
run_openjd_cli_tests.py: the openjd-rs release CLI, and the Python CLI running the fixed RFC 0007/0008 branches. Both.invalidtemplates confirmed decode-clean (openjd checkpasses on both CLIs), failing only at job creation.