diff --git a/PLAN.md b/PLAN.md index b094ac1b7..4570e0bac 100644 --- a/PLAN.md +++ b/PLAN.md @@ -124,8 +124,8 @@ manpages, and SDKs — never a runtime dependency of somebody else's program. `env`, defaults, delimiters, the `double_dash` modes, global flags, flatten, boxed subcommand variants, headings, `cfg`-gated variants. - [x] **The post-binding layer** — `required`, `choices`, `env` fallback, defaults, - `var_min`/`var_max`, `conflicts`, `required_if`, `required_unless`, and - `overrides`. All of them need a value's type, so they belong with the derive + `var_min`, `conflicts`, `required_if`, `required_unless`, and `overrides` — + `var_max` moved to the binder, see the decision below. All of them need a value's type, so they belong with the derive rather than in the parser. `overrides` is the one that happens _during_ the parse instead of after it: it asks which of two flags came last, which only the arriving token knows. @@ -151,6 +151,14 @@ carrying them rather than being worked around. shape mise uses on `run`, `exec` and `git`: `[ARGS]…` for the words before the separator and `[-- ARGS_LAST]…` for the ones after. Both parsers already bound it; only the derive's validation disagreed. Found by compiling the mise shadow. +- [x] **Is `var_max` a limit or a check?** — usage-argv let a variadic take every word and + judged the count afterwards; usage-lib stopped the variadic at the bound and gave the + rest to the next argument. Two coherent readings, and nothing had recorded that the + two implementations disagreed. Settled as a **limit**, matching usage-lib and clap's + `num_args`, since every spec in the fleet is generated from a clap command and it is + the only reading under which `[a]… [b]` can be filled. `var_max` therefore moved into + the table binding reads; `var_min` stays a check, because no single word tells you a + variadic will end up short. Costs 55 instructions per parse, or 0.1%. - [ ] **A mount on the root command** — the spec accepts `mount` only inside a `cmd` block, so a CLI whose _top-level_ subcommands are discovered by running something cannot say so. Worth deciding whether that is a gap or a deliberate @@ -192,9 +200,9 @@ checked-in `mise.usage.kdl` for both parsers. same properties. Three release binaries — one per parser, one that does everything except parse — measured by `tak`, which gates the counts in CI. - [x] **Perf report** — at mise's full scale, a cold parse costs **50.9k instructions - against clap's 5.96M: 117× fewer**, and **2.0µs against 500µs of wall clock**. + against clap's 5.96M: 117× fewer**, and **2.1µs against 490µs of wall clock**. Measured by differencing two runs of the _same_ binary over how many parses it - does, so nothing but the parse varies. clap's 500µs is 315µs building its command + does, so nothing but the parse varies. clap's 490µs is 305µs building its command tree, ~160µs validating it, and ~24µs actually parsing — so even against clap's parse alone, with the tree already built and paid for, this is 12× faster. - [ ] **Differential fuzzing** — proptest over argv against usage-lib on the mise @@ -206,14 +214,14 @@ Runtime targets, which gate: | measurement | clap, measured | target | result | | ---------------------------------- | -------------- | ------ | ---------------- | | instructions, route + parse | 5.96M | < 100k | 50.9k, 117× | -| wall time, argv to parsed struct | 500µs | < 50µs | 2.0µs, 245× | +| wall time, argv to parsed struct | 490µs | < 50µs | 2.1µs, 238× | | heap allocations, successful parse | thousands | 0 | not yet measured | Both gating targets are met, and not narrowly. The one number still owed is allocations on the derive path — usage-argv's own are asserted at zero, but the derive allocates a `String` per value, and nothing counts them yet. -An earlier draft of these numbers said 48–58× rather than 103×, because it subtracted a +An earlier draft of these numbers said 48–58× rather than 117×, because it subtracted a baseline measured in a _separate_ no-op binary. Two binaries do measurably different amounts of setup before `main`, and that difference had been landing in what was attributed to parsing. Differencing two runs of one binary over how many parses it does diff --git a/argv/src/lib.rs b/argv/src/lib.rs index 7f3189a8f..43b985d6c 100644 --- a/argv/src/lib.rs +++ b/argv/src/lib.rs @@ -155,6 +155,13 @@ pub struct Flag<'a> { /// parser, since it already reports every occurrence separately. Conflating /// the two makes a merely repeatable flag greedy enough to eat a positional. pub variadic: bool, + /// How many values one variadic occurrence may take, after which the next word + /// belongs to whatever comes next. + /// + /// Only for [`variadic`](Self::variadic). A merely repeatable flag — the spec's + /// `var=#true` — is bounded on how many times it was *given*, which no single token + /// can decide, so that bound stays with the metadata and is checked after the parse. + pub var_max: ::core::option::Option, /// Whether the flag is recognized by every command beneath the one that /// declares it. pub global: bool, @@ -170,6 +177,7 @@ impl Flag<'_> { negate: None, takes_value: false, variadic: false, + var_max: ::core::option::Option::None, global: false, }; @@ -188,6 +196,14 @@ pub struct Arg<'a> { pub key: u64, /// Whether this argument keeps taking values once it has one. pub var: bool, + /// How many words a variadic may take before the next argument gets the rest. + /// + /// A bound belongs here, in the table binding reads, rather than with the metadata: + /// it decides *where* a word lands, not whether what landed is acceptable. clap's + /// `num_args` works the same way, and every spec in the wild is generated from a clap + /// command. `u32` rather than `usize` because a CLI that bounds a variadic above four + /// billion has other problems, and this table is read on the hot path. + pub var_max: ::core::option::Option, /// This argument's relationship to the `--` separator. pub double_dash: DoubleDash, /// Unused by binding, kept so a table entry can carry its own name for @@ -200,6 +216,7 @@ impl Arg<'_> { pub const REQUIRED: Arg<'static> = Arg { key: 0, var: false, + var_max: ::core::option::Option::None, double_dash: DoubleDash::Optional, name: "", }; @@ -386,8 +403,12 @@ pub struct Parser<'t, 'v> { bundle_token: &'v [u8], /// A variadic flag that is still collecting values. collecting: Option<&'t Flag<'t>>, + /// How many values it has taken, so a bound can stop it. + collected: u32, /// Which of `cmd.args` is next to fill. arg_pos: usize, + /// How many words the variadic at `arg_pos` has taken, for the same reason. + arg_taken: u32, /// Whether any word has been bound to a positional of `cmd`. Once one has, /// no further word can select a subcommand. arg_filled: bool, @@ -420,7 +441,9 @@ impl<'t, 'v> Parser<'t, 'v> { bundle: &[], bundle_token: &[], collecting: None, + collected: 0, arg_pos: 0, + arg_taken: 0, arg_filled: false, flags_stopped: false, separator_seen: false, @@ -478,6 +501,12 @@ impl<'t, 'v> Parser<'t, 'v> { match self.argv.get(self.pos) { Some(next) if !is_flag_like(bytes(next)) && bytes(next) != b"--" => { self.pos += 1; + self.collected += 1; + // Same rule as a positional: a bounded occurrence takes that many and + // leaves the rest to whatever follows. + if flag.var_max.is_some_and(|max| self.collected >= max) { + self.collecting = None; + } return Some(Ok(Event::Flag { flag, value: Some(bytes(next)), @@ -512,7 +541,12 @@ impl<'t, 'v> Parser<'t, 'v> { .iter() .position(|a| a.double_dash == DoubleDash::Required) { + // The count belongs to the argument at `arg_pos`, so jumping past it has + // to leave the count behind: a bounded variadic before the separator would + // otherwise lend its total to the argument after it, which then stops + // early or at once. self.arg_pos += idx; + self.arg_taken = 0; } return self.step(); } @@ -558,7 +592,7 @@ impl<'t, 'v> Parser<'t, 'v> { None }; if flag.variadic { - self.collecting = Some(flag); + self.start_collecting(flag); } return Ok(Event::Flag { flag, @@ -633,7 +667,7 @@ impl<'t, 'v> Parser<'t, 'v> { rest }; if flag.variadic { - self.collecting = Some(flag); + self.start_collecting(flag); } Ok(Event::Flag { flag, @@ -682,9 +716,16 @@ impl<'t, 'v> Parser<'t, 'v> { if arg.double_dash == DoubleDash::Automatic { self.flags_stopped = true; } - // A variadic keeps taking values, so the cursor stays put. - if !arg.var { - self.arg_pos += 1; + // A variadic keeps taking values, so the cursor stays put — until it reaches its + // bound, at which point the words after it belong to whatever comes next. That is + // what makes `[a]… [b]` expressible at all. + if arg.var { + self.arg_taken += 1; + if arg.var_max.is_some_and(|max| self.arg_taken >= max) { + self.advance_arg(); + } + } else { + self.advance_arg(); } Ok(Event::Arg { arg, value: token }) } @@ -697,10 +738,30 @@ impl<'t, 'v> Parser<'t, 'v> { self.depth += 1; self.cmd = sub; self.arg_pos = 0; + self.arg_taken = 0; self.arg_filled = false; Ok(()) } + /// Move to the next positional, forgetting what the last one took. + fn advance_arg(&mut self) { + self.arg_pos += 1; + self.arg_taken = 0; + } + + /// A variadic flag occurrence begins, counting from zero. + /// + /// The value it was given on the same token counts, which is why this starts at one: + /// `--include a b` with `var_max=2` takes `a` and `b`, not three words. + fn start_collecting(&mut self, flag: &'t Flag<'t>) { + self.collected = 1; + self.collecting = if flag.var_max.is_some_and(|max| max <= 1) { + None + } else { + Some(flag) + }; + } + fn next_arg(&self) -> Option<&'t Arg<'t>> { self.cmd.args.get(self.arg_pos).copied() } diff --git a/conformance/src/argv.rs b/conformance/src/argv.rs index c1d806648..55e03d6ee 100644 --- a/conformance/src/argv.rs +++ b/conformance/src/argv.rs @@ -236,6 +236,17 @@ fn build( // single-value argument is repeatable instead: one value per // occurrence, which the parser gets by not collecting. variadic: f.arg.as_ref().is_some_and(|a| a.var), + // The bound on one occurrence's values, which is the argument's. A + // repeatable flag's own `var_max` counts occurrences and is checked after + // the parse, so it does not belong in this table. + var_max: f + .arg + .as_ref() + .filter(|a| a.var) + .and_then(|a| a.var_max) + // Saturating rather than truncating: `4294967296 as u32` is zero, + // which would read as "stop at once" rather than "no real limit". + .map(|max| u32::try_from(max).unwrap_or(u32::MAX)), global: f.global, })) }) @@ -249,6 +260,10 @@ fn build( key: 0, name: leak(&a.name), var: a.var, + var_max: a + .var_max + .filter(|_| a.var) + .map(|max| u32::try_from(max).unwrap_or(u32::MAX)), double_dash: match a.double_dash { usage::SpecDoubleDashChoices::Required => DoubleDash::Required, usage::SpecDoubleDashChoices::Preserve => DoubleDash::Preserve, diff --git a/conformance/tests/derive.rs b/conformance/tests/derive.rs index 40cfc18a1..2fa89c048 100644 --- a/conformance/tests/derive.rs +++ b/conformance/tests/derive.rs @@ -669,3 +669,40 @@ fn a_double_dash_argument_can_follow_a_variadic() { assert_eq!(last.double_dash, usage::SpecDoubleDashChoices::Required); assert!(last.var); } + +/// A bounded variadic, with an argument after it. +/// +/// The bound is what makes the second one reachable: `[first]…` takes two words and hands +/// the rest over. An unbounded variadic there would be a compile error, because nothing +/// after it could ever be filled. +#[derive(Cli, Debug)] +#[usage(bin = "bounded")] +struct Bounded { + /// The first two words + #[usage(arg, name = "FIRST", var_max = 2)] + first: Vec, + /// Whatever follows them + #[usage(arg, name = "REST")] + rest: Option, +} + +#[test] +fn a_bound_hands_the_rest_to_the_next_argument() { + let a = argv(["x", "y", "z"]); + let b = Bounded::parse_from(&a).expect("should parse"); + assert_eq!(b.first, ["x", "y"]); + assert_eq!(b.rest.as_deref(), Some("z")); + + // Fewer words than the bound: the variadic simply takes what there is. + let a = argv(["x"]); + let b = Bounded::parse_from(&a).expect("should parse"); + assert_eq!(b.first, ["x"]); + assert!(b.rest.is_none()); + + // And the spec says the bound, so docs and completions agree with the parse. + assert!( + Bounded::to_kdl().contains("var_max=2"), + "{}", + Bounded::to_kdl() + ); +} diff --git a/conformance/tests/post_binding.rs b/conformance/tests/post_binding.rs index 6bf922abb..975d1fcb2 100644 --- a/conformance/tests/post_binding.rs +++ b/conformance/tests/post_binding.rs @@ -398,3 +398,35 @@ fn displacing_one_flag_leaves_the_others_alone() { ); assert!(ovr.stdin); } + +/// A flag whose one occurrence collects, bounded. +/// +/// `var_max` is a binding limit here — the occurrence stops at two values and the words +/// after belong to the positional — so a *second* occurrence starts its own count. Checking +/// the total afterwards would fail an invocation that never broke the limit. +#[derive(Cli)] +#[usage(bin = "bounded")] +struct BoundedFlag { + /// Patterns, two at a time + // `variadic` alone: one occurrence takes several values. `var` would be the other + // shape — one value per occurrence — and the derive refuses both at once. + #[usage(long, variadic, var_max = 2)] + include: Vec, + /// Where to write + #[usage(arg, name = "OUT")] + out: Option, +} + +#[test] +fn each_occurrence_of_a_bounded_flag_counts_for_itself() { + // Two occurrences, each within its bound, four values in total. + let a = argv(["--include", "a", "b", "--include", "c", "d"]); + let bounded = BoundedFlag::parse_from(&a).expect("two occurrences should parse"); + assert_eq!(bounded.include, ["a", "b", "c", "d"]); + + // And the bound still stops one occurrence, leaving the third word to the positional. + let a = argv(["--include", "a", "b", "c"]); + let bounded = BoundedFlag::parse_from(&a).expect("should parse"); + assert_eq!(bounded.include, ["a", "b"]); + assert_eq!(bounded.out.as_deref(), Some("c")); +} diff --git a/corpus/03-positionals.json b/corpus/03-positionals.json index b13d11456..3e37bb4c3 100644 --- a/corpus/03-positionals.json +++ b/corpus/03-positionals.json @@ -7,14 +7,22 @@ "doc": "A required argument takes the first non-flag word.", "spec": "name \"ex\"\nbin \"ex\"\narg \"\"\n", "argv": ["a.txt"], - "expect": { "ok": { "args": { "file": "a.txt" } } } + "expect": { + "ok": { + "args": { + "file": "a.txt" + } + } + } }, { "id": "arg-required-missing", "doc": "A required argument with nothing to fill it is an error.", "spec": "name \"ex\"\nbin \"ex\"\narg \"\"\n", "argv": [], - "expect": { "error": "missing_required_arg" }, + "expect": { + "error": "missing_required_arg" + }, "layer": "post-binding" }, { @@ -22,14 +30,23 @@ "doc": "An optional argument left unfilled is simply absent.", "spec": "name \"ex\"\nbin \"ex\"\narg \"[file]\"\n", "argv": [], - "expect": { "ok": {} } + "expect": { + "ok": {} + } }, { "id": "arg-order", "doc": "Two arguments fill in declaration order, regardless of what the values look like.", "spec": "name \"ex\"\nbin \"ex\"\narg \"\"\narg \"\"\n", "argv": ["from", "to"], - "expect": { "ok": { "args": { "src": "from", "dst": "to" } } } + "expect": { + "ok": { + "args": { + "src": "from", + "dst": "to" + } + } + } }, { "id": "arg-interleaved-with-flags", @@ -38,8 +55,13 @@ "argv": ["from", "-f", "to"], "expect": { "ok": { - "flags": { "force": true }, - "args": { "src": "from", "dst": "to" } + "flags": { + "force": true + }, + "args": { + "src": "from", + "dst": "to" + } } } }, @@ -48,54 +70,132 @@ "doc": "More words than there are arguments to hold them is an error, not silent truncation.", "spec": "name \"ex\"\nbin \"ex\"\narg \"\"\n", "argv": ["a", "b"], - "expect": { "error": "unexpected_arg" } + "expect": { + "error": "unexpected_arg" + } }, { "id": "arg-variadic-collects-rest", "doc": "A variadic argument collects every remaining word.", "spec": "name \"ex\"\nbin \"ex\"\narg \"...\"\n", "argv": ["a", "b", "c"], - "expect": { "ok": { "args": { "files": ["a", "b", "c"] } } } + "expect": { + "ok": { + "args": { + "files": ["a", "b", "c"] + } + } + } }, { "id": "arg-variadic-after-fixed", "doc": "A fixed argument is filled before a following variadic starts collecting.", "spec": "name \"ex\"\nbin \"ex\"\narg \"\"\narg \"[rest]...\"\n", "argv": ["run", "a", "b"], - "expect": { "ok": { "args": { "cmd": "run", "rest": ["a", "b"] } } } + "expect": { + "ok": { + "args": { + "cmd": "run", + "rest": ["a", "b"] + } + } + } }, { "id": "arg-variadic-empty", "doc": "An optional variadic given nothing is absent rather than an empty list.", "spec": "name \"ex\"\nbin \"ex\"\narg \"[rest]...\"\n", "argv": [], - "expect": { "ok": {} } + "expect": { + "ok": {} + } }, { "id": "arg-variadic-min", "doc": "`var_min` is enforced: too few values is an error.", "spec": "name \"ex\"\nbin \"ex\"\narg \"\" var=#true var_min=2\n", "argv": ["a"], - "expect": { "error": "var_too_few" }, + "expect": { + "error": "var_too_few" + }, "layer": "post-binding" }, { "id": "arg-variadic-max", - "doc": "`var_max` is enforced: too many values is an error.", + "doc": "A variadic stops at its bound, so the words past it belong to whatever comes next \u2014 and with nothing after it, the third word has nowhere to go. `var_too_many` cannot arise for a variadic any more: it describes a repeatable flag given more times than its bound allows. This used to be recorded as a divergence \u2014 the corpus wanted `var_too_many` and usage-lib said `unexpected_arg` \u2014 and adopting the limit reading is what made the two agree.", "spec": "name \"ex\"\nbin \"ex\"\narg \"\" var=#true var_max=2\n", "argv": ["a", "b", "c"], - "expect": { "error": "var_too_many" }, - "reference": { - "diverges": "usage-lib reports `unexpected_arg`. The outcome is the same rejection; the distinct code is what tells a caller the limit was a `var_max` rather than an absent argument." - }, - "layer": "post-binding" + "expect": { + "error": "unexpected_arg" + } }, { "id": "arg-value-looks-like-negative-number", "doc": "A negative number is a value, not an unknown flag. A leading `-` followed by a digit cannot begin a flag name, so the token is offered to the next argument.", "spec": "name \"ex\"\nbin \"ex\"\narg \"\"\n", "argv": ["-1"], - "expect": { "ok": { "args": { "offset": "-1" } } } + "expect": { + "ok": { + "args": { + "offset": "-1" + } + } + } + }, + { + "id": "a-bound-stops-a-variadic", + "doc": "`var_max` is a limit on how many words a variadic takes, not only a check on how many it got: once it is full the next argument gets the rest. Without that, `[a]\u2026 [b]` could never be filled, and clap's `num_args` \u2014 which every generated spec comes from \u2014 behaves this way.", + "spec": "name \"ex\"\nbin \"ex\"\narg \"[a]\" var=#true var_max=1\narg \"[b]\"\n", + "argv": ["x", "y"], + "expect": { + "ok": { + "args": { + "a": ["x"], + "b": "y" + } + } + } + }, + { + "id": "a-bound-does-not-invent-somewhere-to-put-the-rest", + "doc": "The bound stops the variadic even when nothing follows it, so the extra word has nowhere to go and is an error rather than a silently dropped value.", + "spec": "name \"ex\"\nbin \"ex\"\narg \"[a]\" var=#true var_max=1\n", + "argv": ["x", "y"], + "expect": { + "error": "unexpected_arg" + } + }, + { + "id": "an-unbounded-variadic-still-takes-everything", + "doc": "Unbounded is the common case and unchanged: nothing after it can be filled, which is why declaring an argument there is refused.", + "spec": "name \"ex\"\nbin \"ex\"\narg \"[a]\" var=#true\n", + "argv": ["x", "y", "z"], + "expect": { + "ok": { + "args": { + "a": ["x", "y", "z"] + } + } + } + }, + { + "id": "a-bound-stops-a-variadic-flag", + "doc": "The same rule for a flag whose one occurrence collects: `--include` takes the two words its bound allows, and the third is the positional's.", + "spec": "name \"ex\"\nbin \"ex\"\nflag \"--include \u2026\" {\n arg \"\u2026\" var=#true var_max=2\n}\narg \"[out]\"\n", + "argv": ["--include", "a", "b", "c"], + "expect": { + "ok": { + "flags": { + "include": ["a", "b"] + }, + "args": { + "out": "c" + } + } + }, + "reference": { + "diverges": "usage-lib does not collect for a variadic flag at all \u2014 it takes one value, so `b` fills [out] and `c` is unexpected. The same gap as `long-variadic-flag-arg`, which the spec reference documents; bounding what usage-argv collects is consistent with it collecting in the first place." + } } ] } diff --git a/corpus/06-double-dash.json b/corpus/06-double-dash.json index 1b269d25b..02a0e5019 100644 --- a/corpus/06-double-dash.json +++ b/corpus/06-double-dash.json @@ -173,6 +173,20 @@ } } } + }, + { + "id": "a-bound-before-the-separator-is-not-charged-after-it", + "doc": "Each variadic counts its own words. A bounded one before the `--` must not lend its count to the argument after it, which would otherwise stop early or at once.", + "spec": "name \"ex\"\nbin \"ex\"\narg \"[first]\" var=#true var_max=3\narg \"[-- rest]\" var=#true var_max=2\n", + "argv": ["a", "b", "--", "c", "d"], + "expect": { + "ok": { + "args": { + "first": ["a", "b"], + "rest": ["c", "d"] + } + } + } } ] } diff --git a/corpus/08-choices-and-required.json b/corpus/08-choices-and-required.json index 1f5b5085a..1916500d0 100644 --- a/corpus/08-choices-and-required.json +++ b/corpus/08-choices-and-required.json @@ -218,19 +218,6 @@ "error": "var_too_many" }, "layer": "post-binding" - }, - { - "id": "a-bound-does-not-stop-a-variadic", - "doc": "A variadic collects every word still available; `var_max` is a check on how many it got, not a limit that hands the extra words to the next argument. So an argument after a bounded variadic is as unreachable as one after an unbounded variadic, and giving two words here is too many rather than one each.", - "spec": "name \"ex\"\nbin \"ex\"\narg \"[a]\" var=#true var_max=1\narg \"[b]\"\n", - "argv": ["x", "y"], - "expect": { - "error": "var_too_many" - }, - "layer": "post-binding", - "reference": { - "diverges": "usage-lib stops the variadic at var_max while binding, so it fills [a] with x, [b] with y, and reports no error. clap's num_args behaves the way usage-lib does here, so which of the two is right is a question for the grammar rather than a bug either way." - } } ] } diff --git a/derive/src/codegen.rs b/derive/src/codegen.rs index ccf07e93e..2d8b2b6a5 100644 --- a/derive/src/codegen.rs +++ b/derive/src/codegen.rs @@ -255,6 +255,19 @@ fn flag_table(i: usize, field: &Field) -> TokenStream { let shorts: Vec = shorts.iter().map(|c| *c as u8).collect(); let negate = option_str(negate.as_deref()); let takes_value = field.takes_value(); + // The bound on one occurrence's values. A repeatable flag's bound counts occurrences + // instead, which no single token can decide, so that one stays a post-binding check. + let var_max = match field.var_max.filter(|_| *variadic) { + Some(max) => { + // Saturating, not `as`: on a 64-bit target a bound above `u32::MAX` would + // narrow, and `4294967296` narrows to zero — a limit of "none" read as a limit + // of "stop immediately". A variadic bounded above four billion is unbounded in + // every practical sense, so clamping says the same thing safely. + let max = u32::try_from(max).unwrap_or(u32::MAX); + quote!(::std::option::Option::Some(#max)) + } + None => quote!(::std::option::Option::None), + }; quote! { pub static #name: Flag = Flag { @@ -265,6 +278,7 @@ fn flag_table(i: usize, field: &Field) -> TokenStream { negate: #negate, takes_value: #takes_value, variadic: #variadic, + var_max: #var_max, global: #global, }; } @@ -286,12 +300,22 @@ fn arg_table(i: usize, field: &Field) -> TokenStream { } else { quote!(DoubleDash::Optional) }; + // A bound stops the variadic while binding, so the argument after it is reachable. + let var_max = match field.var_max.filter(|_| var) { + Some(max) => { + // Saturating, for the reason given on the flag above. + let max = u32::try_from(max).unwrap_or(u32::MAX); + quote!(::std::option::Option::Some(#max)) + } + None => quote!(::std::option::Option::None), + }; quote! { pub static #name: Arg = Arg { key: #key, name: #field_name, var: #var, + var_max: #var_max, double_dash: #double_dash, }; } @@ -1367,7 +1391,22 @@ fn post_binding(cli: &Cli) -> TokenStream { }, None => quote!(), }; - let max = match f.var_max { + // `var_max` is a binding limit for anything that *collects*: a variadic stops at it + // and the next field takes the rest, so a total above it cannot arise and this + // check would be unreachable. Worse than unreachable for a repeatable flag whose + // argument is variadic — each occurrence respects the limit, and the total across + // occurrences would fail a check the invocation never broke. + // + // What is left is the repeatable flag with a single-value argument, where the bound + // counts occurrences: nothing about one token can decide that, so it stays here. + let counts_occurrences = matches!( + &f.kind, + Kind::Flag { + variadic: false, + .. + } + ) && f.repeatable; + let max = match f.var_max.filter(|_| counts_occurrences) { Some(max) => quote! { if got > #max { return ::std::result::Result::Err( diff --git a/derive/src/lib.rs b/derive/src/lib.rs index cecd2c5aa..0c5392ce5 100644 --- a/derive/src/lib.rs +++ b/derive/src/lib.rs @@ -131,6 +131,7 @@ //! | `count` | count occurrences instead of collecting values | //! | `var` | the flag may be repeated, taking one value each time | //! | `variadic` | one occurrence keeps taking values, until a flag-like token or `--` | +//! | `var_max = n` | how many values a variadic takes before the next field gets the rest | //! | `global` | subcommands inherit the flag | //! | `env = "X"` | an environment variable that can supply the value | //! | `default = "x"` | the value when the command line does not supply one | diff --git a/derive/src/model.rs b/derive/src/model.rs index 5015e4963..adcd9edd0 100644 --- a/derive/src/model.rs +++ b/derive/src/model.rs @@ -281,38 +281,34 @@ impl Cli { Kind::Arg { double_dash_required, } => { - // A variadic takes everything left, so anything after it can never - // be filled — unless it requires a `--`, which is precisely what - // stops the variadic. mise declares that on `run`, `exec` and `git`: - // `[ARGS]…` for the words before the separator and `[-- ARGS_LAST]…` - // for the ones after. - if let Some(first) = variadic_arg.filter(|_| !double_dash_required) { + // A variadic takes every remaining word, so anything after it can + // never be filled — with two exceptions, both of which are something + // that stops the variadic. An argument only fillable after a `--`, + // because the separator ends the collecting; and any argument at all + // when the variadic is *bounded*, because it hands over the words past + // its bound. mise relies on the first on `run`, `exec` and `git`. + if let Some(first) = variadic_arg.filter(|_| !*double_dash_required) { return Err(dup( field.span, first, - "an argument after a variadic one can never be filled, \ - because the variadic takes every remaining word — unless it \ - is only fillable after a `--`, which stops the variadic", + "an argument after an unbounded variadic can never be filled, \ + because the variadic takes every remaining word — give the \ + variadic a `var_max`, or make this one fillable only after a \ + `--`, either of which stops it", )); } - // A command line has one `--`, so the exemption is good for one - // argument. A second one after the separator would be as unreachable - // as the first case, and there is no third place to put it. - // - // A `var_max` does not change this: a bound is a check on how many - // words a variadic ended up with, not a limit that hands the rest to - // the next argument — see `a-bound-does-not-stop-a-variadic` in the - // corpus, which also records that usage-lib disagrees. + // The separator comes once, so an unbounded variadic behind it holds + // the rest of the command line and nothing can follow. if let Some(first) = spent_separator { return Err(dup( field.span, first, - "nothing can follow a variadic that takes the words after a \ - `--`: it has both the rest of the command line and the only \ - separator there is", + "nothing can follow an unbounded variadic that takes the words \ + after a `--`: it has both the rest of the command line and \ + the only separator there is", )); } - if field.shape == Shape::Many { + if field.shape == Shape::Many && field.var_max.is_none() { if *double_dash_required { spent_separator = Some(field.span); } else { @@ -1335,8 +1331,8 @@ mod tests { "unhelpful message: {err}" ); - // Including a plain argument, which the variadic behind the separator would - // already have taken. + // Including a plain argument, which the unbounded variadic behind the separator + // would already have taken. let err = rejection( r#" struct Ex { @@ -1353,6 +1349,48 @@ mod tests { ); } + #[test] + fn a_bounded_variadic_lets_an_argument_follow_it() { + // The bound is what stops it, so what comes after is reachable — the rule the + // grammar now states and clap's `num_args` has always had. + cli(r#" + struct Ex { + #[usage(arg, var_max = 2)] + first: Vec, + #[usage(arg)] + rest: Option, + } + "#) + .expect("a bounded variadic should allow an argument after it"); + + // Unbounded, and the argument after it is unreachable. + let err = rejection( + r#" + struct Ex { + #[usage(arg)] + first: Vec, + #[usage(arg)] + rest: Option, + } + "#, + ); + assert!( + err.contains("unbounded variadic"), + "unhelpful message: {err}" + ); + + // And a bounded variadic behind the separator does not spend it either. + cli(r#" + struct Ex { + #[usage(arg, double_dash = "required", var_max = 1)] + first: Vec, + #[usage(arg)] + rest: Option, + } + "#) + .expect("a bounded variadic behind the separator should allow one after it"); + } + #[test] fn a_relationship_needs_a_flag_to_hold_between() { // The spec records these on a flag and has nowhere to put them on an argument, diff --git a/docs/spec/argv.md b/docs/spec/argv.md index d9451fb2a..ab02b1cf7 100644 --- a/docs/spec/argv.md +++ b/docs/spec/argv.md @@ -90,12 +90,13 @@ A flag needing a value that ends the command line is an error. A flag whose argument is variadic (`--include ...`) keeps taking values from one occurrence: it consumes following tokens until one is flag-like, or a -`--` arrives, or the command line ends. So `--include a b` gives it both, while -`--include a --force` gives it only `a`. +`--` arrives, or its `var_max` is reached, or the command line ends. So +`--include a b` gives it both, while `--include a --force` gives it only `a`. This is greedy, and a command that declares both a variadic flag and positionals will find the flag eating them. That is inherent to the feature rather than a -quirk of this grammar; the way to end the run explicitly is `--`. +quirk of this grammar; the ways to end the run are `--` and a `var_max`, which +stops the flag and leaves the following words to whatever comes next. A flag declared `var` without a variadic argument is the other shape: it takes one value per occurrence and may be repeated, so `--include a --include b` @@ -179,10 +180,12 @@ declaration order. Each argument takes one word, except a variadic (`var=#true`, or a trailing `...`), which collects every word still available. - A word offered when no argument can hold it is an error, not silently dropped. -- `var_min` and `var_max` are enforced. They are checks on how many words an argument - ended up with, not limits that stop it collecting: a bounded variadic still takes - every word available, so an argument after one is as unreachable as an argument after - an unbounded variadic. +- `var_max` is a limit rather than a check: a variadic stops once it is full and the next + argument takes the rest, which is what makes `[a]… [b]` fillable at all. An argument + after an _unbounded_ variadic still can never be filled. clap's `num_args` behaves this + way, and specs are commonly generated from clap commands. +- `var_min` is a check, since nothing about a word tells you a variadic will end up short. + It is enforced once the last token has been read. - An unfilled required argument is an error. An unfilled optional one is absent. Flags and words may interleave freely: `ex from -f to` fills the same arguments @@ -267,7 +270,7 @@ told apart mechanically. | `invalid_choice` | a value outside the declared `choices` | | `arg_requires_double_dash` | a `double_dash="required"` argument got a value too early | | `var_too_few` | fewer values than `var_min` | -| `var_too_many` | more values than `var_max` | +| `var_too_many` | more occurrences than a repeatable flag's `var_max` | | `conflicting_flags` | two flags declared to `conflict` were both given | Choices match exactly; case-insensitive matching would have to be declared rather @@ -305,8 +308,7 @@ Any implementation in any language can run these. In this repository, Each vector says which layer of a parser it is a question for. Most are `binding` — which token becomes which flag or argument — and a parser that reads argv is expected to answer all of those. The rest are `post-binding`: `required`, -`choices`, `env` fallback, defaults, `var_min`/`var_max`, `overrides`, and -`conflicts` are +`choices`, `env` fallback, defaults, `var_min`, `overrides`, and `conflicts` are decided once the last token has been read, and need to know a value's type, so a binding-only parser leaves them to the layer that owns the target struct.