Skip to content

resolve_query's decomposition filters bypass the tool-argument repair pipeline #2185

Closed

Description

@malibio

Summary

resolve_query builds its filters from an internal decomposition LLM call and deserializes them straight into AgentFilterItem, never passing through the tool-argument repair pipeline. So none of the four repairs — repair_over_quoted_keys, repair_leaked_special_token_keys, repair_spliced_object_values, repair_scalar_in_operator_values — cover that path.

packages/agent/src/local_agent/tools.rs ~2094:

let filters: Vec<query_ops::AgentFilterItem> = resolved
    .get("filters")
    .and_then(|v| v.as_array())
    .map(|arr| {
        arr.iter()
            .filter_map(|f| match serde_json::from_value(f.clone()) {
                Ok(item) => Some(item),
                Err(e) => { tracing::warn!(...); None }
            })
            .map(|item| coerce_filter_value_to_field_type(item, schema.as_ref()))

The repairs run at the agent loop's tool-call parse boundary (repair_tool_call_arguments, and the execution backstop via repair_parsed_tool_arguments). This is a nested sub-call whose output is parsed locally, so it never reaches either.

Why this is low priority

Surfaced during review of #2183 and deliberately not fixed there — it is orthogonal to that change and pre-existing, not a regression from it. Two things bound the impact:

  • The decomposition prompt never instructs in. It only ever asks for equals and date comparison operators (tools.rs ~2020-2024), so the specific malformation search_nodes 'in' operator gets a comma-joined string, not an array: filter silently matches nothing #2182 is about is not reachable through this path today. That is a property of a prompt, though, not a guarantee — a future edit widening the decomposition's operator vocabulary would open it without anything failing.
  • Unparseable filters are dropped, not fatal. filter_map skips them with a tracing::warn! rather than erroring the turn. Note this cuts both ways: a dropped filter can silently turn a would-be unique match into a multiple_matches, which the existing comment there already flags as a real hazard.

Where the fix belongs

coerce_filter_value_to_field_type (tools.rs:289) is the natural seam — it is already the local normalization step on this path, already schema-aware, and already coerces string→number and string→boolean. Adding the in-array case there is a small, consistent extension.

Two other shapes to consider at the same time, since they share the seam: AgentFilterItem uses deny_unknown_fields, so a decomposition emitting an over-quoted or leaked-token key is dropped entirely rather than repaired, and that drop is exactly the silent-narrowing hazard above.

The alternative — routing the decomposition output through repair_parsed_tool_arguments before deserializing — is more uniform but drags in repairs shaped for the model's tool-call quirks, which may not be the decomposition sub-call's failure modes. Worth measuring which malformations this path actually produces before choosing; #2182's own lesson was that the assumed cause and the measured one differed.

Acceptance criteria

  • Decide, with evidence, whether the decomposition sub-call actually produces malformed filter shapes — dump its raw output across the existing golden corpus before writing a fix
  • If it does: normalize at coerce_filter_value_to_field_type, covering at minimum the scalar-in case
  • If it does not: record that as a negative result and close, rather than adding a repair for a shape nothing emits
  • Either way, note the coverage boundary at the resolve_query filter-parsing site so the next reader does not assume the repair pipeline protects it

Context

Activity

  1. malibio commented on Aug 20, 2026

    @malibio
    CollaboratorAuthor

    Baseline (worktree issue-2185-resolve-query-filter-repair): Frontend 227 test files passed, 4794 tests passed.

  2. self-assigned this
    on Aug 20, 2026
  3. malibio commented on Aug 20, 2026

    @malibio
    CollaboratorAuthor

    Measurement result — the predicted malformation does not occur, a different one does

    Ran the first acceptance criterion before writing any fix: drove the real exec_resolve_query over a corpus of 11 phrasings × 3 reps (33 decompositions, 21 emitted filters) against the locked E4B model, capturing each decomposition's raw text through the existing NODESPACE_PROMPT_DUMP hook — which already names resolve_query as a covered caller, so no new production instrumentation was needed and the prompt measured is the one production builds.

    Harness: packages/agent/tests/live_resolve_query_decomposition_shapes.rs (#[ignore]d, loads the GGUF).

    Tally

    well_formed:            21
    dropped_unknown_field:   0
    over_quoted_key:         0
    leaked_token_key:        0
    scalar_in_value:         0     <- the shape #2182/#2183 repaired
    used_in_operator:        6     <- but `in` IS emitted
    unknown_property:        0
    coerced_scalar_type:     0
    total_filters:          21
    unparseable_output:      3     <- see below
    total_decompositions:   33
    

    Finding 1 — the issue's stated bound is wrong, but its conclusion holds

    The issue reasons that the scalar-in malformation is unreachable here because "the decomposition prompt never instructs in". The prompt doesn't, and the model emits in anyway — 6 of 21 filters, on multi-value enum phrasings:

    {"property": "status", "operator": "in", "value": ["checked_out", "in_repair"]}

    So the bound as written does not hold. But every single one carried a proper JSON array, never a comma-joined scalar — scalar_in_value: 0. The conclusion (this path doesn't produce #2182's shape) survives; the reason given for it does not. Worth correcting in place, because the stated reason is what a future reader would rely on when widening the prompt's operator vocabulary — and it would not protect them.

    This also lines up with #2182's own finding: production's declaration works, and the comma-joined string came from a corpus case rather than from the model.

    Likewise zero over-quoted keys, zero leaked special-token keys, zero deny_unknown_fields drops, and zero type-coercions needed. On this path, on this model, all four repairs plus the existing coercion are inert.

    Finding 2 — a real, deterministic defect the issue did not anticipate

    unparseable_output: 3 is not noise. All three are the same request, 3/3 reps, byte-identical — "Show me the overdue ones":

    {"query": "", "filters": [{"type":"property","operator":"lt","property":"due_date","value":2026-08-19}]}

    value:2026-08-19 — a bare unquoted date. JSON parses 2026 as a number and then chokes on -08-19:

    Expecting ',' delimiter: line 1 column 96
    

    The consequence is a full-resolution failure, not a dropped filter. serde_json::from_str fails on the whole object, exec_resolve_query falls back to {"query":"","filters":[]}, and the search degrades to a bare type listing. Observed end to end: the request returned multiple_matches over both invoices — including the one that is not overdue — instead of resolving the single overdue one. That is the silent-narrowing hazard the issue flags, arriving through a wider door than the one it was watching.

    Two things make this worse than the malformations the issue predicted:

    • It is upstream of every repair. All four repairs and coerce_filter_value_to_field_type operate on an already-parsed Value. This fails at from_str. Routing the decomposition output through repair_parsed_tool_arguments — the issue's "more uniform" alternative — would not catch it either, for the same reason.
    • The proposed seam cannot reach it. coerce_filter_value_to_field_type runs per-filter-item, and there are no filter items — parsing failed before any existed.

    It is also directly provoked by the prompt: the decomposition instructs resolving relative dates ("overdue") to "a concrete YYYY-MM-DD value", and YYYY-MM-DD unquoted is exactly what it produced.

    Recommendation

    Answer the acceptance criteria as:

    • AC1 (decide with evidence): done, above. The predicted shapes: measured absent. A different shape: measured present, deterministic.
    • AC2 (normalize at coerce_filter_value_to_field_type, covering scalar-in): decline. Nothing on this path emits that shape, and search_nodes 'in' operator gets a comma-joined string, not an array: filter silently matches nothing #2182's own lesson was not to add a repair for a shape nothing emits. Adding it would be the negative-result mistake AC3 exists to prevent.
    • AC3 (record the negative result): done for the four repairs — all inert here.
    • AC4 (note the coverage boundary): yes, and now with a concrete reason: the boundary is not just "repairs don't run here", it is "repairs couldn't help here, because this path's real failure is a parse failure".

    And add the one repair the evidence actually supports: quote bare YYYY-MM-DD date literals in the decomposition's raw text before serde_json::from_str. Narrow, mechanically unambiguous (an unquoted YYYY-MM-DD is not valid JSON under any reading, so nothing legitimate is rewritten), applied only to this sub-call's output, and it converts a total-resolution failure into a working date filter.

    Proceeding on that basis unless redirected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions