Skip to content

feat(agents): humanize uncurated Databricks model ids with a label grammar - #7844

Merged
wpfleger96 merged 8 commits into
mainfrom
duncan/databricks-generative-labels
Sep 23, 2026
Merged

wpfleger96 merged 8 commits into
mainfrom
duncan/databricks-generative-labels

Conversation

@wpfleger96

@wpfleger96 wpfleger96 commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

New Databricks endpoints (Claude Opus 5.5, GPT-6 Astra/Sol, etc.) showed as raw FQNs in the model picker until someone added a curated corpus record for each. This adds a strict token-class grammar as a third label tier inside the Databricks label helpers (Rust model_capabilities.rs + TS modelCapabilities.ts): exact record → unique alias → generated label → raw id. In TS it fires only for an explicit databricks_v2 provider; providerless ids stay raw.

Grammar

Takes the last component of a UC FQN (or strips one wrapper: databricks-, goose-, kgoose-, builderbot-; an id that is neither stays raw), splits on -, and classifies each token as version, letter+digit version, size, trailing date (dropped), or word. Labels are built only from the id's own text, and the grammar never reinterprets a version. When in doubt it returns raw:

  • Known families only. The family (leading letters of the first token) must be in the manifest's existing label_family_tokens. Custom endpoints such as prod.ai.rag-agent-3 or builderbot-pr-reviews-v2 stay raw. Supporting a new vendor means adding one entry to that list, not a record per model.
  • Attached digits stay whole (gpt10 → GPT10). If the family already carries digits and the next token is a version (goose-qwen3-5-122b-a10b, system.ai.llama3-1-8b, system.ai.claude3-5-sonnet), the id stays raw; this is checked before any reorder.
  • A third consecutive number (gemini-3-7-1-flash) returns raw rather than splitting off a patch version.
  • Any non-ASCII input returns raw, checked before trimming or case folding.
  • Any unclassifiable token, or no version-bearing token (builderbot-pr-reviews), returns raw. (latest)/Preview never appear unless an exact record supplies them.

Special cases, each justified by an unseen-id shape rather than the corpus score:

  • meta-llama- → Llama: a namespace convention.
  • Claude numeric-first reorder: Goose names Claude endpoints claude-4-7-opus. Only 1–2 version tokens followed by opus/sonnet/haiku reorder (claude-5-6-opus → Claude Opus 5.6); anything else keeps its order (claude-5-6-thinking → Claude 5.6 Thinking). Both current numeric-first ids also have exact records.
  • Qwen compact decimal: Qwen's own naming qwen35 → Qwen3.5 when the second digit is nonzero.

Colliding labels in the picker

The same model is often served from several sources, so humanized labels collide (four rows are all GPT-5.5). disambiguateModelLabels in formatAgentModelLabel.ts suffixes only colliding Databricks labels with their source:

  • UC FQN: its catalog.schema, e.g. GPT-6 Astra (system.ai) / GPT-6 Astra (sandbox.public).
  • Workspace endpoint: its wrapper, e.g. GPT-5.5 (databricks) / GPT-5.5 (goose) / GPT-5.5 (kgoose).
  • Still colliding after that (e.g. a date variant in the same schema): the full id, e.g. Gemini 3.7 Flash (system.ai.gemini-3-7-flash-0925).

Unique labels, raw-id rows, the default row, and other providers are unchanged. It runs in getDiscoveredPersonaModelOptions (covering AgentDropdownSelect, PersonaModelCombobox, PersonaDropdownField, whose closed triggers read the option label) and in ModelPicker. Model search in AgentDropdownSelect and PersonaModelCombobox also matches the raw id, so searching system.ai finds those rows. Rust ModelEntry.name is unchanged.

What labels drive

Picker values, persistence, routing, catalog dedup, resolve() and capabilities all stay keyed on the raw id; two ids sharing a label both remain selectable. Labels do drive picker search (alongside the raw id), the workspace catalog's equal-time sort, and TeamIdentityCard's "Mixed models" summary, which compares lowercased base labels, so goose- and system.ai GPT-6 Astra read as one model there (correct, since they are the same model).

Tests

Shared fixtures in scripts/databricks-label-fixtures.json are replayed by both Rust and TS. Each row asserts its tier: exact rows match a record, alias rows resolve with generation off, generated rows return nothing with generation off (so a future alias or record cannot mask them), and raw rows return nothing either way. A curator guard requires that across all 57 exact records, the ids the grammar does not reproduce equal exactly the 10 curator-only ids. disambiguateModelLabels has unit tests for namespace, wrapper, and full-id-fallback collisions, untouched rows, other providers, and case-insensitive grouping. The defaults-picker e2e checks the suffixed rows and trigger, raw-id search, and that choosing the second uncurated FQN by its visible name persists its raw id and restores its label. An edit-agent e2e covers the existing-agent model dropdown (#edit-agent-model): both suffixed rows render, the system.ai row is chosen by name, and after save and reopen the agent stores the system.ai id and shows GPT-6 Astra (system.ai).

…ammar

New Databricks endpoints (Claude Opus 5.5, GPT-6 Astra/Sol) rendered as raw
FQNs in the model picker until someone added a curated record for each one.
A strict token-class grammar now labels any uncurated id after the exact and
unique-alias tiers miss; unparseable or versionless ids still show raw. It is
display-only and fires in TS only for an explicit databricks_v2 provider.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96
wpfleger96 requested a review from a team as a code owner September 23, 2026 16:15
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Sep 23, 2026
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Note: This is an automated, security-focused review generated by Codex.
Use it as a supplement to human review; false positives are possible.

Scope

  • Exact PR diff: 48884848f566d02c42ce07636636c4ad5f164c27...18f874ee2235aae6330672910d689fa3fd2b9aaa
  • Model: gpt-5.6-sol

💡 Click "edited" above to see earlier reviews for this PR.


Review Summary

Overall Risk: NONE

No concrete security, correctness, or reliability findings. The changes affect Databricks display labels and collision disambiguation; model IDs remain unchanged for selection, persistence, capability resolution, and routing.

Findings

No concrete security, correctness, or reliability findings were identified.

Notes

  • Review used read-only inspection only; repository scripts, builds, and tests were not executed as required.

Generated by Codex Security Review |
Requested by: @wpfleger96 |
Workflow run

Duncan and others added 2 commits September 23, 2026 13:04
Attached family digits stay whole (qwen compact decimal excepted), the Claude
reorder fires only for a tier word, a third consecutive number and non-ASCII
input return raw. Shared fixtures now assert each row's tier with generation off.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Mirror Rust: only a version or letter-version token consuming a minor can be
followed by a refused third number; a word before two numbers is unaffected.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Sep 23, 2026
Comment thread crates/buzz-agent/src/databricks_label_grammar.rs
Comment thread crates/buzz-agent/src/databricks_label_grammar.rs
Duncan and others added 3 commits September 23, 2026 15:45
…digit versions

Custom endpoints such as rag-agent-3 no longer get invented names: the grammar
now requires a label_family_tokens family. qwen3-5 / llama3-1 style ids stay raw
instead of splitting an attached version, checked before the Claude reorder.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The same model served from several catalogs or wrappers showed identical picker
rows. Colliding labels now get (catalog.schema) or (wrapper), falling back to the
full id, and model search matches the raw id.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…erative-labels

* origin/main:
  fix: route databricks claude fqns to anthropic messages (#7829)
  feat(relay): add opt-in newest-first thread windows (#7823)
  refactor: move agent Git bootstrap into ACP harness (#7819)
  Use worker snapshots for relay storage metrics (#7845)
  fix(hooks): strip repo-local git env from pre-push test lanes (#7841)

Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Changes requested

One P2 blocks this head: generated labels hide which namespace-distinct model target the user is selecting. Raw IDs correctly survive discovery, selection, persistence and routing, but that does not make identical picker rows distinguishable. The inline finding independently corroborates the existing collision report.

The separate attached-digit finding is P3/nonblocking: both interpreters agree, but can display ambiguous version grouping instead of taking the conservative raw fallback. It does not expand the blocking exit criteria.

Exit criterion: keep colliding choices and the selected target visibly/accessibly distinguishable, preserve raw-ID selection/persistence, and bind regressions to user-visible selection/search in Defaults and the existing-agent picker. Collision-only qualification or conservative fallback is sufficient; a model-family allowlist is not required by this finding.

Reviewed head 5f130546aabb6dd8479174e07049915fdf735543, exact base 3b2e50b15c6afba3ff3b8fbe1bdf0a7a69d29f7b, and the ten-file feature delta from their real merge-base 0ef7a2222ef4e9b221f310ac60ca6a535d9c840d. Source coverage includes both label interpreters, curated/provider precedence, catalog filtering/dedup/order, Defaults and existing-agent selection, raw-ID persistence/routing/capabilities, card/team consumers, and shared-fixture/E2E binding. The base-only hook commit is unrelated.

Validation is source-only on the pinned Blox host; no checkout, install, build, test or live reproduction. Existing exact-head CI snapshot: 53 successful, 27 skipped, one cancelled security-review check; no reruns or monitoring. This is not approval.

Comment thread crates/buzz-agent/src/databricks_label_grammar.rs
Comment thread crates/buzz-agent/src/databricks_label_grammar.rs
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Sep 23, 2026
Carl's review asks that e2e rows be chosen by accessible name and that the existing-agent picker be covered, not only Defaults.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@github-actions github-actions Bot added codex-security-review-current The posted Codex security review matches its recorded range. and removed codex-security-review-current The posted Codex security review matches its recorded range. labels Sep 23, 2026
…test

The inherited-default trigger now uses the same disambiguated row label as an explicit model, and the search query only occurs in the raw id so it falsifies removal of that filter branch.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Sep 23, 2026
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Sep 23, 2026
@wpfleger96
wpfleger96 merged commit d01e5f8 into main Sep 23, 2026
81 checks passed
@wpfleger96
wpfleger96 deleted the duncan/databricks-generative-labels branch September 23, 2026 21:12
bradseiler pushed a commit that referenced this pull request Sep 23, 2026
…in-gate

* origin/main: (30 commits)
  feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844)
  fix: route databricks claude fqns to anthropic messages (#7829)
  feat(relay): add opt-in newest-first thread windows (#7823)
  refactor: move agent Git bootstrap into ACP harness (#7819)
  Use worker snapshots for relay storage metrics (#7845)
  fix(hooks): strip repo-local git env from pre-push test lanes (#7841)
  fix(mobile-infra): render push grant lifetimes as decimal in chart 0.3.2 (#7820)
  fix(agent): preserve Databricks Opus UC reasoning and tool continuation (#7840)
  feat(canvas): add version history with atomic restore (#6780)
  chore(release): release Buzz Desktop version 0.5.24 (#7817)
  test(desktop): stabilize unread and audio release smoke fixtures (#7821)
  fix(desktop): remember Inbox unread-only choice (#7672)
  feat(mobile-infra): support development App Attest (#7744)
  docs(nip-fi): clarify federated identity amendments (#7803)
  fix(desktop): refresh channels after access-revoked closure (#7784)
  fix(desktop): bound startup request bursts and recover quota refusals (#7790)
  fix(audit): frame hash inputs with TLV (#7492)
  fix(admin): allow cold storage worker DB startup (#7770)
  feat(relay): add admin HTTP routes for member restriction management (#7302)
  fix(relay): fire kick live side effects at convergence; persist target; fence re-add race with held lock (#7298)
  ...

Signed-off-by: coder 1 <a93f3b1decd199cec83848f116ff60c20776cdd03861b9bba2610acae7c9eeeb@buzz.block.builderlab.xyz>
michaelneale added a commit that referenced this pull request Sep 24, 2026
* origin/main:
  feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844)
  fix: route databricks claude fqns to anthropic messages (#7829)
  feat(relay): add opt-in newest-first thread windows (#7823)
  refactor: move agent Git bootstrap into ACP harness (#7819)
  Use worker snapshots for relay storage metrics (#7845)
  fix(hooks): strip repo-local git env from pre-push test lanes (#7841)
  fix(mobile-infra): render push grant lifetimes as decimal in chart 0.3.2 (#7820)
  fix(agent): preserve Databricks Opus UC reasoning and tool continuation (#7840)
  feat(canvas): add version history with atomic restore (#6780)
  chore(release): release Buzz Desktop version 0.5.24 (#7817)
  test(desktop): stabilize unread and audio release smoke fixtures (#7821)
  fix(desktop): remember Inbox unread-only choice (#7672)
  feat(mobile-infra): support development App Attest (#7744)
  docs(nip-fi): clarify federated identity amendments (#7803)
  fix(desktop): refresh channels after access-revoked closure (#7784)
  fix(desktop): bound startup request bursts and recover quota refusals (#7790)
  fix(audit): frame hash inputs with TLV (#7492)
  fix(admin): allow cold storage worker DB startup (#7770)

Signed-off-by: Michael Neale <michael.neale@gmail.com>
wpfleger96 pushed a commit that referenced this pull request Sep 24, 2026
…n-surface

* origin/main:
  fix(ci): consume the published MinIO image (#7870)
  fix(mobile): converge sidebar managers on relay head with resume re-read (#7806)
  fix(ci): bootstrap the reusable MinIO image in GHCR (#7869)
  Discover alternate Buzz ACP commands (#6948)
  fix(hooks): surface nextest failures and stale pnpm deps in pre-push (#7850)
  feat(acp): run one prepared task from a file or stdin (#7851)
  Fix mobile heart and warning emoji with native font fallback (#7842)
  chore(mesh): upgrade MeshLLM to 0.76.2 (#7559)
  feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844)
  fix: route databricks claude fqns to anthropic messages (#7829)
  feat(relay): add opt-in newest-first thread windows (#7823)
  refactor: move agent Git bootstrap into ACP harness (#7819)
  Use worker snapshots for relay storage metrics (#7845)
  fix(hooks): strip repo-local git env from pre-push test lanes (#7841)
  fix(mobile-infra): render push grant lifetimes as decimal in chart 0.3.2 (#7820)

Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
wpfleger96 pushed a commit that referenced this pull request Sep 24, 2026
…rcement

* origin/main:
  fix(ci): consume the published MinIO image (#7870)
  fix(mobile): converge sidebar managers on relay head with resume re-read (#7806)
  fix(ci): bootstrap the reusable MinIO image in GHCR (#7869)
  Discover alternate Buzz ACP commands (#6948)
  fix(hooks): surface nextest failures and stale pnpm deps in pre-push (#7850)
  feat(acp): run one prepared task from a file or stdin (#7851)
  Fix mobile heart and warning emoji with native font fallback (#7842)
  chore(mesh): upgrade MeshLLM to 0.76.2 (#7559)
  feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844)

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

# Conflicts:
#	Cargo.lock
#	crates/buzz-relay/Cargo.toml
wpfleger96 pushed a commit that referenced this pull request Sep 24, 2026
…c-agent-commit-identity

* origin/main:
  fix(mobile): keep retired sections manager out of successor cache (#7873)
  Select one feature flag provider at compile time (#7677)
  chore(release): release Buzz Desktop version 0.5.25 (#7867)
  fix(ci): consume the published MinIO image (#7870)
  fix(mobile): converge sidebar managers on relay head with resume re-read (#7806)
  fix(ci): bootstrap the reusable MinIO image in GHCR (#7869)
  Discover alternate Buzz ACP commands (#6948)
  fix(hooks): surface nextest failures and stale pnpm deps in pre-push (#7850)
  feat(acp): run one prepared task from a file or stdin (#7851)
  Fix mobile heart and warning emoji with native font fallback (#7842)
  chore(mesh): upgrade MeshLLM to 0.76.2 (#7559)
  feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844)

Signed-off-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>

# Conflicts:
#	Cargo.lock
TheSentinel454 added a commit that referenced this pull request Sep 24, 2026
…undation-local

* origin/main: (50 commits)
  feat(desktop): relay admin console for the /api/admin/v1 operator surface (#4768)
  fix(mobile): keep retired sections manager out of successor cache (#7873)
  Select one feature flag provider at compile time (#7677)
  chore(release): release Buzz Desktop version 0.5.25 (#7867)
  fix(ci): consume the published MinIO image (#7870)
  fix(mobile): converge sidebar managers on relay head with resume re-read (#7806)
  fix(ci): bootstrap the reusable MinIO image in GHCR (#7869)
  Discover alternate Buzz ACP commands (#6948)
  fix(hooks): surface nextest failures and stale pnpm deps in pre-push (#7850)
  feat(acp): run one prepared task from a file or stdin (#7851)
  Fix mobile heart and warning emoji with native font fallback (#7842)
  chore(mesh): upgrade MeshLLM to 0.76.2 (#7559)
  feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844)
  fix: route databricks claude fqns to anthropic messages (#7829)
  feat(relay): add opt-in newest-first thread windows (#7823)
  refactor: move agent Git bootstrap into ACP harness (#7819)
  Use worker snapshots for relay storage metrics (#7845)
  fix(hooks): strip repo-local git env from pre-push test lanes (#7841)
  fix(mobile-infra): render push grant lifetimes as decimal in chart 0.3.2 (#7820)
  fix(agent): preserve Databricks Opus UC reasoning and tool continuation (#7840)
  ...
Signed-off-by: tornquist <tornquist@squareup.com>
TheSentinel454 added a commit that referenced this pull request Sep 24, 2026
…t/osc-event-write-chokepoint

* commit 'a6a3032e446e6e66e8e41a229ef655ea79f36202': (50 commits)
  feat(desktop): relay admin console for the /api/admin/v1 operator surface (#4768)
  fix(mobile): keep retired sections manager out of successor cache (#7873)
  Select one feature flag provider at compile time (#7677)
  chore(release): release Buzz Desktop version 0.5.25 (#7867)
  fix(ci): consume the published MinIO image (#7870)
  fix(mobile): converge sidebar managers on relay head with resume re-read (#7806)
  fix(ci): bootstrap the reusable MinIO image in GHCR (#7869)
  Discover alternate Buzz ACP commands (#6948)
  fix(hooks): surface nextest failures and stale pnpm deps in pre-push (#7850)
  feat(acp): run one prepared task from a file or stdin (#7851)
  Fix mobile heart and warning emoji with native font fallback (#7842)
  chore(mesh): upgrade MeshLLM to 0.76.2 (#7559)
  feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844)
  fix: route databricks claude fqns to anthropic messages (#7829)
  feat(relay): add opt-in newest-first thread windows (#7823)
  refactor: move agent Git bootstrap into ACP harness (#7819)
  Use worker snapshots for relay storage metrics (#7845)
  fix(hooks): strip repo-local git env from pre-push test lanes (#7841)
  fix(mobile-infra): render push grant lifetimes as decimal in chart 0.3.2 (#7820)
  fix(agent): preserve Databricks Opus UC reasoning and tool continuation (#7840)
  ...

Signed-off-by: tornquist <tornquist@squareup.com>

# Conflicts:
#	crates/buzz-db/src/store/event.rs
brow added a commit that referenced this pull request Sep 25, 2026
…ction

* origin/main: (21 commits)
  docs(vision): add /buzz/v1 read endpoints to the protocol contract (#7879)
  🤖 fix(justfile): point just staging at the current staging relay (#7881)
  fix(relay-admin): make thread deletions atomic and fence expired action leases under row lock (#7853)
  feat(desktop): relay admin console for the /api/admin/v1 operator surface (#4768)
  fix(mobile): keep retired sections manager out of successor cache (#7873)
  Select one feature flag provider at compile time (#7677)
  chore(release): release Buzz Desktop version 0.5.25 (#7867)
  fix(ci): consume the published MinIO image (#7870)
  fix(mobile): converge sidebar managers on relay head with resume re-read (#7806)
  fix(ci): bootstrap the reusable MinIO image in GHCR (#7869)
  Discover alternate Buzz ACP commands (#6948)
  fix(hooks): surface nextest failures and stale pnpm deps in pre-push (#7850)
  feat(acp): run one prepared task from a file or stdin (#7851)
  Fix mobile heart and warning emoji with native font fallback (#7842)
  chore(mesh): upgrade MeshLLM to 0.76.2 (#7559)
  feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844)
  fix: route databricks claude fqns to anthropic messages (#7829)
  feat(relay): add opt-in newest-first thread windows (#7823)
  refactor: move agent Git bootstrap into ACP harness (#7819)
  Use worker snapshots for relay storage metrics (#7845)
  ...

Signed-off-by: Tom Brow <tomb@block.xyz>

This branch was successfully deployed

1 active deployment
codex-review — 18f874ee Deployed Sep 23, 2026 by wpfleger96 via Run Codex Security Review #5366
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-security-review-current The posted Codex security review matches its recorded range.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants