Skip to content

descriptor: resolve a name shared by two fields deterministically - #818

Open
AkshayShah03 wants to merge 3 commits into
anthropics:mainfrom
AkshayShah03:fix/field-by-name-proto-name-first
Open

AkshayShah03 wants to merge 3 commits into
anthropics:mainfrom
AkshayShah03:fix/field-by-name-proto-name-first

Conversation

@AkshayShah03

Copy link
Copy Markdown

Closes #817.

Where JSON-name conflicts are allowed (proto2, or deprecated_legacy_json_field_conflicts), a proto name and another field's JSON name can be equal. Both lived in one index that was sorted with sort_unstable_by and searched with binary_search_by, so field_by_name and the reflective JSON parser resolved such a name to an arbitrary field.

Change

  • Index order (pool.rs): equal names now have a total order. A field's own proto name sorts before another field's JSON name, and ties are then broken by declaration order. A field's own proto and JSON entries never share a key, because the JSON entry is only pushed when it differs from the proto name.
  • MessageDescriptor::field_by_name (desc.rs): returns the first match. That is the field with that proto name, as C++ FindFieldByName returns it, or the first declared field among those sharing a JSON name. It is still a single partition_point.
  • New crate-internal field_by_json_key, behind reflect + json and used by reflect/json.rs: resolves an object key JSON name first, taking the first declared field, and otherwise falls back to the proto name, as protobuf-go's protojson does. The common case of a single matching entry returns immediately.
  • No conflict: messages without such a conflict resolve exactly as before, and no public API is added.

Tests (buffa-descriptor/tests/pool_e2e.rs)

  • field_by_name with fooBar/foo_bar in both declaration orders → the field with that proto name.
  • field_by_name with two fields sharing only a JSON name, in both orders → the first declared.
  • JSON {"fooBar": 5} in both orders → the first declared field with JSON name fooBar.
  • A custom json_name case (fooBar with JSON name custom, and x with JSON name fooBar): JSON takes x, while field_by_name takes fooBar.
  • Fallback to the proto name for a JSON key.
  • Three of the five fail on main; the fallback and custom cases pin behaviour.

Checks

  • cargo test --workspace: 4322 passed (4317 on main).
  • cargo test -p buffa-descriptor --all-features: 570 passed.
  • cargo clippy -D warnings is clean for the workspace and for buffa-descriptor with reflect, json, reflect,json and --all-features.
  • task doc passes.
  • Changelog fragment added.
  • The two pre-commit reviews from AGENTS.md (correctness and API ergonomics) were run. Their Medium finding, that one shared tie-break can't match protojson for JSON keys, is why the JSON parser has its own lookup.

The open question from #817, whether to add separate public proto-name and JSON-name lookups, is left out of this PR.

I used Claude Code to help find this bug and write the change. I reviewed and tested it.

Where JSON-name conflicts are allowed (proto2, or
deprecated_legacy_json_field_conflicts), a proto name and another
field's JSON name can be equal. Both lived in one index sorted with an
unstable sort and searched with a plain binary search, so
field_by_name and the reflective JSON parser resolved such a name to
an arbitrary field: for proto2 fields fooBar = 1 and foo_bar = 2,
field_by_name("fooBar") returned foo_bar.

Give equal names a total order: a field's own proto name before
another field's JSON name, then declaration order. field_by_name takes
the first match, the field with that proto name as C++'s
FindFieldByName returns it. The reflective JSON parser uses a new
crate-internal field_by_json_key that resolves JSON name first (the
first declared field), then the proto name, as protobuf-go's
protojson does.
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@AkshayShah03

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Nov 9, 2026

@mangeshraut712 mangeshraut712 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making the order total (proto name before another field's JSON name, then declaration order) and switching to partition_point gives deterministic results without adding a second index. field_by_json_key's "min index among exact json_name matches, else the proto-name entry" also matches how protobuf-go's lazily built byJSON map behaves (first declared wins) before protojson falls back to ByTextName. The tests cover both declaration orders and the custom-json_name case.

}
} else {
md.field_by_name(&key)
md.field_by_json_key(&key)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A related gap, possibly for a follow-up. The serializer still emits fd.json_name for each field (map.serialize_entry(&fd.json_name, …) around line 131), so a proto2 message with both fooBar = 1 and foo_bar = 2 set serialises to two "fooBar" keys. On the way back in, both keys now resolve deterministically to field 1, so the second one hits the seen_fields check and from_json returns duplicate field "fooBar". That means to_json → from_json fails for any such message with both fields populated. It was already broken before this PR (just non-deterministically), so it's not a regression. A test that documents the intended behaviour would help, whether that's erroring at serialisation, emitting the proto name for the later field, or accepting the failure as protojson parity.

@AkshayShah03

Copy link
Copy Markdown
Author

Thanks for the review. I confirmed the round-trip gap. With proto2 fooBar = 1 and foo_bar = 2 both set, to_json() gives {"fooBar":1,"fooBar":2}, and from_json on that returns duplicate field "fooBar" on message .... Main (1047169) gives the same output and the same error, so it isn't a regression from this PR. On main both keys happened to resolve to field 2 instead of field 1.

I'd keep it out of this PR. The fix is a choice about the serializer's output, not about lookup, and the options look like separate decisions for the maintainers:

  • reject at serialization time,
  • write the later field under its proto name, or
  • keep the current output and document it.

If that sounds right, I'll open a follow-up issue with the repro and those options. I can also add a test here that pins the current error, if a maintainer would like that in this PR.

@agammann agammann left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new conflict lookup tests all construct proto2 messages. The existing deprecated_legacy_json_field_conflicts_opts_a_proto3_message_out test only checks that a proto3 descriptor is accepted, though the new lookup docs also promise deterministic behavior for that opt-out. Could it assert field_by_name and the reflective JSON parser resolve the shared name in the expected order?

…opt-out

Cover a proto3 message with deprecated_legacy_json_field_conflicts in
both declaration orders, for field_by_name and the reflective JSON
parser, as the proto2 tests do.
@AkshayShah03

Copy link
Copy Markdown
Author

Good catch, thanks. 04a8058 adds proto3_legacy_json_conflict_opt_out_resolves_shared_names_deterministically. It builds a proto3 message with deprecated_legacy_json_field_conflicts = true in both declaration orders and checks that:

  • field_by_name("fooBar") returns field 1, the one with that proto name;
  • field_by_name("foo_bar") returns field 2;
  • {"fooBar": 5} sets the first declared field with JSON name fooBar.

The test fails on main and passes here. The proto2 helper now delegates to a pool_with_message(syntax, options, fields) helper so both tests share the setup. buffa-descriptor with --all-features: 571 passed. Clippy is clean for the default, reflect, json, reflect,json and --all-features builds.

@agammann

Copy link
Copy Markdown

I rechecked head 04a8058. The new proto3 legacy-conflict test exercises both declaration orders with the opt-out enabled and checks both lookup paths. That covers the case I asked about. I reviewed the diff; I did not run tests locally. AI-assisted review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

descriptor: field_by_name and the reflective JSON parser resolve a name shared by two fields to an arbitrary one

3 participants