Repository navigation
descriptor: resolve a name shared by two fields deterministically - #818
AkshayShah03 wants to merge 3 commits into
Conversation
Where JSON-name conflicts are allowed (proto2, or
deprecated_legacy_json_field_conflicts), a proto name and another
field's JSON name can be equal. Both lived in one index sorted with an
unstable sort and searched with a plain binary search, so
field_by_name and the reflective JSON parser resolved such a name to
an arbitrary field: for proto2 fields fooBar = 1 and foo_bar = 2,
field_by_name("fooBar") returned foo_bar.
Give equal names a total order: a field's own proto name before
another field's JSON name, then declaration order. field_by_name takes
the first match, the field with that proto name as C++'s
FindFieldByName returns it. The reflective JSON parser uses a new
crate-internal field_by_json_key that resolves JSON name first (the
first declared field), then the proto name, as protobuf-go's
protojson does.
|
All contributors have signed the CLA ✍️ ✅ |
|
I have read the CLA Document and I hereby sign the CLA |
mangeshraut712
left a comment
There was a problem hiding this comment.
Making the order total (proto name before another field's JSON name, then declaration order) and switching to partition_point gives deterministic results without adding a second index. field_by_json_key's "min index among exact json_name matches, else the proto-name entry" also matches how protobuf-go's lazily built byJSON map behaves (first declared wins) before protojson falls back to ByTextName. The tests cover both declaration orders and the custom-json_name case.
| } | ||
| } else { | ||
| md.field_by_name(&key) | ||
| md.field_by_json_key(&key) |
There was a problem hiding this comment.
A related gap, possibly for a follow-up. The serializer still emits fd.json_name for each field (map.serialize_entry(&fd.json_name, …) around line 131), so a proto2 message with both fooBar = 1 and foo_bar = 2 set serialises to two "fooBar" keys. On the way back in, both keys now resolve deterministically to field 1, so the second one hits the seen_fields check and from_json returns duplicate field "fooBar". That means to_json → from_json fails for any such message with both fields populated. It was already broken before this PR (just non-deterministically), so it's not a regression. A test that documents the intended behaviour would help, whether that's erroring at serialisation, emitting the proto name for the later field, or accepting the failure as protojson parity.
|
Thanks for the review. I confirmed the round-trip gap. With proto2 I'd keep it out of this PR. The fix is a choice about the serializer's output, not about lookup, and the options look like separate decisions for the maintainers:
If that sounds right, I'll open a follow-up issue with the repro and those options. I can also add a test here that pins the current error, if a maintainer would like that in this PR. |
agammann
left a comment
There was a problem hiding this comment.
The new conflict lookup tests all construct proto2 messages. The existing deprecated_legacy_json_field_conflicts_opts_a_proto3_message_out test only checks that a proto3 descriptor is accepted, though the new lookup docs also promise deterministic behavior for that opt-out. Could it assert field_by_name and the reflective JSON parser resolve the shared name in the expected order?
…opt-out Cover a proto3 message with deprecated_legacy_json_field_conflicts in both declaration orders, for field_by_name and the reflective JSON parser, as the proto2 tests do.
|
Good catch, thanks. 04a8058 adds
The test fails on main and passes here. The proto2 helper now delegates to a |
|
I rechecked head |
Closes #817.
Where JSON-name conflicts are allowed (proto2, or
deprecated_legacy_json_field_conflicts), a proto name and another field's JSON name can be equal. Both lived in one index that was sorted withsort_unstable_byand searched withbinary_search_by, sofield_by_nameand the reflective JSON parser resolved such a name to an arbitrary field.Change
pool.rs): equal names now have a total order. A field's own proto name sorts before another field's JSON name, and ties are then broken by declaration order. A field's own proto and JSON entries never share a key, because the JSON entry is only pushed when it differs from the proto name.MessageDescriptor::field_by_name(desc.rs): returns the first match. That is the field with that proto name, as C++FindFieldByNamereturns it, or the first declared field among those sharing a JSON name. It is still a singlepartition_point.field_by_json_key, behindreflect+jsonand used byreflect/json.rs: resolves an object key JSON name first, taking the first declared field, and otherwise falls back to the proto name, as protobuf-go's protojson does. The common case of a single matching entry returns immediately.Tests (
buffa-descriptor/tests/pool_e2e.rs)field_by_namewithfooBar/foo_barin both declaration orders → the field with that proto name.field_by_namewith two fields sharing only a JSON name, in both orders → the first declared.{"fooBar": 5}in both orders → the first declared field with JSON namefooBar.json_namecase (fooBarwith JSON namecustom, andxwith JSON namefooBar): JSON takesx, whilefield_by_nametakesfooBar.Checks
cargo test --workspace: 4322 passed (4317 on main).cargo test -p buffa-descriptor --all-features: 570 passed.cargo clippy -D warningsis clean for the workspace and forbuffa-descriptorwithreflect,json,reflect,jsonand--all-features.task docpasses.The open question from #817, whether to add separate public proto-name and JSON-name lookups, is left out of this PR.
I used Claude Code to help find this bug and write the change. I reviewed and tested it.