Repository navigation
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
This was referenced Nov 13, 2026
Author
|
recheck |
vanderbr
force-pushed
the
decode-hooks
branch
from
November 17, 2026 14:30
512ddd5 to
a04ac69
Compare
vanderbr
added a commit
to reallyme/buffa
that referenced
this pull request
Nov 18, 2026
Merge feature/native-adapters at ce3fd6e and port the published corrections from anthropics#825 (a04ac69) and anthropics#828 (8ffbf8a). Keep decode-hook windows bounded after fallible conversion, reject length prefixes that cross message boundaries, and select test maps from the runtime configuration. Preserve fork cleanup, compiler requirements, package versions and dependency pins.
Let custom bytes and string representations decode directly from bounded field payloads. Keep hooks within their message and stream frame, including entry points that dispatch to handwritten merge overrides. Preserve borrowed decoding and allocation-sharing defaults.
vanderbr
force-pushed
the
decode-hooks
branch
from
November 18, 2026 23:27
a04ac69 to
316b2b8
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #822.
A custom
ProtoBytesorProtoStringrepresentation builds itself fromWirePayloadinfrom_wire. Fragmented input first passes through an ordinary temporaryBytes, so a representation that needs to control every copy cannot protect that allocation. The new overridabledecode_length_delimited(&mut FieldPayload<impl Buf>)lets it copy directly into its own storage.FieldPayloadimplementsBuf, includingcopy_to_bytesand, withstd,chunks_vectored. Its underlying buffer and limit are private. The decoder validates the length before invoking the hook, and skips unread field bytes on a successful return. Existing implementations inherit defaults that preservefrom_wire,PREFERS_OWNED_PAYLOAD, allocation sharing and the borrowed UTF-8 fast path.Generated singular, optional, repeated, oneof and map paths use
decode_string_to_ctx/decode_bytes_to_ctx. These validate against the innermost enclosing message, preventing a malformed field from exposing the parent's following fields or later stream frames to a hook. Sub-message and map-entry lengths are checked at runtime too. An incomplete prefix is rejected even if bytes outside the frame would complete it as zero; valid empty fields and supported noncanonical zero prefixes wholly inside the frame remain accepted.The bounded window stays synchronized if a delegated default conversion rejects an owned or fragmented payload. A hook that inspects, retries or recovers from that error still cannot access the next field. Borrowed conversion errors leave their original field bytes unread, so a successful fallback skips exactly those bytes. Propagated errors retain an unspecified cursor position within the field.
DecodeContextrecords the enclosing message end through the defaultMessage::merge_to_limit. Hand-written overrides and code decoding a different buffer must record the appropriate boundary withwith_message_end(zero for a buffer holding exactly that message), or enter throughmerge/merge_to_limit. These compatibility requirements and the context-free helpers' buffer-only bounds are documented. Downstream generated custom-field code needs regeneration to use the context-aware entry points.The stronger field-window guarantee applies to custom decode hooks. Ordinary
String/Vec<u8>/Bytesfields, unknown fields and MessageSet payload copies retain their existing checks and can consume bytes past an enclosing boundary before the parent reports an error. Extending the bounds to those paths is separate follow-up work; this PR does not claim whole-decoder frame isolation.Length-delimited entry points record the frame end before invoking a handwritten
merge_to_limitoverride. Context-aware hooks therefore keep the enclosing frame boundary even when that override supplies the tag loop. The manual-decoder documentation distinguishes this context guarantee from an override that deliberately ignores it.Validation:
The external protobuf conformance suite was not rerun for this follow-up.
Contributed by ReallyMe Technologies Limited.