Skip to content

Binary notation: self-referencing links take 3 bytes, the spec says 2 #51

Closed

Description

@konard

Summary

docs/BINARY-NOTATION-SPEC.md says a self-referencing link is encoded with its id written once:

Self-Referencing Link (5: 5)

Text notation: (5: 5 5) (10 bytes)
Binary encoding:

Type: 0x0F (SOURCE_IS_ID | TARGET_IS_ID | SELF_REF | HAS_ID)
ID: 0x05 (varint 5)
Total: 2 bytes (80% reduction)

Both encoders write 3 bytes instead: the type, the id, and then the shared source/target again.

Reproduction (at bdc7631)

import { BinaryNotation, HEADER_SIZE } from './js/src/protocol/binary-notation.js';
const body = BinaryNotation.encode({ id: 5, source: 5, target: 5 }).slice(HEADER_SIZE + 1);
console.log(Buffer.from(body).toString('hex')); // 0f0505, the spec says 0f05

The Rust encoder does the same: encode_link_body in rust/src/backends/binary_notation.rs writes the id when link.id.is_some(), then, for is_self_ref, writes source_id again.

Expected

Either the encoders and decoders follow the spec (when SELF_REF and HAS_ID are both set and the id equals the source, write the id only, saving one varint per point link — a store of 1000 points (i: i i) takes 4759 bytes today), or the spec's example is corrected to 0F 05 05, 3 bytes. Changing the encoding is a wire-format change, so the decoder and the spec version would need to move together.

A smaller point in the same example: (5: 5 5) is 8 bytes of text, not 10.

Context

Found while comparing this notation with the binary links notation of link-cli (link-foundation/link-cli#106, spec). The comparison and the scripts used are in research-notes.md §8.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions