Repository navigation
LiNo over TCP: text and binary links notation protocols, remote links store and binary store archive in Rust and C# - #106
Conversation
Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #105
Add the protocol module for issue #105: a dot-stuffed UTF-8 text protocol and a binary links packet whose header carries the link count and whose reference width grows with it (8/16/32/64 bits). External (Hybrid) references, the sequence section and progressive widths are optional and independently switchable. Includes golden vectors and hostile-input limits.
LinksServer detects the protocol of every message and replies in kind, executing requests serially against the store; LinksClient runs substitution queries over any LinoProtocol. Replies mirror clink --changes and the empty request reads the whole store.
clink --serve <addr> exposes a database over TCP, accepting text and binary messages; clink --connect <addr> sends a query with the chosen protocol and binary options and prints the reply.
Port of the Rust protocol module: LinksPacket codec, LiNo <-> packet mapping with optional external references, sequence section and progressive widths, dot-stuffed text framing, protocol detection and a LinksServer/LinksClient pair that run substitution queries over TCP.
…server links-notation 0.16.1 parsed nested groups in time exponential in the depth (link-foundation/links-notation#314, fixed in 0.21.3): a 90-byte message nested 40 deep kept a server thread busy for minutes. 0.22.0 is linear and rejects nesting deeper than 64. 0.22.0 wraps a lone reference in a group in both ports (`a` parses as `(a)`), so the Rust parse_document now removes that wrapper, as C# already did. Golden vectors are unchanged. Deep-nesting regression tests are in the Rust packet tests and the C# codec tests.
… golden vectors Also add lino-structure-probe examples in C# and Rust that print how each links-notation port structures LiNo text, for checking that they agree.
Mirrors rust/tests/protocol_tcp_tests.rs: CRUD over text and all eight binary option sets, identical replies across protocols, remote errors, a raw CRLF text session, malformed headers, protocol restriction, concurrent clients and shutdown.
The C# CLI now serves a database over TCP and queries a served one, matching the Rust options: --serve, --connect, --protocol and the binary decorators --external-references, --sequences and --progressive-widths. examples/tcp/run-interop.sh drives a server written in one language with a client written in the other.
konard
changed the title
Working session summaryIssue #105 is done: both the Rust and C# versions can now serve a links database over TCP and run LiNo queries against it. PR #106 is marked ready for review. All CI checks pass on the latest commit, the full test suites pass locally, and a Rust server talking to a C# client works in both directions. How it works:
Rest of the working session summary (1 KB)This summary was automatically extracted from the AI working session output. |
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost: $17.640115📊 Context and tokens usage:Claude Opus 5.5: (8 sub-sessions)
Total: (19.7K new + 823.3K cache writes + 21.8M cache reads) input tokens, 344.3K output tokens, $17.601906 cost Claude Haiku 4.5:
Total: 33.3K input tokens, 984 output tokens, $0.038209 cost 🤖 Models used:
📎 Log file uploaded as Gist (10207KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
✅ Ready to mergeThis pull request is now ready to be merged:
Monitored by hive-mind with --auto-restart-until-mergeable flag |
|
We need to make sure that links remote client is implemented as decorator with ILinks interface in both C# and Rust, so for the end user it is easy to swap local direct usage of the data store with remote data store. Double check that everything done perfectly, and everything is covered with 100% tests, with guarantee both client and server of all protocols work perfectly for all CRUD operations as described in our original docs of substitution operation. Double check the docs are perfectly expressive, no contradictions of the code, and polish all the code in the entire repository codebase, so it is as perfect as symmetrical as logical and correct as possible, if you see any false positives, false negatives, warnings or errors fix them all. Everything is wrong or has errors should be proven with tests, so no regressions in the future. Try to make everything as perfect as possible in all supported languages. Make sure we reported issues to all potential dependencies repositories from our organizations, so we can also improve codebase there later. And make sure exactly all dependencies in the codebase are updated to all latest versions, and we guarantee with CI/CD it will be always the case. We should make sure we do as less code duplication as possible. The code is fully self explanatory, if we need comments we might replace them with better names for variables and functions. Also the packet format of binary links notation should be reusable, we may also put it to github.com/link-foundation/links-notation make binary format fully configurable and so on. Like which fixed length links we should support, by default it is range from 2 to 2 inclusive, yet we should also allow 1 to infinity, test carefully with 2 - 3 (duplets and triples), and variable length links. So it uses as little disk space as possible and has good algorithms for packing links in such a way so we minimize empty space in the data format, for example we should try to fit as much links to 256 block as possible, so they reference only each other, and all others links reference at at least one link with bigger addresses and so on. We first fully implement it here, and later propose copy/migrate/expand on implementation in github.com/link-foundation/links-notation so we will have not only text version of universal links notation, but also universal standard for binary links notation, that can be used as data archives, binary messaging protocols and so on. It should be possible also to fully import and export links from the data store in and out of binary links notation. All previous issues must be solved, that we noticed, so we have a perfect state of codebase which is carefully 100% covered with tests. |
|
🤖 AI Work Session Started Starting automated work session at 2026-10-04T23:23:40.556Z The PR has been converted to draft mode while work is in progress. This comment marks the beginning of an AI work session. Please wait for the session to finish, and provide your feedback. Runtime: solve |
…medTypeLinks over the LiNo protocols
NamedTypeLinks gains update_observed next to delete_observed; every layer (LinkStorage, NamedTypesDecorator, transactions, version control, persistent transformations, RemoteLinks) forwards the observer, so an update that merges into an existing doublet reports the deletion of the updated link, exactly as AdvancedMixedQueryProcessor does in C#. The server's update operation returns every change, and RemoteLinks is tested against a local store over every protocol.
A create followed by a match of the same link reported both '(1: 1 1) -> (1: 1 1)' and '() -> (1: 1 1)'. Both simplifiers now keep an unchanged pair only when no other change touches that link.
The simplifier walked a graph of link states in which the null link was a single node, so every deletion chained into every creation: deleting link 3 and creating link 4 was reported as '(3: 3 3) -> (4: 4 4)', and C# reported both '((1: 1 1)) ((1: 3 3))' and '() ((1: 3 3))' when '((1: 1 1)) ((3 3))' deleted and recreated address 1. Both ports now reduce the raw steps of each address to (first before, last after), drop links created and deleted within the query, and keep ordering by the after state. The C# tests that were commented out describe exactly this behaviour and are enabled again; the debug-only test now asserts its result.
… the store Named point links and leaves created on the way were not reported, or were reported as an empty link that was filled in later. Every creation is now reported once as the link it ends up being, in both ports. Creating under an existing name redefines that link, and a new name for an existing doublet names it instead of copying it. Reference validation assumed that a new link gets the lowest free address. Both stores reuse the address freed last first, so a reference to a freed address the query never refills was accepted and then silently dropped, and C# EnsureCreated threw on a freed address above its target. The validator now asks the store for the next addresses by creating and deleting them in reverse, and repeats the plan until auto-created references settle.
… stack trace A missing reference or a malformed query ended clink with an unhandled exception and its stack trace. Both are now printed as 'Error: Query error:' or 'Error: Parse error:' like the Rust clink, with the stack trace kept for --trace. Tests in both ports pin the output, and the CLI tests share one helper that runs the built clink.
… them An update into an existing doublet merges into it. Platform.Data.Doublets 0.18.1 MergeUsages builds new Link<T>(a, b), which is (index: a, source: b, target: 0), so C# left (1: 1 0) where Rust re-points the usage to (1: 1 1) (linksplatform/Data.Doublets#515). Stores are now composed with DecorateWithAutomaticUniquenessAndUsagesRepointing(), whose LinksUniquenessAndUsagesRepointingResolver re-points both halves of each usage in one update. The issue-100 parity harness drops its last exemption: 41 scenarios agree.
It never reads or writes the file system: save, flush and reload write nothing, and has_external_changes is always false. The browser has no file system (fs::metadata returns Unsupported on wasm32), so this is the store the WebAssembly workbench needs.
The WebAssembly crate stopped compiling (E0046) when NamedTypeLinks began requiring update_observed and delete_observed, which failed the WebAssembly and Docs workflows. Instead of growing its own simplified copy of the store, the workbench now uses the CLI's LinkStorage kept in memory, so deletes cascade to usages and updates merge into an existing pair exactly as in the CLI. Tests pin both behaviours natively and on wasm32.
…t the exact error - AddressTier throws for an address beyond the internal range - drop the unused LinoFormat.IsReference - a stopping server answers 'server is shutting down' instead of hanging up - trace 'client hung up' on a clean close - three malformed-packet cases passed for the wrong reason; every case now checks its detail
…ct error - address_tier fails as unencodable beyond the internal range - a stopping server replies (error: "server is shutting down") to a read request - operation errors print links in LiNo, not Debug syntax - width_from_code is total over two-bit codes; dead fallbacks removed
…ing branch of the accept loop
konard
changed the title
Working session summaryThe tests now run every line of the protocol code in both ports, and PR #106 is marked ready for review: #106. All 7 CI workflows pass on the latest commit ( Coverage. Changes in this round:
Rest of the working session summary (1 KB)This summary was automatically extracted from the AI working session output. |
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost: $63.452679📊 Context and tokens usage:Claude Opus 5.5: (28 sub-sessions)
Total: (127.5K new + 3.1M cache writes + 85.3M cache reads) input tokens, 1.1M output tokens, $63.452679 cost 🤖 Models used:
📎 Log file uploaded as Repository (40320KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
✅ Ready to mergeThis pull request is now ready to be merged:
Monitored by hive-mind with --auto-restart-until-mergeable flag |
2 participants
Closes #105.
LiNo substitution queries (create, read, update and delete) now run over TCP in both ports. There are two interchangeable protocols:
Rust and C# produce identical bytes, and either port's client works with either port's server. Following the review of 2026-10-04, the PR also adds a remote store (
RemoteLinks), a binary store archive, a dependency-freshness check, full protocol test coverage and upstream bug reports.The full analysis is in
docs/case-studies/issue-105. It covers the requirements, prior art, the wire format, golden vectors, sizes, coverage, upstream findings and the remaining limits. The format specification isdocs/protocol/binary-links-notation.md.Try it
What changed
Protocols (
rust/src/protocol/,csharp/…Library/Protocol/)TextLinoProtocol: canonical LiNo. A message ends with a line holding., with dots doubled as in SMTP, soncworksBinaryLinoProtocol/LinksPacket. The header is0x10 | flags; a packet is sections of links that share one reference width and one arity range--packed-widthsgives each section the narrowest width it needs, and is never larger than uniform output(Number unary(n))and(String code-points…).--external-referencesencodes values asPlatform.Data.Hybrid<T>--arity 2..3/1..: links of any allowed length, addressed after the fixed onesLinoProtocol/ILinoProtocol.LinoConnectiondecorates any stream with either one. The server detects the protocol per message and replies in kindBinaryLinoOptions: every option is off by default.--protocol text|binaryrestricts a serverThe binary links notation is reusable outside TCP (the store archive uses it). Shared golden vectors (
docs/protocol/binary-links-notation-vectors.txt) pin the format: 84 documents × 12 option sets, plus raw packets, asserted byte for byte in both ports.Server, client, remote store, CLI
LinksServer/LinksClient.(before) (after)like--changes.(error: 'message').(error: 'server is shutting down')and never touches the store again.DecodeLimitsbound untrusted input.RemoteLinksis a links-interface decorator. It implementsINamedTypesLinks<uint>(C#) andLinks+Doublets+NamedTypeLinks(Rust). Code written against those interfaces, including the query processors, switches from a local file to a server by swapping one value. Each call is oneLinksOperationdocument ((count: …),(update: …),(set-name: …), …). Both suites replay the same recorded conversation (docs/protocol/links-operations.txt).--export-binary/--import-binarywrite and read the whole store (every address and hole, plus names) as binary links notation. The bytes are identical across ports.--serve,--connect,--protocol,--external-references,--arityand--packed-widthswork the same in both CLIs.Fixes found while proving CRUD end to end (each with a test)
--changescreation reports, in both ports.() ((name: name name)).AddressTier/address_tierreject an address beyond the internal range instead of silently returning width 8.LinkStorage::in_memory, the CLI's own store without a file.js/incomplete-sanitizationin the dependency script.Latest dependencies, enforced by CI
The new
Dependenciesworkflow runs on every PR, on main and daily. It fails while any Cargo crate, lock entry, NuGet package, npm package, GitHub Action, Node.js or .NET version is behind its latest release. Dependabot opens the bumps.Reported upstream
Each issue has a minimal repro, and §6.5 of the case study lists how this repo steers around each one.
Tests and coverage
100% of lines in
rust/src/protocol/andcsharp/…Library/Protocol/are covered. Measured withcargo llvm-covand coverlet; the commands are in the case study, §5.protocol_packet_tests(27)BinaryLinksNotationTests(17),LinoProtocolCodecTests(11)protocol_tcp_tests(11),cli_tcp_tests(4)LinksServerTests(15),CliTcpIntegrationTests(5)remote_links_tests(14)RemoteLinksTests(16)store_archive_tests(16)StoreArchiveTests(11),CliStoreArchiveTests(3)The C# counts are test methods; many are theories. Full C# suite: 440 passed.
Local checks, all passing:
cargo fmt --checkcargo clippy --all-targets -D warningscargo testdotnet format --verify-no-changesdotnet build -c Releasedotnet testcheck-file-sizenpm run test:jsnpm run test:wasmnode --testfor the dependency scriptRelease notes
rust/changelog.d/2026100{4,5}_*_issue_105_*.md: TCP protocols (minor), creation reports, remote links, store archive, latest dependencies, browser storage, protocol coverage.csharp/.changeset/issue-105-*.md: the same topics, plus merge re-pointing usages.README.mddocs/ARCHITECTURE.mddocs/HOW-IT-WORKS.mddocs/REQUIREMENTS.mddocs/protocol/rust/README.mdandcsharp/README.mdREADMEsRemaining limits
127.0.0.1or tunnel the connection.The case study lists next steps for each of these.