Skip to content

M6: the twelve edges, traversal and export - #25

Merged
tamnd merged 1 commit into
mainfrom
m6-edges-traversal-export
Aug 15, 2026
Merged

tamnd merged 1 commit into
mainfrom
m6-edges-traversal-export

Conversation

@tamnd

@tamnd tamnd commented Aug 15, 2026

Copy link
Copy Markdown
Owner

Closes #14.

The edge set is closed

goodread/predicates.go holds the twelve predicates from 04_graph.md section 4 as constants, each with its endpoints, the surfaces it comes from and what it carries. PutEdge refuses anything else. The guard is at the write, where the name is still in front of somebody, rather than at the read, where a typo looks like missing data.

Four names the M5 indexer was writing were not the spec's:

was is why
wrote, author to book contributed_by, book to author the spec's name and the spec's direction
contains, list to book listed_in, book to list same
in_genre shelved_as same
edition_of, in_series unchanged already right

Directions are the spec's rather than the ones that read best in a sentence. Edges answers either way, and having one canonical direction is what stops the same fact being stored twice under two names.

What rides on an edge

contributed_by carries the role, in_series carries the position raw and parsed, listed_in carries the rank, won carries the category and the year. The rank is on the edge because a book sits at 3 on one list and 900 on another, and the rank belongs to neither of them alone.

best_edition, set_in, features, won and quoted_from are written now where the record carries the data. Places, characters and awards got tables: they have no id on Goodreads and no page of their own, so the slug is the id, and two places with the same name in different countries collide. That limit is written down rather than papered over with a synthetic id nothing else in the world shares.

Traversal

goodread graph <uri> walks the local store and fetches nothing.

  • --depth defaults to 1, per section 5.
  • Above 500 nodes it prints the count and asks for --yes.
  • Cycles are the normal case. Book to Work to BestBook closes in three hops on most popular titles, so the walk carries a visited set from the first hop rather than growing one after something goes wrong.
  • Each node is reported at the shortest depth it was reached at.
  • --edges prints the relationships instead of the nodes.
  • A node with an edge and no record is still reported, because that is the most useful thing a walk can say to somebody deciding what to crawl next.

Export

goodread export, section 6.

  • --to jsonl is the default, one node per line, the whole record and not the columns. The columns are an index over the record and a consumer that got only them would be missing every field this version has no column for.
  • --edges writes the edge table instead.
  • --to rdf is Turtle: schema.org where it fits, grv: where it does not. schema:exampleOfWork for book to work, which is the standard's own edition versus work distinction and lines up exactly with ours. The aggregate rating goes on a blank schema:AggregateRating because that is where schema.org puts it, and the rating distribution stays a grv: list because nothing in schema.org holds five numbers in rating order.
  • SQLite is not a choice here. The store already is a SQLite file and the way to export it is to copy it.
  • --to and not --format, because --format is the global flag that picks how records print.

Tests

  • TestTheEdgeSetIsTheSpecsTwelve, exact names, endpoints that are real node kinds, and the two the spec calls out for their properties.
  • TestAnUnknownPredicateIsRefused, and the refusal names what to use instead.
  • TestEveryEdgeHasAnRDFPredicate, so no edge can vanish from the export without a word.
  • TestBothIDSpacesSurviveTheRoundTrip, reading the kca id and the legacy id back out of the columns off a real capture.
  • TestWalkFinishesOnACycle, six deep through a two hop cycle, on a timeout so a hang is a failure and not a hung suite.
  • TestWalkSizeAgreesWithWalk, because a count that disagreed with the walk would gate the wrong walks.
  • TestExportRDFMapsOntoSchemaOrg and TestExportJSONLWritesTheWholeRecord, both off the real book_show_2767052 capture.
  • CLI coverage for graph and export.

gofmt -l . && go vet ./... && go test ./... is clean.

The edge set is closed now. 04_graph.md section 4 names twelve predicates and
predicates.go holds them as constants with their endpoints, their surfaces and
what they carry, and PutEdge refuses anything else. The refusal is at the write,
where the name is still in front of somebody, rather than at the read, where a
typo looks like missing data.

Four of the names the indexer was writing were not the spec's. `wrote` was an
author to book edge and is now `contributed_by` running book to author, `contains`
was list to book and is now `listed_in` running book to list, and `in_genre` is
now `shelved_as`. Directions are the spec's rather than the ones that read best
in a sentence, because Edges answers either way and one canonical direction is
what stops the same fact being written twice under two names.

The properties that ride on an edge are there too. `contributed_by` carries the
role, which is the difference between an author and an illustrator, and
`in_series` carries the position both raw and parsed, because the raw string is
the only one that holds 1-3 on an omnibus and the number is the only one that
sorts. `listed_in` carries the rank, which belongs to the edge because a book
sits at 3 on one list and 900 on another.

Places, characters and awards got tables. They have no id on Goodreads and no
page of their own, so the slug is the id and two places with the same name in
different countries collide. That limit is in the spec rather than papered over
with a synthetic id nothing else in the world shares.

`goodread graph <uri>` walks the local store, defaults to depth 1, and asks for
--yes above 500 nodes. Cycles are the normal case and not the error case: Book
to Work to BestBook closes in three hops on most popular titles, so the walk
carries a visited set from the first hop. A node with an edge and no record is
still reported, because that is the most useful thing a walk can say to somebody
deciding what to crawl next.

`goodread export` writes JSONL or Turtle. JSONL is the whole record and not the
columns, since the columns are an index over it. The RDF is schema.org where it
fits and grv: where it does not, with schema:exampleOfWork for the book to work
edge, which is the standard's own edition versus work distinction and lines up
exactly with ours. A wrong predicate is worse than a local one.
@tamnd
tamnd merged commit 02dca60 into main Aug 15, 2026
4 of 7 checks passed
@tamnd
tamnd deleted the m6-edges-traversal-export branch August 15, 2026 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

M6: identity, the gr: URI space, edges and export

1 participant