[tamnd]

Closes #11.

What this is

The record model from ~/notes/Spec/3008/03_model.md, plus the mapper from the extractor onto it.

Two rules run through all of it.

A field the read did not look at is absent, not zero.
That is why so much of model.go is pointers.
A book with no page count and a book with zero pages are not the same fact, and a model that cannot tell them apart pushes the ambiguity onto every consumer downstream.

Every number carries provenance.
A book page and a work page publish different numbers for the same named thing, edition against all editions, and a record that does not say which population it holds is not usable for anything quantitative.
Stats.Via is not omitempty for that reason.

The rename

Book, Review and User in types.go become ScrapedBook, ScrapedReview and ScrapedUser.
They are still here because every v0.2.0 command still returns them, and they go when M4 ports those commands onto the extractor.
The prefix is deliberate rather than cosmetic: two types called Book in one package is how a call site ends up building the wrong one.

The four states of a missing field

Extractor now records fields the page published blank in Empty, separately from fields it never mentioned at all.
That is rows two and three of the table in section 8 of the spec, and they mean genuinely different things.
A book with a blank publisher is a book Goodreads has no publisher for.
A book with no publisher key at all, read at a depth that never fetched details, is a book nobody asked about.
TestFourStatesOfAMissingField has a subtest per row, and TestEmptyListsAreRowThreeOnTheRealPage anchors the same rule to a capture: Pride and Prejudice has an empty places list where The Hunger Games has a full one.

New in the extractor

• series, with userPosition kept as written. Goodreads uses 2.5 for novellas and 1-3 for omnibus editions, so the model keeps the raw string and parses Number only when it is a single number.
• links kept whole and untyped. Its shape moves, it is not worth a struct, and dropping it loses the ebook and audiobook availability.
• genre URLs, because the slug in the URL is the only stable handle on a genre.
• the work's editions pointer and choiceAwards.
• Extra for every cache field the model has no home for, counted through NoteUnknown so verify reports it. A new field appearing is Goodreads shipping something, which is a chance to model it rather than a failure.
• awardsWon carries nominations as well as wins, so HasWon is derived from the designation. A nomination reported as a win is a wrong claim about somebody's book.

goodread book <id> --check

Reads a book into the model and reconciles it two ways.
The histogram has to sum to the ratings count, which catches a truncated or misaligned distribution.
The mean derived from the histogram has to match the published average, which catches a reversed slice: five integers with no labels are exactly the thing that ends up reversed and nothing about the values gives it away.

Measured live:

id       title                fields  buckets sum  ratings   derived  published  ok
2767052  The Hunger Games     40      10232418     10232418  4.3548   4.35       true
1885     Pride and Prejudice  40      4983594      4983594   4.2965   4.30       true

Both sums are exact and both means match to two decimals.
TestStatsCheckCatchesAReversedDistribution reverses the real distribution and asserts the check fails and says why.

goodread verify --sample N

Carried over from M2.
Reads live book pages instead of the pinned captures, from a deterministic spread across the id space.
Deterministic rather than random because the point is to find drift, and the same books every run means a difference is drift rather than a different book.
A capture keeps extracting cleanly forever, including on the day Goodreads changes the page, which is exactly when somebody needs to hear about it.

CAPTURE       SURFACE  FIELDS  L1  L2  L3  NOTE
live 2767052  s1       41      40  1   0   -
live 1885     s1       39      38  1   0   -
live 5107     s1       40      39  1   0   -

Gate

gofmt -l . clean, go vet ./... clean, go test ./... green for both packages.