host top and host get report zero for numbers nothing measured #144

host top and host get read the web graph rank table and nothing else, but every row they print carries the full enriched schema, and the numeric half of it comes out as zero:

$ ccrawl host top -n 1 -o jsonl
{"host":"www.facebook.com","host_rev":"com.facebook.www","tld":"com","harmonic_pos":1,"harmonic_val":34375268,"pagerank_pos":3,"pagerank_val":0.0055143537769673755,"in_degree":0,"out_degree":0,"url_count":0,"status_2xx":0,"status_3xx":0,"status_4xx":0,"status_5xx":0,"total_bytes":0}

Facebook does not have zero inbound links and zero URLs in the crawl. Nothing looked. A reader has no way to tell that from the output, and neither does a script: url_count is a number, 0 is a number, and the row says the host was measured and came back empty.

The same for host get without --cdx, which is the default:

$ ccrawl host get commoncrawl.org
{"host":"commoncrawl.org",...,"in_degree":0,"out_degree":0,"url_count":0,"status_2xx":0,...,"total_bytes":0}

host get has no way at all to fill in_degree, so those two are always zero from that command.

The string fields already do the right thing, because they carry omitempty: top_mime, language, first_seen and last_seen are simply absent when nothing measured them. It is only the integers that lie, since a zero int is indistinguishable from an unset one.

Zero is a measurement. Absent is not. -o csv has the same problem in the other direction, where the columns are printed with empty or zero cells for every host.

Done when

• a number nothing measured is absent from the JSON rather than zero
• a number that was measured and is genuinely zero still prints as zero
• -o csv and -o table leave the cell blank rather than writing 0
• host enrich and host get --cdx still fill in what they measured
• measured live against the real web graph