Totals/Daily report $0 for a single date even after a full cache rebuild, while live per-tool/per-model breakdown for the same date is correct #1127

Activity

1. iamtoruk commented on Aug 24, 2026

   

   iamtoruk

   on Aug 24, 2026

   Member

   More actions

   Thanks — that's an unusually good report, and the categories.conversation.turns: 1 detail was the key.

   I can confirm what that cache entry is: I reproduced its exact shape synthetically. {cost: 0, calls: 0, sessions: 0, categories.conversation.turns: 1} is precisely what aggregateProjectsIntoDays emits for a day touched by exactly one midnight-straddling turn whose assistant calls all landed on the following day — turn-level counts anchor on the turn (day-aggregator.ts:116), cost/calls/tokens anchor per call (:187). Also worth knowing: the sessions 16 you see on the zeroed Totals line is the live scan count leaking in via Math.max (usage-aggregator.ts:513), not a cache value — so the display and your sessions: 0 dump are consistent.

   What that means: the backfill parse that built the cache saw zero calls timestamped 2026-08-20. They weren't mis-bucketed to a neighbour day; they were missing from that parse's output. The daily cache is built by a separate, earlier parse (usage-aggregator.ts:432) than the one behind By tool / Top models (:448), which is how the two can disagree in a single command.

   One command settles where it goes wrong. codeburn export never touches the daily cache — it's a pure live parse — so a narrow vs. wide export isolates parser from cache:

   codeburn export --format json --from 2026-08-20 --to 2026-08-20 -o /tmp/cb-narrow.json
   codeburn export --format json --from 2026-07-20 --to 2026-08-23 -o /tmp/cb-wide.json

   Then bucket the per-call rows by local day in each and compare the 2026-08-20 bucket:

   for f in /tmp/cb-narrow.json /tmp/cb-wide.json; do
     node -e '
       const r=require(process.argv[1]).records, d={};
       for (const x of r) { if(!x.timestamp) continue;
         const t=new Date(x.timestamp); if(isNaN(t)) continue;
         const k=`${t.getFullYear()}-${String(t.getMonth()+1).padStart(2,"0")}-${String(t.getDate()).padStart(2,"0")}`;
         (d[k]??=(d[k]={c:0,n:0})); d[k].c+=x.cost||0; d[k].n++; }
       for (const k of ["2026-08-19","2026-08-20","2026-08-21"])
         console.log(process.argv[1], k, JSON.stringify(d[k]??{c:0,n:0}));
     ' "$f"
   done

   • Wide shows ~$78 / ~1,437 calls on 08-20 → the parse is fine, the bug is in the daily-cache derive/merge.
   • Wide shows ~$0 on 08-20 → the parse loses it at wide range, and the cache is only reporting that faithfully. This is the outcome I expect.

   A few more things that would narrow it fast:

   1. Your timezone (TZ if set, otherwise the system one) — the bucketing is local-midnight and the straddling turn tells me your 08-20/08-21 boundary matters.
   2. Was CodeBurnMenubar running during those codeburn overview runs? When it holds the cache-refresh lock, a concurrent CLI parse falls back to read-only, and in read-only mode any session file with no cache entry is dropped from that parse entirely (parser.ts:2001-2022). That is my leading hypothesis and it would explain why the cache under-reports while the live panels don't. Please try once with the menubar app quit, and once with CODEBURN_VERBOSE=1 codeburn overview --from 2026-08-20 --to 2026-08-20.
   3. Poisoned per-file cache entries. Deleting daily-cache.v17.json does not clear session-cache.v7.json, and a file that once failed to parse is stored there as {"turns":[],"failed":true} and never retried until the file changes — which would explain why your rebuild reproduced identically. Could you check:

      node -e 'const c=require(process.env.HOME+"/.cache/codeburn/session-cache.v7.json");
        const f=c.providers?.claude?.files??{};
        for (const [p,v] of Object.entries(f)) if (v.failed || (v.turns??[]).length===0) console.log(p, v.failed?"failed":"empty");' | head -50

      and, separately, whether the ~16 session files that cover 2026-08-20 appear in that cache with a non-zero turns count.
   4. A redacted day dump — for 2026-08-19, 08-20 and 08-21 from the daily cache, so I can confirm the neighbours aren't inflated:

      node -e 'const c=require(process.env.HOME+"/.cache/codeburn/daily-cache.v17.json");
        for (const d of c.days) if (d.date>="2026-08-19"&&d.date<="2026-08-21")
          console.log(JSON.stringify({...d, projects:undefined,
            providers:Object.fromEntries(Object.entries(d.providers||{}).map(([k,v])=>[k,{...v,projects:undefined,models:undefined}]))}));'

   Independently of which of those it is, there's a real hole I'm fixing: the gap-fill path writes days into durable history with a blind overwrite and no completeness gate (daily-cache.ts:1350-1360 → addNewDays), so a degraded parse can replace a good cached day with an undercount; and nothing ever re-derives a day that came out as turn-residue-only. Both get guards.
2. unstko commented on Aug 24, 2026

   

   unstko

   on Aug 24, 2026

   Last edited by unstko

   Author

   More actions

   Thanks — that pins it down a lot, but the export test came out the opposite of what you expected: the wide parse sees 2026-08-20 just fine. So the parse is exonerated and the bug is in the daily-cache derive.

   Export comparison

   cb-narrow.json  2026-08-19 {"c":0,"n":0}
   cb-narrow.json  2026-08-20 {"c":78.41,"n":1445}
   cb-narrow.json  2026-08-21 {"c":0,"n":0}
   cb-wide.json    2026-08-19 {"c":31.36,"n":497}
   cb-wide.json    2026-08-20 {"c":78.41,"n":1445}
   cb-wide.json    2026-08-21 {"c":33.92,"n":534}
   

   I then widened it to the exact range the derive uses ({start: today − BACKFILL_DAYS, end: yesterdayEnd}, daily-cache.ts:1068) — --from 2025-08-24 --to 2026-08-23, 48,070 records — and 08-20 is still {"c":78.41,"n":1445}. Same for --from 2025-08-24 --to 2026-08-24. Every range width I tried gives the correct day.

   That's apples-to-apples: export calls fp(await parseAllSessions(customRange, pf)) (main.ts:1268) — the same parseAllSessions(range, 'all') that hydrateCache hands to ensureCacheHydrated (usage-aggregator.ts:91-92).

   Your four questions

   1. Timezone. TZ unset; system zone Europe/Berlin (UTC+2 in August). Cache tzKey is Europe/Berlin, so no tz change is in play.

   2. Menubar. Reproduces with CodeBurnMenubar quit. Two things argue against the read-only-lock path: the rebuilt cache finalizes with complete: true / watermarkTrusted: true / lastComputedDate: 2026-08-23, which a degraded parse shouldn't reach; and all 152 other days come out correct (below). CODEBURN_VERBOSE=1 codeburn overview --from 2026-08-20 --to 2026-08-20 emitted no diagnostics — 0.9.20's overview has no --verbose flag either, so if there's another env var that turns the derive tracing on, tell me and I'll rerun.

   3. Session cache. Clean — nothing poisoned:

   total files: 1623   failed: 0   empty: 174
   

   Zero failed: true. The 174 empty ones are all under my home project dir and its subagents/ dirs, none covering 08-20. All 16 session ids the narrow export attributes to 08-20 are present in session-cache.v7.json with non-zero turns.

   4. Day dump. Neighbours are not inflated — they match the live parse call-for-call:

   date	cache	live parse
   2026-08-19	497 calls / $31.32	497 calls / $31.36
   2026-08-20	0 calls / $0.00	1445 calls / $78.41
   2026-08-21	534 calls / $33.91	534 calls / $33.92

   {"date":"2026-08-20","cost":0,"savingsUSD":0,"calls":0,"sessions":0,"inputTokens":0,"outputTokens":0,
    "cacheReadTokens":0,"cacheWriteTokens":0,"editTurns":0,"oneShotTurns":0,"models":{},
    "categories":{"conversation":{"turns":1,"cost":0,"savingsUSD":0,"editTurns":0,"oneShotTurns":0}},
    "providers":{"claude":{"calls":0,"cost":0,"savingsUSD":0,"sessions":0,"inputTokens":0,"outputTokens":0,
    "cacheReadTokens":0,"cacheWriteTokens":0,"editTurns":0,"oneShotTurns":0,
    "categories":{"conversation":{"turns":1,"cost":0,"savingsUSD":0,"editTurns":0,"oneShotTurns":0}}}}}

   The calls aren't mis-bucketed to a neighbour; they're gone.

   Full-window diff

   Diffing every cached day against the same-range live export:

   2026-08-20  live 1445 calls/$78.41 | cache 0 calls/$0.00 | delta $-78.41
   ---
   days compared: 152 | deviating days: 1
   

   Exactly one bad day out of 152.

   Where that leaves it

   The derive ran the full re-derive branch (I deleted daily-cache.v17.json, so c.complete !== true). baseline there could only come from adoptOlderDailyCaches over daily-cache.v15.json — which has 143 days, watermark 2026-08-10, and no 08-20 entry at all. With mergeDayEntries(freshDays, baseline, true, …) and nothing in the baseline to win against, the residue-only day has to be what aggregateDays(projects) returned — off a projects array that, via export, demonstrably contains all 1,445 calls.

   So the surviving suspects are aggregateProjectsIntoDays and the merge/retention step, not the parser — which lines up with the addNewDays blind-overwrite and missing-completeness-gate holes you're already fixing, though neither of those is on the path that produced this one.

   Happy to run anything further — a patched build with tracing in the derive, a dump of freshDays[2026-08-20] before the merge, whatever's most useful. The machine still has the state (daily-cache.v17.json.bak-20260824, all session logs for that day intact), so this is reproducible on demand.
3. iamtoruk commented on Aug 24, 2026

   

   iamtoruk

   on Aug 24, 2026

   Member

   More actions

   Found it, and reproduced it byte-for-byte. You were right that the parse is exonerated — the bug is entirely in the cache write, and your cost: 0 inside categories.conversation was the tell that cracked it.

   Root cause. Two rules collide:

   1. aggregateProjectsIntoDays anchors turn-level counts on turn.timestamp (the user-message time) while cost/calls/tokens anchor on each call's own timestamp (day-aggregator.ts:114 vs :174 on 0.9.20). A turn can therefore contribute to a day the parse range never covered.
   2. The range slicer returns a turn unchanged, keeping its original anchor, when all its calls are in range — parser.ts:2845, if (inRangeCalls.length === turn.assistantCalls.length) return turn. Re-anchoring only happens on a partial slice.

   So a turn whose user message is at 2026-08-20 23:5x and whose first assistant token lands after midnight survives a parse of 2026-08-21 whole, still anchored on 08-20. aggregateDays emits a 2026-08-20 entry holding nothing but that turn's category count.

   Then the gap-fill path (daily-cache.ts:1139) writes it with addNewDays → byDate.set(day.date, day) (:475) — a blind overwrite, no merge, no completeness gate. Your correct, already-sealed 08-20 is replaced by the residue.

   Reproduction. Feeding one such straddling turn (user msg 2026-08-20T23:58+02:00, both calls at 00:03/00:04 on 08-21, zero cost) through the real filterProjectsByDateRange → aggregateProjectsIntoDays under TZ=Europe/Berlin:

   turn anchors surviving a 2026-08-21 gap parse: [ '2026-08-20T23:58:00+02:00', '2026-08-21T10:00:00+02:00' ]
   days emitted:                                  [ '2026-08-20', '2026-08-21' ]
   2026-08-20 entry: {"date":"2026-08-20","cost":0,"savingsUSD":0,"calls":0,"sessions":0,"inputTokens":0,
     "outputTokens":0,"cacheReadTokens":0,"cacheWriteTokens":0,"editTurns":0,"oneShotTurns":0,"models":{},
     "categories":{"conversation":{"turns":1,"cost":0,"savingsUSD":0,"editTurns":0,"oneShotTurns":0}},
     "providers":{"claude":{...identical...}}}
   after addNewDays -> cost 0  calls 0  sessions 0
   

   That is your dump, field for field — including categories.conversation.cost: 0, which means the straddling turn was a zero-cost exchange (interrupted, or fully cache-served).

   Everything you measured now lines up: the parse was never wrong; the session cache was never poisoned; the gap parse was complete (parseWasComplete gates only the complete/watermark markers, never the write, which is why you correctly saw complete: true); 08-21 is written correctly by the same parse so the neighbours stay exact; and it hits exactly one day in 152 because it needs a turn whose user message lands in the last few minutes before local midnight and whose first response token lands after it — any pre-midnight call re-anchors the slice and no residue day is produced — landing precisely on a run seam.

   Why your delete-and-rebuild "reproduced" it. It never rebuilt. adoptOlderDailyCaches adopts every file matching startsWith('daily-cache') && includes('.json') (daily-cache.ts:391), which includes your daily-cache.v17.json.bak-20260824. Candidates sort by version then mtime (:408), so that backup — same version 17, newest mtime — became the fully trusted base and kept its complete: true (:444). The run adopted it, republished it verbatim, saw no gap, and did no re-derive. A real re-derive would have fixed the day.

   One command to confirm — defeat adoption by moving all the daily-cache files, not just the versioned one:

   mkdir -p ~/cb-cache-quarantine && mv ~/.cache/codeburn/daily-cache* ~/cb-cache-quarantine/
   codeburn overview --from 2026-08-20 --to 2026-08-20

   That should print Cost $78.41 / Calls 1,445 on your existing 0.9.20. And the tell-tale prediction: after that, the next day's run should zero out whichever day immediately precedes the new watermark, if a straddler happens to sit on that seam — the bug always eats the day just before gapStart.

   Fix. Already merged in #1129 (b0a42172), which replaced the gap path's blind addNewDays with the same guarded merge the re-derive path uses. Re-running the fixture against that code restores cost 78.41 / calls 1445 / sessions 16 instead of zeros. It also adds a codeburn doctor Cache health section that lists residue-only days and failed/empty session-cache entries, so this is a one-command diagnosis next time. Landing in 0.9.21.

   Your cache self-heals on upgrade — no manual step needed. 0.9.21 ships daily-cache v28; with no v28 file present, adoption from your v17 yields complete: false, which forces one full lossless re-derive over the whole retention window. The broken 08-20 slice is skipped (it has no cost/calls/sessions to defend), the fresh parse wins, and the new gap guard stops it recurring.

   Thank you for this one — the export matrix, the neighbour call counts and the full-window diff are what made it findable. Genuinely one of the best bug reports we've had.
4. timothybrush

   added a commit that references this issue on Aug 24, 2026

   fix(daily-cache): never-lose guards for degraded parses and residue-o…

   ...

   b0a4217
5. pull

   added a commit that references this issue on Aug 24, 2026

   fix(daily-cache): ranged parses never write days outside their range

   ...

   99d9ec8
6. unstko commented on Aug 26, 2026

   

   unstko

   on Aug 26, 2026

   Author

   More actions

   Confirmed on my end — upgraded to 0.9.21 and the daily-cache re-derived automatically on first run. 2026-08-20 now shows the correct values, no manual cache clearing needed. Thanks for the quick fix!

   ❤️React with ❤️1Reacted by Resham Joshi
7. unstko commented on Sep 9, 2026

   

   unstko

   on Sep 9, 2026

   Author

   More actions

   From my side, the issue can be closed.
8. iamtoruk

   closed this as completedon Sep 9, 2026