Repository navigation
Conversation
…#183) main_zig.zig is the 4,692-line code generator every backend × platform combo flows through. Splitting it carries real risk: any drift in emission order, allocator lifetime, or template-slot string content breaks bit-identical output for every downstream backend, and there are ~18 named slots wired through one orchestrator. This PR does NOT attempt the full split. It lands: 1. `docs/REFACTOR-PLAN-main-zig.md` — section-by-section map of the current file, proposed module layout (`src/codegen/{scan, idents, validate, blocks/*, lifecycle/*, preview, main_template}.zig`), dependency graph, per-section risk assessment, and the ordering for future extraction PRs. 2. `src/codegen/scan.zig` — the lowest-risk extraction from the plan: the AST-walk discovery helpers (`PluginEvent`, `PluginFlowDecls`, `PluginPinStyle`, `PluginCoercion`, `discoverPluginEvents`, `discoverPluginFlowDecls`, `dedupePinStyles`, `sanitizePluginIdent`, `pathToIdent`) plus their existing tests. Pure data collection with no template-state coupling and a stable external API already used from `root.zig` + `test/tests.zig`. 3. `main_zig.zig` re-exports the moved symbols so every existing `@import("main_zig.zig").Plugin*` / `.discover*` site keeps working unchanged. No call sites moved; no logic touched. Net: 4,692 → 3,880 lines (-812). 4. `scripts/gen_all_examples.sh` — regenerates every bundled example (raylib, sokol, sokol_imgui, null, plugin-controllers, flows-smoke, asset-streaming-smoke, bgfx, wgpu) so before/after diffs can verify the bit-identical contract. Verification: `zig build test` exits 0 (same as origin/main). Across all 9 examples, every generated `main.zig` is byte-identical pre/post extraction; the only `diff -r` drift is the `generated_at` timestamp inside `flow_catalog.json`. Draft: requesting human review of the cut plan before any further extraction PRs. Refs #183.
There was a problem hiding this comment.
Code Review
This pull request initiates a refactoring plan to split the large src/main_zig.zig file by extracting discovery and scanning helpers into a new submodule, src/codegen/scan.zig, while maintaining backward compatibility via re-exports. It also introduces a verification script to ensure bit-identical output generation across examples. The review feedback highlights critical memory leak risks in the newly extracted scanning code where partial allocations are not properly cleaned up with block-scoped errdefers on failure. Additionally, it recommends replacing std.debug.print with standard logging facilities (std.log.err) for user-facing errors in CLI contexts.
| try entries.append(allocator, .{ | ||
| .plugin_import_name = try allocator.dupe(u8, module_name), | ||
| .plugin_sanitized = try allocator.dupe(u8, sanitized), | ||
| .event_name = try allocator.dupe(u8, event_name), | ||
| }); |
There was a problem hiding this comment.
If any of the subsequent allocator.dupe allocations or the entries.append call fail, the previously allocated strings in the current iteration will be leaked. Using block-scoped errdefers ensures that these temporary allocations are cleaned up if the iteration fails before ownership is transferred to entries.
const duped_import_name = try allocator.dupe(u8, module_name);
errdefer allocator.free(duped_import_name);
const duped_sanitized = try allocator.dupe(u8, sanitized);
errdefer allocator.free(duped_sanitized);
const duped_event_name = try allocator.dupe(u8, event_name);
errdefer allocator.free(duped_event_name);
try entries.append(allocator, .{
.plugin_import_name = duped_import_name,
.plugin_sanitized = duped_sanitized,
.event_name = duped_event_name,
});
| const init_src = ast.getNodeSource(member_init); | ||
| const constructs_value = extractConstructsString(allocator, init_src); | ||
| try flow_nodes_out.append(allocator, .{ | ||
| .module_import_path = try allocator.dupe(u8, module_import_path), | ||
| .module_sanitized = try allocator.dupe(u8, module_sanitized), | ||
| .node_name = try allocator.dupe(u8, member_name), | ||
| .is_script = is_script, | ||
| .constructs = constructs_value, | ||
| }); |
There was a problem hiding this comment.
If any of the allocator.dupe calls or the append call fail, constructs_value (which is allocated by extractConstructsString) and any previously allocated strings in the current iteration will be leaked. Block-scoped errdefers should be used to prevent these memory leaks on allocation failure.
const init_src = ast.getNodeSource(member_init);
const constructs_value = extractConstructsString(allocator, init_src);
errdefer if (constructs_value) |c| allocator.free(c);
const duped_path = try allocator.dupe(u8, module_import_path);
errdefer allocator.free(duped_path);
const duped_sanitized = try allocator.dupe(u8, module_sanitized);
errdefer allocator.free(duped_sanitized);
const duped_name = try allocator.dupe(u8, member_name);
errdefer allocator.free(duped_name);
try flow_nodes_out.append(allocator, .{
.module_import_path = duped_path,
.module_sanitized = duped_sanitized,
.node_name = duped_name,
.is_script = is_script,
.constructs = constructs_value,
});
| const append = struct { | ||
| fn f(b: *[256]u8, idx: *usize, bytes: []const u8) void { | ||
| if (idx.* + bytes.len > b.len) { | ||
| std.debug.print("labelle: path too long for identifier (max {d} chars): too many escaped chars\n", .{b.len}); |
There was a problem hiding this comment.
According to the general rules, debug-specific print functions like std.debug.print should not be used for user-facing warnings or errors in CLI tools. Standard logging facilities (such as std.log.err) should be used instead.
std.log.err("labelle: path too long for identifier (max {d} chars): too many escaped chars", .{b.len});
References
- In CLI tools, use standard logging facilities (e.g. std.log.warn) or write to stderr for user-facing warnings, rather than using debug-specific print functions (e.g. std.debug.print).
Three findings from gemini on #193, all pre-existing bugs the PoC extraction surfaced. Fixed only in src/codegen/scan.zig — main_zig.zig already re-exports these symbols (no duplicate copies to keep in sync). - :191 — errdefer per dupe before append, prevents leak when one step in the chain fails. - :504 — same shape around extractConstructsString + 3 dupes; the optional constructs allocation gets its own errdefer guard. Also applied to the parallel pin_styles_out and coercions_out appends in the same loop for consistency. - :757 — std.debug.print -> local stderrPrint helper (matches the writeStderr pattern in main.zig / cache_cmd.zig; std.log.err can't be used because it trips the test-runner log interceptor, same rationale documented in src/scene_manifest.zig). Bit-identical generated output across all bundled examples (8/8 of the working ones — sokol_imgui was already failing pre-fix and is unrelated). zig build test green: 506/510 (4 skipped), same as baseline.
Now under umbrella PR #202The content from this PR is already on the umbrella integration branch You can either:
Either way the actual code on main ends up identical. |
Round-N follow-up to a02c6cd. The earlier pass fixed the dupe-chain sites that gemini flagged inline; this pass walks every remaining allocator.dupe loop and every readFileAlloc call in the file, plus the toOwnedSlice cascade in discoverPluginFlowDecls. - PluginEvent doc comment (line 46) updated: was "both strings are heap-allocated dupes" but the struct has three string fields. - discoverEventsFromRoot readFileAlloc (line ~165) now propagates error.OutOfMemory instead of swallowing every error as "this module contributes nothing" — same applied to the plugin pass (line ~663) and the script pass (line ~693) in discoverPluginFlowDecls. - discoverPluginFlowDecls return chain (line ~711): three back-to-back toOwnedSlice calls — each one detaches its ArrayList from the enclosing errdefer, so a mid-sequence OOM leaked the previously detached slice. Stage each slice into a local with a slice-shaped errdefer so OOM stays clean across all three. Dismissed as already fixed in a02c6cd: - per-dupe errdefer at lines 209-220 (events loop) - per-dupe errdefer at lines 525-547 (flow_nodes loop) - per-dupe errdefer at lines 549-561 (pin_styles loop) - per-dupe errdefer at lines 573-585 (coercions loop) - std.debug.print at line 814 (already routes through stderrPrint helper a02c6cd added at the top of the file) zig build test exit 0, 346/350 passing (4 skips, unchanged from baseline). scripts/gen_all_examples.sh: 8/8 main.zig files bit-identical to baseline (sokol_imgui pre-existing soft-fail unchanged).
…ig (part of #183) Per the plan in #193. Moved LoadStyle + emitResourceLoad to src/codegen/blocks/resource_loader.zig (113 lines). Re-exported from main_zig.zig so callers in both lifecycle paths (buildSetupCode + the callback init builder) keep their existing references. Depends on idents.zig from #195 (extWithoutDot, isValidZigIdentifier). Bit-identical generated output across all 8 working bundled examples (scripts/gen_all_examples.sh green; sokol_imgui pre-existing skip because labelle-imgui sibling not present). Refs: #183 Builds on: #195 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ig (part of #183) Per the plan in #193. Moved writeSceneAssetManifests and writeSceneInitialStateManifests to src/codegen/blocks/scene_manifests.zig (137 lines). Re-exported from main_zig.zig so callers don't change. Depends on idents.zig from #195 (writeZigString, pathToIdent). Bit-identical generated output across all 8 working bundled examples (scripts/gen_all_examples.sh green). Refs: #183 Builds on: #195 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
….zig (part of #183) Per the plan in #193. Moved 5 plugin-registry block writers (controllers, events, flow_nodes, pin_styles, coercions) to src/codegen/blocks/plugin_registries.zig (353 lines). Re-exported from main_zig.zig so the orchestrator's call sites stay unchanged. Depends on scan types (PluginEvent, PluginFlowNode, PluginPinStyle, PluginCoercion) from #193 and ident helpers from #195. Bit-identical generated output across all 8 working bundled examples (scripts/gen_all_examples.sh green). Refs: #183 Builds on: #193, #195 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…(part of #183) Fourth extraction per the plan in #193. Moved writeImageBackendWiring/writeAudioBackendWiring/writeFontBackendWiring to src/codegen/blocks/asset_wiring.zig (377 lines of pure emit, no state). Re-exported from main_zig.zig so callers don't change. Bit-identical generated output across all 8 working bundled examples (scripts/gen_all_examples.sh green; sokol_imgui soft-fails on missing labelle-imgui sibling, same as baseline). Refs: #183 Builds on: #195 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…part of #183) Per the plan in #193. Moved buildCallbackInitCode + buildImmersiveEntryCode + buildCallbackCleanupCode (sokol/wasm/Android callback path) to src/codegen/lifecycle/callback.zig (252 lines). Re-exported from main_zig.zig so callers in the orchestrator stay unchanged. HIGH-risk extraction per the plan -- preserves call order across the preview-mode wiring constants and the wasm/Android conditional branches. Verified bit-identical across all 8 working bundled examples (scripts/gen_all_examples.sh). The asset-wiring + resource-loader helpers (writeImageBackendWiring, writeAudioBackendWiring, writeFontBackendWiring, emitResourceLoad, LoadStyle) are still in main_zig.zig because the matching blocks/asset_wiring.zig + blocks/resource_loader.zig extractions (plan steps 4-5) haven't landed yet; the new submodule reaches them through @import("../../main_zig.zig") in the meantime. When those blocks land, the import flips to the new module paths and the pub markers in main_zig.zig revert to private. Refs: #183 Builds on: #202 (umbrella) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… of #183) Per the plan in #193. Moved buildSetupCode (raylib/sdl/bgfx/wgpu loop-backend setup) + buildGuiDrawCode to src/codegen/lifecycle/loop.zig (235 lines). Re-exported from main_zig.zig so the orchestrator's references stay unchanged. HIGH-risk extraction per the plan -- preserves call order across writeXxxWiring helpers + emitResourceLoad. Verified bit-identical across all 8 working bundled examples (scripts/gen_all_examples.sh); sokol_imgui still soft-fails because the labelle-imgui sibling repo is absent, same as on origin/main. On this branch the helpers reached by buildSetupCode (writeImageBackendWiring, writeAudioBackendWiring, writeFontBackendWiring, emitResourceLoad, LoadStyle) are still inline in main_zig.zig -- pubified here so the new module can reach them. When the parallel blocks/asset_wiring + blocks/resource_loader cuts (PRs #197, #199) land, lifecycle/loop.zig's imports flip to the new modules and the pub markers in main_zig.zig retire alongside the moved bodies. main_zig.zig shrank 3817 -> 3628 lines (-189). zig build test: 506/510 passed (4 skipped) -- unchanged from baseline. Refs: #183 Builds on: #202 (umbrella) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three findings from gemini on #193, all pre-existing bugs the PoC extraction surfaced. Fixed only in src/codegen/scan.zig — main_zig.zig already re-exports these symbols (no duplicate copies to keep in sync). - :191 — errdefer per dupe before append, prevents leak when one step in the chain fails. - :504 — same shape around extractConstructsString + 3 dupes; the optional constructs allocation gets its own errdefer guard. Also applied to the parallel pin_styles_out and coercions_out appends in the same loop for consistency. - :757 — std.debug.print -> local stderrPrint helper (matches the writeStderr pattern in main.zig / cache_cmd.zig; std.log.err can't be used because it trips the test-runner log interceptor, same rationale documented in src/scene_manifest.zig). Bit-identical generated output across all bundled examples (8/8 of the working ones — sokol_imgui was already failing pre-fix and is unrelated). zig build test green: 506/510 (4 skipped), same as baseline.
Summary
src/main_zig.zigis the 4,692-line code generator that every backend × platform combo flows through. A full split is high-risk: bit-identical generated output must be preserved for every example, and there are ~18 template slots wired through a single orchestrator.This PR is an RFC for the cut plan plus one proof-of-concept extraction. The rest of the cuts are deferred until reviewers vet the approach. Refs #183.
What's in this PR
docs/REFACTOR-PLAN-main-zig.md— section-by-section map ofmain_zig.zig, proposed module layout (src/codegen/{scan, idents, validate, blocks/*, lifecycle/*, preview, main_template}.zig), dependency graph, per-section risk assessment, and ordering for future extraction PRs.src/codegen/scan.zig(875 lines) — the PoC extraction. Holds the AST-walk discovery helpers (PluginEvent,PluginFlowDecls,PluginPinStyle,PluginCoercion,discoverPluginEvents,discoverPluginFlowDecls,dedupePinStyles,sanitizePluginIdent,pathToIdent) plus the existingpathToIdenttests. Chosen because it's pure data collection — no template state, no codegen-orchestrator coupling, well-defined public API already used externally.src/main_zig.zig(4,692 → 3,880 lines, -812) — re-exports the moved symbols soroot.zigandtest/tests.zigkeep their existing imports unchanged. Pure move + import; no call sites moved, no logic touched.scripts/gen_all_examples.sh— regenerates every bundled example so before/after diffs can verify the bit-identical contract.Verification
zig buildclean.zig build testexits 0 (same asorigin/main; "failed command" log lines are pre-existing diagnostic-output tests, not test failures).diff -rdrift across the entire generated tree is thegenerated_atISO-8601 timestamp insideflow_catalog.json(expected and unrelated).Constraints honored
@import("main_zig.zig").Plugin*/.discover*site keeps working via re-exports.Asks for review
src/codegen/blocks/*.zigper-slot or oneblocks.zig? Per-backendlifecycle/*vs one file?preview.zig(~1200 lines of inline templates) andlifecycle/*as medium-to-high. The orchestrator itself stays last.If the plan looks wrong, the PoC reverts cleanly (one re-export shim, one new file).
Test plan
docs/REFACTOR-PLAN-main-zig.mdand reacts to the proposed module layout / ordering.src/codegen/scan.zigagainst the deleted regions ofsrc/main_zig.zigto confirm pure move.zig build+zig build test.bash scripts/gen_all_examples.sh /tmp/xon this branch vsorigin/mainto reproduce the bit-identical check.DRAFT — do not merge until plan is reviewed.