Skip to content

feat(codegen): route compressed (ASTC) atlases through the catalog adapter - #348

Merged
apotema merged 4 commits into
mainfrom
feat/catalog-compressed-adapter
Jun 16, 2026
Merged

apotema merged 4 commits into
mainfrom
feat/catalog-compressed-adapter

Conversation

@apotema

@apotema apotema commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Problem

The async streaming asset catalog (labelle-engine#450) loads atlases via the assembler-generated ImageBackendAdapter, whose decode/upload call BackendGfx.decodeImage/uploadTexture directly — bypassing gfx's synchronous compressed seam (loadTextureFromMemory). So ASTC blobs always hit the CPU decoder and fail.

Proven on the real flying-platform-labelle game (sokol / Android, Galaxy Tab A7): the menu atlas returned error.LoadFailed until this fix. With it, cold start dropped from ~24s to ~1s.

Fix

The generated adapter now branches on isCompressed:

  • decode: if BackendGfx.isCompressed(data), dupe the raw bytes verbatim (the catalog frees them after upload), read dims from the ASTC header via BackendGfx.compressedDims, set .compressed = true, and skip the CPU decoder. Otherwise CPU-decode as before.
  • upload: .compressed blobs route through BackendGfx.uploadCompressed (uploaded as-is); RGBA8 goes through uploadTexture unchanged.

Both branches are @hasDecl-guarded (isCompressed/compressedDims for decode, uploadCompressed for upload), so any backend without ASTC support — or a future headless backend — still generates valid code and always CPU-decodes.

compressedDims added to all 4 backends

Added a header-only pub fn compressedDims(data) ?struct { width: u32, height: u32 } helper + gfx.zig re-export to sokol, bgfx, raylib, and wgpu, each reusing its existing validateAstc to read width/height from the ASTC header without decoding. This lets the adapter set correct DecodedImage dims before the (later, main-thread) upload.

Tests

  • zig build test passes (assembler codegen + backend_wiring_tests, updated to assert the new compressed branch + @hasDecl guards in the generated text).
  • All four backend packages build + pass their unit tests with the new helper: zig build --build-file backends/{sokol,bgfx,raylib,wgpu}/build.zig test.

Required paired PRs (cross-repo)

The generated code references engine.DecodedImage.compressed and the backends' compressed methods. For consumers to compile, these must be released / pinned:

Epic

Part of the ASTC cold-start epic: labelle-gfx#269.

…apter

The async streaming asset catalog (engine#450) loads atlases via the
assembler-generated ImageBackendAdapter, whose decode/upload call
BackendGfx.decodeImage/uploadTexture directly — bypassing gfx's
synchronous compressed seam (loadTextureFromMemory). So ASTC blobs
always hit the CPU decoder and fail.

The generated adapter now:
  - decode: branches on BackendGfx.isCompressed(data); for compressed
    blobs it dupes the raw bytes (the catalog frees them after upload),
    reads dims from the header via BackendGfx.compressedDims, sets
    .compressed = true, and skips the CPU decoder.
  - upload: routes .compressed blobs through BackendGfx.uploadCompressed
    (verbatim), RGBA8 through uploadTexture as before.

Both branches are @hasDecl-guarded (isCompressed/compressedDims and
uploadCompressed), so backends without ASTC support — or a future
headless backend — still generate valid code and always CPU-decode.

Adds a header-only compressedDims(data) ?struct{width,height} helper +
gfx.zig re-export to all four backends (sokol/bgfx/raylib/wgpu), reusing
each backend's existing validateAstc to read dims without decoding, so
the adapter can set correct DecodedImage dims before upload.

Part of the ASTC cold-start epic (labelle-gfx#269). Proven on the real
flying-platform-labelle game (sokol/Android, Tab A7): the menu atlas
returned error.LoadFailed until this fix; with it, cold start dropped
from ~24s to ~1s.
@cursor

cursor Bot commented Jun 16, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes the default image load path for streaming catalogs and ASTC atlases; mitigated by comptime @hasDecl gating and requires paired engine/gfx releases for DecodedImage.compressed.

Overview
The generated ImageBackendAdapter now handles ASTC on the async asset-catalog path (engine#450), which previously always CPU-decoded via decodeImage/uploadTexture and bypassed gfx’s synchronous compressed seam.

When a backend exposes the full compressed API (isCompressed, compressedDims, uploadCompressed), decode dupes raw ASTC bytes, reads width/height from the header with compressedDims, sets engine.DecodedImage.compressed = true, and skips CPU decode. upload sends .compressed blobs to uploadCompressed; RGBA8 still uses uploadTexture with an inline struct literal. A comptime supports_compressed flag wraps both branches so backends missing any decl never reference absent symbols at compile time.

compressedDims is added and re-exported on bgfx, raylib, sokol, and wgpu (shared validateAstc logic). backend_wiring_tests assert the new emitted strings and upload call shape.

Reviewed by Cursor Bugbot for commit cbb951b. Bugbot is set up for automated code reviews on this repo. Configure here.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a header-only dimensions reading mechanism (compressedDims) for compressed ASTC textures across multiple backends (bgfx, raylib, sokol, and wgpu), allowing the async asset-catalog adapter to retrieve image dimensions without CPU decoding. Additionally, the code generator in asset_wiring.zig is updated to route compressed textures directly to uploadCompressed when supported. The review feedback highlights a potential issue where a backend implementing isCompressed and compressedDims but lacking uploadCompressed could cause a runtime crash. It is recommended to strengthen the @hasDecl guard in the generated code to also verify the presence of uploadCompressed and update the corresponding test assertions.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/codegen/blocks/asset_wiring.zig Outdated
// worker-thread decode from main-thread upload, so the divert happens here.
// Guarded by `@hasDecl` so backends without compressed support (or a future
// headless backend) still generate valid code and just always CPU-decode.
try w.print("{s} if (@hasDecl(BackendGfx, \"isCompressed\") and @hasDecl(BackendGfx, \"compressedDims\")) {{\n", .{indent});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If a backend implements isCompressed and compressedDims but does not implement uploadCompressed (or if it is temporarily disabled/removed), the decode function will still divert and return raw compressed bytes in decoded.pixels. Then, in upload, since uploadCompressed is missing, it will fall back to calling BackendGfx.uploadTexture with the raw compressed bytes, leading to runtime crashes or rendering corruption.

To prevent this, the decode guard should also check for the presence of uploadCompressed to ensure the backend fully supports the entire compressed texture lifecycle before diverting.

    try w.print("{s}        if (@hasDecl(BackendGfx, \"isCompressed\") and @hasDecl(BackendGfx, \"compressedDims\") and @hasDecl(BackendGfx, \"uploadCompressed\")) {{\n", .{indent});

Comment thread test/backend_wiring_tests.zig Outdated
try std.testing.expect(std.mem.indexOf(u8, main_zig, "BackendGfx.unloadTexture(tex)") != null);
// Compressed (ASTC) blobs route through the @hasDecl-guarded path:
// decode diverts before the CPU decoder, upload uses uploadCompressed.
try std.testing.expect(std.mem.indexOf(u8, main_zig, "@hasDecl(BackendGfx, \"isCompressed\") and @hasDecl(BackendGfx, \"compressedDims\")") != null);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Update the test assertion to match the strengthened @hasDecl guard in the generated code.

        try std.testing.expect(std.mem.indexOf(u8, main_zig, "@hasDecl(BackendGfx, \"isCompressed\") and @hasDecl(BackendGfx, \"compressedDims\") and @hasDecl(BackendGfx, \"uploadCompressed\")") != null);

…iew)

The generated ImageBackendAdapter previously gated the compressed-upload
path on isCompressed+compressedDims only, while the upload step
referenced BackendGfx.uploadCompressed via a runtime `if`. A backend
exposing the probe decls but NOT uploadCompressed would generate an
adapter that fails to compile (dangling reference to the missing decl).

Introduce a single `supports_compressed` comptime flag requiring the
ENTIRE lifecycle (isCompressed + compressedDims + uploadCompressed), and
wrap both the decode-divert and upload arms in `if (comptime
supports_compressed)` so the inner blocks are not semantically analyzed
when the flag is comptime-false — backends lacking compressed support
generate code that still compiles and always CPU-decodes.

Update backend_wiring_tests to assert the strengthened 3-decl guard and
the comptime-gated upload arm.

Verified: zig build test green; sokol/bgfx/raylib/wgpu backends EXIT 0;
the null backend (no compressed decls) also compiles, exercising the
comptime-false gate.
@apotema

apotema commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Addressed both Gemini HIGH findings in 2360004.

Fix: The generated ImageBackendAdapter now derives a single comptime flag

const supports_compressed = @hasDecl(BackendGfx, "isCompressed") and @hasDecl(BackendGfx, "compressedDims") and @hasDecl(BackendGfx, "uploadCompressed");

requiring the entire compressed lifecycle (probe + dims + upload). Both the decode divert and the upload arm are wrapped in if (comptime supports_compressed), so a backend that exposes isCompressed+compressedDims but not uploadCompressed no longer emits a dangling reference to the missing decl — the inner block isn't semantically analyzed when the flag is comptime-false, so it compiles and just always CPU-decodes. The previous runtime if (@hasDecl(...) and decoded.compressed) form still analyzed the BackendGfx.uploadCompressed arm and would fail to compile for such a backend.

test/backend_wiring_tests.zig updated to assert the strengthened 3-decl guard and the comptime-gated upload arm.

Verification:

  • zig build test green (backend_wiring + codegen tests).
  • All 4 real backends standalone EXIT 0: sokol, bgfx, raylib, wgpu.
  • The null backend (no compressed decls at all) also compiles EXIT 0 — exercises the comptime-false gate, proving a backend lacking uploadCompressed generates code that still compiles.

…t (CI)

The generated catalog adapter referenced engine.DecodedImage.compressed
and BackendGfx.uploadCompressed whenever the backend exposed the full
compressed seam (true for raylib). But the released/main engine's
DecodedImage lacks the compressed field (only present in engine#632),
so the Examples integration test failed building the raylib example
against the released engine:
  error: no field named 'compressed' in struct DecodedImage

Add @Hasfield(engine.DecodedImage, "compressed") to the
supports_compressed conjunction so the whole compressed decode/upload
path is comptime-pruned when building against an engine without the
field (falls back to CPU decode). With engine#632 + a compressed
backend it stays true and the ASTC fast path is active. The assembler
PR no longer requires the engine release to land first.

Update backend_wiring_tests assertion to match the new expression.
@apotema

apotema commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

The Examples integration test failure was a forward-reference to an unreleased engine field: the generated catalog adapter referenced engine.DecodedImage.compressed and BackendGfx.uploadCompressed for any compressed-capable backend (raylib), but the released/main engine's DecodedImage doesn't have the compressed field yet (it only lands in engine#632). Hence error: no field named 'compressed' in struct DecodedImage at main.zig:486 / :504.

Fixed by adding @hasField(engine.DecodedImage, "compressed") to the supports_compressed conjunction in src/codegen/blocks/asset_wiring.zig. Building against an engine without the field makes the flag comptime-false → the entire compressed decode/upload path is pruned and the example falls back to CPU decode (compiles clean). Against engine#632 + a compressed backend it stays true → ASTC fast path active. This makes the adapter back-compatible, so this PR no longer needs the engine release to land first.

Verified locally (Zig 0.16.0):

  • zig build test green (incl. backend_wiring_tests + codegen).
  • All 4 backend packages standalone zig build test exit 0 (sokol/bgfx/raylib/wgpu).
  • Reproduced the CI Examples integration step (labelle generate + zig build) for the raylib example against a fresh main engine checkout (no compressed field): builds clean now. Confirmed the old gate still fails with the exact no field named 'compressed' error at main.zig:486/:504, and the new @hasField gate fixes it. asset-streaming-smoke (also raylib) builds clean too.

Reverts a78b339. Silent CPU-decode fallback against an old engine hides
misconfigurations; the maintainer prefers the assembler PR to depend on
the engine version that ships DecodedImage.compressed (engine#632).

Restores supports_compressed to gate on exactly the three backend decls
(isCompressed + compressedDims + uploadCompressed). All other PR work
(comptime-gated decode/upload divert from 2360004, compressedDims on all
4 backends) is intact. Content of both touched files now equals 2360004.
@apotema

apotema commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Reverted the `@hasField(engine.DecodedImage, "compressed")` back-compat gate (commit a78b339) per maintainer preference. A silent CPU-decode fallback against an old engine hides misconfigurations, so the codegen now gates `supports_compressed` on exactly the three backend decls (`isCompressed` + `compressedDims` + `uploadCompressed`) again, and this PR simply depends on the engine version that ships `DecodedImage.compressed`.

The Examples integration check will stay RED until engine #632 (which adds `DecodedImage.compressed`) is released and this PR's example engine pin is bumped. That is an intentional stacked-PR forward dependency, not a defect — all other PR work (the comptime-gated decode/upload divert, `compressedDims` on all 4 backends) is intact, `zig build test` is green, and all 4 backend packages build standalone (EXIT 0).

@apotema
apotema merged commit 4e4f284 into main Jun 16, 2026
8 of 9 checks passed
@apotema
apotema deleted the feat/catalog-compressed-adapter branch June 16, 2026 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant