Repository navigation
setup: the SYCL setup on an integrated Arc (xe): native launcher, a RAM fit rule, no KV streaming - #2115
Open
andersson82 wants to merge 2 commits into
Open
setup: the SYCL setup on an integrated Arc (xe): native launcher, a RAM fit rule, no KV streaming#2115andersson82 wants to merge 2 commits into
andersson82 wants to merge 2 commits into
Conversation
jielin99
added a commit
to jielin99/Strata-codex
that referenced
this pull request
Oct 13, 2026
…p reports The NPU plugin returns EXECUTION_DEVICES as the plain string "NPU" (CPU and GPU return a list), so the probe split it into ['N', 'P', 'U'] and rejected a valid placement. Wrap a string in a list first; a CPU fallback or an empty list still fails. Rerun on the laptop: the 64x64 FP32 MatMul passes on NPU (max abs error 4.9e-05). This only shows the NPU runs a small graph; it is not Strata on the NPU. The handoff doc gains the Ubuntu dev machine inventory (no NPU, 15 GiB RAM: no real model baseline possible there), the review of upstream PRs Niko1221#2079, Niko1221#2110, Niko1221#2114 and Niko1221#2115 against this branch, and the laptop's hardware and OpenVINO reports (Core Ultra 9 285H, Arc 140T, Intel AI Boost, 63.5 GiB). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
jielin99
added a commit
to jielin99/Strata-codex
that referenced
this pull request
Oct 13, 2026
probe_sycl.cpp reports, per SYCL device, what the native Windows engine (PR Niko1221#2079) gates on - ext_intel_free_memory and ext_oneapi_graph - plus host_unified_memory, memory sizes, the largest allocation, sub-group sizes and fp16/fp64, and runs one 32-wide sub-group kernel per GPU. Read-only, no large allocation. PROBE_SYCL.bat builds it with icx and runs it three ways (no switches, ZES_ENABLE_SYSMAN=1, plus UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1). Built with icpx 2025.3.2 (intel/oneapi-basekit image) and run on a UHD 730 over Level Zero: valid JSON, sub-group kernel ok in all three. There the iGPU reports ~270 MiB free with ~13 GiB MemAvailable, and no ext_oneapi_graph. The Windows build and the Arc 140T are not tested. The handoff doc records the laptop toolchain, the cherry-picked upstream PRs (Niko1221#2114, Niko1221#2110, Niko1221#2079; Niko1221#2115 left out), the setup test results and the probe run. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…/meminfo, no prompt-slot borrowing, a native launcher On the Arc 140T (Core Ultra 9 285H, 62 GiB, Fedora 44, xe, oneAPI 2026.0 on the PC) the engine did not start: the device has no ext_intel_free_memory (also with ZES_ENABLE_SYSMAN=1), so the expert cache was sized from 0. - sycl/include/dpct/device.hpp: get_memory_info on a device with host_unified_memory and no free-memory query reads MemAvailable from /proc/meminfo (Linux); STRATA_SYCL_FREE_MIB sets it by hand. Other devices: unchanged. - sycl/src/program/generate.cpp: no prompt-slot borrowing on a host_unified_memory device. With it the answer after a borrowing prompt was token 0 (expert activations NaN from the 2nd chunk; 0.1.40, any context). - sycl/serve/strata-sycl-native.sh (new): the config's exe for the engine on the PC's own oneAPI, no Docker. - docs/INTEL.md: "Integrated Arc: the Arc 140T"; docs/INTEL_ARC.md: the 140T is no longer "not tested". Measured on the 140T, Coder IQ1_M, engine 0.1.43 with Niko1221#2110, through the server: 12.0-17.4 tok/s decode on 100-token answers (all 12,288 experts cached), a 72,648-token prompt read at 156.5 tok/s with its first-line fact recalled. Built with sycl/tools/build.sh (native oneAPI): 0 errors. No discrete card was run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…AM fit rule, no KV streaming ./setup.sh --backend sycl on a PC whose only Intel GPU is integrated (PCI 00:02.0, xe) now writes a config for sycl/serve/strata-sycl-native.sh (the engine on the PC's own oneAPI, no Docker) instead of failing to find a card. Depends on the engine PR (the launcher and the iGPU memory query). - sycl/setup_intel.py: intel_gpus lists the iGPU with setup's unified-memory fields; sycl_engine checks for oneAPI's setvars.sh; to_sycl_igpu writes the PC's paths, --stream-experts, no --kv-resident, the reserve of a card, --draft-vocab en, SYCL_CACHE_PERSISTENT=1 on the 140T only, STRATA_SYCL_BIN for an AOT build. The size menu and --check use shard 1 + 20 GiB <= RAM (measured: 10-13 GiB left with 36.5 GiB shard 1, 3-4 GiB with 43.8 GiB did not run well) and ask before installing a size that does not fit. An iGPU on i915 stops with the kernel command line for xe. The AMD wording setup prints is rewritten for an iGPU. - tools/test_setup_sycl.py: 6 new tests (detection, config, fit rule, cache, launcher); 23 of 23 pass. - docs/INTEL.md: setup's bullet and "With setup" in "Integrated Arc". Discrete cards: their path is unchanged (the new branches need igpu set, which a card behind a PCIe port never gets). Checked: all tools/test_setup_*.py pass except test_setup_engine_hash.py, which fails the same 2 tests on origin/main (a release download returns 404). On the Arc 140T, setup from this branch with --family coder --model IQ1_M --context 98304 --no-start --yes wrote the hand-made config of the engine PR's docs, less "fit_max_tokens". No discrete Intel card was run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
andersson82
force-pushed
the
sycl-igpu-setup
branch
from
October 15, 2026 05:31
592d332 to
59a5ba8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title
setup: the SYCL setup on an integrated Arc (xe): native launcher, a RAM fit rule, no KV streaming
Issue: none. Depends on #2114 (the launcher and the iGPU memory query): until that one is merged this PR's diff
also shows its commit; the change of this PR is the last commit.
Summary
./setup.sh --backend syclon a PC whose only Intel GPU is integrated (an Intel GPU at PCI00:02.0onxe) stoppedwith "no Intel Arc found". It now finds the GPU, checks that oneAPI and the engine build are on the PC, offers the
sizes that fit the RAM, and writes a config for
sycl/serve/strata-sycl-native.sh(no Docker).What changed
sycl/setup_intel.py:intel_gpuslists the integrated GPU (onxe) with setup's unified-memory fields;i915_igpusfinds one oni915,for which setup stops and names the kernel command line that moves it to
xe.sycl_engine(igpu=True)also needs/opt/intel/oneapi/setvars.sh(orONEAPI_ROOT) and says how to build natively.to_sycl_igpu: the PC's paths, the launcher asexe,--stream-experts, no--kv-resident(the KV is in the sameRAM either way), the reserve as on a card (1,024 MiB up to 32K, 2,048 above; a given one is kept),
--draft-vocab en,SYCL_CACHE_PERSISTENT=1on the 140T only (it crashed on Xe2),STRATA_SYCL_BINfor an AOT build.did not run well). The size menu marks the sizes that need more and recommends the first that fits; a size that does
not fit is asked about (
--model X --yesinstalls it anyway);--checksays the same per model.tools/test_setup_sycl.py: 6 new tests (detection on xe and i915, the config, the fit rule, the persistent cache, the launcher's binary path).docs/INTEL.md: setup's bullet and a "With setup" paragraph in "Integrated Arc".Left alone: discrete cards. Every new branch needs the GPU at
00:02.0, where a card behind a PCIe port never is, sotheir config and messages are unchanged.
setup.pyis not touched.How I checked
python tools/test_setup_sycl.py: 23 of 23 pass. All othertools/test_setup_*.pypass excepttest_setup_engine_hash.py, which fails the same 2 tests on a cleanorigin/main(a release download returns 404).xe, oneAPI 2026.0), from this branch:./setup.sh --backend sycl --family coder --model IQ1_M --context 98304 --no-start --yesran through and wrote theconfig of the engine PR's docs (the one measured there) less
"fit_max_tokens", andrun-coder-iq1_m.sh.Not tested:
i915on real hardware (the message is unit-tested only), Windows.--model IQ1_M --yes).🤖 Generated with Claude Code