Skip to content

setup: the SYCL setup on an integrated Arc (xe): native launcher, a RAM fit rule, no KV streaming - #2115

Open
andersson82 wants to merge 2 commits into
Niko1221:mainfrom
andersson82:sycl-igpu-setup
Open

andersson82 wants to merge 2 commits into
Niko1221:mainfrom
andersson82:sycl-igpu-setup

Conversation

@andersson82

Copy link
Copy Markdown

Title

setup: the SYCL setup on an integrated Arc (xe): native launcher, a RAM fit rule, no KV streaming

Issue: none. Depends on #2114 (the launcher and the iGPU memory query): until that one is merged this PR's diff
also shows its commit; the change of this PR is the last commit.

Summary

./setup.sh --backend sycl on a PC whose only Intel GPU is integrated (an Intel GPU at PCI 00:02.0 on xe) stopped
with "no Intel Arc found". It now finds the GPU, checks that oneAPI and the engine build are on the PC, offers the
sizes that fit the RAM, and writes a config for sycl/serve/strata-sycl-native.sh (no Docker).

What changed

  • sycl/setup_intel.py:
    • intel_gpus lists the integrated GPU (on xe) with setup's unified-memory fields; i915_igpus finds one on i915,
      for which setup stops and names the kernel command line that moves it to xe.
    • sycl_engine(igpu=True) also needs /opt/intel/oneapi/setvars.sh (or ONEAPI_ROOT) and says how to build natively.
    • to_sycl_igpu: the PC's paths, the launcher as exe, --stream-experts, no --kv-resident (the KV is in the same
      RAM either way), the reserve as on a card (1,024 MiB up to 32K, 2,048 above; a given one is kept), --draft-vocab en,
      SYCL_CACHE_PERSISTENT=1 on the 140T only (it crashed on Xe2), STRATA_SYCL_BIN for an AOT build.
    • The fit rule: shard 1 + 20 GiB <= RAM (measured: 10-13 GiB left with a 36.5 GiB shard 1, 3-4 GiB left with 43.8 GiB
      did not run well). The size menu marks the sizes that need more and recommends the first that fits; a size that does
      not fit is asked about (--model X --yes installs it anyway); --check says the same per model.
    • setup's AMD wording (BIOS carve-out, "this AMD card") is rewritten for an iGPU.
  • tools/test_setup_sycl.py: 6 new tests (detection on xe and i915, the config, the fit rule, the persistent cache, the launcher's binary path).
  • docs/INTEL.md: setup's bullet and a "With setup" paragraph in "Integrated Arc".

Left alone: discrete cards. Every new branch needs the GPU at 00:02.0, where a card behind a PCIe port never is, so
their config and messages are unchanged. setup.py is not touched.

How I checked

  • python tools/test_setup_sycl.py: 23 of 23 pass. All other tools/test_setup_*.py pass except
    test_setup_engine_hash.py, which fails the same 2 tests on a clean origin/main (a release download returns 404).
  • On the Arc 140T (Core Ultra 9 285H, 62 GiB, Fedora 44, xe, oneAPI 2026.0), from this branch:
    ./setup.sh --backend sycl --family coder --model IQ1_M --context 98304 --no-start --yes ran through and wrote the
    config of the engine PR's docs (the one measured there) less "fit_max_tokens", and run-coder-iq1_m.sh.

Not tested:

  • A discrete Intel card with this branch (I have none); the unit tests for the card path still pass.
  • A PC with a discrete card and an iGPU together (setup should pick the card; not run).
  • Another integrated GPU than the 140T, an iGPU on i915 on real hardware (the message is unit-tested only), Windows.
  • The interactive size menu by hand (unit-tested; the real run used --model IQ1_M --yes).

🤖 Generated with Claude Code

jielin99 added a commit to jielin99/Strata-codex that referenced this pull request Oct 13, 2026
…p reports

The NPU plugin returns EXECUTION_DEVICES as the plain string "NPU" (CPU and
GPU return a list), so the probe split it into ['N', 'P', 'U'] and rejected a
valid placement. Wrap a string in a list first; a CPU fallback or an empty
list still fails. Rerun on the laptop: the 64x64 FP32 MatMul passes on NPU
(max abs error 4.9e-05). This only shows the NPU runs a small graph; it is not
Strata on the NPU.

The handoff doc gains the Ubuntu dev machine inventory (no NPU, 15 GiB RAM:
no real model baseline possible there), the review of upstream PRs Niko1221#2079,
Niko1221#2110, Niko1221#2114 and Niko1221#2115 against this branch, and the laptop's hardware and
OpenVINO reports (Core Ultra 9 285H, Arc 140T, Intel AI Boost, 63.5 GiB).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
jielin99 added a commit to jielin99/Strata-codex that referenced this pull request Oct 13, 2026
probe_sycl.cpp reports, per SYCL device, what the native Windows engine
(PR Niko1221#2079) gates on - ext_intel_free_memory and ext_oneapi_graph - plus
host_unified_memory, memory sizes, the largest allocation, sub-group sizes
and fp16/fp64, and runs one 32-wide sub-group kernel per GPU. Read-only, no
large allocation. PROBE_SYCL.bat builds it with icx and runs it three ways
(no switches, ZES_ENABLE_SYSMAN=1, plus UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1).

Built with icpx 2025.3.2 (intel/oneapi-basekit image) and run on a UHD 730
over Level Zero: valid JSON, sub-group kernel ok in all three. There the iGPU
reports ~270 MiB free with ~13 GiB MemAvailable, and no ext_oneapi_graph.
The Windows build and the Arc 140T are not tested.

The handoff doc records the laptop toolchain, the cherry-picked upstream PRs
(Niko1221#2114, Niko1221#2110, Niko1221#2079; Niko1221#2115 left out), the setup test results and the probe run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
andersson82 and others added 2 commits October 15, 2026 07:27
…/meminfo, no prompt-slot borrowing, a native launcher

On the Arc 140T (Core Ultra 9 285H, 62 GiB, Fedora 44, xe, oneAPI 2026.0 on the PC) the engine did not start: the
device has no ext_intel_free_memory (also with ZES_ENABLE_SYSMAN=1), so the expert cache was sized from 0.

- sycl/include/dpct/device.hpp: get_memory_info on a device with host_unified_memory and no free-memory query
  reads MemAvailable from /proc/meminfo (Linux); STRATA_SYCL_FREE_MIB sets it by hand. Other devices: unchanged.
- sycl/src/program/generate.cpp: no prompt-slot borrowing on a host_unified_memory device. With it the answer
  after a borrowing prompt was token 0 (expert activations NaN from the 2nd chunk; 0.1.40, any context).
- sycl/serve/strata-sycl-native.sh (new): the config's exe for the engine on the PC's own oneAPI, no Docker.
- docs/INTEL.md: "Integrated Arc: the Arc 140T"; docs/INTEL_ARC.md: the 140T is no longer "not tested".

Measured on the 140T, Coder IQ1_M, engine 0.1.43 with Niko1221#2110, through the server: 12.0-17.4 tok/s decode on
100-token answers (all 12,288 experts cached), a 72,648-token prompt read at 156.5 tok/s with its first-line
fact recalled. Built with sycl/tools/build.sh (native oneAPI): 0 errors. No discrete card was run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…AM fit rule, no KV streaming

./setup.sh --backend sycl on a PC whose only Intel GPU is integrated (PCI 00:02.0, xe) now writes a config for
sycl/serve/strata-sycl-native.sh (the engine on the PC's own oneAPI, no Docker) instead of failing to find a card.
Depends on the engine PR (the launcher and the iGPU memory query).

- sycl/setup_intel.py: intel_gpus lists the iGPU with setup's unified-memory fields; sycl_engine checks for
  oneAPI's setvars.sh; to_sycl_igpu writes the PC's paths, --stream-experts, no --kv-resident, the reserve of a
  card, --draft-vocab en, SYCL_CACHE_PERSISTENT=1 on the 140T only, STRATA_SYCL_BIN for an AOT build. The size
  menu and --check use shard 1 + 20 GiB <= RAM (measured: 10-13 GiB left with 36.5 GiB shard 1, 3-4 GiB with
  43.8 GiB did not run well) and ask before installing a size that does not fit. An iGPU on i915 stops with the
  kernel command line for xe. The AMD wording setup prints is rewritten for an iGPU.
- tools/test_setup_sycl.py: 6 new tests (detection, config, fit rule, cache, launcher); 23 of 23 pass.
- docs/INTEL.md: setup's bullet and "With setup" in "Integrated Arc".

Discrete cards: their path is unchanged (the new branches need igpu set, which a card behind a PCIe port never
gets). Checked: all tools/test_setup_*.py pass except test_setup_engine_hash.py, which fails the same 2 tests on
origin/main (a release download returns 404). On the Arc 140T, setup from this branch with --family coder
--model IQ1_M --context 98304 --no-start --yes wrote the hand-made config of the engine PR's docs, less
"fit_max_tokens". No discrete Intel card was run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant