Skip to content

Offer Recursive Flint as a model on this Mac - #562

Closed
brunobergher wants to merge 3 commits into
mainfrom
claude/mac-model-option-516837
Closed

brunobergher wants to merge 3 commits into
mainfrom
claude/mac-model-option-516837

Conversation

@brunobergher

@brunobergher brunobergher commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Adds Recursive Flint to the models Lamp can download and run on this Mac (Settings → AI Providers → Native Mac AI, and the welcome's picker).

  • Catalog entry (LocalModels.catalog): RecursiveSI/flint-int4-gguf, the official Q4_1 GGUF (26 GB, needs 48 GB of memory per its card). Text only, so the blurb says it can't see screenshots. It sits after Qwen3.6 35B-A3B, which stays the recommended pick on 48 GB+ Macs because look_at_app needs vision.
  • llama.cpp b11346 → b11814: Flint's architecture landed in models: Add Flint model (FlintForCausalLM) ggml-org/llama.cpp#30928 (merged yesterday); b11814 contains it. Checksums are the release's published digests, and build.sh verified the arm64 one.
  • A context of its own: Lamp's prompt alone is over 20K tokens, so the usual 32K overflowed after a few turns (request (33506 tokens) exceeds the available context size). LocalModelSpec.contextSize gives Flint 131,072 (its cache is about 3.4 GB there: all but 13 of its 49 layers look back only 513 tokens). The size goes to llama-server through a presets file (models.ini, [*] for the rest at 32K, a section per model), since a -c on the command line overrides presets, and to pi's models.json.

Tested on a 64 GB M-series Mac, with the bundled llama-server started using Lamp's own router flags (--models-dir, --jinja, -ngl 999, -c 32768):

  • Flint loaded and answered an OpenAI tool-calling request with a correct tool call, at about 105 tokens/s.
  • With the presets file, a 51.5K-token request to Flint answered correctly; the server used about 28 GB in total.
  • Qwen3.5 4B GGUF still loads and calls tools on the new server (regression check).

Engines: only pi runs local models, so pi is the only engine this touches. I didn't run a full build in Lamp with Flint selected.

🤖 Generated with Claude Code

Flint is an open-weight coding model (40B MoE, 2.5B active) with an official
Q4_1 GGUF for Apple Silicon. It needs llama.cpp's new flint architecture, so the
bundled llama-server moves to b11814.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
forge-updates Ready Ready Preview Oct 31, 2026 7:27am UTC
lamp Ready Ready Preview Oct 31, 2026 7:27am UTC
lamp-updates Ready Ready Preview Oct 31, 2026 7:27am UTC

Request Review

@roomote-roomote

roomote-roomote Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

No new code issues found. See session

Reviewed the merge with current main, including conflict resolution in server startup and pi registration. Flint's 128K presets remain intact alongside the API key and worker environment; the b11814 pin feeds the new signed runtime-asset packaging and installer consistently.

CI: current-head macOS Swift build and website checks passed; regression tests were skipped by the PR workflow. Full app packaging and native Flint inference were not exercised in this review.

Reviewed 14e6809

Lamp's prompt alone is over 20K tokens, so the usual 32K ran out within a few
turns. Each catalog model now has its own context, given to llama-server through
its presets file (a size on its command line would override it) and to pi.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…el-option-516837

# Conflicts:
#	Sources/Lamp/LocalModels.swift
#	build.sh

This branch was successfully deployed

3 active (1 outdated) deployments
Preview – lamp — 14e68093 Deployed Oct 31, 2026 by vercel[bot]
Preview – lamp-updates — 14e68093 Deployed Oct 31, 2026 by vercel[bot]
Preview – forge-updates — 2de41f96 Deployed Oct 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant