Repository navigation
Congratulations for the project, finally able to run something useful locally using this projectΒ #373
Description
Congratulations for the team and the contributors π‘οΈ
I was testing the limits of this project, for that I used Deepseek Harness along with ornith 1.5 35B A3B and this FreeToken.
Got a good cache 262144 and 15 to 19 tokens per seconds (RTX 5070ti 16GB, 96GB system ram)
The project https://github.com/rodjjo/pimbalgame
I validated using this #354
Not related to this project the only thing I saw was this funny thing (related to the model, not the project):
At the end the model just created a python script to concatenate "set" + "Scale" to fix the C++ files :D
Thanks, Hope we have support for vision and MTP for this model :)
Activity
MT-z commentedon Sep 4, 2026
Congratulations β I built it and ran it here, and read the repo alongside it. The thing that stayed
with me is that you got here without vision.
A pinball table is close to the least forgiving thing to build with a model that cannot see the
screen. Every bug in it is something you watch happen and then have to render into prose before the
model can act on it. Your commit messages are the proof:
Those are one screenshot each, and instead they are paragraphs β carefully measured ones, with
coordinates. A complete C++17 game with its own SVG-to-atlas asset pipeline, out of a 35B model with
~3B active on a 16 GB card, built that way. Thank you for posting it rather than just running it β I would not have seen it otherwise.
Which brings me to vision
I see vision is listed on the roadmap (#79). In the meantime I have been running the 3.5-family
path locally, and reading your issue I could not resist: I took a screenshot of your table and fed it to Ornith.
I would rather leave what came back for you to find β I suspect you will enjoy it.
The branch is
feat/vision-qwen3_5-fullβnine commits on current
main, served with--vision. It is not upstream.What I can tell you is only what I have seen here: the vision tower is ~0.8 GiB of resident bf16 on
this checkpoint.
Written with AI assistance; everything above was run on my own hardware (RTX 4090, sm_89).
rodjjo commentedon Sep 4, 2026
@MT-z Thanks! I'll tryout your branch.
By the way I added audio support to pinball game :D. When I said "I did" I mean the "model did", I did nothing except typing prompt on to create that program, even the commits, the pushes, the docs were created by the model itself. I installed https://en.wikipedia.org/wiki/Xvfb to have a "x environment" inside the sandbox, so the graphical application could run there, the model could map the screen into png then open with python read the pixels in numpy to see the pixels values.
The SVG were created by the model as well, i told it to create a tool to convert into png and put it inside the program as a header.
I'm just trying the limits of it.
rodjjo commentedon Sep 5, 2026
@MT-z Thanks, I was able to test your changes π‘οΈ
MT-z commentedon Sep 5, 2026
Your Xvfb-and-numpy account stayed with me, and it turned into two commits on that branch.
Reporting them here since they came out of reading it.
feat/vision-qwen3_5-fullis nowa08437f(wasc08372d); a fetch of the same branchpicks both up.
/imgin the terminal. The server takes images only as inlinedata:URLs, so untilnow sending one meant writing a client or assembling the base64 by hand.
ft shellnow has/img <path> [text]-- it reads the file, guesses the mime type, and puts the image aheadof the text; repeat it to attach several. So a screenshot goes in directly:
FREETOKEN_LOAD_VISION=1works again without--vision. All three processes thatbuild a model were pinning the gate to False on a bare
ft serve, so the environmentvariable could never be read. They now pin only when the flag is given, which means the
upstream env gate is enough on its own.
Both verified here on Ornith-1.5-35B-A3B-NVFP4 (RTX 4090, 24 GiB): a 448x448 PNG of three
shapes comes back named and coloured correctly, one image or several, with or without text
alongside it.
Assisted-by: Claude Opus 5
rodjjo commentedon Sep 8, 2026
Nice I merged several branchs and yours, so I could use .gguf models https://github.com/rodjjo/FreeToken/tree/test-freetoken-merges
I tried to make MTP to work too, no success (architectural related issues when we need to have a heavy offloading).
Also tried yarn scheduler for the KV cache to scale it with 2.0 (other branch).
I got a sweet spot here at test-freetoken-merges, Thanks again!
rodjjo commentedon Sep 8, 2026
@MT-z I also made the agent to create this to test vision https://github.com/rodjjo/agentic-book-writer