Selfhosting and AI nerd from Germany 🇩🇪
I play Minecraft, RPGs and survival games, and I build things around them.
- ⚡ optllama: a llama.cpp fork that I contribute to heavily, focused on KV cache offload to RAM, unbalanced dual-GPU setups and partial KV cache residency.
- 🧪 Piggidragon/llama.cpp: native flash attention kernels for Nvidia turing and upwards that decrease vram usage and inccrease speed on long context. Prompt caching is not only ram but on ssd too now so you dont need to recalculate old chats again.
- 🤖 Pithagoras: I'm an active contributor to this AI agent project. Image generation and editing with a gallery, voice improvements, UI polish and docs, and most recently the Devices add-on, which lets chats work with files and a shell on paired computers.
- 🦀 Nerve-Protocol: a single Rust binary that lets any harness reach any remote device natively, without SSH.
- 🧩 OpenWebUI-plugins: tools, functions and pipes for Open WebUI.
- ⛏️ Minecraft modding with NeoForge: a magic and dimension mod, plus an Ender Dragon fight overhaul.
| Project | What I contributed |
|---|---|
| generelschwerz/llama.cpp (optllama) | Host-resident KV cache: pipelined delivery, head-split cache under split mode tensor, partial KV residency budget on the slowest link first. Multi-GPU: attention split separate from the tensor split, tied output projection split, meta transport ring per device share. CUDA: quantized-native MMA FlashAttention. Also: DFlash speculative decoding, llama-bench placement and sweep flags, scheduler fixes and the arch-coverage test suite |
| thecodacus/pithagoras | Image generation and editing (several reference pictures, gallery, in-chat viewer), voice fillers, extension screens, animations, the Devices add-on, docs and release fixes |
Ask me about selfhosting, local AI, agents and Minecraft modding.