Repository navigation
Intermittent 'int' object has no attribute 'ranks' in tokenizer #1385
Description
During an OpenCode coding session, Strata logged this exception 12 times while preparing requests. OpenCode showed Bad Gateway, retried, and eventually finished. The Strata service did not restart and its cgroup OOM counters stayed at zero.
serve/server.py: prepare -> encode_prompt -> self.tok.encode
tools/strata_tokenizer.py: _encode_plain -> self._bpe(mapped)
tools/strata_tokenizer.py: _bpe
r = self.ranks.get((parts[i], parts[i + 1]))
AttributeError: 'int' object has no attribute 'ranks'
Environment
- Ubuntu 26.04, x86_64
- CPython 3.14.4, built Aug 20 2026, GCC 15.2.0
- regex 2026.9.10
- RTX 4090, Qwen3.8-Flash-Next IQ3_S, context limit 131072
- Original deployment: commit 6f32ec0, engine 0.1.39
- Update tested: v0.1.40.1, commit 82f46a8, engine 0.1.40
The same exception also occurred during the update test, in a separate Python client process calling the tokenizer to count a long synthetic prompt. It happened before that process sent the long-prompt HTTP request. The client did not load Strata's native engine or NVML bindings. The server was running in another process at the time.
What we checked
- The old and new
_bpeimplementations are identical. The failing line is in the short-word scan._bpereadsself.HEAP_MINearlier and does not assign toself. - Fresh CPU-only processes encoded the same synthetic prompt successfully on both commits: 41829 tokens. Another 100 suffix variants passed on each commit.
- A later full update test passed, including a 41864-token model request. This did not establish that the intermittent error was fixed.
- Separate fresh Python probes reported JIT available but disabled, and GIL enabled. We did not measure those states inside the original failing server process.
Synthetic workload
Synthetic workload
This describes the workload used in the CPU retests. It is not a reliable reproducer: those retests passed.
From the Strata checkout, with the affected pack's tokenizer loaded as tok:
unit = "record: 1234567890 abcdefghijklmnop\n" * 100
unit_tokens = len(tok.encode(unit))
filler = unit * (40000 // unit_tokens + 1)
halfway = len(filler) // 2
content = (
"SENTINEL=CEDAR917\n"
+ filler[:halfway]
+ "\nSENTINEL=MAPLE428\n"
+ filler[halfway:]
+ "\nSENTINEL=PINE652\n"
)
print(len(tok.encode(content)))
for case in range(100):
tok.encode(content + "\nCPU_CASE=" + str(case))We found #268, but that reports long-word tokenization latency rather than this exception. CPython python/cpython#156319 reports frame corruption with Tier 2 and active monitoring; we have not established those conditions here.
Is this a known tokenizer/runtime issue? If not, which process-level diagnostics or isolated Python-version comparison would help narrow it down? We currently have no confirmed root cause.
Activity
tokenizer: _bpe reads self once instead of per symbol pair, and a thr…
Niko1221 commentedon Oct 7, 2026
Thanks for the careful report, and for ruling out so much before sending it.
We read tools/strata_tokenizer.py end to end for a way
selfcould become an int inside_bpe: there is no cache or decorator on a method, no__slots__or__getattr__, no staticmethod/classmethod mix-up, no callback or thread that passes an argument in, and the regex module is only called forfindall/finditeron plain strings._bpeis pure Python, takesselffrom the callself._bpe(mapped)and does not assign to it. We also ran 8 threads sharing one tokenizer against a single-threaded run (new test in serve/test_detok.py) and every id matched. So we cannot reproduce it and we do not see a path in our code that produces an intself.That leaves what you pointed at: a fault in the interpreter. A local variable slot holding the wrong object in the middle of a loop, in a function that has run millions of times, is what a broken Tier 2 / JIT optimisation or frame bug looks like, and the CPython issue you linked (frame corruption with Tier 2 and active monitoring) matches that shape. It also fits that fresh processes on the same text pass.
What we changed in 0.1.40.3:
_bpenow readsself.ranks.getonce before the scan instead of re-readingselfon every symbol pair, which removes the line that failed from the hot loop. We want to be clear that this is hardening, not a confirmed fix.If you see it again, the most useful single test is to run the server with
PYTHON_JIT=0(and, if you can, once on CPython 3.13) and see whether the error goes away. If it still happens with the JIT off, please send the log and we will look at it again. Leaving this open until then.Merge Niko1221/Strata main (0.1.40.2 + 0.1.40.3) into main
tokenizer: _bpe reads self once instead of per symbol pair, and a thr…
mustafaerdinc commentedon Oct 8, 2026
I tested v0.1.40.3 (d5ea713) in a separate checkout with PYTHON_JIT=0.
The upstream CPU test now reproduces failures intermittently, without loading a model or sending an HTTP request:
STRATA_TOKENIZER was unset: four synthetic tests execute and five pack-only tests are skipped.
_bpe, line 167, duringtest_words_directly.TypeError: 'int' object is not callableat line 162,r = ranks_get((parts[i], parts[i + 1])), duringtest_synthetic_vocab.PYTHONMALLOC=debug.Both Linux environments use regex 2026.9.10 and MarkupSafe 3.0.3. Faulthandler lists only
markupsafe._speedupsandregex._regexas extension modules. This does not establish which component caused the failure.The 3.13 build is Astral python-build-standalone 20261003, configured with
--enable-experimental-jit=yes-off. PYTHON_JIT=0 was supplied at startup; ignore_environment=0, optimize=0 and GIL enabled=true. This build has nosys._jitAPI, so I cannot report its JIT state through that API.For comparison, the same source and package versions passed five fresh processes on macOS arm64 with CPython 3.14.8. A separate stdlib-only check using the exact Tokenizer class and bound
_bpe/_bpe_heapmethods also passed 22,981 synthetic words on both Linux interpreters. These controls differ in execution history or platform and do not rule out an intermittent fault.The Linux host has an i9-13900K, microcode 0x133. Readable kernel logs showed EDAC initialization and “No ECC support,” with no matching machine-check error. That does not establish hardware health.
I have kept the production service unchanged. Would the sanitized faulthandler logs or the bounded test runner be useful for the next investigation step?
Niko1221 commentedon Oct 13, 2026
Thanks for the report. To keep the tracker current we are closing issues older than three days: 0.1.44 is out with a long list of fixes, and many older reports are covered by it or by 0.1.43.1.
If this still happens on 0.1.44, please leave a comment here with your engine log (and the version line it prints), and we will reopen it. A new issue that links this one works too.