[samgsx]

Overview

Fixes #30834

Additional information

Problem

from_json() in common/jinja/value.cpp recursed once per nesting level of request-derived JSON (chat_template_kwargs, messages, tools) with no depth limit. A single unauthenticated POST /v1/chat/completions carrying a deeply nested value overflows the stack — uncatchable SIGSEGV, whole-server crash. global_from_json() converts every top-level key unconditionally, so the crash happens during context setup even when the template never references the payload.

Reproduced on master (Windows, MSVC, stories260K.gguf, --jinja)

• GET /health → {"status":"ok"}
• POST /v1/chat/completions with chat_template_kwargs.x = 100k nested arrays (200 KB body) → connection dropped, server process dies (stack overflow, no log output)
• GET /health → connection refused

Fix

Carry a depth counter through the from_json recursion and throw std::runtime_error past 128 levels (generous — real chat contexts are a handful of levels deep):

static constexpr size_t k_from_json_max_depth = 128;

After the fix, the same request returns cleanly and the server stays up:

{"error":{"code":400,"message":"Unable to generate parser for this template. Automatic parser generation failed: JSON nesting depth exceeds 128","type":"invalid_request_error"}}

GET /health → {"status":"ok"}, normal POST /v1/chat/completions still returns 200.

Note: the same request body could in principle also stress nlohmann::json::parse of the raw HTTP body; that path survived the depth used here (the crash was in from_json), but if maintainers want a hard cap on body-parse nesting too, that would be a separate guard at the HTTP boundary.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES — an AI coding assistant was used to analyze the codebase, implement the fix, and run the verification described above

[ggml-gh-bot]

Hi @origin-labs-ai, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

• Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 3 open PRs.

• AI-generated content: While code is allowed to be generated by AI, please write the PR description and commit messages on your own without the help of AI.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[nikwen]

There's a prior PR here with better test coverage: #31032

[ServeurpersoCom]

Thanks for the fix. #30242 covers the same crash and more: it checks the depth once at the entry, before every recursive walker, not only from_json. I tested both on master: a nested chat_template_kwargs is fixed by both, but a nested response_format schema still segfaults with this PR (it overflows in the schema builder) and returns a 400 with #30242. So I would focus on #30242.

[samgsx]

Thanks @ServeurpersoCom for testing both — agreed, #30242 is the more complete fix (entry-point depth check covering every recursive walker, including the response_format schema path this PR misses). Closing in favor of #30242. (Also thanks @nikwen for the #31032 pointer.)