generate_embedding_llama never checks the tokenized text's length against context size before sizing the batch/context #2969

Problem

The originally reported trigger (the document-ingestion chunker under-counting tokens) does not actually hold up — that chunker bounds chunk size to 1,536 characters, which would require an implausible <0.19 characters/token to overflow an 8,192-token context, so document ingestion is not the realistic exposure. The real, live exposure is different: EmbeddingService::semantic_search/semantic_search_nodes_of_type and the workspace-context assembly path (context_ops.rs) call generate_embedding(query) directly with no length cap at all on the query text, and context_ops.rs runs semantic schema retrieval against the user's own chat-message text on ordinary turns — meaning a user simply pasting a long block of text into chat is a realistic, everyday way to reach this unguarded path with an arbitrarily large, unchunked token count.

Root cause

get_or_create_context (packages/nlp-engine/src/embedding.rs:614, called via LlamaState::get_or_create_context) sizes LlamaContextParams::with_n_batch/with_n_ubatch off required_batch_size = max(tokens.len(), 512) with no comparison against self.context_size (default 8192) anywhere. ChatEngine::generate_blocking has an explicit, tested ContextOverflow guard for the identical class of input; the embedding path has no equivalent. Tracing llama.cpp's own source shows non-causal (embedding) models don't clamp n_batch/n_ubatch to n_ctx, and the pre-reserved compute buffer (sized for min(n_ctx, n_ubatch)) is smaller than what an oversized request actually needs — the likely outcome is a larger dynamic allocation (succeeding at extra memory/time cost, or failing gracefully as EncodeError), not a guaranteed hard crash, but it is genuinely unguarded.

Suggested fix

Add an explicit if tokens.len() > self.context_size { ... } guard before sizing the batch/context, mirroring ChatEngine::generate_blocking's ContextOverflow check, applied specifically to the actually-exposed unbounded callers (semantic_search/semantic_search_nodes_of_type's raw query text, and context_ops.rs's workspace-context query) rather than just the already-safely-chunked document-ingestion path.

Found via the 2026-09-25 full-repo code sweep (finding #19 of 84 in CODE_SWEEP_FINDINGS_2026-09-25.md). Severity: medium. Category: risky.