Repository navigation
/v1/embeddings ignores prompt_name / task_type, so every input is embedded as a query #758
Description
What happens
/v1/embeddings and /api/embeddings always embed with task_query, whatever the request asks for. For embed-gemma this silently selects the wrong prefix. A retrieval index built through the server stores every document with the query prefix (task: search result | query: ) instead of title: none | text: .
flm serve llama3.2:1b --embed 1
for t in query document; do
curl -s http://127.0.0.1:52625/v1/embeddings -H 'Content-Type: application/json' \
-d "{\"model\":\"embed-gemma:300m\",\"input\":\"The capital of France is Paris.\",\"prompt_name\":\"$t\"}" \
| jq -c '.data[0].embedding[:4]'
done
# identical vectorsOn embed-gemma:300m, cosine(query, document) for the same text is 1.000000. It should differ, because the prefixes differ. With the task passed through it is 0.957706.
An unknown name ("prompt_name":"not_a_task") or a non-string ("prompt_name":7) is also accepted, and the server answers 200 with a query embedding.
Cause
The engine already supports every task. embedding_task_type_t has ten values (auto_embedding_model.hpp:33-44), and Gemma_Embedding::_get_task_prefix maps each one to its prefix (modeling_gemma_embedding.hpp:13-54). The REST handler never reads the request's task and hardcodes embedding_task_type_t::task_query (rest_handler.cpp:900).
Expected
Read prompt_name, with task_type accepted as an alias, and pass the matching task to embed(). Refuse an unknown or non-string value with a 400 instead of embedding with a different task. A request with no task keeps today's task_query default.
Fix we are carrying
It maps the REST names (query/search_query, document/search_document, clustering, classification, STS, and so on) onto embedding_task_type_t and passes the result to embed(). It refuses unknown, non-string, or conflicting prompt_name/task_type values with 400 invalid_value. One resolved task applies to every element of an input array. The mapping and validation are ported from OpenFlowLM-Next's SERVER-EMBED-TASK-PROMPT (openai_compat::task_names/resolve_task/task_policy, Vegard Berget). The helpers need a FASTFLOWLM_LINUX_LIMITED_MODELS guard, as the patch shows. I can send this as a PR against main if that's useful.
Probe logs, before and after: https://github.com/noamsto/nix-amd-ai/blob/46d7a73c1044435cec0ff08410e1a3bc87cbc544/bench-logs/oflm-api-conformance-2026-09-28-after-174/README.md
Environment
FLM v1.0.6 built from the tag, with our downstream request-handling patches, none of which touched this call before the fix. Model: embed-gemma:300m via --embed 1.
Ryzen AI MAX+ 395 (Strix Halo), XDNA2 NPU, 8 columns. NixOS, Linux 7.2.8, amdxdna 0.10, NPU FW 1.1.2.65.
This is separate from the embedding-quality reports in #661, #647 and #732. It is about which prefix gets applied, not about the kernel output.
Activity