Repository navigation
[Bug] Qwen3 Q4 DWQ Models Fail with Errors (qwen3-4b/8b/14b-dwq-053125 All Fail) #108
Description
The code functions properly with standard Qwen3 Q4 MLX models. Nevertheless, when I switch to the Q4 DWQ models, errors occur. I've tested several models such as qwen3-4b-dwq-053125, qwen3-8b-dwq-053125, and qwen3-14b-dwq-053125, but all of them result in errors.:
Traceback (most recent call last):
File "/Users/Larkin/workspace/gitee/ai/lmstudio-json.py", line 17, in <module>
result = model.respond("/no_think\nTell me about The Hobbit", response_format=BookSchema)
lmstudio.LMStudioServerError: Chat response error: Error rendering prompt with jinja template: "Parser Error: Expected closing statement token. OpenSquareBracket !== CloseStatement.".
This is usually an issue with the model's prompt template. If you are using a popular model, you can try to search the model under lmstudio-community, which will have fixed prompt templates. If you cannot find one, you are welcome to post this issue to our discord or issue tracker on GitHub. Alternatively, if you know how to write jinja templates, you can override the prompt template in My Models > model settings > Prompt Template.
The script:
"""Example script demonstrating an interactive LLM chatbot."""
import json
import lmstudio as lms
class BookSchema(lms.BaseModel):
"""Structured information about a published book."""
title: str
author: str
year: int
# model = lms.llm("qwen3-8b-mlx") # runs well
model = lms.llm("qwen3-8b-dwq-053125") # runs with error
# # add /no_think to the prompt to disable qwen3 thinking
result = model.respond("/no_think\nTell me about The Hobbit", response_format=BookSchema)
book = result.parsed
print(json.dumps(book, indent=2))env：m4 16g ram
Activity
12bitmisfit commentedon Jun 30, 2025
Have you tried copying the jinja template from a working model and setting that in lm studio?
larria commentedon Jul 4, 2025
No, actually, the dwq model runs perfectly well in LM Studio App with no preset values at all. That’s why I figure the same default template is being used when calling the Python API. And even if I needed to specify it manually, I wouldn't know what that default template is. Any guide for me to try more will be appreciated~
ncoghlan commentedon Jul 24, 2025
@larria Would you be able to check if this problem still affects you with version 0.3.20 of the main LM Studio app providing the server API?
(This template issue was also reported against the mlx-engine component: lmstudio-ai/mlx-engine#169)