Repository navigation
Model Request:Tencent-Hunyuan/Hy-MT2 #560
Description
Activity
The vocab_size being 120818 isn't divisible by 32, so the lm_head can never be packed into Q4NX tiles. Or currently, anyway.
At least, for the 1.7B. I haven't looked at the others yet. The 7B would be the most likely fit if any of them, but I wouldn't get your hopes up on the hunyuan-dense models.
The vocab_size being 120818 isn't divisible by 32, so the lm_head can never be packed into Q4NX tiles. Or currently, anyway.
At least, for the 1.7B. I haven't looked at the others yet. The 7B would be the most likely fit if any of them, but I wouldn't get your hopes up on the hunyuan-dense models.
well, i just thought that 1.7b will be faster, 7b
it's ok tooReacted by Atomic-GermIt looks like something equivalent to ggml-org/llama.cpp#22836 would need to be implemented.
Hy-MT2 is a family of “fast-thinking” multilingual translation models designed for complex real-world scenarios. I think 1.8b is the best way to run on npu, then we will have a translation api