Support for orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 #738

[10:51:15] cmd/INFO  detecting CPU/PCIe bandwidth to pick the MoE backend (ft bench bw)…
[10:51:45] cmd/INFO  ft serve --model D:\Cache\FreeToken_Model\Qwen3.8-Flash-Next-Uncensored-NVFP4 --port 1919 --moe-backend auto --max-running-requests 4 --memory-ratio 0.95 --moe-cpu-threads 16 --host 172.20.1.121 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420 --gpu GPU-caeb6cf1-3a04-7a6a-11e5-c93dc9bc561e
[10:51:45] stdout/INFO  serve started (pid=8096 model=D:\Cache\FreeToken_Model\Qwen3.8-Flash-Next-Uncensored-NVFP4 port=1919)
[10:51:49] stdout/WARN  [2026-10-19|10:51:49] WARNING  --moe-backend is deprecated; use --moe-strategy
[10:51:49] stdout/INFO  [2026-10-19|10:51:49] INFO     Parsed arguments:
[10:51:49] stdout/INFO  ServerArgs(model_path='D:\\Cache\\FreeToken_Model\\Qwen3.8-Flash-Next-Uncensored-NVFP4', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, hf_overrides={}, max_running_req=4, attention_backend='auto', moe_strategy='auto', quant_backend=None, ple_backend='disk', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=False, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=16, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.95, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, mm=MultimodalConfig(disabled_encoders=frozenset(), embed_cache_device='cpu', encoder_weights='host', image_min_tokens=None, image_max_tokens=None, processor_kwargs={}), max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=13324', _ipc_base_port=53143, server_host='172.20.1.121', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='Qwen3.8-Flash-Next-Uncensored-NVFP4', tool_call_parser='qwen3_coder', reasoning_parser='qwen3', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, allowed_media_domains='', allowed_local_media_path='', cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=('GPU-caeb6cf1-3a04-7a6a-11e5-c93dc9bc561e',), gpu_assigned=None)
[10:51:49] stdout/INFO  [2026-10-19|10:51:49|initializer] INFO     --gpu GPU-caeb6cf1-3a04-7a6a-11e5-c93dc9bc561e -> GPU-caeb6cf1-3a04-7a6a-11e5-c93dc9bc561e
[10:51:49] stdout/INFO  [2026-10-19|10:51:49|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=20, top_p=0.95
[10:51:49] stdout/INFO  INFO:     Started server process [13324]
[10:51:49] stdout/INFO  INFO:     Waiting for application startup.
[10:51:49] stdout/INFO  INFO:     Application startup complete.
[10:51:49] stdout/INFO  INFO:     Uvicorn running on http://172.20.1.121:1919 (Press CTRL+C to quit)
[10:51:52] stdout/INFO  D:\Cache\FreeToken_Engine\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py:81: FutureWarning: torch.cuda._set_allocator_settings is deprecated. Use torch._C._accelerator_setAllocatorSettings instead.
[10:51:52] stdout/INFO    scheduler = Scheduler(args)
[10:51:52] stdout/INFO  [2026-10-19|10:51:52|core|rank=0] INFO     Enabled expandable_segments (override via PYTORCH_ALLOC_CONF)
[10:51:52] stdout/INFO  [2026-10-19|10:51:52|core|rank=0] INFO     Auto-selected attention backend: qsa_sparse
[10:51:52] stdout/WARN  [2026-10-19|10:51:52|core|rank=0] WARNING  Page size is overridden to 64 for the qsa_sparse backend
[10:51:52] stdout/INFO  [2026-10-19|10:51:52|core|rank=0] INFO     Auto-selected MoE strategy: offload
[10:51:52] stdout/INFO  [2026-10-19|10:51:52|core|rank=0] INFO     No MoE cache sizing flag given; defaulting to --moe-cache-auto for auto-selected strategy 'offload'
[10:51:52] stdout/INFO  [2026-10-19|10:51:52|core|rank=0] INFO     Resolved config: moe_strategy='offload', attention_backend='qsa_sparse', cache_type='hybrid_radix', page_size=64
[10:51:52] stdout/INFO  [2026-10-19|10:51:52|core|rank=0] INFO     Free memory before loading model: 30.25 GiB
[10:51:52] stdout/INFO  [2026-10-19|10:51:52|core|rank=0] INFO     kernel triton selected; skipped torch: the checkpoint has no static activation scale  (×14)
[10:51:52] stdout/INFO  D:\Cache\FreeToken_Engine\FreeToken\venv\Lib\site-packages\torch\utils\_device.py:116: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:39.)
[10:51:52] stdout/INFO    return func(*args, **kwargs)
[10:51:52] stdout/INFO  [2026-10-19|10:51:52|core|rank=0] INFO     kernel triton selected; skipped torch: the checkpoint has no static activation scale  (×178)
[10:51:55] stdout/INFO  Loading weights: 100%|██████████| 18/18 [00:02<00:00,  6.41it/s]
[10:51:55] stdout/INFO  Process freetoken-TP0-scheduler:
[10:51:55] stdout/ERROR  [2026-10-19|10:51:55|FrontendAPI] ERROR    Backend supervisor: WeightLoadError: KeyError: 'model.layers.0.linear_attn.in_proj_qkvz.weight_scale'
[10:51:55] stdout/INFO  Traceback (most recent call last):
[10:51:55] stdout/INFO    File "python/freetoken/engine/engine.py", line 347, in freetoken.engine.engine._weight_load_context
[10:51:55] stdout/INFO    File "python/freetoken/engine/engine.py", line 604, in freetoken.engine.engine.Engine._load_weights
[10:51:55] stdout/INFO    File "D:\Cache\FreeToken_Engine\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 48, in load_state_dict
[10:51:55] stdout/INFO      param.load_state_dict(
[10:51:55] stdout/INFO    File "D:\Cache\FreeToken_Engine\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 48, in load_state_dict
[10:51:55] stdout/INFO      param.load_state_dict(
[10:51:55] stdout/INFO    File "D:\Cache\FreeToken_Engine\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 96, in load_state_dict
[10:51:55] stdout/INFO      op.load_state_dict(state_dict, prefix=_concat_prefix(prefix, str(i)), _internal=True)
[10:51:55] stdout/INFO    File "D:\Cache\FreeToken_Engine\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 48, in load_state_dict
[10:51:55] stdout/INFO      param.load_state_dict(
[10:51:55] stdout/INFO    File "D:\Cache\FreeToken_Engine\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 48, in load_state_dict
[10:51:55] stdout/INFO      param.load_state_dict(
[10:51:55] stdout/INFO    File "D:\Cache\FreeToken_Engine\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 43, in load_state_dict
[10:51:55] stdout/INFO      item = state_dict.pop(_concat_prefix(prefix, name))
[10:51:55] stdout/INFO             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[10:51:55] stdout/INFO  KeyError: 'model.layers.0.linear_attn.in_proj_qkvz.weight_scale'
[10:51:55] stdout/INFO  
[10:51:55] stdout/INFO  The above exception was the direct cause of the following exception:
[10:51:55] stdout/INFO  
[10:51:55] stdout/INFO  Traceback (most recent call last):
[10:51:55] stdout/INFO    File "C:\Users\annawz\AppData\Roaming\uv\python\cpython-3.12.11-windows-x86_64-none\Lib\multiprocessing\process.py", line 314, in _bootstrap
[10:51:55] stdout/INFO      self.run()
[10:51:55] stdout/INFO    File "C:\Users\annawz\AppData\Roaming\uv\python\cpython-3.12.11-windows-x86_64-none\Lib\multiprocessing\process.py", line 108, in run
[10:51:55] stdout/INFO      self._target(*self._args, **self._kwargs)
[10:51:55] stdout/INFO    File "D:\Cache\FreeToken_Engine\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 81, in _run_scheduler
[10:51:55] stdout/INFO      scheduler = Scheduler(args)
[10:51:55] stdout/INFO                  ^^^^^^^^^^^^^^^
[10:51:55] stdout/INFO    File "python/freetoken/scheduler/scheduler.py", line 68, in freetoken.scheduler.scheduler.Scheduler.__init__
[10:51:55] stdout/INFO    File "python/freetoken/engine/engine.py", line 420, in freetoken.engine.engine.Engine.__init__
[10:51:55] stdout/INFO    File "python/freetoken/engine/engine.py", line 603, in freetoken.engine.engine.Engine._load_weights
[10:51:55] stdout/INFO    File "C:\Users\annawz\AppData\Roaming\uv\python\cpython-3.12.11-windows-x86_64-none\Lib\contextlib.py", line 158, in __exit__
[10:51:55] stdout/INFO      self.gen.throw(value)
[10:51:55] stdout/INFO    File "python/freetoken/engine/engine.py", line 354, in _weight_load_context
[10:51:55] stdout/INFO  freetoken.engine.engine.WeightLoadError: KeyError: 'model.layers.0.linear_attn.in_proj_qkvz.weight_scale'
[10:52:03] health/ERROR  WeightLoadError: KeyError: 'model.layers.0.linear_attn.in_proj_qkvz.weight_scale'  (×5)
[10:52:05] stdout/ERROR  [2026-10-19|10:52:05|FrontendAPI] ERROR    Backend worker is gone and cannot be restarted; stopping the API server
[10:52:05] stdout/INFO  serve exited with code 15 (pid=8096)