Repository navigation
Sometimes crashes with UTF8 error #22
Description
Some prompts fail with:
thread 'main' panicked at 'called Result::unwrap() on an Err value: Utf8Error { valid_up_to: 0, error_len: None }', /home/kimt/.cargo/registry/src/index.crates.io-6f17d22bba15001f/llama_cpp_rs-0.3.0/src/lib.rs:528:46
I'm using this model: https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGUF
Anything I can do to try to fix this problem?
Activity
markcda commentedon Nov 9, 2023
Yes, see this commit in fork.
kimtore commentedon Nov 10, 2023
@markcda thanks for the workaround. Are some words now going to be dropped from the model's outputif they contain illegal UTF-8 characters?
markcda commentedon Nov 11, 2023
The above commit only made specific deployment of the bug. Roughly speaking, the response will be reset if it contains at least one incorrect character. But the application will not crash. To solve your specific problem, I advise you to read the following post.