Illegal hardware instruction in quantize step #41

Activity

1. tunglee35

   changed the title [-]Illegal hardware instruction in quantize[/-] [+]Illegal hardware instruction in quantize step[/+] on Mar 12, 2023
2. prusnak commented on Mar 12, 2023

   

   prusnak

   on Mar 12, 2023

   Contributor

   More actions

   What does cc --version and c++ --version say?
3. brozkrut commented on Mar 12, 2023

   

   brozkrut

   on Mar 12, 2023

   More actions

   This error happens when binary is compiled for x86_64 instead of arm64.

   This is common mistake if you are compiling under conda like environment.

   Check binary quantize with:
   file quantize

   On Apple Silicon Macs it should print:
   quantize: Mach-O 64-bit executable arm64

   If it shows x86_64 then you have to recompile after exiting conda environment:
   make clean
make

   👍React with 👍8Reacted by Seungwon, tung.le, Yi Hein Chai, Marcus, Joaquim Picó Mora, Sam Pullara, Stephan Janssen and Simon_NT

   ❤️React with ❤️5Reacted by Seungwon, Yi Hein Chai, Joaquim Picó Mora, Daniel Manners and Simon_NT

   🚀React with 🚀6Reacted by Oleksandr Nikitin, Andrew Harper, Seungwon, Yi Hein Chai, Joaquim Picó Mora and Simon_NT
4. tunglee35 commented on Mar 13, 2023

   

   tunglee35

   on Mar 13, 2023

   Author

   More actions

   @prusnak it said

   Target: x86_64-apple-darwin21.6.0
   Thread model: posix
   InstalledDir: /Library/Developer/CommandLineTools/usr/bin
   
5. tunglee35 commented on Mar 13, 2023

   

   tunglee35

   on Mar 13, 2023

   Author

   More actions

   Solved by re-install conda env. Thanks guys
6. tunglee35

   closed this as completedon Mar 13, 2023
7. bowbowbow commented on Mar 14, 2023

   

   bowbowbow

   on Mar 14, 2023

   More actions

   @brozkrut

   Thank you, it works as you said.
8. gjmulder

   added

   buildCompilation issuesCompilation issues

   on Mar 15, 2023
9. JerryYao80 commented on May 29, 2023

   

   JerryYao80

   on May 29, 2023

   Last edited by JerryYao80

   More actions

   @brozkrut I got the error in Windows 7:

   ERROR: /app/.devops/tools.sh: line 40 6 Illegal instruction ./main $arg2

   when I executed :

   --quantize "/models/7B/ggml-model-f16.bin" "/models/7B/ggml-model-q4_0.bin" 2

   and I checked my quantize file:

   quantize: ELF 64-bit LSB shared object, x86-64, version 1 <GNU/Linux> ...... for GNU/Linux 3.2.0, not stripped

   But the architecture of docker image named ghcr.io/ggerganov/llama.cpp:full is amd64:

   docker image ghcr.io/ggerganov/llama.cpp:full|grep Architecture
   "Architecture": "amd64"

   and my environment is:

   Docker Toolbox 1.13.1
   docker client: 1.13.1 os/arch: windows 7 /amd64
   docker server:19.03.12 os/arch:ubuntu 22.04 /amd64

   SO I'M CONFUSED:
   1 Must I re-compile quantize?
   2 How to re-compile if I use docker image?
   3 Is there anywhere that I can download directly?
10. amanreddy77 commented on Jan 19, 2024

    

    amanreddy77

    on Jan 19, 2024

    More actions

    This error happens when binary is compiled for x86_64 instead of arm64.

    This is common mistake if you are compiling under conda like environment.

    Check binary quantize with: file quantize

    On Apple Silicon Macs it should print: quantize: Mach-O 64-bit executable arm64

    If it shows x86_64 then you have to recompile after exiting conda environment: make clean make can you explain me clearly
11. amanreddy77 commented on Jan 19, 2024

    

    amanreddy77

    on Jan 19, 2024

    More actions

    Solved by re-install conda env. Thanks guys

    can you please explain me
12. phuongncn

    added a commit that references this issue on Apr 28, 2026

    iqk_mul_mat(ARM_NEON): adding bf16 support (ggml-org#41)

    ...

    fc89202
13. Anyesh

    added a commit that references this issue on May 20, 2026

    EVOKE: iSWA dual-cache support in kv_block_save/load (ggml-org#41)

    ...

    1f37e75
14. Anyesh

    added a commit that references this issue on Jun 29, 2026

    EVOKE: iSWA dual-cache support in kv_block_save/load (ggml-org#41)

    ...

    aed1071
15. zhsy12345689

    added 2 commits that reference this issue on Jul 30, 2026

    Merge pull request ggml-org#41 from tc-mb/feat/web-demo-readme

    ...

    d461145

    Merge pull request ggml-org#41 from tc-mb/feat/web-demo-readme

    ...

    ff224f9
16. polivitus

    added a commit that references this issue on Sep 16, 2026

    feat(l10n): patches 36-42 + ru.json 246 фраз — финальная русификация …

    ...

    608be91