Skip to content

tensor 'tok_embeddings.weight' has wrong size in model file #14

Closed

Description

@shaunabanana

When trying to run the 13B model the following output is given:

main: seed = 1678543550
llama_model_load: loading model from './models/13B/ggml-model-q4_0.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx   = 512
llama_model_load: n_embd  = 5120
llama_model_load: n_mult  = 256
llama_model_load: n_head  = 40
llama_model_load: n_layer = 40
llama_model_load: n_rot   = 128
llama_model_load: f16     = 2
llama_model_load: n_ff    = 13824
llama_model_load: ggml ctx size = 8559.49 MB
llama_model_load: memory_size =   800.00 MB, n_mem = 20480
llama_model_load: tensor 'tok_embeddings.weight' has wrong size in model file
main: failed to load model from './models/13B/ggml-model-q4_0.bin'

I have followed the commands in the readme to quantize the model, i.e.:

python3 convert-pth-to-ggml.py models/13B/ 1
./quantize ./models/13B/ggml-model-f16.bin   ./models/13B/ggml-model-q4_0.bin 2
./quantize ./models/13B/ggml-model-f16.bin.1 ./models/13B/ggml-model-q4_0.bin.1 2

I am using a M1 MacBook Pro. Any thoughts on how to resolve this issue?

Activity

  1. shaunabanana commented on Mar 11, 2023

    @shaunabanana
    Author

    I just realized that I had been using binaries (quantize and main) compiled from a previous version. Recompiling solved the issue. Thank you for your awesome work!

  2. added a commit that references this issue on Apr 10, 2023
  3. added a commit that references this issue on Jun 2, 2023
  4. added a commit that references this issue on Oct 22, 2025
  5. added a commit that references this issue on Nov 4, 2025
  6. added a commit that references this issue on Nov 27, 2025
  7. added a commit that references this issue on Feb 12, 2026
  8. added a commit that references this issue on Mar 14, 2026
  9. added a commit that references this issue on Mar 17, 2026
  10. 14 remaining items

  11. added a commit that references this issue on May 22, 2026
    c16fe6d
  12. added a commit that references this issue on Jul 16, 2026
  13. added 2 commits that reference this issue on Jul 30, 2026
  14. added a commit that references this issue on Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildCompilation issues

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions