Skip to content

transformers version is not correct #25

Description

@Navid-visual

Running the code faces an error because it cannot import LlamaTokenizer.

File "/local/home/.../SparseGPT/datautils.py", line 6, in <module>
    from transformers import AutoTokenizer, LlamaTokenizer
ImportError: cannot import name 'LlamaTokenizer' from 'transformers' 

If you update to the latest version you run into.a different issue:raise OSError(

OSError: Unable to load weights from pytorch checkpoint file for '/home/.../.cache/huggingface/hub/models--facebook--opt-125m/snapshots/27dcfa74d334bc871f3234de431e71c6eeba5dd6/pytorch_model.bin' at '/home/.../.cache/huggingface/hub/models--facebook--opt-125m/snapshots/27dcfa74d334bc871f3234de431e71c6eeba5dd6/pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

The best workaround is to use transformers v4.21.2 and remove the LlamaTokenizer from line 6 of datautils.py.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions