Repository navigation
Conversation
Three bugs caused predict() to always return empty clusters: 1. model_cross.py: independent if-blocks overwrote soft-dot/soft-mc results The three if-blocks at lines 1732-1807 were independent, so the final else branch (preds["full_coreferences"] = []) always ran for soft-dot and soft-mc models because coref_hs is always empty for those representations. Fixed by converting the second and third if to elif. 2. xcore_model.py: CLS/SEP tokens missing with transformers >= 5 transformers 5.x changed DeBERTa-v3 tokenizer behaviour so add_special_tokens=True no longer inserts [CLS]/[SEP]. The model was trained with these boundary tokens, so without them the encoder produces nearly identical representations for all tokens and the start_token_classifier outputs ~0 probability for every token (logit ~-14), yielding zero detected mentions and empty clusters. Fixed by detecting the missing tokens and manually injecting them while patching word_ids() and word_to_tokens() for the +1 offset. 3. xcore_model.py: checkpoint loading fails with PyTorch >= 2.6 PyTorch 2.6 changed torch.load to use weights_only=True by default, which blocks PL checkpoints containing non-tensor globals (AttributeDict, DictConfig, etc.). Since PyTorch Lightning passes weights_only explicitly downstream, functools.partial cannot override it. Fixed by temporarily replacing torch.load with a wrapper that forces weights_only=False only for the duration of the checkpoint load. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix broken imports in evaluate.py (missing xcore. prefix) - Add torch.load weights_only=False patch to evaluate.py for PyTorch >=2.6 - Fix all Hydra _target_ paths across conf/data/ and conf/model/ to use fully-qualified xcore.* module paths (previously used bare module names that only worked when running from inside the xcore/ subdirectory) - Fix xdeberta-large_80000steps.yaml: set num_training_steps=80000 and num_warmup_steps=8000 (was incorrectly identical to the 10k-step config) - Fix conf/evaluation/default_evaluation.yaml: replace hardcoded absolute checkpoint path with Hydra ??? (required override) placeholder - Add conf/data/xanimalfarm.yaml for zero-shot Animal Farm evaluation - Add REPLICATION.md with step-by-step instructions for all paper experiments (Tables 3 and 4), expected results, and hyperparameter table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This change fixes the issue with xcore returning empty clusters. But there are still performance issues.
Two transformers 5.x changes broke xCoRe: - transformers 5.0-5.3 stop inserting [CLS]/[SEP] for DeBERTa-v3 with add_special_tokens=True. Add encode_words(), which tokenizes without special tokens and wraps [CLS] ... [SEP] itself, and use it for both inference (xcore_model.py) and training (datasets.py). Training had no fallback before this change. - transformers 5.x loads deberta-v3-large in float16 by default, so the published checkpoints crashed with a Half/Float dtype mismatch. Load the encoder with dtype=torch.float32. With both fixes, encoder hidden states are bit-identical between 4.57.6 and 5.19.0, and sapienzanlp/xcore-litbank predicts the same clusters on 4.56.0, 4.57.6, 5.0.0 and 5.19.0, so the checkpoints need no retraining. Also reject processed-dataset caches built without [CLS], and replace the transformers<5 pin with >=4.56,<6 (4.56 adds the dtype keyword). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add Superset workspace setup and teardown scripts Setup builds a uv-managed .venv (torch, requirements.txt, editable xcore), symlinks data/ to the root checkout's datasets, and smoke-tests imports. Teardown removes only the data symlink. numpy is capped below 2 so the spaCy 3.7 / thinc 8.2 stack imports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Link datasets per entry so processed caches stay per workspace Address review: keep data/ a real directory with one symlink per root dataset (skipping cache_*), repair dangling links on re-run, recreate a broken .venv with --clear, report dataset status, and drop the now unneeded .gitignore rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Move dataset linking into link-data.sh with tests Address second review: teardown removes only links that point at the root's data/<name> and never runs in the root checkout, the root path falls back to git's common dir when SUPERSET_ROOT_PATH is unset, and the missing-data hint says to download into the root checkout. .superset/tests/test_link_data.sh covers linking, repair, migration, and removal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Harden link-data.sh against foreign links and path spellings Address third review: never write through a data symlink the script did not create, run from the git toplevel, canonicalize the root path, find the root via git worktree list, prune only dangling links into the root's data/, and recreate a broken .venv without uv's newer --clear flag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Simplify link-data.sh and fix dangling-link collision Address final review: a user's dangling data/<name> link no longer makes ln -s abort setup, one is_root_link helper decides ownership, the never-shipped whole-directory migration path is dropped (any data symlink is now left alone), setup.sh runs from the git toplevel, Python is pinned to 3.12, and constraints.txt caps transformers<5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Install in one resolve and tighten link-data.sh edge cases Address review: overrides.txt settles the sentencepiece conflict so torch, requirements.txt and the package resolve once with no churn on re-run; setup rebuilds a .venv on the wrong Python; link-data.sh rejects unknown arguments, cleans links to a deleted root data/, and warns about dangling links it leaves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep user state safe in setup and link-data.sh Address review: setup refuses to delete a .venv on the wrong Python, link-data.sh keeps links while the root data/ is missing (e.g. an unmounted drive), tolerates a regular file named data, and warns about dangling user links in the same pass that prunes root links. Tests pin git discovery to the temp dir. Drop the transformers cap that the single resolve made redundant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Read venv version once and test the git worktree root fallback Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Resolve conflicts with SapienzaNLP#13: keep transformers>=4.56,<6 instead of the ==4.34 pin, and keep the fork's equivalent torch.load weights_only patch. Take upstream's 0.1.3 version bump and .float() call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The .superset/ directory configures the fork maintainer's local development workspaces and is not part of xCoRe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes xCoRe run correctly on current PyTorch and on transformers 4.56 through 5.x, using the published checkpoints as they are. It also fixes the bug where
predict()always returned empty clusters, adds paper-replication configs and docs, and adds tests.Transformers 5.x support
Two transformers 5.x changes break xCoRe:
[CLS]/[SEP]. Transformers 5.0.0–5.3.0 stop inserting them for DeBERTa-v3 whenadd_special_tokens=True; 5.4.0 restores them. Without them the encoder output collapses and no mentions are detected. A newencode_words()inxcore/common/tokenization.pytokenizes without special tokens and wraps[CLS] … [SEP]itself. Inference (xcore_model.py) and training (datasets.py) both use it, so the input layout no longer depends on the installed version.deberta-v3-largein float16 by default, and checkpoint loading keeps that dtype, so inference fails withmat1 and mat2 must have the same dtype, but got Half and Float.model_cross.pynow loads the encoder withdtype=torch.float32.With both fixes, encoder hidden states are bit-identical between transformers 4.57.6 and 5.19.0.
sapienzanlp/xcore-litbankpredicts the same clusters on 4.56.0, 4.57.6, 5.0.0 and 5.19.0 as the original code does on 4.57.6, so the checkpoints do not need retraining.Other changes in this group:
transformers>=4.56,<6. Version 4.56 is the first that acceptsdtype=.xcoreDatasetrefuses a processed-data cache that was built without[CLS]and asks you to rebuild it.Other fixes
model_cross.py, three independentifblocks let the finalelseoverwrite the soft-dot and soft-mc predictions with[]. They are nowif/elif.xcore_model.pyandevaluate.pyforcesweights_only=Falsefor the trusted Lightning checkpoint.evaluate.pyhad broken imports, which are fixed. Hydra_target_paths now use fullxcore.*module paths, so commands run from the repository root.xdeberta-large_80000steps.yamlnow really uses 80,000 steps with 8,000 warm-up steps; it was a copy of the 10k config.default_evaluation.yamlno longer hardcodes an absolute checkpoint path.Additions
REPLICATION.md, with commands and expected scores for Tables 3 and 4, andconf/data/xanimalfarm.yaml.tests/test_tokenization.py, covering:collate_fnrun;XCORE_SLOW_TESTS=1).Testing
All 11 tests pass on transformers 4.56.0, 4.57.6 and 5.19.0. On 5.0.0, 10 pass and one comparison test skips, because that version's tokenizer omits the special tokens.
Relation to #13
This branch already merges #13, so there are no conflicts:
transformers==4.34with>=4.56,<6.torch.loadpatch, equivalent to the one in Fix PyTorch 2.6 checkpoint loading and pin transformers to 4.34 #13..float()call. That call is now redundant because the encoder loads in float32, but it is harmless.🤖 Generated with Claude Code