Repository navigation
Fix xCoRe always returning empty coreference clusters - #1
Merged
Merged
Conversation
Three bugs caused predict() to always return empty clusters: 1. model_cross.py: independent if-blocks overwrote soft-dot/soft-mc results The three if-blocks at lines 1732-1807 were independent, so the final else branch (preds["full_coreferences"] = []) always ran for soft-dot and soft-mc models because coref_hs is always empty for those representations. Fixed by converting the second and third if to elif. 2. xcore_model.py: CLS/SEP tokens missing with transformers >= 5 transformers 5.x changed DeBERTa-v3 tokenizer behaviour so add_special_tokens=True no longer inserts [CLS]/[SEP]. The model was trained with these boundary tokens, so without them the encoder produces nearly identical representations for all tokens and the start_token_classifier outputs ~0 probability for every token (logit ~-14), yielding zero detected mentions and empty clusters. Fixed by detecting the missing tokens and manually injecting them while patching word_ids() and word_to_tokens() for the +1 offset. 3. xcore_model.py: checkpoint loading fails with PyTorch >= 2.6 PyTorch 2.6 changed torch.load to use weights_only=True by default, which blocks PL checkpoints containing non-tensor globals (AttributeDict, DictConfig, etc.). Since PyTorch Lightning passes weights_only explicitly downstream, functools.partial cannot override it. Fixed by temporarily replacing torch.load with a wrapper that forces weights_only=False only for the duration of the checkpoint load. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This was referenced Mar 24, 2026
- Fix broken imports in evaluate.py (missing xcore. prefix) - Add torch.load weights_only=False patch to evaluate.py for PyTorch >=2.6 - Fix all Hydra _target_ paths across conf/data/ and conf/model/ to use fully-qualified xcore.* module paths (previously used bare module names that only worked when running from inside the xcore/ subdirectory) - Fix xdeberta-large_80000steps.yaml: set num_training_steps=80000 and num_warmup_steps=8000 (was incorrectly identical to the 10k-step config) - Fix conf/evaluation/default_evaluation.yaml: replace hardcoded absolute checkpoint path with Hydra ??? (required override) placeholder - Add conf/data/xanimalfarm.yaml for zero-shot Animal Farm evaluation - Add REPLICATION.md with step-by-step instructions for all paper experiments (Tables 3 and 4), expected results, and hyperparameter table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bug 1 (
model_cross.py): Three independentifblocks in the cluster-representation dispatch meant the finalelse: preds["full_coreferences"] = []always ran forsoft-dotandsoft-mcmodels, silently overwriting the correctly computed result. Fixed by converting the second and thirdiftoelif.Bug 2 (
xcore_model.py): transformers 5.x changed the DeBERTa-v3 tokenizer soadd_special_tokens=Trueno longer inserts[CLS]/[SEP]tokens. Without them the encoder produces nearly identical representations for every token and thestart_token_classifieroutputs ~0 for all positions, yielding zero detected mentions and empty clusters. Fixed by detecting the missing tokens after tokenization and manually injecting them, patchingword_ids()andword_to_tokens()to account for the +1 offset.Bug 3 (
xcore_model.py): PyTorch ≥ 2.6 changedtorch.loadto useweights_only=Trueby default, blocking PL checkpoints that contain non-tensor globals. Fixed by temporarily wrappingtorch.loadto forceweights_only=Falseonly during checkpoint load.Bug 4 (
setup.py): Pintransformersto>=4.34,<5to prevent accidental upgrade to 5.x until full compatibility work is complete (see issue Upgrade to transformers >= 5.x and retrain model checkpoints #2). Closes issue Pin transformers to 4.x to restore full model accuracy #3.Verification
Note: the
['Barack Obama', 'the president']cluster is not returned — this is a pre-existing model quality issue unrelated to these fixes (the model scores 'Barack' at 0.472, just below the 0.5 threshold, in both transformers 4.x and 5.x). Tracked in issue #2.Test plan
crossandlongmodes still work🤖 Generated with Claude Code