Skip to content

fix: ignore masked variable labels when matching constraints - #895

Merged
FabianHofmann merged 3 commits into
masterfrom
fix-remove-variables-masked
Aug 18, 2026
Merged

fix: ignore masked variable labels when matching constraints#895
FabianHofmann merged 3 commits into
masterfrom
fix-remove-variables-masked

Conversation

@FabianHofmann

Copy link
Copy Markdown
Collaborator

Closes #883. Builds on @YassineAbdelouadoud's #886, whose fix commit is included here unchanged.

Note

Everything below this line was generated by AI.

Changes proposed in this Pull Request

Model.remove_variables removed constraints that never referenced the removed
variable. A masked variable stores -1 in its label array, and -1 is also the
marker for an empty term slot in a constraint, so every masked variable looked
like it was used by every constraint that carried padded terms. Models built
with mask= could silently lose constraints and solve to a wrong optimum. The
fix (from #886) filters the sentinel out before matching labels, in
Constraint.has_variable, CSRConstraint.has_variable and the objective
update.

On top of that, this PR makes the matching faster and fixes a second, unrelated
failure in the same function.

Performance. The label matching runs once per constraint group for every
variable removal, so it is worth doing well.

  • ConstraintBase.has_variable is now concrete: it strips the sentinels once
    and delegates to a new abstract has_labels(labels). remove_variables
    calls has_labels directly, so the filtering pass happens once per removal
    instead of once per constraint group.
  • The new contains_labels helper in linopy/common.py replaces the plain
    isin. Labels are handed out in ascending blocks, so restricting the values
    to the label range is a cheap prefilter — and when the block turns out to be
    gap-free it is already the complete answer, which reduces the test to two
    comparisons instead of a sort-based membership check. Correctness was fuzzed
    against np.isin over 20k random cases.
  • CSRConstraint.has_labels maps the variable's labels to dense positions and
    tests those against csr.indices, rather than gathering
    vlabels[csr.indices] first. That trades an O(nnz) materialisation per
    constraint group for an O(n_labels) gather. Masked labels are absent from
    vlabels entirely, so positions are gap-free even for a masked variable and
    the fast path applies where the label-space one cannot.

Measured on 200k labels across 40 constraint groups, remove_variables end to
end:

case before after
unmasked, unfrozen 40.1 ms 8.6 ms
unmasked, frozen 54.4 ms 28.7 ms
masked, unfrozen 49.3 ms 9.9 ms
masked, frozen 47.5 ms 21.7 ms

The frozen cases gain less because most of their remaining time is spent
rebuilding the label index, not scanning.

Quadratic objectives. remove_variables raised
IndexError: Boolean array size 2 is used to index array with shape (2, 6) for
any model with a quadratic objective, because the term mask was built over the
factor dimension as well as the term dimension. The mask is now reduced with
any(FACTOR_DIM) when that dimension is present, so a quadratic term is
dropped as soon as any of its factors references the removed variable. This
failed on master too and is independent of the masking bug.

The regression test is parametrised over freeze and over linear versus
quadratic objectives.

Known limitation, not addressed here

A masked variable inside a quadratic term leaves a [-1, other_label] factor
pair, and the quadratic encoding reads a -1 factor as "this term is linear".
So (a * c).sum() with a masked silently contributes a linear cost on c.
This is present on master before any removal happens and deserves its own
issue rather than a fix folded in here.

Checklist

  • AI-generated content is marked (see AGENTS.md).
  • Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
  • Unit tests for new features were added (if applicable).
  • A note for the release notes doc/release_notes.rst of the upcoming release is included.
  • I consent to the release of this PR's code under the MIT license.

YassineAbdelouadoud and others added 3 commits August 11, 2026 15:36
`Model.remove_variables` deleted every constraint whose `has_variable`
matched the removed variable. For a masked variable that was every
constraint with a padded term slot: `-1` marks a masked entry on the
variable side and an empty term slot on the constraint side, and the
two were compared without filtering the sentinel.

Filter the -1 entries out of the variable labels before matching, via a
new `common.assigned_labels` helper, in both `has_variable`
implementations and in the objective cleanup of `remove_variables`.
Only the dense `Constraint` path was actually affected -- a CSR matrix
stores no empty slots -- but the same guard now holds for both.

Fixes #883

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Filter masked labels once per removal instead of once per constraint,
match via the contiguous label range, and test CSR positions instead of
gathering labels. Reduce the objective term mask over the factor
dimension so remove_variables works with a quadratic objective.
@codspeed-hq

codspeed-hq Bot commented Aug 17, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 1.22%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 2 regressed benchmarks
✅ 171 untouched benchmarks
⏩ 175 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory test_to_lp[storage-n=250] 28.2 MB 36.3 MB -22.27%
Memory test_to_lp[expression_arithmetic-n=250] 41 MB 46.9 MB -12.4%
Memory test_to_lp[knapsack-n=10000] 2.7 MB 2.3 MB +21.4%
Memory test_to_lp[storage-n=10] 2.6 MB 2.3 MB +15.16%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix-remove-variables-masked (19288f0) with master (5d22e35)

Open in CodSpeed

Footnotes

  1. 175 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@FabianHofmann
FabianHofmann merged commit 65e36de into master Aug 18, 2026
21 of 23 checks passed
@FabianHofmann
FabianHofmann deleted the fix-remove-variables-masked branch August 18, 2026 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

remove_variables deletes unrelated constraints: -1 means "masked" in a variable but "empty term slot" in a constraint

2 participants