Skip to content

fix(sos): order a set by its declaration, not by its coordinate labels - #893

Merged
FabianHofmann merged 4 commits into
PyPSA:masterfrom
fluxopt:feat/sos-weights
Aug 18, 2026
Merged

fix(sos): order a set by its declaration, not by its coordinate labels#893
FabianHofmann merged 4 commits into
PyPSA:masterfrom
fluxopt:feat/sos-weights

Conversation

@FBumann

@FBumann FBumann commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

(your intent here, handwritten)

Note

The following content was generated by AI.

Closes #892.

An SOS set now runs in the order its members are declared along sos_dim, rather than the order its coordinate labels sort into — the array order linopy reads everywhere else (isel, shift, roll, diff).

Why

Making the labels the SOS weights had two consequences:

  • a string-labelled dimension could carry no set at all (ValueError: SOS constraint requires numeric coordinates), reaching piecewise the day its breakpoints are labelled [low, mid, high];
  • numeric labels declared out of order silently redefined which members are adjacent. Two models identical member-for-member reached different optima depending only on what the coordinates were named, both reported optimal.

What does not change

Ascending numeric coordinates state the same order and are still passed through as the member weights, so a model whose SOS dimension is sorted is unaffected down to the bytes of its LP file — every piecewise formulation among them, since _validate_numeric_breakpoint_coords already requires strictly increasing breakpoints.

Descending coordinates are also meaning-preserving: reversing a set maps its consecutive pairs {i, i+1} to {n-i, n+1-i}, which are consecutive too, so SOS2's feasible set is unchanged. Only a sos_type=2 set whose numeric coordinates neither ascend nor descend genuinely regroups; it now warns, and sorting the index restores the previous behaviour. An sos_type=1 set cannot change meaning at all — its feasible region never depended on the order, reformulate_sos1 being x <= M * y and sum(y) <= 1 — so it stays silent.

Notes for review

  • The numeric-coordinate requirement is dropped for both SOS types.
  • reformulate_sos_constraints loses its argsort: that permutation is now always the identity, so it is deleted rather than rewritten. reformulate_sos2 selects the neighbouring segment indicators with isel instead of sel.
  • test_sos2_unsorted_coords_matches_sorted passes both before and after, its best pair happening to coincide either way. That is why this was never caught, and why the new tests pick coefficients that separate the two orders.

Commits

  1. test(sos) — five tests pinning the overlap where labels and declaration agree, plus seven strict xfails marking where they part. Self-verifying at that commit.
  2. fix(sos) — the rule, with the xfail markers removed.

@codspeed-hq

codspeed-hq Bot commented Aug 15, 2026

Copy link
Copy Markdown

Merging this PR will regress 2 benchmarks

⚡ 6 improved benchmarks
❌ 2 regressed benchmarks
✅ 167 untouched benchmarks
⏩ 175 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory test_to_lp[sos-n=1000] 2.4 MB 2.8 MB -17.08%
Memory test_to_lp[knapsack-n=10000] 2.2 MB 2.7 MB -15.96%
Memory test_to_lp[masked-n=100] 2.7 MB 2.1 MB +29.25%
Memory test_to_lp[merge_balance-severity=0] 3.2 MB 2.6 MB +21.84%
Memory test_to_lp[rolling-severity=50] 429.7 MB 356.2 MB +20.63%
Memory test_to_lp[nodal_balance-severity=50] 3.2 MB 2.9 MB +10.99%
Memory test_to_lp[storage-n=10] 2.9 MB 2.6 MB +10.33%
Memory test_to_lp[storage-n=250] 36.1 MB 32.8 MB +10.13%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fluxopt:feat/sos-weights (b90e1ee) with master (65e36de)

Open in CodSpeed

Footnotes

  1. 175 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@FBumann
FBumann force-pushed the feat/sos-weights branch 4 times, most recently from ffef6dc to fb79d90 Compare August 15, 2026 08:47
FBumann and others added 2 commits August 15, 2026 10:49
An SOS set's weights are its coordinate labels, so a solver orders the
members by label value rather than by declaration. Where the labels ascend
the two coincide — the common case, and every piecewise model.

Five tests pin that overlap; seven strict xfails mark where they part.

Refs PyPSA#892

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An SOS set now runs in the order its members are declared along sos_dim,
the array order linopy reads everywhere else. Ascending numeric coordinates
state the same order and are still passed through as the member weights, so
a model whose SOS dimension is sorted is unaffected, every piecewise
formulation among them. Anything else is weighted by position, which lifts
the numeric-coordinate requirement.

A sos_type=2 set whose numeric coordinates do not ascend changes meaning
and warns. An sos_type=1 set cannot: its feasible region never depended on
the order.

The reformulation loses its argsort, now always the identity, and selects
the segment indicators positionally.

Closes PyPSA#892

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@FabianHofmann

Copy link
Copy Markdown
Collaborator

this looks good. I would include this in a bugfix release

np.diff underflows on unsigned coords, so a non-ascending uint index
looked ascending: no warning, and the LP/native weights disagreed with
the reformulation order. Excludes bool from the numeric weight dtypes,
renames the helpers to coords_ascend/coords_reorder_set, and tidies the
release note and ordering docs.

@FabianHofmann FabianHofmann left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a commit about for negative values and some smaller refactorings. this is good to go

@FabianHofmann
FabianHofmann marked this pull request as ready for review August 18, 2026 10:30
@FabianHofmann
FabianHofmann merged commit 9129b35 into PyPSA:master Aug 18, 2026
33 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add_sos_constraints conflates coordinate labels with SOS weights: string dims rejected, unsorted numeric labels silently reorder SOS2 adjacency

2 participants