Skip to content

feat(constraint.py): Auto-convert hard to soft constraints through soften method - #904

Draft
isanchez-ng wants to merge 6 commits into
PyPSA:masterfrom
isanchez-ng:master
Draft

feat(constraint.py): Auto-convert hard to soft constraints through soften method#904
isanchez-ng wants to merge 6 commits into
PyPSA:masterfrom
isanchez-ng:master

Conversation

@isanchez-ng

Copy link
Copy Markdown

Closes #782
Hi! This is my first contribution. I created a draft just to let you know that I'm working on this. I'll try to complete this code the next weekend 🤓


Changes proposed in this Pull Request

Building soft constraints requires a lot of manual work that can be avoided through a method that modifies both the variables (to add the slacks) and the objective function (to add the penalty term).

Implementation

I followed the proposed structure for the method, but changed some details that I mention on the next section.
Also, I changed slightly the typehints for objective.__add__. to avoid some mypy errors that weren't actually errors.

How did I test this new feature?

For now this is just a draft, so the test has been mainly by code written on a notebook. At some point I'll add unit tests and maybe a notebook. On the mean time, I'll leave the testing code here:

import linopy
import pandas as pd

linopy.options["semantics"] = "v1"

add_budget_slack = True
add_risk_slack = False
budget_penalty = 2
penalty_risk = 0

# --- Toy data: just 3 investments ---
investments = pd.Index(["Investment A", "Investment B", "Investment C"], name="investments")
risk = pd.Series([0.5, 0.2, 0.8], index=investments, name="risk")
maximum_risk = 0.1 # 0.1 forces the slack on the budget to act.
expected_return = pd.Series([0.08, 0.03, 0.1], index=investments, name="expected_return")

# --- Model ---
m = linopy.Model()

# --- Variables ---
# Fraction of the portfolio allocated to each asset, between 0 and 1
w = m.add_variables(lower=0, upper=1, coords=[investments], name="weights")
m.add_objective((expected_return * w).sum(), sense="max")


# ---- Constraints ----
# Budget constraint: allocations must sum to 1 (fully invested). Option with and without slack.
if add_budget_slack:
    # easy version:
    budget_constraint = m.add_constraints(w.sum() == 1, name="budget")
    slack = budget_constraint.soften(penalty=budget_penalty)
else:
    budget_constraint = m.add_constraints(w.sum() == 1, name="budget_constraint")

# Risk constraint: keep the risk to be lower than 0.5. Option with and without slack.
if add_risk_slack:
    # easy version:
    # m.add_constraints((w * risk).sum() <= 0.5, name="total_risk", penalty=1e4)

    # complicated version:
    penalty_risk = 0.5
    risk_slack = m.add_variables(lower=0, name="risk_slack")
    risk_constraint = m.add_constraints((w * risk).sum() - risk_slack <= maximum_risk, name="total_risk")
else:
    risk_constraint = m.add_constraints((w * risk).sum() <= maximum_risk, name="total_risk")



m.solve(solver_name="highs", output_flag=False)

w.solution

Open questions/discussions

  • I decided to change the proposed behaviour of returning either a VariableLike or a tuple of variables, because in my experience this behaviour gets messy when the codebase grows (for example, people would have to manage the doubled behaviour through isinstance(var_name, tuple) through their own codes. Instead of that, I proposed a NamedTuple.
  • I decided to raise an assertion at the start of soften() if the model's objective hasn't been defined yet, since soften() adds a penalty term to the existing objective rather than replacing it.
    • It would be technically possible to let soften() run before add_objective(). But doing so creates a weird condition where the line model.objective += penalty things asserts the objective is still empty, so calling a second model.add_objective after soften() would raise an error telling the user to pass overwrite=True. Doing that, however, replaces the whole objective expression, and silently discards the penalty term soften() had already added.
    • soften() also relies on model.sense to pick the correct sign for the penalty term. Since sense defaults to "min" until model.add_objective() is called with a different value, calling soften() first risks silently penalizing in the wrong direction if the user later sets sense="max".

I'm open to discussion on both bullet points if someone else has a better proposal.

To-Dos:

  • Build unit tests for .soften method
  • Build wrapper of soften inside model.add_constraint
  • Maybe create an example of this in one of the notebooks, or create a new notebook. I don't know exactly what to do here so, any guidance from the mantainers will be appreciated.

Checklist

  • AI-generated content is marked (see AGENTS.md).
  • Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
  • Unit tests for new features were added (if applicable).
  • A note for the release notes doc/release_notes.rst of the upcoming release is included.
    • Don't know exactly what they mean with this 🤔 (it's first time colllaborating on this open source repo). If someone checks this draft, I'd appreaciate a small guidance. Does this just mean to add a small phrase of what's this doing?
  • I consent to the release of this PR's code under the MIT license.

Ignacia and others added 6 commits August 22, 2026 23:38
…oid alternating between a bare Variable and a tuple of Variables depending on the constraint's sign.

Negative is now None for inequality constraints instead of being absent from the return.
… a scalar operand) to accept ConstantLike.

The narrow annotation caused mypy to flag valid code
@codspeed-hq

codspeed-hq Bot commented Aug 23, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 1.2%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 3 regressed benchmarks
✅ 170 untouched benchmarks
⏩ 175 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory test_to_lp[milp-n=50] 2 MB 2.6 MB -23.08%
Memory test_to_lp[merge_balance-severity=0] 2.6 MB 3.2 MB -17.85%
Memory test_to_lp[nodal_balance-severity=50] 2.8 MB 3.3 MB -16.1%
Memory test_to_lp[rolling-severity=50] 429.7 MB 305.6 MB +40.61%
Memory test_to_lp[knapsack-n=10000] 2.8 MB 2.2 MB +26.3%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing isanchez-ng:master (f094077) with master (09c34dd)

Open in CodSpeed

Footnotes

  1. 175 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto-convert hard to soft constraints

1 participant