Skip to content

feat(models): add learning curves for XGBoost and GBLinear - #1183

Open
fahad-ee wants to merge 1 commit into
OpenSTEF:release/v4.0.0from
fahad-ee:feat/947-learning-curves
Open

fahad-ee wants to merge 1 commit into
OpenSTEF:release/v4.0.0from
fahad-ee:feat/947-learning-curves

Conversation

@fahad-ee

@fahad-ee fahad-ee commented Nov 9, 2026

Copy link
Copy Markdown

Relates to #947. Covers XGBoost and GBLinear; LightGBM will follow in a separate PR.

What this adds

  • LearningCurveMixin with a learning_curves table (train and validation metric per boosting round) and plot_learning_curves().
  • LearningCurvePlotter: a Plotly chart of the two curves with the best validation round marked.
  • Support in XGBoostForecaster and GBLinearForecaster.
  • A "Learning curves" section in the model explainability tutorial (notebook kept in sync).

Design notes

  • XGBoost and LightGBM name their evaluation sets by position, so train and validation are mapped by order, not by name.
  • The metric name is not hardcoded; the first recorded metric is used (mean_pinball_loss in OpenSTEF).
  • Training behavior and forecasts are unchanged.

Why LightGBM is not included

MultiQuantileRegressor.fit() accepts eval_set but does not pass it to the base learners, so evals_result_ is empty after fitting. I plan a separate PR for that fix, then LightGBM curves. Happy to change the plan if you prefer otherwise.

Testing

  • 19 new unit tests; the models test suite passes.
  • I ran the tutorial on the Liander data: the validation curve flattens after about 20 rounds and does not rise, so there is no overfitting in this example.
  • On Windows, one existing MLflow path test fails (test_normalize_tracking_uri[absolute-posix]). It is unrelated to this change.

@github-actions github-actions Bot added the feature New feature or request label Nov 9, 2026
Record the per round train and validation metric history and expose it as a learning_curves table and a plot_learning_curves() figure, so overfitting is easy to spot. Adds tests and a section in the model explainability tutorial.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AXEaoRCwBuFiLDca7M47FT
Signed-off-by: Engr. Muhammad Fahad <muhammadfahadx2.0@gmail.com>
@fahad-ee
fahad-ee force-pushed the feat/947-learning-curves branch from 670e013 to 9d2a1c7 Compare November 9, 2026 07:10

@JanMaartenvanDoorn JanMaartenvanDoorn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good, could you please target Main instead of OpenSTEF:release/v4.0.0 ?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants