wave energy converters and new wind and wave data modules by lmezilis · Pull Request #475 · PyPSA/atlite

lmezilis · 2025-11-05T12:53:00Z

Closes # (if applicable).

Changes proposed in this Pull Request

Checklist

Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
Unit tests for new features were added (if applicable).
Newly introduced dependencies are added to environment.yaml, environment_docs.yaml and setup.py (if applicable).
A note for the release notes doc/release_notes.rst of the upcoming release is included.
I consent to the release of this PR's code under the MIT license.

for more information, see https://pre-commit.ci

euronion · 2025-11-06T09:00:57Z

Thanks a lot for this nice feature!

Can you let us know when you are ready for us to review it?
It would also be nice to have a short example, e.g. a ipynb to include into the documentation.

@brynpickering Can I ask you to review it if you have capacity? Thanks!

lmezilis · 2025-11-06T09:23:13Z

Thank you!

I think we can review it right away, and I can prepare some files for documentation. Apologies for being very new to github procedures. I am slowly starting to get the hang of it.

euronion · 2025-11-06T09:38:30Z

[...] Apologies for being very new to github procedures. I am slowly starting to get the hang of it.

No worries, good that you mention it! We'll help you get settled in and don't hesitate to ask questions if something is unclear or to ping us (e.g. using @euronion or @brynpickering).

brynpickering

Thanks for opening this PR @lmezilis ! As @euronion has mentioned, some documentation is needed. My comments are about implementation. I have no comments on the method, I'm sure you've got that right! There's lots of scope to clean things up, though, which will help us maintain the feature in future.

If you want any clarification on comments/suggestions I've made, just reply directly on the comment. If you are happy with a suggestion, you can just accept it and it will automatically update the code for you. You can always accept a suggestion but then make edits to it later.

One thing missing from the new files is a REUSE header. You can find equivalents in other files. The easiest is to just copy the one across from era5.py. Then, make sure your name is in AUTHORS.rst so you count as one of the "Contributors to atlite".

brynpickering · 2025-11-06T10:36:30Z

 features = {
    "height": ["height"],
-    "wind": ["wnd100m", "wnd_shear_exp", "wnd_azimuth", "roughness"],
+    "wind": ["wnd100m", "wnd_azimuth", "roughness"],


you probably didn't mean to delete "wnd_shear_exp". I assume this is needed by other atlite methods.

Yes I was working with an older version of atlite, this was accidental.

brynpickering · 2025-11-06T10:37:44Z

+    "wave_height": ["wave_height"],
+    "wave_period": ["wave_period"],


Collapse this into a wave: ["wave_height", "wave_period"] option. Then users can request the wave feature and get both of these variables. It's unlikely they'd ever want one but not the other.

Correct, thank you.

brynpickering · 2025-11-06T10:39:22Z

+    Optionally (add_lon_lat, default:True) preserves latitude and
+    longitude columns as 'lat' and 'lon'.


you mention this option in the docstring but it isn't in the method signature. Do you want to include this option or not?

this option was not necessary for the mrel wave files. I eventually renamed the dimentions in the last part of the script. I would say that the function "_rename_and_clean_coords" can be deleted here. do you think we should include this option?

EDIT: I saw the rest of the comments now, I will include the function and try to make it as similar to era5 as possible.

brynpickering · 2025-11-06T10:49:26Z

+def get_data_wave_height(ds):
+    ds = ds.rename({"hs": "wave_height"})
+    ds["wave_height"] = ds["wave_height"].clip(min=0.0)
+
+    return ds
+
+
+def get_data_wave_period(ds):
+    ds = ds.rename({"tp": "wave_period"})
+    # ds["wave_period"] = (1 / ds["wave_period"])
+    ds["wave_period"] = ds["wave_period"].clip(min=0.0)
+
+    return ds


Keep the process the same as in era5.py - split this into data retrieval and then sanitisation and call the sanitisation function only if requested in get_data (you can copy most of the functionality directly over from era5.py)

EDIT: you don't actually need these get_data_... methods in this module, but having sanitize_... methods would be good. They can then be called iteratively in get_data as is done in era5.py

you are correct, I was working with these files a long time ago, figuring out how which functions I need, will correct this!

brynpickering · 2025-11-06T10:50:49Z

+
+def get_data_wave_period(ds):
+    ds = ds.rename({"tp": "wave_period"})
+    # ds["wave_period"] = (1 / ds["wave_period"])


It's always better to rely on version history to recover lines of code you no longer need, rather than commenting them out. So, feel free to delete all your commented out lines!

Yes, sorry for this, I thought I did that for every file but forgot this one.

brynpickering · 2025-11-06T11:32:05Z

+def convert_wave(ds, wec_type):
+    power_matrix = pd.DataFrame.from_dict(wec_type["Power_Matrix"])


You should have a docstring here

brynpickering · 2025-11-06T11:32:54Z

+    """
+    Generate wave generation time series
+
+    evaluates the significant wave height (Hs) and wave peak period (Tp)


Since Hs and Tp are MREL-specific and wouldn't make sense if using ERA5 data, it might make more sense to not reference them by these acronyms in this file.

brynpickering · 2025-11-06T11:35:33Z

 # Ignore IDE project files
 .idea/
 .vscode
+.vs


It's generally best to add these pointers to your own "global" gitignore, rather than to every project you work on. That way, it never accidentally slips in without you realising it!

brynpickering · 2025-11-06T11:45:37Z

Since CERRA is available via the Climate Data Store, we should follow the same approach to data retrieval as with ERA5. You could probably just copy era5.py directly and delete all but the wind getter method, then adapt the wind getter method to match the data available from CERRA.

It might be best to drop this from this PR, though, and bring it in separately later. I can see a benefit to changing the features we bring in for wind since we can retrieve wind speed at various height/pressure levels from CDS, which would allow us to create a wind vertical profile (as we do with ERA5 data).

The problem with CERRA here is that there is a lot of preprocessing of data in order for atlite to be able to read it. I agree that it is best to review this later. Should I take an action on this or can you simply reject this file?

The easiest is for you to simply delete this file (and reference to cerra elsewhere). If we want to revisit it, this file will still be in the commit history so we can bring it back if we want anything from it!

lmezilis · 2025-11-07T14:44:21Z

Hello @brynpickering, I have made all of the changes locally, should I commit changes in the forked branch or is there another way to continue?

…/atlite into wecmatrices-datamodules

for more information, see https://pre-commit.ci

…/atlite into wecmatrices-datamodules

for more information, see https://pre-commit.ci

…/atlite into wecmatrices-datamodules

for more information, see https://pre-commit.ci

…/atlite into wecmatrices-datamodules

for more information, see https://pre-commit.ci

…/atlite into wecmatrices-datamodules

for more information, see https://pre-commit.ci

brynpickering · 2025-11-12T14:58:01Z

Hello @brynpickering, I have made all of the changes locally, should I commit changes in the forked branch or is there another way to continue?

Yes, in the forked branch. You should be able to just always work in the forked branch and push to your own repository ("origin") whenever you make changes. Those changes will then be made visible in this PR

brynpickering

Thanks for the changes @lmezilis. Just a couple of extra comments.

I still need to check the processing method, which I'll do now. In the meantime, it would be great to add some documentation. The easiest would be to add a jupyter notebook to examples and then link that into the docs by adding an entry into doc/examples (following the example of the other entries). The example notebook could load from era5 and mrel separately and maybe compare the results for a specific gridcell?

brynpickering · 2025-11-12T14:59:03Z

The easiest is for you to simply delete this file (and reference to cerra elsewhere). If we want to revisit it, this file will still be in the commit history so we can bring it back if we want anything from it!

brynpickering · 2025-11-12T15:01:25Z

+    """
+    Rename and sanitize retrieved wave height data.
+    """
+    ds = ds.rename({"hs": "wave_height"})


No need, this renaming is happening in the main function now (rename(features)).

brynpickering · 2025-11-12T15:01:30Z

+    """
+    Rename and sanitize retrieved wave height data.
+    """
+    ds = ds.rename({"tp": "wave_period"})


No need, this renaming is happening in the main function now (rename(features)).

…/atlite into wecmatrices-datamodules

for more information, see https://pre-commit.ci

lmezilis · 2025-11-20T17:52:45Z

I made all of these similar commits because there are some things that I need to change in the syntax, but the pre-commit auto fix changes them back. I don't know why.

brynpickering · 2025-11-24T11:07:21Z

@lmezilis no worries. We'll probably squash all these commits when we merge it in, so it'll all be cleaned up.

You could install pre-commit locally so the fixes are managed locally. In your atlite working environment call pre-commit install. Then pre-commit will fix things before you try to commit.

RE allowing data downloads, I've found that the OpenDAP fails when trying to download more than a few MB of data at once (DAP failure or Authorization failure). Not sure if you get this issue @lmezilis but it seems to me that it's too volatile to rely upon as a way to access the data.

lmezilis · 2025-11-25T09:25:57Z

Yes I had the same problem the last few days even though last week I could complete it. I say for now lets keep it manual, and I will contact the server to see what we can do.

FabianHofmann · 2026-08-13T10:49:44Z

@lmezilis we have some merge conflicts now. Do you want to resolve them by yourself or should I take over? I would review afterwards

…odules

for more information, see https://pre-commit.ci

…/atlite into wecmatrices-datamodules

for more information, see https://pre-commit.ci

…/atlite into wecmatrices-datamodules

…atrices-datamodules

for more information, see https://pre-commit.ci

…atrices-datamodules

lmezilis · 2026-08-25T13:50:36Z

@FabianHofmann I made some adjustments and finally the pre-commit check is completed. I think we are ready to finalize this PR.

lmezilis and others added 2 commits October 16, 2025 12:52

atlite-mrel first commit

5f87745

[pre-commit.ci] auto fixes from pre-commit.com hooks

20045e1

for more information, see https://pre-commit.ci

euronion added type: enhancement status: in progress labels Nov 6, 2025

brynpickering requested changes Nov 6, 2025

View reviewed changes

lmezilis and others added 4 commits November 7, 2025 18:27

code resolutions for merge

ffb156d

Merge branch 'wecmatrices-datamodules' of https://github.com/lmezilis…

2b0380e

…/atlite into wecmatrices-datamodules

commit to fork

dcffaef

[pre-commit.ci] auto fixes from pre-commit.com hooks

3383ebc

for more information, see https://pre-commit.ci

lmezilis requested a review from brynpickering November 10, 2025 09:13

lmezilis and others added 14 commits November 10, 2025 10:21

fix count message error

7b6872f

Merge branch 'wecmatrices-datamodules' of https://github.com/lmezilis…

335a67c

…/atlite into wecmatrices-datamodules

[pre-commit.ci] auto fixes from pre-commit.com hooks

1b7efd7

for more information, see https://pre-commit.ci

fix docstring

7067490

Merge branch 'wecmatrices-datamodules' of https://github.com/lmezilis…

3f0c0df

…/atlite into wecmatrices-datamodules

[pre-commit.ci] auto fixes from pre-commit.com hooks

7aaae50

for more information, see https://pre-commit.ci

fix syntax

a1a2aaa

Merge branch 'wecmatrices-datamodules' of https://github.com/lmezilis…

963896d

…/atlite into wecmatrices-datamodules

copyrights cerra

76f94fd

fix syntax

5f6cfe2

[pre-commit.ci] auto fixes from pre-commit.com hooks

39eefe6

for more information, see https://pre-commit.ci

convert func

ba748f4

Merge branch 'wecmatrices-datamodules' of https://github.com/lmezilis…

f2acfc4

…/atlite into wecmatrices-datamodules

[pre-commit.ci] auto fixes from pre-commit.com hooks

1de851d

for more information, see https://pre-commit.ci

brynpickering reviewed Nov 12, 2025

View reviewed changes

lmezilis and others added 9 commits November 20, 2025 16:20

update commit

0c50fa9

correct syntax

d53fdd6

Merge branch 'wecmatrices-datamodules' of https://github.com/lmezilis…

0c0b726

…/atlite into wecmatrices-datamodules

[pre-commit.ci] auto fixes from pre-commit.com hooks

5cc55a8

for more information, see https://pre-commit.ci

syntax corrections

8b4ae18

[pre-commit.ci] auto fixes from pre-commit.com hooks

9b7cae6

for more information, see https://pre-commit.ci

delete picture

b06dfd3

[pre-commit.ci] auto fixes from pre-commit.com hooks

0363202

for more information, see https://pre-commit.ci

Merge branch 'master' into wecmatrices-datamodules

270951b

fneum mentioned this pull request Dec 29, 2025

Wave and floating solar updates PyPSA/pypsa-eur#1883

Open

7 tasks

brynpickering mentioned this pull request Mar 31, 2026

[SPIKE] Source of Marine energy capacity factor profiles open-energy-transition/gb-dispatch-model#49

Open

lmezilis and others added 14 commits August 25, 2026 12:57

Merge remote-tracking branch 'upstream/master' into wecmatrices-datam…

f2f5cbb

…odules

[pre-commit.ci] auto fixes from pre-commit.com hooks

cfc84a3

for more information, see https://pre-commit.ci

[pre-commit.ci] auto fixes from pre-commit.com hooks

c6b442a

for more information, see https://pre-commit.ci

resolve pre-commit conflicts

3ace155

Merge branch 'wecmatrices-datamodules' of https://github.com/lmezilis…

934fbc7

…/atlite into wecmatrices-datamodules

[pre-commit.ci] auto fixes from pre-commit.com hooks

4e493e8

for more information, see https://pre-commit.ci

Fix docstrings

79fbd73

Merge branch 'wecmatrices-datamodules' of https://github.com/lmezilis…

f44b77b

…/atlite into wecmatrices-datamodules

Fix convert_wave docstring

d8b0602

Merge remote-tracking branch 'fork/wecmatrices-datamodules' into wecm…

c92f9db

…atrices-datamodules

[pre-commit.ci] auto fixes from pre-commit.com hooks

e3bd195

for more information, see https://pre-commit.ci

fix pre-commit issues

c7d9fef

mypy issues

e3d1a3d

Merge remote-tracking branch 'fork/wecmatrices-datamodules' into wecm…

cc35f64

…atrices-datamodules

		"wave_height": ["wave_height"],
		"wave_period": ["wave_period"],

		Optionally (add_lon_lat, default:True) preserves latitude and
		longitude columns as 'lat' and 'lon'.

		def convert_wave(ds, wec_type):
		power_matrix = pd.DataFrame.from_dict(wec_type["Power_Matrix"])

Uh oh!

Conversation

lmezilis commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes proposed in this Pull Request

Checklist

Uh oh!

euronion commented Nov 6, 2025

Uh oh!

lmezilis commented Nov 6, 2025

Uh oh!

euronion commented Nov 6, 2025

Uh oh!

brynpickering left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lmezilis Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lmezilis commented Nov 7, 2025

Uh oh!

brynpickering commented Nov 12, 2025

Uh oh!

brynpickering left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lmezilis commented Nov 20, 2025

Uh oh!

brynpickering commented Nov 24, 2025

Uh oh!

lmezilis commented Nov 25, 2025

Uh oh!

FabianHofmann commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lmezilis commented Aug 25, 2026

Uh oh!

Reviewers

Assignees

Labels

lmezilis commented Nov 5, 2025 •

edited

Loading

lmezilis Nov 6, 2025 •

edited

Loading

FabianHofmann commented Aug 13, 2026 •

edited

Loading