Skip to content

Commit d94504b

Browse files
Toni-SMkellyguo11
Toni-SM
and
kellyguo11
authored
Update skrl integration for version 2.0.0 (isaac-sim#5311)
# Description Updates the integration of skrl for the version 2.0.0. This also solves isaac-sim#5300. ## Checklist - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there <!-- As you go through the checklist above, you can mark something as done by putting an x character in it For example, - [x] I have done this task - [ ] I have not done this task --> --------- Signed-off-by: Kelly Guo <kellyg@nvidia.com> Co-authored-by: Kelly Guo <kellyg@nvidia.com>
1 parent 2957920 commit d94504b

15 files changed

Lines changed: 73 additions & 42 deletions

File tree

‎scripts/reinforcement_learning/skrl/play.py‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -46,15 +46,17 @@
4646
"--ml_framework",
4747
type=str,
4848
default="torch",
49-
choices=["torch", "jax", "jax-numpy"],
49+
choices=["torch", "jax"],
5050
help="The ML framework used for training the skrl agent.",
5151
)
5252
parser.add_argument(
5353
"--algorithm",
5454
type=str,
5555
default="PPO",
56-
choices=["AMP", "PPO", "IPPO", "MAPPO"],
57-
help="The RL algorithm used for training the skrl agent.",
56+
help=(
57+
"Name of the RL algorithm to use (e.g. AMP, DDPG, IPPO, MAPPO, PPO, SAC, TD3, etc.) "
58+
"when several algorithms exist for the same task. For a more specific selection, use the argument --agent."
59+
),
5860
)
5961
parser.add_argument("--real-time", action="store_true", default=False, help="Run in real-time, if possible.")
6062

@@ -84,7 +86,7 @@
8486
from packaging import version
8587

8688
# check for minimum supported skrl version
87-
SKRL_VERSION = "1.4.3"
89+
SKRL_VERSION = "2.0.0"
8890
if version.parse(skrl.__version__) < version.parse(SKRL_VERSION):
8991
skrl.logger.error(
9092
f"Unsupported skrl version: {skrl.__version__}. "
@@ -207,10 +209,11 @@ def main(env_cfg: ManagerBasedRLEnvCfg | DirectRLEnvCfg | DirectMARLEnvCfg, expe
207209
print(f"[INFO] Loading model checkpoint from: {resume_path}")
208210
runner.agent.load(resume_path)
209211
# set agent to evaluation mode
210-
runner.agent.set_running_mode("eval")
212+
runner.agent.enable_training_mode(False, apply_to_models=True)
211213

212214
# reset environment
213215
obs, _ = env.reset()
216+
states = env.state()
214217
timestep = 0
215218
# simulate environment
216219
while simulation_app.is_running():
@@ -219,7 +222,7 @@ def main(env_cfg: ManagerBasedRLEnvCfg | DirectRLEnvCfg | DirectMARLEnvCfg, expe
219222
# run everything in inference mode
220223
with torch.inference_mode():
221224
# agent stepping
222-
outputs = runner.agent.act(obs, timestep=0, timesteps=0)
225+
outputs = runner.agent.act(obs, states, timestep=0, timesteps=0)
223226
# - multi-agent (deterministic) actions
224227
if hasattr(env, "possible_agents"):
225228
actions = {a: outputs[-1][a].get("mean_actions", outputs[0][a]) for a in env.possible_agents}
@@ -228,6 +231,7 @@ def main(env_cfg: ManagerBasedRLEnvCfg | DirectRLEnvCfg | DirectMARLEnvCfg, expe
228231
actions = outputs[-1].get("mean_actions", outputs[0])
229232
# env stepping
230233
obs, _, _, _, _ = env.step(actions)
234+
states = env.state()
231235
if args_cli.video:
232236
timestep += 1
233237
# exit the play loop after recording one video

‎scripts/reinforcement_learning/skrl/train.py‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,15 +44,17 @@
4444
"--ml_framework",
4545
type=str,
4646
default="torch",
47-
choices=["torch", "jax", "jax-numpy"],
47+
choices=["torch", "jax"],
4848
help="The ML framework used for training the skrl agent.",
4949
)
5050
parser.add_argument(
5151
"--algorithm",
5252
type=str,
5353
default="PPO",
54-
choices=["AMP", "PPO", "IPPO", "MAPPO"],
55-
help="The RL algorithm used for training the skrl agent.",
54+
help=(
55+
"Name of the RL algorithm to use (e.g. AMP, DDPG, IPPO, MAPPO, PPO, SAC, TD3, etc.) "
56+
"when several algorithms exist for the same task. For a more specific selection, use the argument --agent."
57+
),
5658
)
5759
parser.add_argument(
5860
"--ray-proc-id", "-rid", type=int, default=None, help="Automatically configured by Ray integration, otherwise None."
@@ -85,7 +87,7 @@
8587
from packaging import version
8688

8789
# check for minimum supported skrl version
88-
SKRL_VERSION = "1.4.3"
90+
SKRL_VERSION = "2.0.0"
8991
if version.parse(skrl.__version__) < version.parse(SKRL_VERSION):
9092
skrl.logger.error(
9193
f"Unsupported skrl version: {skrl.__version__}. "

‎source/isaaclab_rl/config/extension.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[package]
22

33
# Note: Semantic Versioning is used: https://semver.org/
4-
version = "0.5.0"
4+
version = "0.5.1"
55

66
# Description
77
title = "Isaac Lab RL"

‎source/isaaclab_rl/docs/CHANGELOG.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,14 @@
11
Changelog
22
---------
33

4+
0.5.1 (2026-04-21)
5+
~~~~~~~~~~~~~~~~~~
6+
7+
Changed
8+
^^^^^^^
9+
10+
* Updated skrl wrapper to support the new version of skrl 2.0.
11+
412
0.5.0 (2026-3-04)
513
~~~~~~~~~~~~~~~~~~
614

‎source/isaaclab_rl/isaaclab_rl/skrl.py‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@
3838

3939
def SkrlVecEnvWrapper(
4040
env: ManagerBasedRLEnv | DirectRLEnv | DirectMARLEnv,
41-
ml_framework: Literal["torch", "jax", "jax-numpy"] = "torch",
41+
ml_framework: Literal["torch", "jax", "warp"] = "torch",
4242
wrapper: Literal["auto", "isaaclab", "isaaclab-single-agent", "isaaclab-multi-agent"] = "isaaclab",
4343
):
4444
"""Wraps around Isaac Lab environment for skrl.
@@ -77,9 +77,11 @@ def SkrlVecEnvWrapper(
7777
from skrl.envs.wrappers.torch import wrap_env
7878
elif ml_framework.startswith("jax"):
7979
from skrl.envs.wrappers.jax import wrap_env
80+
elif ml_framework.startswith("warp"):
81+
from skrl.envs.wrappers.warp import wrap_env
8082
else:
81-
ValueError(
82-
f"Invalid ML framework for skrl: {ml_framework}. Available options are: 'torch', 'jax' or 'jax-numpy'"
83+
raise ValueError(
84+
f"Invalid ML framework for skrl: {ml_framework}. Available options are: 'torch', 'jax', 'warp'"
8385
)
8486

8587
# wrap and return the environment

‎source/isaaclab_rl/setup.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@
4141
# Extra dependencies for RL agents
4242
EXTRAS_REQUIRE = {
4343
"sb3": ["stable-baselines3>=2.6", "tqdm", "rich"], # tqdm/rich for progress bar
44-
"skrl": ["skrl>=1.4.3"],
44+
"skrl": ["skrl>=2.0.0"],
4545
"rl-games": [
4646
"rl-games @ git+https://github.com/isaac-sim/rl_games.git@python3.11",
4747
"gym",

‎source/isaaclab_tasks/config/extension.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[package]
22

33
# Note: Semantic Versioning is used: https://semver.org/
4-
version = "0.11.14"
4+
version = "0.11.16"
55

66
# Description
77
title = "Isaac Lab Environments"

‎source/isaaclab_tasks/docs/CHANGELOG.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,14 @@
11
Changelog
22
---------
33

4+
0.11.16 (2026-04-21)
5+
~~~~~~~~~~~~~~~~~~~~
6+
7+
Changed
8+
^^^^^^^
9+
10+
* Updated some agents' configuration files for the skrl library to support the new version of skrl 2.0.
11+
412
0.11.15 (2026-03-07)
513
~~~~~~~~~~~~~~~~~~~~
614

‎source/isaaclab_tasks/isaaclab_tasks/direct/cart_double_pendulum/agents/skrl_ippo_cfg.yaml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,9 @@ agent:
5454
learning_rate_scheduler: KLAdaptiveLR
5555
learning_rate_scheduler_kwargs:
5656
kl_threshold: 0.008
57-
state_preprocessor: RunningStandardScaler
57+
observation_preprocessor: RunningStandardScaler
58+
observation_preprocessor_kwargs: null
59+
state_preprocessor: null
5860
state_preprocessor_kwargs: null
5961
value_preprocessor: RunningStandardScaler
6062
value_preprocessor_kwargs: null

‎source/isaaclab_tasks/isaaclab_tasks/direct/cart_double_pendulum/agents/skrl_mappo_cfg.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ models:
2828
clip_actions: False
2929
network:
3030
- name: net
31-
input: OBSERVATIONS
31+
input: STATES
3232
layers: [32, 32]
3333
activations: elu
3434
output: ONE
@@ -54,10 +54,10 @@ agent:
5454
learning_rate_scheduler: KLAdaptiveLR
5555
learning_rate_scheduler_kwargs:
5656
kl_threshold: 0.008
57+
observation_preprocessor: RunningStandardScaler
58+
observation_preprocessor_kwargs: null
5759
state_preprocessor: RunningStandardScaler
5860
state_preprocessor_kwargs: null
59-
shared_state_preprocessor: RunningStandardScaler
60-
shared_state_preprocessor_kwargs: null
6161
value_preprocessor: RunningStandardScaler
6262
value_preprocessor_kwargs: null
6363
random_timesteps: 0

0 commit comments

Comments
 (0)