@mshooter and I would like to add an example to Robotics Academy using Pollen Robotics’ microduck. To do that, @jmplaza advised to first add support for microduck in this repo.
In this issue, we would like to confirm with the community that our understanding and approach is correct. We have been doing a bit of research but we are new to robotics, so feel free to correct any understanding that may be off!
RoboticsInfrastructure and RoboticsAcademy
RoboticsInfrastructure is where the robots and worlds live. RoboticsAcademy is where exercise code and documentation live. RoboticsAcademy pulls the RoboticsInfrastructure repo as a dependency.
The RoboticsInfrastructure repository holds robots not included in the official ROS or Gazebo packages. Most (or is it all?) of the RoboticsAcademy examples rely on Gazebo as the “backend” simulator. The examples show the student a live view of the simulated robot in the browser. That view is produced by Gazebo and piped through the existing plumbing
microduck policies may need re-validating
The microduck runs policies trained in MuJoCo (specifically, mjlab see here). If I understand correctly, this means that even though a URDF for the duck exists (see here for example) and the geometry converts easily, physics may not. In particular, the linked repo seems to aim for visualisation in Gazebo, not simulation. In practice, this means that the MuJoCo-trained policies provided by Pollen Robotics may need re-validating for Gazebo.
Since re-validating all current and future microduck shipped policies would mean a significant effort (and affect long-term sustainability), we were wondering: is there a way to quickly assess how likely this risk is?
Claude suggested loading the duck URDF in Gazebo and try running one of the shipped policies against it. Is this what you would recommend?
Non-Gazebo backend
Alternatively, is a non-Gazebo backend something you'd consider, given the policies are MuJoCo-trained? Maybe supporting MuJoCo-based approaches would expand the space of possible exercises.
However, it seems that a MuJoCo exercise would need to render frames in a way that does not rely on Gazebo, and get them into the browser. Is there a simple way to do this?
Would it be worth it to draft a standalone demo? E.g. the duck walking in MuJoCo, streamed to a browser, driven by Robotics Academy-style Python with HAL.setVelocity(). This was suggested by Claude too.
I understand that even on the MuJoCo route we'd likely still touch RoboticsInfrastructure, because launchers and the database registration live here (and probably the MJCF and the ONNX policy too?).
Let us know what you think about these two possible paths. We are happy to try a few things before committing to one of the two.
Btw @jmplaza also suggested checking the Unitree case as a reference, but I couldn’t find it in the repo. Would you mind pointing us to it?
Thanks!
@mshooter and I would like to add an example to Robotics Academy using Pollen Robotics’ microduck. To do that, @jmplaza advised to first add support for microduck in this repo.
In this issue, we would like to confirm with the community that our understanding and approach is correct. We have been doing a bit of research but we are new to robotics, so feel free to correct any understanding that may be off!
RoboticsInfrastructure and RoboticsAcademy
RoboticsInfrastructure is where the robots and worlds live. RoboticsAcademy is where exercise code and documentation live. RoboticsAcademy pulls the RoboticsInfrastructure repo as a dependency.
The RoboticsInfrastructure repository holds robots not included in the official ROS or Gazebo packages. Most (or is it all?) of the RoboticsAcademy examples rely on Gazebo as the “backend” simulator. The examples show the student a live view of the simulated robot in the browser. That view is produced by Gazebo and piped through the existing plumbing
microduck policies may need re-validating
The microduck runs policies trained in MuJoCo (specifically, mjlab see here). If I understand correctly, this means that even though a URDF for the duck exists (see here for example) and the geometry converts easily, physics may not. In particular, the linked repo seems to aim for visualisation in Gazebo, not simulation. In practice, this means that the MuJoCo-trained policies provided by Pollen Robotics may need re-validating for Gazebo.
Since re-validating all current and future microduck shipped policies would mean a significant effort (and affect long-term sustainability), we were wondering: is there a way to quickly assess how likely this risk is?
Claude suggested loading the duck URDF in Gazebo and try running one of the shipped policies against it. Is this what you would recommend?
Non-Gazebo backend
Alternatively, is a non-Gazebo backend something you'd consider, given the policies are MuJoCo-trained? Maybe supporting MuJoCo-based approaches would expand the space of possible exercises.
However, it seems that a MuJoCo exercise would need to render frames in a way that does not rely on Gazebo, and get them into the browser. Is there a simple way to do this?
Would it be worth it to draft a standalone demo? E.g. the duck walking in MuJoCo, streamed to a browser, driven by Robotics Academy-style Python with
HAL.setVelocity(). This was suggested by Claude too.I understand that even on the MuJoCo route we'd likely still touch RoboticsInfrastructure, because launchers and the database registration live here (and probably the MJCF and the ONNX policy too?).
Let us know what you think about these two possible paths. We are happy to try a few things before committing to one of the two.
Btw @jmplaza also suggested checking the Unitree case as a reference, but I couldn’t find it in the repo. Would you mind pointing us to it?
Thanks!