Typical Physics Simulation Pipeline

  • Broad phase collision detection
  • Narrow phase collision detection
  • Generate islands
  • Solve constraints
    • Constraint based method
    • Impulse based method
    • Penalty based method
  • Integrate position: velocity * dt

ref

  • Box2D
  • Jolt (Box3D)

RL.Env(legged_robot) Pipeline

  • __init__
    • _parse_cfg()
    • super().__init__()
      • create_sim()
        • _create_ground_plane()
        • _create_heightfield()
        • _create_trimesh()
        • _create_envs()
          • _get_env_origins()
          • _process_rigid_shape_props()
          • _process_dof_props()
          • _process_rigid_body_props()
      • set_camera()
      • _init_buffers()
      • _prepare_reward_function()
  • step
    • actions clip
    • render()
    • for decimation: _compute_torques() & simulate
    • post_physics_step()
      • _post_physics_step_callback()
        • _resample_commands()
        • _get_heights()
        • _push_robots()
      • check_termination()
      • compute_reward()
      • reset_idx()
        • _update_terrain_curriculum()
        • update_command_curriculum()
        • _reset_dofs()
        • _reset_root_states()
        • _resample_commands()
      • compute_observations()
        • add perceptive
        • add noise
      • _draw_debug_vis()
    • obs+states clip

isaacgym supported functions

  • sdf
  • vhacd
  • attractor
  • image
  • xml(mjcf, support stiffness|damping|armature) better than urdf?
Name Data type Description
hasLimits bool Whether the DOF has limits or has unlimited motion.
lower float32 Lower limit.
upper float32 Upper limit.
driveMode gymapi.DofDriveMode DOF drive mode, see below.
stiffness float32 Drive stiffness.
damping float32 Drive damping.
velocity float32 Maximum velocity.
effort float32 Maximum effort (force or torque).
friction float32 DOF friction.
armature float32 DOF armature.

SlimeVR server/core/src/main/java/dev/slimevr/tracking/processor/skeleton

HumanSkeleton.updatePose()

fun updatePose() {
 tapDetectionManager.update()
 updateTransforms()
 updateBones()
 if (enforceConstraints) {
  // TODO re-enable toggling correctConstraints once
  // https://github.com/SlimeVR/SlimeVR-Server/issues/1297 is solved
  headBone.updateWithConstraints(false)
 }
 updateComputedTrackers()
 // Don't run post-processing if the tracking is paused
 if (pauseTracking) return
 legTweaks.tweakLegs()
 localizer.update()
}

Reset: TrackerResetsHandler::resetFull(reference):

offset content local/global
mountingOrientation HalfHorizontal  
attachmentFix pitch=roll=0 global
tposeDownFix   global
gyroFix yaw=0 local
yawFix align yaw to headTracker local

R_ref = yawFix @ gyroFix @ R_raw @ mountingOrientation @ attachmentFix @ tposeDownFix

private fun adjustToReference(rotation: Quaternion): Quaternion {
    var rot = rotation
    if (!tracker.isHmd || tracker.trackerPosition != TrackerPosition.HEAD) {
        rot *= mountingOrientation
    }
    rot = gyroFix * rot
    rot *= attachmentFix
    // rot = mountRotFix.inv() * (rot * mountRotFix)
    rot *= tposeDownFix
    rot = yawFix * rot
    // rot = constraintFix * rot
    return rot
}

HARL

'''
IN
  actions: (n_threads, n_agents, action_dim)
OUT
  obs: (n_threads, n_agents, obs_dim)
  share_obs: (n_threads, n_agents, share_obs_dim)
  rewards: (n_threads, n_agents, 1)
  dones: (n_threads, n_agents)
  infos: (n_threads)
  available_actions: (n_threads, ) of None or (n_threads, n_agents, action_number)
'''
class Env:
    def __init__(self, args):
        self.env = ...
        self.n_agents = ...
        self.share_observation_space = ...
        self.observation_space = ...
        self.action_space = ...

    def step(self, actions):
        return obs, state, rewards, dones, info, available_actions

    def reset(self):
        return obs, state, available_actions

    def seed(self, seed):
        pass

    def render(self):
        pass

    def close(self):
        self.env.close()

OnPolicyBaseRunner

NxActor + 1xCritic