Control your robot#

Without actuation, the arm from Hello, Genesis World falls under gravity. Genesis World has a built-in PD controller that takes a target joint position or velocity, plus a force path that bypasses it. This tutorial covers both, plus writing the joint state without going through physics. The complete script is examples/tutorials/control_your_robot.py.

Scene setup#

The scene is the same single-arm setup from Hello, Genesis World: a ground plane and a Franka arm loaded from MJCF, simulated at dt=0.01 s.

gs.init(backend=gs.gpu)

scene = gs.Scene(
    viewer_options=gs.options.ViewerOptions(
        camera_pos=(0, -3.5, 2.5),
        camera_lookat=(0.0, 0.0, 0.5),
        camera_fov=30,
    ),
    sim_options=gs.options.SimOptions(
        dt=0.01,
    ),
    show_viewer=True,
)

plane = scene.add_entity(gs.morphs.Plane())
franka = scene.add_entity(
    gs.morphs.MJCF(
        file="xml/franka_emika_panda/panda.xml",
    ),
)
scene.build()

Every snippet below issues its control command after the build, to hold or move the arm.

Joints and degrees of freedom#

A joint connects two links, and a dof (degree of freedom) is one independent coordinate that a joint contributes. On the Franka arm the two line up: 7 revolute arm joints and 2 prismatic gripper joints, each carrying a single dof, make a 9-dof articulated body. Other joint types carry more, so a free joint contributes 6 dofs and a ball joint 3.

The distinction matters because every control method addresses dofs by index. Start by mapping the joint names in the MJCF/URDF file to the dof indices the solver assigned them:

joints_name = (
    "joint1",
    "joint2",
    "joint3",
    "joint4",
    "joint5",
    "joint6",
    "joint7",
    "finger_joint1",
    "finger_joint2",
)
motors_dof_idx = [franka.get_joint(name).dofs_idx_local[0] for name in joints_name]

dofs_idx_local is the dof index relative to this entity, and each single-dof joint exposes a one-element list, hence the [0]. Use joint.dofs_idx when you need the dof’s global index within the scene.

Control gains#

Position and velocity control run through a PD controller with two gains. kp (stiffness) scales the force by the distance to the target, and kv (damping) opposes the motion in proportion to the current velocity, which keeps the dof from overshooting. Genesis World parses both from the MJCF or URDF file when the model provides them, but setting them explicitly makes the behavior reproducible. set_dofs_force_range caps the controller’s output.

franka.set_dofs_kp(
    kp=np.array([4500, 4500, 3500, 3500, 2000, 2000, 2000, 100, 100]),
    dofs_idx_local=motors_dof_idx,
)
franka.set_dofs_kv(
    kv=np.array([450, 450, 350, 350, 200, 200, 200, 10, 10]),
    dofs_idx_local=motors_dof_idx,
)
franka.set_dofs_force_range(
    lower=np.array([-87, -87, -87, -87, -12, -12, -12, -100, -100]),  # N*m / N
    upper=np.array([87, 87, 87, 87, 12, 12, 12, 100, 100]),
    dofs_idx_local=motors_dof_idx,
)

These methods share the pattern used throughout the control API: a tensor of values paired with the dof indices they apply to, lining up element by element.

Setting state versus controlling#

set_* and control_* look similar and do different things.

  • set_* writes the robot state directly, without consulting the dynamics. The dof lands on the requested value in a single step.

  • control_* sends a target to the controller, which produces forces that move the robot toward it over several steps, respecting the dynamics and the force range set above.

Use set_dofs_position to reset an episode or place a robot at a starting configuration, and keep it out of a control loop. On a rigid entity it zeroes the dof velocities as well, so the robot starts from rest; pass zero_velocity=False to keep the velocities it had:

for i in range(150):
    if i < 50:
        franka.set_dofs_position(np.array([1, 1, 0, 0, 0, 0, 0, 0.04, 0.04]), motors_dof_idx)
    elif i < 100:
        franka.set_dofs_position(np.array([-1, 0.8, 1, -2, 1, 0.5, -0.5, 0.04, 0.04]), motors_dof_idx)
    else:
        franka.set_dofs_position(np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]), motors_dof_idx)
    scene.step()

With the viewer on, the arm snaps to a new configuration every 50 steps.

PD and force control#

Switching from set_* to the matching control_* method turns a state assignment into an actuated command. The controller holds a position target until you replace it, so a single call is enough.

franka.control_dofs_position(
    np.array([1, 1, 0, 0, 0, 0, 0, 0.04, 0.04]),
    motors_dof_idx,
)

Different dofs can run under different control modes at the same time, because passing a subset of indices leaves the remaining dofs on their previous command. Here the first dof follows a velocity target while the rest stay under position control:

# control first dof with velocity, and the rest with position
franka.control_dofs_position(
    np.array([0, 0, 0, 0, 0, 0, 0, 0, 0])[1:],
    motors_dof_idx[1:],
)
franka.control_dofs_velocity(
    np.array([1.0, 0, 0, 0, 0, 0, 0, 0, 0])[:1],
    motors_dof_idx[:1],
)

control_dofs_force applies a torque (or force, for prismatic dofs) directly, skipping the PD controller. Commanding zero force lets gravity take over and the arm falls:

franka.control_dofs_force(
    np.array([0, 0, 0, 0, 0, 0, 0, 0, 0]),
    motors_dof_idx,
)

Reading forces back#

Two accessors report the forces at each dof after a step:

# force applied by the controller
print("control force:", franka.get_dofs_control_force(motors_dof_idx))
# actual force experienced by each dof
print("internal force:", franka.get_dofs_force(motors_dof_idx))

get_dofs_control_force is what the controller commanded, computed from the target and gains under position or velocity control, and equal to your input under force control. get_dofs_force is the total force the dof actually experiences, combining that control force with internal effects such as contact and Coriolis forces.

Running the full example produces this sequence:

Applying external forces#

The control_* methods act in joint space, through the dofs. Sometimes you want to push or twist a link directly in Cartesian space instead: a thruster, wind, a scripted tug on a payload, or a disturbance to see how a controller recovers. A link takes a wrench, a linear force and a torque together, through apply_external_wrench.

A wrench lasts for a single step and is then cleared, so reapply it on every step you want it active:

hand = franka.get_link("hand")

for i in range(150):
    # push the hand straight up in the world frame, 50 N
    hand.apply_external_wrench(force=(0.0, 0.0, 50.0))
    scene.step()

Both components are optional, and apply_external_force and apply_external_torque are shortcuts for the one-sided cases. Forces are in newtons and torques in newton-meters. To drive many links in one call, RigidEntity.apply_links_external_wrench and RigidSolver.apply_links_external_wrench take the same arguments plus a links_idx, with force and torque of shape ([n_envs,] n_links, 3).

Three arguments decide where the wrench acts and in which coordinates it is read:

  • ref: the reference frame, either gs.link_ref_frame.link_origin (default) or gs.link_ref_frame.link_COM, the link’s center of mass. It selects the frame the input coordinates are expressed in, and the point the force acts at when you give no pos.

  • local: by default the wrench is expressed in world coordinates. Set local=True to express it in the reference frame’s own coordinates instead, so the force rotates with the link.

  • pos: the point the linear force acts at, which sets the moment arm of the torque it induces. With local=True it is an offset from the reference frame’s origin, so the point follows the link as it moves. Otherwise it is a world position. A torque acts the same wherever it is applied, so pos requires a force.

Applying a force away from the center of mass is what makes a body swing rather than translate. The viewer’s mouse interaction plugin works this way: it pulls at the point you grabbed, and the moment about the center of mass turns the body to follow the cursor.

Pick and place with a suction cup#

An industrial suction gripper behaves like an instant rigid grasp, which you reproduce by welding two links together for the duration of the grasp. The rigid solver exposes add_weld_constraint and delete_weld_constraint, each taking the two link indices to attach or detach. The runnable version is examples/rigid/suction_cup.py, which moves the end-effector above a cube, welds them, transports the cube, and releases it.

Move to a pose above the cube using inverse kinematics, then activate the “suction” by welding the cube’s link to the gripper’s hand link:

# ... arm moved above the cube via inverse_kinematics + control_dofs_position ...
rigid = scene.sim.rigid_solver
link_cube = cube.get_link("box_baselink").idx
link_franka = franka.get_link("hand").idx
rigid.add_weld_constraint(link_cube, link_franka)

While the weld is active the cube tracks the gripper, so transporting it means sending more IK targets. Releasing is a single call:

rigid.delete_weld_constraint(link_cube, link_franka)

Note

The weld is an ideal rigid attachment: it fixes the relative pose of the two links and models neither compliance nor a grasp-force limit. For a physically grounded grasp, control the gripper fingers against the object instead.

See also#