Tactile sensors#

Tactile sensors turn a link’s surface into a grid of sensing points, or taxels, and read contact geometry directly from the scene’s signed-distance fields (SDFs) or sampled point clouds rather than from solver contact impulses. They give you a dense taxel field at arbitrary locations without adding contacts to the solver, at the cost of being an approximation. Because the layout is link-local, a regular grid imitates a taxel array on a fingertip or a sensor pad.

You describe the layout once with probe_local_pos, a set of (x, y, z) offsets in the link-local frame (an (N, 3) set or an (M, N, 3) planar grid), and the probes move rigidly with the link. Genesis World provides a helper for a planar grid, genesis.utils.geom.generate_grid_points_on_plane(lo, hi, normal, nx, ny), which returns an (ny, nx, 3) array; n_probes is the flattened probe count.

Two families share this interface but estimate contact differently:

SDF-query probes (ContactProbe, ContactDepthProbe, and KinematicTaxel) query the signed distance from each probe to nearby collision geometry directly. They need no list of target links.
Point-cloud probes (ElastomerTaxel and ProximityTaxel) sample a point cloud from the meshes named in track_link_idx (global link indices) and measure against those points. n_sample_points sets the sample budget.

Readings are geometric estimates, not solver impulses, and are uncalibrated. Treat them as relative signals unless you tune the coefficients to your setup. The taxels also expose hardware-style imperfections for sim-to-real robustness. For how sensors are sampled, read back, and batched, see the sensors overview.

Choosing a sensor#

Sensor	`read()` returns	Shape	Frame / units
`ContactProbe`	in-contact flag per probe	`([n_envs,] n_probes)`	bool
`ContactDepthProbe`	penetration depth per probe	`([n_envs,] n_probes)`	m
`KinematicTaxel`	`(force, torque)` per probe	each `([n_envs,] n_probes, 3)`	link frame, N / N·m
`ElastomerTaxel`	marker displacement per probe	`([n_envs,] n_probes, 3)`	link frame, m
`ProximityTaxel`	`(force, torque)` per probe	each `([n_envs,] n_probes, 3)`	link-local, N / N·m

Two interactive demos drive these sensors:

examples/sensors/tactile_franka.py sensorizes both Franka fingertips and lets you teleoperate a grasp (--sensor {depth,kinematic,elastomer,proximity}).
examples/sensors/tactile_sandbox.py presses controllable objects into a fixed taxel pad (box or dome) across four parallel environments (--sensor {contact,depth,kinematic,elastomer,proximity}), selecting the depth backend with --contact-depth-query {sdf,raycast} and enabling the imperfections below with --noise.

Contact depth and contact probes#

The simplest probes report geometry alone. ContactDepthProbe returns the penetration depth at each probe in meters; ContactProbe thresholds that depth into a per-probe boolean.

depth_probe = scene.add_sensor(
    gs.sensors.ContactDepthProbe(
        entity_idx=franka.idx,
        link_idx_local=franka.get_link("left_finger").idx_local,
        probe_local_pos=probe_local_pos.reshape(-1, 3),
        probe_radius=0.002,
    )
)

depth = depth_probe.read()  # shape ([n_envs,] n_probes), m

probe_radius is the taxel’s sensing radius, not a tolerance: each reading is depth = max(0, probe_radius - signed_distance_to_surface), so a larger radius both extends the detection range and adds a constant offset to the reported depth. The SDF-query probes select the contact-depth backend with contact_depth_query: "sdf" (default) queries each geom’s analytic SDF grid, while "raycast" walks the rigid solver’s collision-mesh BVH and takes the signed distance to the nearest triangle (sharing the BVH with RaycasterSensor). The mode is class-wide: all sensors of the same class must agree.

ContactProbe gates its boolean output with a Schmitt trigger to suppress chatter: it latches on when depth reaches contact_threshold (default 1e-4 m) and releases when depth falls back to release_threshold (default equals contact_threshold, i.e. no hysteresis; may be negative to require separation before release).

Kinematic taxels#

KinematicTaxel adds a spring-damper force model on top of the depth query. For each taxel it estimates a force from penetration along the probe normal and a torque from the twist, using the probe’s motion relative to the object it touches:

s = penetration ** normal_exponent
F = -normal_stiffness * s * n  -  normal_damping * s * v_n  -  shear_scalar * v_t

where n is the probe normal, v_n / v_t are the normal and tangential relative velocities. Use normal_exponent=1.5 for Hertzian (spherical) contact; the default 1.0 is a linear spring.

taxel = scene.add_sensor(
    gs.sensors.KinematicTaxel(
        entity_idx=franka.idx,
        link_idx_local=franka.get_link("left_finger").idx_local,
        probe_local_pos=probe_local_pos.reshape(-1, 3),
        probe_radius=0.002,
        normal_stiffness=5000.0,
        normal_exponent=1.5,
    )
)

data = taxel.read()
data.force   # shape ([n_envs,] n_probes, 3), N, link frame
data.torque  # shape ([n_envs,] n_probes, 3), N·m, link frame

read() returns a KinematicTaxelReturnType named tuple, so data.force and data.torque unpack by name.

Elastomer taxels#

ElastomerTaxel models a soft tactile skin without simulating deformation. Each probe reports a 3D marker displacement caused by local indentation and shear against the tracked geometry, computed with a HydroShear-style model. It is the right choice when you want the visual “dot displacement” signal of a vision-based tactile sensor.

tactile = scene.add_sensor(
    gs.sensors.ElastomerTaxel(
        entity_idx=franka.idx,
        link_idx_local=franka.get_link("left_finger").idx_local,
        probe_local_pos=probe_local_pos,  # (ny, nx, 3) grid, or (N, 3)
        probe_local_normal=(0.0, -1.0, 0.0),
        probe_radius=0.002,
        track_link_idx=(cube.base_link_idx,),  # global link idx to sense against
        dilate_scale=10.0,   # gain on normal indentation
        shear_scale=100.0,   # gain on tangential slip
    )
)

displacement = tactile.read()  # shape ([n_envs,] n_probes, 3), m

dilate_scale and shear_scale scale the indentation and shear response; lambda_d and lambda_s control how far each effect spreads across neighboring markers. When probe_local_pos is a regular planar grid with a single shared normal, the dilation term is computed with an FFT to keep large arrays fast. The shear anchor is gated by the same contact_threshold / release_threshold Schmitt trigger (a tracked point begins anchoring shear at contact_threshold penetration and releases once it separates back to release_threshold).

Proximity taxels#

ProximityTaxel estimates per-taxel force and torque from a point cloud sampled on the tracked meshes, within a spherical sensing volume of radius probe_radius around each taxel. It reads before hard contact, so it captures near-touch as well as touch.

proximity = scene.add_sensor(
    gs.sensors.ProximityTaxel(
        entity_idx=pad.idx,
        link_idx_local=0,
        probe_local_pos=probe_local_pos.reshape(-1, 3),
        probe_local_normal=(0.0, 0.0, 1.0),
        probe_radius=0.008,
        track_link_idx=(obj.base_link_idx,),
        n_sample_points=4000,
        stiffness=200.0,
        shear_coupling=100.0,  # 0.0 disables shear, leaving only the normal channel
    )
)

data = proximity.read()
data.force   # shape ([n_envs,] n_probes, 3), N, link-local
data.torque  # shape ([n_envs,] n_probes, 3), N·m, link-local

Like KinematicTaxel, it returns a named tuple (ProximityTaxelReturnType) with force and torque fields.

Sensor imperfections#

On top of the generic per-sensor imperfections (noise, bias, resolution, delay, jitter, available on every sensor), the tactile probe and taxel sensors expose hardware-style imperfections. They apply to the measured branch only: read() returns the imperfect signal, while read_ground_truth() stays clean.

Option(s)	Models	Applies to
`hysteresis_strength`, `hysteresis_tau`	Viscoelastic (single-Maxwell) hysteresis: a step input overshoots by `hysteresis_strength` and relaxes with time constant `hysteresis_tau` seconds; cyclic input traces a loading-unloading loop (equilibrium gain 1).	All probes and taxels
`probe_radius_noise`	Additive uniform noise (meters) on the effective sensing radius used by the measured branch.	All probes and taxels
`probe_gain`, `probe_gain_resample_range`	Per-taxel multiplicative gain on the measured contact depth (scalar or per-taxel array); with a range, resampled uniformly on each `scene.reset()`.	All probes and taxels
`dead_taxel_probability`, `dead_taxel_value_range`	Per-taxel Bernoulli chance of going dead on each `scene.reset()`; a dead taxel’s output is replaced by a constant drawn from `dead_taxel_value_range`.	All probes and taxels
`crosstalk_strength`, `crosstalk_sigma`	Gaussian spatial crosstalk: each taxel’s force/torque bleeds onto its grid neighbors (`crosstalk_strength=1` is a pure Gaussian blur of width `crosstalk_sigma`).	Grid taxels: `KinematicTaxel`, `ProximityTaxel`
`crosstalk_kernel`	Explicit point-spread kernel for spatial crosstalk (odd dims; center tap is the self weight). Mutually exclusive with the Gaussian options. `genesis.utils.misc.gaussian_crosstalk_kernel(n_rows, n_cols, sigma)` builds an L1-normalized Gaussian kernel.	Grid taxels: `KinematicTaxel`, `ProximityTaxel`

Spatial crosstalk requires a regular planar (M, N, 3) grid probe_local_pos; pad irregular layouts with probe_radius=0 filler taxels, which read zero and are skipped.

import genesis.utils.misc as misc

taxel = scene.add_sensor(
    gs.sensors.KinematicTaxel(
        entity_idx=platform.idx,
        link_idx_local=0,
        probe_local_pos=grid_positions,  # (M, N, 3) grid
        probe_radius=0.004,
        # measured-branch imperfections:
        hysteresis_strength=0.5,
        hysteresis_tau=0.1,
        probe_radius_noise=0.001,
        probe_gain=1.5,
        crosstalk_kernel=misc.gaussian_crosstalk_kernel(3, 3, sigma=1.0),
    )
)

Pass --noise to tactile_sandbox.py to enable these imperfections in the interactive demo.