Raycaster sensor#

The RaycasterSensor provides ray-based distance measurements, useful for LIDAR simulation, proximity sensing, and obstacle detection.

Overview#

The Raycaster sensor:

Casts rays from a link’s frame into the scene
Returns hit points and distances to intersecting geometry
Supports configurable ray patterns (spherical, grid, custom)
Efficiently uses GPU-accelerated BVH traversal

Usage#

import genesis as gs

gs.init()
scene = gs.Scene()
robot = scene.add_entity(gs.morphs.URDF(file="robot.urdf"))
scene.add_entity(gs.morphs.Box(pos=(2, 0, 0.5), size=(1.0, 1.0, 1.0)))  # Obstacle

# Add raycaster sensor (using Lidar options)
lidar = scene.add_sensor(
    gs.sensors.Lidar(
        pattern=gs.sensors.SphericalPattern(),
        entity_idx=robot.idx,
        pos_offset=(0.3, 0.0, 0.1),
        return_world_frame=True,
    )
)

scene.build()

# Simulation loop
for i in range(100):
    scene.step()

    # Get raycast data (RaycasterReturnType NamedTuple)
    data = lidar.read()
    print(f"Min distance: {data.distances.min():.2f} m")

Output format#

read() returns a RaycasterReturnType NamedTuple:

Field	Type	Shape	Description
`points`	`torch.Tensor` (float32)	`([n_envs,] *pattern_shape, 3)`	Intersection points in world or local frame
`distances`	`torch.Tensor` (float32)	`([n_envs,] *pattern_shape)`	Distance to intersection (`max_range` if no hit)

The pattern_shape depends on the ray pattern (e.g. (n_horizontal, n_vertical) for spherical, (height, width) for depth camera).

Ray patterns#

Circular (2D LIDAR)#

lidar_2d = scene.add_sensor(
    gs.sensors.Raycaster(
        pattern=gs.sensors.SphericalPattern(),
        entity_idx=robot.idx,
    )
)

Planar (3D LIDAR)#

lidar_3d = scene.add_sensor(
    gs.sensors.Raycaster(
        pattern=gs.sensors.SphericalPattern(
            fov=(360.0, 30.0),  # (horizontal, vertical) field of view, degrees
        ),
        entity_idx=robot.idx,
    )
)

Custom pattern#

There is no ray_directions argument on Raycaster. To cast an arbitrary set of rays, subclass gs.sensors.RaycastPattern and fill in _ray_dirs (unit direction vectors in the sensor frame), then pass an instance as pattern.

import torch
import genesis as gs

class CrossPattern(gs.sensors.RaycastPattern):
    def _get_return_shape(self):
        return (4,)

    def compute_ray_dirs(self):
        self._ray_dirs[:] = torch.tensor(
            [
                [1.0, 0.0, 0.0],   # forward
                [0.0, 1.0, 0.0],   # left
                [-1.0, 0.0, 0.0],  # back
                [0.0, -1.0, 0.0],  # right
            ],
            dtype=gs.tc_float,
            device=gs.device,
        )

sensor = scene.add_sensor(
    gs.sensors.Raycaster(
        pattern=CrossPattern(),
        entity_idx=robot.idx,
    )
)

Performance#

The Raycaster uses a GPU-accelerated Linear BVH (LBVH) for efficient ray-scene intersection:

Scales well with scene complexity
Efficient for hundreds to thousands of rays
Batched across parallel environments

Depth camera#

The DepthCameraSensor is a raycaster driven by a DepthCameraPattern. It exposes everything the raycaster does, plus read_image(), which reshapes the per-ray hit distances into a depth image of shape ([n_envs,] height, width) (misses carry no_hit_value). See the raycaster guide for usage.

depth_cam = scene.add_sensor(
    gs.sensors.DepthCamera(
        pattern=gs.sensors.DepthCameraPattern(
            res=(96, 72),         # (width, height) in pixels
            fov_horizontal=90.0,  # degrees
        ),
        entity_idx=robot.idx,
        link_idx_local=0,
    )
)

scene.build()
scene.step()
depth = depth_cam.read_image()  # shape ([n_envs,] 72, 96), meters

API reference#

`gs.sensors.Raycaster`#

Also available as gs.sensors.Lidar.

class genesis.options.sensors.options.Raycaster(*, history_length: int = 0, delay: float = 0.0, jitter: float = 0.0, draw_debug: bool = False, entity_idx: int = -1, resolution: tuple[float, ...] | float = 0.0, bias: tuple[float, ...] | float = 0.0, noise: tuple[float, ...] | float = 0.0, random_walk: tuple[float, ...] | float = 0.0, link_idx_local: int = 0, pos_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), euler_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), pattern: genesis.options.sensors.raycaster.RaycastPattern, min_range: float = 0.0, max_range: float = 20.0, no_hit_value: float | None = None, return_world_frame: bool = False, debug_sphere_radius: float = 0.02, debug_ray_start_color: tuple[float, float, float, float] = (0.5, 0.5, 1.0, 1.0), debug_ray_hit_color: tuple[float, float, float, float] = (1.0, 0.5, 0.5, 1.0)) → None[source]#

Raycaster sensor that performs ray casting to get distance measurements and point clouds.

Parameters:

pattern (RaycastPatternOptions) – The raycasting pattern for the sensor.
min_range (float, optional) – The minimum sensing range in meters. Defaults to 0.0.
max_range (float, optional) – The maximum sensing range in meters. Defaults to 20.0.
no_hit_value (float, optional) – The value to return for no hit. Defaults to max_range if not specified.
return_world_frame (bool, optional) – Whether to return points in the world frame. Defaults to False (local frame).
debug_sphere_radius (float, optional) – The radius of each debug sphere drawn in the scene. Defaults to 0.02.
debug_ray_start_color (array-like[float, float, float, float], optional) – The color of each debug ray start sphere drawn in the scene. Defaults to (0.5, 0.5, 1.0, 1.0).
debug_ray_hit_color (array-like[float, float, float, float], optional) – The color of each debug ray hit point sphere drawn in the scene. Defaults to (1.0, 0.5, 0.5, 1.0).

class genesis.engine.sensors.raycaster.RaycasterSensor(options: Raycaster, idx: int, shared_context, shared_metadata, manager: SensorManager)[source]#

Bases: KinematicSensorMixin, SimpleSensor[Raycaster, RaycastContext, RaycasterSharedMetadata, RaycasterReturnType]

build()[source]#

Initialize all shared metadata needed to update all noisy sensors.

Time-related state (delays_ts, jitter_ts) is pushed by Sensor.build(); this method adds the imperfection-parameter state.

`gs.sensors.DepthCamera`#

class genesis.options.sensors.options.DepthCamera(*, history_length: int = 0, delay: float = 0.0, jitter: float = 0.0, draw_debug: bool = False, entity_idx: int = -1, resolution: tuple[float, ...] | float = 0.0, bias: tuple[float, ...] | float = 0.0, noise: tuple[float, ...] | float = 0.0, random_walk: tuple[float, ...] | float = 0.0, link_idx_local: int = 0, pos_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), euler_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), pattern: genesis.options.sensors.raycaster.DepthCameraPattern, min_range: float = 0.0, max_range: float = 20.0, no_hit_value: float | None = None, return_world_frame: bool = False, debug_sphere_radius: float = 0.02, debug_ray_start_color: tuple[float, float, float, float] = (0.5, 0.5, 1.0, 1.0), debug_ray_hit_color: tuple[float, float, float, float] = (1.0, 0.5, 0.5, 1.0)) → None[source]#

Depth camera that uses ray casting to obtain depth images.

Parameters:: pattern (DepthCameraPattern) – The raycasting pattern configuration for the sensor.

class genesis.engine.sensors.depth_camera.DepthCameraSensor(options: Raycaster, idx: int, shared_context, shared_metadata, manager: SensorManager)[source]#

Bases: RaycasterSensor, Sensor[DepthCamera, RaycastContext, RaycasterSharedMetadata, RaycasterReturnType]

build()[source]#

Initialize all shared metadata needed to update all noisy sensors.

Time-related state (delays_ts, jitter_ts) is pushed by Sensor.build(); this method adds the imperfection-parameter state.

read_image() → Tensor[source]#

Read the depth image from the sensor.

This method uses the hit distances from the underlying RaycasterSensor.read() method and reshapes into image.

Returns:: The depth image with shape (height, width).
Return type:: torch.Tensor

Ray patterns#

A pattern is a local description of the rays: it fixes a start point and a unit direction per ray in the sensor’s frame. Pass an instance as the pattern argument, or subclass RaycastPattern for a custom layout.

class genesis.options.sensors.raycaster.RaycastPattern[source]#: Base class for raycast patterns.

class genesis.options.sensors.raycaster.SphericalPattern(fov: tuple[float | tuple[float, float], float | tuple[float, float]] = (360.0, 60.0), n_points: tuple[int, int] = (128, 64), angular_resolution: tuple[float | None, float | None] = (None, None), angles: tuple[Optional[Sequence[float]], Optional[Sequence[float]]] = (None, None))[source]#

Configuration for spherical ray pattern.

Either specify: - (n_points, fov) for uniform spacing by count. - (angular_resolution, fov) for uniform spacing by resolution. - angles for custom angles.

Parameters:

fov (tuple[float | tuple[float, float], float | tuple[float, float]]) – Field of view in degrees for horizontal and vertical directions. Defaults to (360.0, 30.0). If a single float is provided, the FOV is centered around 0 degrees. If a tuple is provided, it specifies the (min, max) angles.
n_points (tuple[int, int]) – Number of horizontal/azimuth and vertical/elevation scan lines. Defaults to (64, 128).
angular_resolution (tuple[float, float], optional) – Horizontal and vertical angular resolution in degrees. Overrides n_points if provided.
angles (tuple[Sequence[float], Sequence[float]], optional) – Array of horizontal/vertical angles. Overrides the other options if provided.

class genesis.options.sensors.raycaster.GridPattern(resolution: float = 0.1, size: tuple[float, float] = (2.0, 2.0), direction: tuple[float, float, float] = (0.0, 0.0, -1.0))[source]#

Configuration for grid-based ray casting.

Defines a 2D grid of rays in the sensor coordinate system.

Parameters:

resolution (float) – Grid spacing in meters.
size (tuple[float, float]) – Grid dimensions (length, width) in meters.
direction (tuple[float, float, float]) – Ray direction vector.

class genesis.options.sensors.raycaster.DepthCameraPattern(res: tuple[int, int] = (128, 96), fx: float | None = None, fy: float | None = None, cx: float | None = None, cy: float | None = None, fov_horizontal: float = 90.0, fov_vertical: float | None = None)[source]#

Configuration for pinhole depth camera ray casting.

You can configure the camera intrinsics in several ways: 1. Provide fx and fy directly (and optionally cx, cy) 2. Provide fov_horizontal only (fy computed to maintain aspect ratio) 3. Provide fov_vertical only (fx computed to maintain aspect ratio) 4. Provide both fov_horizontal and fov_vertical

If cx or cy are not provided, they default to the image center.

Parameters:

res (tuple[int, int]) – The resolution of the camera, specified as a tuple (width, height).
fx (float | None) – Focal length in x direction in pixels. Computed from fov_horizontal if None.
fy (float | None) – Focal length in y direction in pixels. Computed from fov_vertical if None.
cx (float | None) – Principal point x coordinate in pixels. Defaults to image center if None.
cy (float | None) – Principal point y coordinate in pixels. Defaults to image center if None.
fov_horizontal (float) – Horizontal field of view in degrees. Used to compute fx if fx is None.
fov_vertical (float | None) – Vertical field of view in degrees. Used to compute fy if fy is None.

Raycaster sensor#

Overview#

Usage#

Output format#

Ray patterns#

Circular (2D LIDAR)#

Planar (3D LIDAR)#

Custom pattern#

Performance#

Depth camera#

API reference#

`gs.sensors.Raycaster`#

`gs.sensors.DepthCamera`#

Ray patterns#

See also#

This Page

Raycaster sensor#

Overview#

Usage#

Output format#

Ray patterns#

Circular (2D LIDAR)#

Planar (3D LIDAR)#

Custom pattern#

Performance#

Depth camera#

API reference#

gs.sensors.Raycaster#

gs.sensors.DepthCamera#

Ray patterns#

See also#

This Page

`gs.sensors.Raycaster`#

`gs.sensors.DepthCamera`#