Camera sensor#

A camera sensor renders an RGB image of the scene each step and returns it through the sensor read() interface, so a camera reads like any other sensor: attach it, step, read.

Do not confuse it with the visualization camera. The camera sensor is created with scene.add_sensor(gs.sensors.*CameraOptions(...)) and its read() returns a CameraReturnType carrying a single rgb field. The visualization camera is created with scene.add_camera(...) and its render() returns RGB together with depth, segmentation, and surface normals. Reach for the visualization camera when you need those extra channels or the interactive viewer; see Camera.

Minimal example#

import genesis as gs

gs.init(backend=gs.gpu)
scene = gs.Scene()
scene.add_entity(gs.morphs.Plane())
scene.add_entity(gs.morphs.URDF(file="urdf/go2/urdf/go2.urdf"))

camera = scene.add_sensor(
    gs.sensors.RasterizerCameraOptions(
        res=(640, 480),      # (width, height)
        pos=(3.5, 0.0, 1.5),
        lookat=(0.0, 0.0, 0.5),
        fov=60.0,            # vertical field of view, degrees
    )
)

scene.build()
scene.step()

frame = camera.read()  # CameraReturnType
rgb = frame.rgb        # uint8, shape ([n_envs,] height, width, 3)

read() renders lazily: the first call after a step renders the current state, and repeated calls within the same step reuse the cached image. Attach the camera to a link by passing entity_idx and link_idx_local, exactly as for other sensors; pos, lookat, and up are then interpreted relative to the link frame.

Choosing a backend#

Each backend is a separate options class and sensor. All three return the same CameraReturnType(rgb); they differ in the renderer they drive and the hardware they require.

`gs.sensors.*`	Renderer	Multiple environments	Requires
`RasterizerCameraOptions`	OpenGL rasterizer	Supported (set `env_separate_rigid=True` in `VisOptions`)	OpenGL
`RaytracerCameraOptions`	LuisaRender path tracer	Not supported (`n_envs > 1` raises)	Scene built with `renderer=gs.renderers.RayTracer(...)`
`BatchRendererCameraOptions`	Madrona batch renderer	Supported, batched on GPU	CUDA backend

Use the rasterizer for fast RGB with no special setup. Use the raytracer for photorealistic single-environment renders. Use the batch renderer when rendering many parallel environments on a GPU.

Rasterizer camera#

class genesis.options.sensors.camera.RasterizerCameraOptions(*, history_length: int = 0, delay: float = 0.0, jitter: float = 0.0, draw_debug: bool = False, entity_idx: int = -1, link_idx_local: int = 0, pos_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), euler_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), res: tuple[int, int] = (512, 512), pos: tuple[float, float, float] = (3.5, 0.0, 1.5), lookat: tuple[float, float, float] = (0.0, 0.0, 0.0), up: tuple[float, float, float] = (0.0, 0.0, 1.0), fov: float = 60.0, lights: list[dict[str, Any]] = [], offset_T: tuple[tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float]] | None = None, near: float = 0.01, far: float = 100.0) → None[source]#

Options for Rasterizer camera sensor (OpenGL-based rendering).

Parameters:

near (float) – Near clipping plane distance. Default is 0.01.
far (float) – Far clipping plane distance. Default is 100.0.

class genesis.engine.sensors.camera.RasterizerCameraSensor(options: RasterizerCameraOptions, idx: int, shared_context, shared_metadata, manager: SensorManager)[source]#

Bases: BaseCameraSensor, Sensor[RasterizerCameraOptions, None, RasterizerCameraSharedMetadata, CameraReturnType]

Rasterizer camera sensor using OpenGL-based rendering.

This sensor renders RGB images using the existing Rasterizer backend, but operates independently from the scene visualizer.

build()[source]#: Initialize the rasterizer and register this camera.

Raytracer camera#

class genesis.options.sensors.camera.RaytracerCameraOptions(*, history_length: int = 0, delay: float = 0.0, jitter: float = 0.0, draw_debug: bool = False, entity_idx: int = -1, link_idx_local: int = 0, pos_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), euler_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), res: tuple[int, int] = (512, 512), pos: tuple[float, float, float] = (3.5, 0.0, 1.5), lookat: tuple[float, float, float] = (0.0, 0.0, 0.0), up: tuple[float, float, float] = (0.0, 0.0, 1.0), fov: float = 60.0, lights: list[dict[str, Any]] = [], offset_T: tuple[tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float]] | None = None, model: Literal['pinhole', 'thinlens'] = 'pinhole', spp: int = 256, denoise: bool = False, aperture: float = 2.8, focal_len: float = 0.05, focus_dist: float = 3.0, env_surface: Any = None, env_radius: float = 15.0, env_pos: tuple[float, float, float] = (0.0, 0.0, 0.0), env_quat: tuple[float, float, float, float] = (1.0, 0.0, 0.0, 0.0)) → None[source]#

Options for Raytracer camera sensor (LuisaRender path tracing).

Parameters:

model (str) – Camera model: “pinhole” or “thinlens”. Default is “pinhole”.
spp (int) – Samples per pixel for path tracing. Default is 256.
denoise (bool) – Whether to apply denoising. Default is False.
aperture (float) – Aperture size for thinlens camera (depth of field). Default is 2.8.
focal_len (float) – Focal length in meters for thinlens camera. Default is 0.05.
focus_dist (float) – Focus distance in meters for thinlens camera. Default is 3.0.
env_surface (gs.surfaces.Surface | None) – Environment surface for skybox. Default is None.
env_radius (float) – Environment sphere radius. Default is 15.0.
env_pos (tuple[float, float, float]) – Environment sphere position. Default is (0, 0, 0).
env_quat (tuple[float, float, float, float]) – Environment sphere quaternion (w, x, y, z). Default is (1, 0, 0, 0).

class genesis.engine.sensors.camera.RaytracerCameraSensor(options: RaytracerCameraOptions, idx: int, shared_context, shared_metadata, manager: SensorManager)[source]#

Bases: BaseCameraSensor, Sensor[RaytracerCameraOptions, None, RaytracerCameraSharedMetadata, CameraReturnType]

Raytracer camera sensor using LuisaRender path tracing.

build()[source]#: Register a raytracer camera that reuses the visualizer pipeline.

move_to_attach()[source]#

Move the camera to follow the currently attached rigid link.

Uses a shared transform computation and delegates to _apply_camera_transform().

Batch renderer camera#

class genesis.options.sensors.camera.BatchRendererCameraOptions(*, history_length: int = 0, delay: float = 0.0, jitter: float = 0.0, draw_debug: bool = False, entity_idx: int = -1, link_idx_local: int = 0, pos_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), euler_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), res: tuple[int, int] = (512, 512), pos: tuple[float, float, float] = (3.5, 0.0, 1.5), lookat: tuple[float, float, float] = (0.0, 0.0, 0.0), up: tuple[float, float, float] = (0.0, 0.0, 1.0), fov: float = 60.0, lights: list[dict[str, Any]] = [], offset_T: tuple[tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float]] | None = None, model: Literal['pinhole', 'thinlens', 'fisheye'] = 'pinhole', near: float = 0.01, far: float = 100.0, use_rasterizer: bool = True) → None[source]#

Options for Batch Renderer camera sensor (Madrona GPU batch rendering).

Note: All batch renderer cameras must have the same resolution.

Parameters:: use_rasterizer (bool) – Whether to use rasterizer mode. Default is True.

class genesis.engine.sensors.camera.BatchRendererCameraSensor(options: BatchRendererCameraOptions, idx: int, shared_context, shared_metadata, manager: SensorManager)[source]#

Bases: BaseCameraSensor, Sensor[BatchRendererCameraOptions, None, BatchRendererCameraSharedMetadata, CameraReturnType]

Batch renderer camera sensor using Madrona GPU batch rendering.