Camera sensor#

A camera sensor renders an RGB image of the scene each step and returns it through the sensor read() interface, so a camera reads like any other sensor: attach it, step, read.

Do not confuse it with the visualization camera. The camera sensor is created with scene.add_sensor(gs.sensors.*CameraOptions(...)) and its read() returns a CameraReturnType carrying a single rgb field. The visualization camera is created with scene.add_camera(...) and its render() returns RGB together with depth, segmentation, and surface normals. Reach for the visualization camera when you need those extra channels or the interactive viewer; see Camera.

Each backend is a separate options class and sensor, all returning the same CameraReturnType(rgb). For usage and how to choose a backend, see the camera sensing guide.

Return type#

class genesis.engine.sensors.camera.CameraReturnType(rgb: Tensor)[source]#

Camera sensor return data.

Rasterizer camera#

class genesis.options.sensors.camera.RasterizerCameraOptions(*, history_length: int = 0, delay: float = 0.0, jitter: float = 0.0, draw_debug: bool = False, entity_idx: int = -1, link_idx_local: int = 0, pos_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), euler_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), res: tuple[int, int] = (512, 512), pos: tuple[float, float, float] = (3.5, 0.0, 1.5), lookat: tuple[float, float, float] = (0.0, 0.0, 0.0), up: tuple[float, float, float] = (0.0, 0.0, 1.0), fov: float = 60.0, lights: list[dict[str, Any]] = [], offset_T: tuple[tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float]] | None = None, near: float = 0.01, far: float = 100.0) → None[source]#

Options for Rasterizer camera sensor (OpenGL-based rendering).

Parameters:

  • near (float) – Near clipping plane distance. Default is 0.01.

  • far (float) – Far clipping plane distance. Default is 100.0.

class genesis.engine.sensors.camera.RasterizerCameraSensor(options: RasterizerCameraOptions, idx: int, shared_context, shared_metadata, manager: SensorManager)[source]#

Bases: BaseCameraSensor, Sensor[RasterizerCameraOptions, None, RasterizerCameraSharedMetadata, CameraReturnType]

Rasterizer camera sensor using OpenGL-based rendering.

This sensor renders RGB images using the existing Rasterizer backend, but operates independently from the scene visualizer.

build()[source]#

Initialize the rasterizer and register this camera.

Raytracer camera#

class genesis.options.sensors.camera.RaytracerCameraOptions(*, history_length: int = 0, delay: float = 0.0, jitter: float = 0.0, draw_debug: bool = False, entity_idx: int = -1, link_idx_local: int = 0, pos_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), euler_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), res: tuple[int, int] = (512, 512), pos: tuple[float, float, float] = (3.5, 0.0, 1.5), lookat: tuple[float, float, float] = (0.0, 0.0, 0.0), up: tuple[float, float, float] = (0.0, 0.0, 1.0), fov: float = 60.0, lights: list[dict[str, Any]] = [], offset_T: tuple[tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float]] | None = None, model: Literal['pinhole', 'thinlens'] = 'pinhole', spp: int = 256, denoise: bool = False, aperture: float = 2.8, focal_len: float = 0.05, focus_dist: float = 3.0, env_surface: Any = None, env_radius: float = 15.0, env_pos: tuple[float, float, float] = (0.0, 0.0, 0.0), env_quat: tuple[float, float, float, float] = (1.0, 0.0, 0.0, 0.0)) → None[source]#

Options for Raytracer camera sensor (LuisaRender path tracing).

Parameters:

  • model (str) – Camera model: “pinhole” or “thinlens”. Default is “pinhole”.

  • spp (int) – Samples per pixel for path tracing. Default is 256.

  • denoise (bool) – Whether to apply denoising. Default is False.

  • aperture (float) – Aperture size for thinlens camera (depth of field). Default is 2.8.

  • focal_len (float) – Focal length in meters for thinlens camera. Default is 0.05.

  • focus_dist (float) – Focus distance in meters for thinlens camera. Default is 3.0.

  • env_surface (gs.surfaces.Surface | None) – Environment surface for skybox. Default is None.

  • env_radius (float) – Environment sphere radius. Default is 15.0.

  • env_pos (tuple[float, float, float]) – Environment sphere position. Default is (0, 0, 0).

  • env_quat (tuple[float, float, float, float]) – Environment sphere quaternion (w, x, y, z). Default is (1, 0, 0, 0).

class genesis.engine.sensors.camera.RaytracerCameraSensor(options: RaytracerCameraOptions, idx: int, shared_context, shared_metadata, manager: SensorManager)[source]#

Bases: BaseCameraSensor, Sensor[RaytracerCameraOptions, None, RaytracerCameraSharedMetadata, CameraReturnType]

Raytracer camera sensor using LuisaRender path tracing.

build()[source]#

Register a raytracer camera that reuses the visualizer pipeline.

move_to_attach()[source]#

Move the camera to follow the currently attached rigid link.

Uses a shared transform computation and delegates to _apply_camera_transform().

Batch renderer camera#

class genesis.options.sensors.camera.BatchRendererCameraOptions(*, history_length: int = 0, delay: float = 0.0, jitter: float = 0.0, draw_debug: bool = False, entity_idx: int = -1, link_idx_local: int = 0, pos_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), euler_offset: tuple[float, float, float] = (0.0, 0.0, 0.0), res: tuple[int, int] = (512, 512), pos: tuple[float, float, float] = (3.5, 0.0, 1.5), lookat: tuple[float, float, float] = (0.0, 0.0, 0.0), up: tuple[float, float, float] = (0.0, 0.0, 1.0), fov: float = 60.0, lights: list[dict[str, Any]] = [], offset_T: tuple[tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float], tuple[float, float, float, float]] | None = None, model: Literal['pinhole', 'thinlens', 'fisheye'] = 'pinhole', near: float = 0.01, far: float = 100.0, use_rasterizer: bool = True) → None[source]#

Options for Batch Renderer camera sensor (Madrona GPU batch rendering).

Note: All batch renderer cameras must have the same resolution.

Parameters:

use_rasterizer (bool) – Whether to use rasterizer mode. Default is True.

class genesis.engine.sensors.camera.BatchRendererCameraSensor(options: BatchRendererCameraOptions, idx: int, shared_context, shared_metadata, manager: SensorManager)[source]#

Bases: BaseCameraSensor, Sensor[BatchRendererCameraOptions, None, BatchRendererCameraSharedMetadata, CameraReturnType]

Batch renderer camera sensor using Madrona GPU batch rendering.

Note: All batch renderer cameras must have the same resolution.

build()[source]#

Initialize the batch renderer and register this camera.

See also#

  • Sensors: sensor overview and return-shape table.

  • Camera: the visualization camera (scene.add_camera().render()) for depth, segmentation, and normals.