> ## Documentation Index
> Fetch the complete documentation index at: https://avala.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Robotics

> Visualize sensor recordings and annotate data for manipulation, navigation, and scene understanding

Robotics teams work with diverse sensor configurations -- depth cameras, stereo rigs, LiDAR, and multi-camera setups that change between robot platforms. Avala handles this variety with native MCAP support for playing back recorded sensor data and a full annotation toolkit for labeling the perception training data that manipulation, navigation, and scene understanding models require.

## Visualization for Robotics Data

Robotics sensor data often arrives as recorded bags or MCAP files from test runs, field deployments, or simulation. Avala's multi-sensor viewer lets you play back these recordings and inspect them before committing to annotation.

<CardGroup cols={2}>
  <Card title="" icon="play">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>MCAP Playback</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Upload MCAP recordings from your robot and play back camera, depth, LiDAR, and IMU streams in a synchronized viewer.</p>
  </Card>

  <Card title="" icon="cube">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Point Cloud Visualization</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Render point clouds from depth cameras and LiDAR with GPU acceleration. Switch between 6 visualization modes to inspect density, intensity, and spatial structure.</p>
  </Card>

  <Card title="" icon="camera-cctv">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Multi-Camera Views</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>View multiple camera streams (RGB, depth, stereo) side by side, synchronized to the same timestamps in the recording.</p>
  </Card>

  <Card title="" icon="clock">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Timeline Navigation</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Step frame-by-frame through robot operations to find key moments -- grasp attempts, navigation decisions, collision events.</p>
  </Card>
</CardGroup>

<Tip>
  If your team is currently using Foxglove or Rerun to review robot recordings, Avala replaces the visualization step and adds annotation, review, and export -- all in one platform.
</Tip>

## Data Types

| Application           | Avala Data Type    | Typical Annotation                            |
| --------------------- | ------------------ | --------------------------------------------- |
| Indoor navigation     | Image, Point Cloud | 2D/3D bounding boxes, segmentation            |
| Pick-and-place        | Image              | Bounding boxes, keypoints, segmentation masks |
| Outdoor mobile robots | MCAP, Point Cloud  | 3D cuboids, polylines                         |
| Manipulation          | Image, Video       | Keypoints, bounding boxes                     |
| Warehouse robots      | Image, MCAP        | Bounding boxes, segmentation, classification  |

## Common Tasks

### Object Detection and Grasping

Label objects on shelves, tables, and conveyor belts with bounding boxes and instance segmentation masks for grasp planning models. For bin-picking tasks, combine bounding boxes with keypoint annotations to mark grasp points on each object.

### Scene Segmentation

Create pixel-level segmentation masks for floors, walls, obstacles, free space, and other surface types. Segmentation data trains navigation models to understand which areas the robot can traverse and which are blocked.

### Keypoint Annotation

Mark joint positions, tool tips, grasping points, and pose landmarks. Keypoint skeletons are configurable -- define the number of points and their connections to match your model's expected input.

### Terrain Classification

For outdoor mobile robots, classify traversable vs. non-traversable surfaces. Combine image-level classification (terrain type, slope) with segmentation masks that delineate safe zones from obstacles.

### Activity and Event Detection

Annotate video recordings of robot operations to label specific events: successful grasp, failed grasp, collision, recovery. Use classification labels on sequences or frame ranges for temporal event annotation.

## Avala Features Used

| Feature                   | Purpose                                                    | Learn More                                                 |
| ------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------- |
| MCAP / ROS integration    | Ingest robot sensor recordings directly                    | [MCAP & ROS](/docs/visualization/mcap-ros/overview)             |
| Multi-sensor viewer       | Synchronized playback of robot sensor streams              | [Multi-Sensor Viewer](/docs/visualization/multi-sensor-viewer)  |
| Point cloud visualization | Inspect depth camera and LiDAR data with 6 rendering modes | [Visualization Overview](/docs/visualization/overview)          |
| Bounding box annotation   | Label objects for detection models                         | [Bounding Box Tool](/docs/annotation/guides/tools/bounding-box) |
| Keypoint annotation       | Mark joint positions and grasp points                      | [Keypoint Tool](/docs/annotation/guides/tools/keypoint)         |
| Segmentation annotation   | Pixel-level masks for scene understanding                  | [Segmentation Tool](/docs/annotation/guides/tools/segmentation) |
| Polygon annotation        | Precise boundaries for irregular objects                   | [Polygon Tool](/docs/annotation/guides/tools/polygon)           |
| Quality control           | Multi-stage review for precision-critical labels           | [Quality Control](/docs/annotation/guides/quality-control)      |
| Slices                    | Organize data by environment, scenario, or robot platform  | [Slices API](/docs/api-reference/slices)                        |

## Example Pipeline

```
Robot sensor recordings (MCAP from test runs)
  -> Upload to Avala dataset
  -> Review recordings in multi-sensor viewer
  -> Identify frames with relevant scenarios (grasps, navigation events)
  -> Create annotation project (bounding boxes + keypoints)
  -> Annotators label objects and grasp points
  -> QC review with spot-checking and targeted review
  -> Export in COCO or custom format
  -> Train manipulation / navigation model
```

## Getting Started

<Steps>
  <Step title="Upload robot recordings">
    Create a dataset and upload your MCAP files. The viewer automatically detects camera, depth, LiDAR, and IMU topics.
  </Step>

  <Step title="Explore the data">
    Play back recordings to understand sensor coverage and data quality. Use frame-by-frame navigation to find key moments.
  </Step>

  <Step title="Define your annotation task">
    Choose the annotation type that matches your model's input: bounding boxes for detection, keypoints for pose estimation, segmentation for scene understanding.
  </Step>

  <Step title="Set up label taxonomy">
    Define object classes and attributes relevant to your robot's task environment (e.g., `cup`, `plate`, `obstacle`, `free_space`).
  </Step>

  <Step title="Annotate, review, and export">
    Your team labels the data, reviewers check quality, and you export in the format your training pipeline expects.
  </Step>
</Steps>

## Fleet Management

For teams operating robot fleets, Avala provides fleet-scale recording management and observability:

* **Device registry** — Track all robots in your fleet with metadata, firmware versions, and health status.
* **Recording browser** — Filter and sort recordings across devices by date, status, and tags.
* **Timeline events** — Mark errors, anomalies, and state changes on recordings for fleet-wide analysis.
* **Recording rules** — Auto-flag recordings matching conditions (e.g., high latency, error frequency).
* **Alerts** — Route notifications to Slack, email, or webhooks when fleet conditions change.

See [Fleet Dashboard](/docs/visualization/fleet/fleet-dashboard) to get started.

## Next Steps

<CardGroup cols={2}>
  <Card title="" icon="satellite-dish" href="/docs/visualization/fleet/fleet-dashboard">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Fleet Dashboard</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Manage devices, recordings, and telemetry across your robot fleet.</p>
  </Card>

  <Card title="" icon="robot" href="/docs/visualization/mcap-ros/overview">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>MCAP & ROS</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Supported formats and how to prepare robot recordings for upload.</p>
  </Card>

  <Card title="" icon="pen-to-square" href="/docs/annotation/guides/annotation-tools">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Annotation Tools</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Overview of all annotation tools available for robotics data.</p>
  </Card>

  <Card title="" icon="circle-check" href="/docs/visualization/guides/recording-best-practices">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Recording Best Practices</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Tips for recording robot data that works well in Avala.</p>
  </Card>
</CardGroup>