> ## Documentation Index
> Fetch the complete documentation index at: https://avala.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Physical AI & Spatial Computing

> Build training datasets for embodied AI, digital twins, and spatial computing applications

Physical AI systems -- embodied agents, digital twins, spatial computing applications, and sim-to-real transfer pipelines -- require training data that captures the 3D structure of real environments. Avala provides the visualization and annotation tools these teams need: Gaussian Splat scene rendering, GPU-accelerated point cloud visualization, multi-sensor MCAP playback, and 3D annotation tools that work directly on spatial data.

## Why Avala for Physical AI

Traditional annotation platforms are built for 2D images. Physical AI teams need to work with 3D scenes, point clouds, multi-sensor recordings, and spatial reconstructions. Avala handles all of these natively.

<CardGroup cols={2}>
  <Card title="" icon="circle-nodes">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Gaussian Splat Viewer</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Load 3D Gaussian Splat scene reconstructions into a WebGPU-rendered viewer with scene hierarchy, properties inspector, and real-time statistics. Navigate and annotate directly in the reconstructed 3D environment.</p>
  </Card>

  <Card title="" icon="cube">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Point Cloud Visualization</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>GPU-accelerated point cloud rendering with 6 visualization modes. Inspect spatial structure, density, and sensor coverage with Neutral, Intensity, Rainbow, Label, Panoptic, and Image Projection views.</p>
  </Card>

  <Card title="" icon="play">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Multi-Sensor MCAP Playback</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Play back recorded sensor data from embodied AI systems -- cameras, depth sensors, LiDAR, IMU -- in a synchronized multi-panel viewer.</p>
  </Card>

  <Card title="" icon="pen-to-square">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>3D Annotation Tools</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Annotate 3D cuboids, classification labels, and object attributes directly on point clouds and Gaussian Splat scenes without switching tools.</p>
  </Card>
</CardGroup>

## Data Types

| Application               | Avala Data Type        | Typical Annotation                    |
| ------------------------- | ---------------------- | ------------------------------------- |
| Scene reconstruction      | Splat (Gaussian Splat) | 3D cuboids, classification            |
| Spatial mapping           | Point Cloud            | 3D cuboids, segmentation              |
| Embodied agent recordings | MCAP                   | Multi-sensor annotation with tracking |
| Navigation training       | Point Cloud, MCAP      | 3D cuboids, polylines                 |
| Object recognition        | Image, Point Cloud     | Bounding boxes, 3D cuboids            |
| Digital twin generation   | Splat, Point Cloud     | Classification, object attributes     |

## Use Cases

### Scene Understanding for Embodied AI

Embodied AI agents need to understand the 3D structure of their environment: where objects are, what surfaces are traversable, and how the space is organized. Avala's point cloud and Gaussian Splat viewers let you visualize captured environments, then annotate objects, regions, and spatial relationships that train scene understanding models.

**Workflow:** Capture environment with LiDAR or depth cameras -> upload point cloud or Gaussian Splat reconstruction -> visualize and inspect in 3D viewer -> annotate objects with 3D cuboids and classification labels -> export for model training.

### 3D Object Recognition

Train models to recognize objects in 3D space using annotated point clouds and scene reconstructions. The 3D cuboid tool lets annotators place precise bounding volumes around objects with full position, dimension, and heading control. Classification attributes add category, material, and state metadata to each object.

### Sim-to-Real Transfer

Teams building simulation environments need labeled real-world data to validate and calibrate their simulations. Avala handles the real-world side of the pipeline:

<Steps>
  <Step title="Capture real-world data">
    Record multi-sensor data from the target environment using your robot or sensor rig. Use MCAP format for multi-sensor recordings or PCD/PLY for standalone point clouds.
  </Step>

  <Step title="Reconstruct and visualize">
    Upload Gaussian Splat reconstructions or raw point clouds. Explore the data in Avala's 3D viewers to understand scene structure.
  </Step>

  <Step title="Annotate ground truth">
    Label objects, regions, and spatial relationships that your simulation needs to replicate accurately.
  </Step>

  <Step title="Export for simulation alignment">
    Export annotations with precise 3D coordinates. Use these as ground truth to validate and tune your simulation parameters.
  </Step>
</Steps>

### Digital Twin Data Annotation

Digital twin applications need annotated data that maps the physical world to its virtual representation. Avala's Gaussian Splat viewer is particularly useful here -- it renders photorealistic 3D scene reconstructions that annotators can navigate and label as if they were in the real environment.

The viewer provides:

* **Scene hierarchy panel** -- Browse and select objects in the scene tree
* **Properties inspector** -- View and edit object attributes
* **Real-time statistics** -- Monitor rendering performance
* **Undo/redo** -- Full edit history for annotation corrections

### Navigation and Path Planning

For robots and autonomous systems that need to navigate physical spaces, annotate traversable regions, obstacles, and waypoints in point cloud data. Use polylines to define paths and boundaries, and 3D cuboids to mark obstacles with size and orientation.

## Avala Features Used

| Feature                   | Purpose                                                        | Learn More                                                     |
| ------------------------- | -------------------------------------------------------------- | -------------------------------------------------------------- |
| Gaussian Splat viewer     | Visualize and annotate 3D scene reconstructions                | [Visualization Overview](/docs/visualization/overview)              |
| Point cloud visualization | Inspect spatial data with 6 rendering modes                    | [Visualization Overview](/docs/visualization/overview)              |
| MCAP / ROS integration    | Ingest multi-sensor recordings from embodied AI systems        | [MCAP & ROS](/docs/visualization/mcap-ros/overview)                 |
| 3D cuboid annotation      | Label objects in 3D space with precise position and dimensions | [3D Cuboid Tool](/docs/annotation/guides/tools/cuboid-3d)           |
| Classification            | Scene-level and object-level categorical labels                | [Classification Tool](/docs/annotation/guides/tools/classification) |
| Python SDK                | Programmatic dataset management and export                     | [Python SDK](/docs/sdks/python)                                     |
| TypeScript SDK            | Integrate with Node.js pipelines                               | [TypeScript SDK](/docs/sdks/typescript)                             |
| Cloud storage             | Connect S3 or GCS for large 3D datasets                        | [Cloud Storage](/docs/integrations/cloud-storage)                   |

## Example Pipeline

```
Real-world environment capture (LiDAR, cameras, depth sensors)
  -> Generate 3D reconstruction (Gaussian Splat or point cloud)
  -> Upload to Avala dataset
  -> Visualize in 3D viewer -- inspect scene structure and quality
  -> Create annotation project with 3D cuboid + classification task
  -> Annotators label objects, regions, and spatial relationships
  -> QC review in the same 3D viewer
  -> Export with 3D coordinates and metadata
  -> Feed into embodied AI training / simulation calibration pipeline
```

## Getting Started

<Steps>
  <Step title="Choose your data format">
    Use Gaussian Splat format for scene reconstructions, PCD/PLY for raw point clouds, or MCAP for multi-sensor recordings from embodied systems.
  </Step>

  <Step title="Upload and visualize">
    Create a dataset with the appropriate data type and upload your files. Open them in the 3D viewer to explore the spatial data.
  </Step>

  <Step title="Define your annotation schema">
    Set up object classes and attributes that match your model's requirements -- object categories, materials, states, spatial relationships.
  </Step>

  <Step title="Annotate in 3D">
    Your team places 3D cuboids and classification labels directly in the point cloud or Gaussian Splat scene.
  </Step>

  <Step title="Export and integrate">
    Export annotations with 3D coordinates via the API or SDK. Integrate into your training pipeline, simulation, or digital twin system.
  </Step>
</Steps>

## Next Steps

<CardGroup cols={2}>
  <Card title="" icon="eye" href="/docs/visualization/overview">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Visualization Overview</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Full overview of Avala's visualization capabilities, including Gaussian Splat and point cloud viewers.</p>
  </Card>

  <Card title="" icon="cube" href="/docs/annotation/guides/tools/cuboid-3d">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>3D Cuboid Tool</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>How to annotate objects in 3D with precise position, dimension, and heading control.</p>
  </Card>

  <Card title="" icon="database" href="/docs/getting-started/data-types">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Data Types</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Supported formats and upload requirements for Splat, Point Cloud, MCAP, and other data types.</p>
  </Card>

  <Card title="" icon="code" href="/docs/sdks/python">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Python SDK</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Programmatic access for managing datasets, creating projects, and exporting annotations.</p>
  </Card>
</CardGroup>