> ## Documentation Index
> Fetch the complete documentation index at: https://avala.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Multi-Camera Setup

> Configure multiple camera streams with calibration data for cross-view projection

Avala's multi-sensor viewer supports recordings with any number of camera topics. When you upload an MCAP file containing multiple image streams, the viewer arranges them in a synchronized grid and -- when calibration data is available -- projects LiDAR point clouds onto each camera view for cross-sensor verification.

## Uploading Multi-Camera Recordings

<Steps>
  <Step title="Upload your MCAP file">
    Navigate to your dataset in Avala and upload the MCAP file containing your camera topics. The platform parses the recording and extracts all available topics.
  </Step>

  <Step title="Select camera topics">
    After parsing, Avala presents a topic selection screen listing every detected topic. Choose the camera streams you want to visualize. You can also select LiDAR, IMU, and other topics at this stage.
  </Step>

  <Step title="Viewer builds the layout">
    The viewer auto-detects image topics and arranges them in a smart grid. Non-image topics (LiDAR, plots, gauges) are placed in separate panels alongside the camera grid.
  </Step>
</Steps>

<Tip>
  You do not need to select every camera topic. If your recording contains 8 surround cameras but you only need front and rear, select just those two. This reduces viewer load and keeps the layout focused.
</Tip>

## Smart Grid Layout

The viewer arranges camera panels based on how many image topics are active:

| Camera Count | Layout       | Description                                    |
| ------------ | ------------ | ---------------------------------------------- |
| **1**        | Full width   | Single camera fills the center area            |
| **2**        | Side-by-side | Two cameras at equal width                     |
| **3**        | 2 + 1        | Two panels on top, one spanning the bottom row |
| **4+**       | Grid         | 2-column grid, rows added as needed            |

Panels can be resized by dragging the dividers between them. The layout persists while you navigate the timeline.

## Calibration and Transforms

For LiDAR-to-camera projection to work, the viewer needs two pieces of information: where each camera is in 3D space (extrinsics) and how each camera forms images (intrinsics).

### Extrinsics from TF Messages

Avala reads coordinate frame transforms from `tf2_msgs/TFMessage` and `foxglove.FrameTransform` messages in your recording. These provide the rigid-body transforms (rotation + translation) between sensor frames.

The viewer resolves the full transform chain from the LiDAR frame to each camera frame automatically. For example, if your recording contains:

```
base_link → lidar_top
base_link → camera_front
base_link → camera_left
```

Avala computes the LiDAR-to-camera transform by chaining `lidar_top -> base_link -> camera_front`.

<Tip>
  Include both `/tf` (dynamic transforms) and `/tf_static` (fixed transforms) topics in your recording. Static transforms are typically published once at the start of the recording and define the fixed mounting positions of sensors on the vehicle.
</Tip>

### Camera Intrinsics

Avala supports two camera models for projection:

| Model             | Parameters                                                             | Use Case                                 |
| ----------------- | ---------------------------------------------------------------------- | ---------------------------------------- |
| **Pinhole**       | `fx`, `fy`, `cx`, `cy` + distortion coefficients `k1`-`k4`, `p1`, `p2` | Standard cameras with rectilinear lenses |
| **Double-sphere** | `fx`, `fy`, `cx`, `cy`, `xi`, `alpha`                                  | Wide-angle and fisheye lenses            |

Camera intrinsics can be provided through:

* **CameraInfo messages** -- `sensor_msgs/CameraInfo` topics published alongside image topics
* **Embedded calibration** -- Calibration data stored in the MCAP file metadata

### Projection Behavior

When calibration data is present, the viewer can project LiDAR points onto camera images. Each projected point is colored by its depth, intensity, or label -- matching the active visualization mode in the 3D panel.

This cross-view projection is useful for:

* Verifying that 3D cuboid annotations align with objects in camera views
* Checking sensor calibration accuracy
* Understanding the spatial relationship between LiDAR returns and visual features

<Warning>
  If calibration data is missing or incorrect, LiDAR projection will be misaligned or unavailable. Always verify your calibration before relying on projected overlays for annotation quality checks.
</Warning>

## Visualization Modes in Camera Panels

When LiDAR projection is active, the projected points inherit the color scheme from the current point cloud visualization mode:

| Mode                 | Projected Point Color                                        |
| -------------------- | ------------------------------------------------------------ |
| **Neutral**          | Uniform color                                                |
| **Intensity**        | LiDAR return intensity (blue→green→yellow→red gradient)      |
| **Rainbow**          | Cycling hue per frame                                        |
| **Label**            | Semantic label color from the annotation class               |
| **Panoptic**         | Instance-level color per annotated object                    |
| **Image Projection** | Camera pixel color back-projected onto LiDAR (3D panel only) |

Switch modes in the 3D panel toolbar and the projected points on camera panels update automatically.

## Independent Panel Controls

Each camera panel supports independent interaction while maintaining timeline synchronization:

* **Zoom** -- Scroll to zoom into a specific region of the camera image
* **Pan** -- Click and drag to pan across zoomed images
* **Reset** -- Press `0` to reset zoom and pan to the default view

Timeline position is always shared. Zooming into one camera does not affect other panels.

## Best Practices for Multi-Camera Recordings

<CardGroup cols={2}>
  <Card title="" icon="arrows-rotate">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Include TF topics</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Always record `/tf` and `/tf_static` topics. Without transforms, the viewer cannot resolve coordinate frames between sensors.</p>
  </Card>

  <Card title="" icon="camera">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Publish CameraInfo</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Publish `sensor_msgs/CameraInfo` alongside each image topic. This provides the intrinsics needed for accurate projection.</p>
  </Card>

  <Card title="" icon="tag">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Use consistent frame IDs</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Ensure each sensor topic references the correct frame ID in its message header. Mismatched frame IDs break the transform chain.</p>
  </Card>

  <Card title="" icon="file-zipper">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Compress images</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Use `sensor_msgs/CompressedImage` with JPEG compression to reduce file sizes. Uncompressed images dramatically increase MCAP file size.</p>
  </Card>
</CardGroup>

## Next Steps

<CardGroup cols={2}>
  <Card title="" icon="circle-check" href="/docs/visualization/guides/recording-best-practices">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Recording Best Practices</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Tips for recording data that works well in Avala, including format, compression, and naming conventions.</p>
  </Card>

  <Card title="" icon="palette" href="/docs/visualization/rendering/visualization-modes">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Rendering Modes</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Deep dive into the 6 point cloud visualization modes and when to use each one.</p>
  </Card>

  <Card title="" icon="clock" href="/docs/visualization/guides/timeline-navigation">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Timeline Navigation</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Playback controls, frame stepping, and timestamp seeking across all panels.</p>
  </Card>

  <Card title="" icon="robot" href="/docs/visualization/mcap-ros/overview">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>MCAP & ROS Overview</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Supported formats, message types, and the upload workflow for multi-sensor recordings.</p>
  </Card>
</CardGroup>