> ## Documentation Index
> Fetch the complete documentation index at: https://avala.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Multi-Sensor Viewer

> Synchronized playback of camera, LiDAR, radar, and IMU data from MCAP/ROS recordings

The multi-sensor viewer is part of Avala's visualization workspace. It provides synchronized playback of MCAP and ROS recordings, automatically detects topics in your recording, assigns the appropriate panel type to each one, and builds an optimized layout -- so you can go from file upload to visual inspection without any manual configuration.

## How It Works

<Steps>
  <Step title="Upload an MCAP file">
    Upload your MCAP or ROS bag file to a dataset. The platform extracts the topic list and schema metadata from the recording.
  </Step>

  <Step title="Topics are detected">
    Each topic is classified by schema type and keyword matching. For example, a `sensor_msgs/PointCloud2` topic is identified as 3D point cloud data, while a topic named `/camera/front/compressed` is identified as an image stream.
  </Step>

  <Step title="Panels are assigned">
    Every detected topic is mapped to one of eight panel types. The viewer selects the panel that best represents the data -- image panels for cameras, 3D panels for LiDAR, plot panels for IMU, and so on.
  </Step>

  <Step title="Layout is built">
    The layout engine arranges panels into a three-column layout optimized for the specific combination of topics in your recording. Image panels are arranged in a smart grid, while plots and other secondary data are placed below.
  </Step>

  <Step title="Playback is ready">
    All panels are synchronized to a shared timeline. Navigate frame-by-frame, seek to any timestamp, or play back the full recording.
  </Step>
</Steps>

## Panel Types

The viewer supports eight panel types. Each panel is automatically assigned based on the topic's schema and name.

| Panel                | Color                                          | Description                                          | Example Topics                            |
| -------------------- | ---------------------------------------------- | ---------------------------------------------------- | ----------------------------------------- |
| **Image**            | <span style={{color: '#4CAF50'}}>Green</span>  | Camera image streams (raw and compressed)            | `/camera/front/image`, `/cam0/compressed` |
| **3D / Point Cloud** | <span style={{color: '#2196F3'}}>Blue</span>   | LiDAR scans, radar returns, 3D point data            | `/lidar/points`, `/velodyne/pointcloud`   |
| **Plot**             | <span style={{color: '#FF9800'}}>Orange</span> | Numeric time-series data rendered as line charts     | `/imu/data`, `/vehicle/speed`             |
| **Raw Messages**     | <span style={{color: '#9E9E9E'}}>Gray</span>   | Structured message data displayed as key-value pairs | Any unrecognized schema                   |
| **Log**              | <span style={{color: '#795548'}}>Brown</span>  | Timestamped log entries                              | `/rosout`, `/diagnostics/log`             |
| **Map**              | <span style={{color: '#00BCD4'}}>Cyan</span>   | Geographic coordinates plotted on a map              | `/gps/fix`, `/navsat`                     |
| **Gauge**            | <span style={{color: '#E91E63'}}>Pink</span>   | Single-value sensor readings with visual indicator   | Manual assignment only                    |
| **State**            | <span style={{color: '#FF5C00'}}>Orange</span> | Discrete state transitions over time                 | `/vehicle/state`, `/system/mode`          |

## Layout System

The viewer uses a three-column layout that adapts to the topics in your recording.

### Column Structure

| Column            | Width | Content                                            |
| ----------------- | ----- | -------------------------------------------------- |
| **Left sidebar**  | 18%   | Topics list -- browse and toggle individual topics |
| **Center area**   | 64%   | Dynamic panel arrangement based on detected data   |
| **Right sidebar** | 18%   | File selector and recording metadata               |

### Smart Image Grid

When multiple image topics are present, the center area arranges them in an optimized grid:

| Image Count | Layout                                     |
| ----------- | ------------------------------------------ |
| 1           | Single panel, full width                   |
| 2           | Side-by-side, equal width                  |
| 3           | Two panels on top, one spanning the bottom |
| 4+          | 2x2 grid                                   |

### Mixed-Type Splitting

When the recording contains both spatial data (images, point clouds) and time-series data (plots, gauges), the center area splits vertically:

* **Top 70%** -- Image and 3D / Point Cloud panels
* **Bottom 30%** -- Plot, Gauge, and other secondary panels

This keeps the primary visual data prominent while still displaying supporting sensor streams.

## Auto-Detection

The viewer classifies topics using two methods, applied in order.

### Schema-Based Detection

When a topic uses a known schema, the viewer assigns the panel type directly:

| Schema                                                                                              | Panel            |
| --------------------------------------------------------------------------------------------------- | ---------------- |
| `sensor_msgs/Image`, `sensor_msgs/CompressedImage`, `foxglove.CompressedImage`, `foxglove.RawImage` | Image            |
| `sensor_msgs/PointCloud2`, `foxglove.PointCloud`                                                    | 3D / Point Cloud |
| `sensor_msgs/NavSatFix`                                                                             | Map              |
| `sensor_msgs/Imu`                                                                                   | Plot             |

### Keyword-Based Detection

When a topic does not match a known schema by exact name, the viewer checks for keywords in the schema name and topic name:

| Keywords                                      | Matched against      | Panel            |
| --------------------------------------------- | -------------------- | ---------------- |
| `image`, `camera`                             | Topic name           | Image            |
| `lidar`, `pointcloud`, `velodyne`, `points`   | Topic name           | 3D / Point Cloud |
| `log`, `rosout`                               | Schema or topic name | Log              |
| `pose`                                        | Schema or topic name | 3D / Point Cloud |
| `transform`, `odometry`                       | Schema name          | 3D / Point Cloud |
| `odom`, `tf`                                  | Topic name           | 3D / Point Cloud |
| `imu`                                         | Schema or topic name | Plot             |
| `twist`, `vector3`, `float`, `int`            | Schema name          | Plot             |
| `cmd_vel`, `battery`, `temperature`, `sensor` | Topic name           | Plot             |
| `navsat`, `gps`                               | Schema or topic name | Map              |
| `fix`                                         | Topic name           | Map              |
| `state`                                       | Schema or topic name | State            |

<Tip>
  If a topic cannot be classified by schema or keyword, it is assigned the **Raw Messages** panel, which displays the structured message content as key-value pairs.
</Tip>

## Synchronized Playback

All panels in the viewer share a single timeline. Playback controls include:

* **Play / Pause** -- Stream all panels at the recording's original rate
* **Frame stepping** -- Advance or rewind one frame at a time across all panels
* **Timestamp seeking** -- Jump to any point in the recording; all panels update simultaneously
* **Playback speed** -- Adjust playback rate for slow-motion or fast-forward review

When you seek to a specific timestamp, every panel displays the message closest to that time, maintaining synchronization across sensors with different publishing rates.

## Multi-Camera Support

Recordings with multiple camera topics are fully supported:

* **Grid display** -- All camera streams are arranged in the smart image grid within the center area
* **Calibration-aware projection** -- When camera intrinsics and extrinsics are present in the recording, LiDAR points can be projected onto camera images for visual verification
* **Independent zoom** -- Each camera panel can be zoomed and panned independently while maintaining timeline synchronization

<Tip>
  Multi-camera projection is one of the most effective ways to verify 3D annotation quality. The camera overlay reveals depth and heading errors that are difficult to spot in the top-down point cloud view alone.
</Tip>

## Next Steps

<CardGroup cols={2}>
  <Card title="" icon="table-cells-large" href="/docs/visualization/mcap-ros/panels">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Panel Types</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Detailed reference for each of the eight panel types and their configuration options.</p>
  </Card>

  <Card title="" icon="robot" href="/docs/visualization/mcap-ros/overview">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>MCAP / ROS Overview</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Supported MCAP and ROS formats, schemas, and data pipeline setup.</p>
  </Card>
</CardGroup>