System Overview
Mission Control
Web UI at avala.ai — Visualization, Annotation, Project Management, Quality Control, Dataset Management
Visualization Engine
GPU-accelerated rendering — Multi-Sensor Viewer, 3D Point Cloud Viewer, Gaussian Splat Viewer, WebGPU/WebGL rendering pipeline
REST API
api.avala.ai/api/v1 — Datasets, Projects, Tasks, Exports, Organizations. Access via direct HTTP/cURL, Python SDK, TypeScript SDK, or CLI.Backend Services
Asynchronous processing — MCAP Parsing, Video Conversion, Export Generation, Task Processing, Inference Pipeline
Components
Mission Control
Mission Control is Avala’s web application at avala.ai. It provides:- Data Visualization — GPU-accelerated viewers for exploring sensor data before, during, and after annotation. Multi-sensor playback, 3D point cloud rendering, and Gaussian Splat scene navigation — all in the browser.
- Annotation Editor — Purpose-built tools for labeling images, video, point clouds, and multi-sensor data with support for bounding boxes, polygons, cuboids, segmentation, polylines, and keypoints.
- Project Management — Create and configure annotation projects, define label taxonomies, assign work to team members, and monitor progress.
- Quality Control — Review submitted annotations, flag issues, track acceptance rates, and run consensus workflows.
- Dataset Management — Upload data, organize items into sequences, browse and search datasets, and manage access permissions.
Visualization Engine
The visualization engine runs in the browser and provides GPU-accelerated rendering for sensor data:- Multi-Sensor Viewer — Synchronized playback of MCAP and ROS recordings with eight panel types: Image, 3D / Point Cloud, Plot, Raw Messages, Log, Map, Gauge, and State Transitions. Automatic topic detection and smart layout composition arrange panels based on your data.
- 3D Point Cloud Viewer — Renders LiDAR data with six visualization modes (Neutral, Intensity, Rainbow, Label, Panoptic, Image Projection). Bird’s-eye view, perspective, and side views. WebGPU compute shaders handle frustum culling and level-of-detail rendering for high frame rates on dense scans.
- Gaussian Splat Viewer — WebGPU-accelerated rendering of 3D Gaussian Splat scene reconstructions. GPU radix sorting, buffer pooling, and pipeline precompilation deliver real-time navigation through photorealistic environments.
- Rendering Pipeline — WebGPU is preferred with automatic WebGL fallback for broader browser support. Feature flags control compute shaders, render bundles, and WGSL shader compilation.
- Multi-Window Layout — Configurable panel arrangements with drag-and-drop. A tree-based layout system with automatic balancing lets you customize how panels are organized, or let the layout composer build an optimized arrangement from your data.
REST API
The REST API atapi.avala.ai/api/v1 provides programmatic access to every platform capability. All requests authenticate with an API key passed in the X-Avala-Api-Key header.
Core resources:
| Resource | Description |
|---|---|
| Datasets | Create, list, and manage datasets and their items |
| Projects | Configure annotation workflows, label taxonomies, and task settings |
| Tasks | Assign, track, and manage individual annotation work units |
| Exports | Generate and download annotated data in various formats |
| Organizations | Manage teams, members, roles, and permissions |
SDKs
You can call the REST API directly from any language using standard HTTP clients (see REST API), or use the official SDKs:- Python SDK (
pip install avala) — Sync and async clients, Pydantic models, automatic pagination, typed exceptions, CLI tool. - TypeScript SDK (
npm install @avala-ai/sdk) — Fully typed client with Promise-based API for Node.js and edge runtimes. - CLI (
curl -fsSL https://avala.ai/install.sh | bash) — Manage datasets, projects, exports, and storage from your terminal.
Backend Services
Backend services handle asynchronous processing that powers the platform:- MCAP Parsing — Extracts and synchronizes sensor streams from MCAP recordings, detects message types, and builds frame indices for the multi-sensor viewer.
- Video Conversion — Splits uploaded video files into frame sequences for visualization and annotation.
- Export Generation — Produces downloadable exports in JSON, COCO, KITTI, and other formats.
- Task Processing — Generates and distributes annotation tasks based on project configuration.
- Inference Pipeline — Runs model inference for AI-assisted annotation and pre-labeling.
Storage
Storage manages all persistent data:- Dataset files — Raw images, video frames, point clouds, and MCAP recordings.
- Annotations — Labeled data produced by annotators (bounding boxes, polygons, cuboids, masks).
- Exports — Generated export files available for download.
- Model artifacts — Weights and configurations for inference models used in AI-assisted labeling.
MCP Server
The MCP (Model Context Protocol) server exposes Avala tools to AI assistants like Claude, Cursor, and VS Code Copilot. It enables natural-language interaction with datasets, projects, and exports without leaving your development environment.The
@avala-ai/mcp-server package is available on npm. Install with npm install -g @avala-ai/mcp-server or use npx. See the MCP setup guide for installation instructions.How Components Interact
- Users interact with Avala through Mission Control (web UI) or programmatically through the REST API and SDKs.
- Mission Control loads data through the REST API and renders it using the visualization engine directly in the browser — no server-side rendering required.
- All client interfaces communicate with the same REST API, so actions taken in Mission Control are immediately visible via the API and vice versa.
- The REST API validates requests, manages resources, and dispatches work to backend services for asynchronous processing.
- Backend services process tasks (MCAP parsing, video conversion, export generation) and write results back to storage.
Next Steps
API Reference
Base URL, authentication, endpoints, and response format.
Use Avala with AI
Connect Avala to Claude, Cursor, and VS Code with MCP.
Data Types
Visualization capabilities and annotation tools for each data type.
Core Concepts
Viewers, panels, layouts, timelines, and other platform concepts.