Avala + LeRobot

Convert an Avala sequence dataset directly into a Hugging Face LeRobot v3 dataset — the de-facto standard format for open robot-learning. The converter reads sequences and frames straight through the SDK (no exports.create, no archive download) and writes a LeRobot dataset on disk, optionally pushing it to the Hub. One Avala sequence becomes one LeRobot episode. Each camera maps to an observation.images.<cam> feature; timestamp/frame_index/episode_index are derived from --fps.

Prerequisites

Install either writer:

pip install "avala[lerobot-core-video]"   # torch-free, Python 3.9+
pip install "avala[lerobot]"              # the lerobot library, Python 3.12+

The torch-free writer (avala.converters.lerobot_v3) needs only pyarrow, numpy, pillow and, for video features, PyAV. It writes the same LeRobot v3 layout as the lerobot library (meta/info.json, meta/stats.json, meta/tasks.parquet, meta/episodes/…, data/chunk-…/file-….parquet, videos/<camera>/chunk-…/file-….mp4); its output loads in LeRobotDataset. By default (--backend auto) the exporter uses the lerobot library when it is installed and the torch-free writer otherwise. --push needs the lerobot library.

The lerobot library requires Python 3.12+; the lerobot extra is unusable on 3.9–3.11. Use avala[lerobot-core-video] there, or avala[lerobot-core] with --no-video to store frames as PNG images and skip video encoding.

Convert from the CLI

avala lerobot export my-org/my-dataset \
  --repo-id my-hf-user/my-dataset \
  --output ./lerobot-out \
  --fps 30 \
  --task "pick up the cube"

This walks every sequence in my-org/my-dataset, writes a LeRobot v3 dataset to ./lerobot-out, and finalizes it (so the parquet footers are written and the dataset is readable).

Flag	Purpose
`--repo-id`	Target dataset id, `<hf_user>/<name>` (required)
`--output`	Local output directory (required)
`--fps`	Frames per second; timestamps are synthesized as `frame_index / fps` (default 30)
`--task`	Instruction string attached to every frame
`--camera`	Restrict to specific camera(s) by positional name (`cam0`, `cam1`, …); repeatable. Default: all cameras on frame 0
`--state-key` / `--action-key`	Dotted path into the raw frame for a numeric `observation.state` / `action` vector (see below)
`--ego-pose-state`	Use the 7-dim camera-rig ego pose as `observation.state` (rig pose, not proprioception)
`--no-video`	Store frames as PNG image features instead of encoded video
`--limit`	Convert at most N sequences
`--push`	Push the result to the Hugging Face Hub (requires HF auth)
`--tag`	Extra dataset-card tag(s) for the Hub; repeatable. `avala` and `LeRobot` are always added
`--license`	License for the pushed dataset card (default: lerobot’s `apache-2.0`)
`--include-outcomes`	Write each episode’s outcome label to `meta/avala_sequence_outcomes.jsonl`. When the label’s `source_metadata` carries the task text (`task`, `task_text`, `instruction` or `language_instruction`), it also becomes that episode’s LeRobot `task` (instead of `--task`)
`--control-source-key`	Dotted path into the raw frame for who was driving (see below)
`--control-source-column`	Column name for `--control-source-key` (default `annotation.avala.control_source`)
`--backend`	`auto` (default), `lerobot` or `core` (the torch-free writer)

Convert from Python

from avala import Client
from avala.lerobot import export_dataset

export_dataset(
    Client(),
    "my-org",
    "my-dataset",
    repo_id="my-hf-user/my-dataset",
    output_dir="./lerobot-out",
    fps=30,
    task="pick up the cube",
)

The result is a standard LeRobot dataset:

from lerobot.datasets import LeRobotDataset

ds = LeRobotDataset("my-hf-user/my-dataset", root="./lerobot-out")
print(ds.num_episodes, ds.num_frames)

The output is standard LeRobot v3, so it also works with StreamingLeRobotDataset (train directly from the Hub with no full download) once pushed. When you --push, the dataset card is tagged LeRobot + robotics (by lerobot) and avala, so it shows up in the LeRobot dataset viewer and filters.

Perception vs. policy datasets (read this)

Avala sequence datasets are annotation-centric: they reliably provide camera frames and calibration, but not robot proprioception. By default this converter therefore produces a perception / vision-language dataset (cameras + timestamps + a task string) and prints a warning saying so. That is a valid LeRobot dataset, but it is not a policy-training dataset — it has no action/observation.state. To produce robot observation.state / action, the source frames must actually carry that data, and you point the converter at it:

# When the raw frame dicts embed numeric vectors (customer-specific schema):
avala lerobot export my-org/my-dataset --repo-id u/d --output ./out \
  --state-key observation.joint_positions \
  --action-key action

# Or use the capture-rig ego pose as observation.state (labelled as rig pose):
avala lerobot export my-org/my-dataset --repo-id u/d --output ./out --ego-pose-state

State/action are all-or-nothing: a configured key that is missing or non-numeric on any frame is an error — the converter never fabricates zeros.

Who was driving: `control_source`

When the raw frames record who was in control — policy, teleop, intervention or hold — point --control-source-key at that field. It is written verbatim as a per-frame string column, annotation.avala.control_source by default:

avala lerobot export my-org/my-dataset --repo-id u/d --output ./out \
  --control-source-key control.source

Like state/action it is all-or-nothing: a frame without the field is an error.

Round-tripping LeRobot data through Avala

A LeRobot v3 dataset imported with avala import lerobot keeps every per-frame column in its MCAP episodes (see the CLI reference), including a control_source column on the /lerobot/control_source topic. To write such MCAP episodes back to LeRobot v3 without torch:

from avala.converters.lerobot_v3.mcap import mcap_to_lerobot

mcap_to_lerobot(
    ["episode_000000.mcap", "episode_000001.mcap"],
    "./lerobot-out",
    repo_id="my-hf-user/my-dataset",
)

Original column names, dtypes, shapes, fps and robot_type are restored from the metadata the importer stores in each MCAP. Numeric and text columns, timestamps and episode boundaries come back exactly; camera frames are re-encoded, so pixels are close but not identical. Other MCAP files are converted too, including robot recordings:

Cameras: foxglove.CompressedVideo and foxglove.CompressedImage topics become video features (/camera/high/video → observation.images.high). H.264 is copied into the LeRobot mp4 without re-encoding when each LeRobot frame maps to one packet, the episode starts on a keyframe and there are no B-frames; otherwise that camera is decoded and re-encoded, with a warning that says why.
Joints: sensor_msgs/JointState on /joint_states becomes observation.state (positions, with joint names), plus observation.velocity / observation.effort when populated. A command topic (/joint_commands, or any joint topic whose name contains command, cmd, action, target or goal) becomes action. Without one there is no action: it is never derived from the state.
Task: a foxglove.Log named task becomes the LeRobot task of every frame.
Timing: frames follow the first camera’s recorded times; other topics contribute their latest value at or before each frame, and frames before every topic has published are dropped. LeRobot timestamps are frame_index / fps, so the largest deviation from the recorded times is reported (a warning past half a frame) and written, with each camera’s pass-through/re-encode mode, to meta/avala_export_report.json.

Next Steps

Avala + PyTorch — stream Avala data into PyTorch with no export step
Python SDK reference
LeRobot — the robot-learning library and dataset format

Resources

Framework Guides

Updates

Prerequisites

Convert from the CLI

Convert from Python

Perception vs. policy datasets (read this)

Who was driving: `control_source`

Round-tripping LeRobot data through Avala

Next Steps

​Prerequisites

​Convert from the CLI

​Convert from Python

​Perception vs. policy datasets (read this)

​Who was driving: control_source

​Round-tripping LeRobot data through Avala

​Next Steps

Prerequisites

Convert from the CLI

Convert from Python

Perception vs. policy datasets (read this)

Who was driving: `control_source`

Round-tripping LeRobot data through Avala

Next Steps