Sequence metadata schema

Every sequence carries two separate kinds of metadata:

Technical validity. Is the recording usable? Avala reports it from acceptance criteria and quality targets. See Technical validity.
Behavioral outcome. What did the person or the robot do, and did the task succeed? Avala stores it as the sequence’s outcome label.

A sequence can be technically valid and still show a failure. A failed sequence is often the most useful one. The two axes stay apart, so a filter for clean data never drops a useful failure, and a filter for success never hides a broken sensor.

The outcome label

Each sequence has at most one current outcome label. Every write records a new version. Earlier versions stay readable in the label history.

Example

This is the label of a human demonstration with a slip and a recovery, as GET .../sequences/<sequence-uid>/outcome/ returns it:

{
  "uid": "<outcome-uid>",
  "sequence_uid": "<sequence-uid>",
  "version": 2,
  "is_current": true,
  "outcome": "mistake_and_recovery",
  "progress": 1.0,
  "quality": 4,
  "speed": 3,
  "subtasks": [
    { "label": "Load the plate into the lower rack", "start_ts": 0.0, "end_ts": 19.8, "outcome": "mistake_and_recovery" },
    { "label": "Pull out the lower rack", "start_ts": 0.0, "end_ts": 6.2, "outcome": "expert_success" },
    { "label": "Place the wet plate in a free slot", "start_ts": 6.2, "end_ts": 19.8, "outcome": "mistake_and_recovery" }
  ],
  "hand_actions": {
    "left": [
      { "start_ts": 0.0, "end_ts": 6.2, "action": "pulling the lower rack out by its front edge", "object": "dishwasher rack", "verb": "pull", "contact": true },
      { "start_ts": 6.2, "end_ts": 19.8, "action": "holding the rack steady", "object": "dishwasher rack", "verb": "hold", "contact": true }
    ],
    "right": [
      { "start_ts": 6.2, "end_ts": 9.1, "action": "carrying the wet plate to the rack", "object": "plate", "verb": "carry", "contact": true },
      { "start_ts": 9.1, "end_ts": 10.4, "action": "the plate slips out of the grasp", "object": "plate", "verb": null, "contact": false },
      { "start_ts": 10.4, "end_ts": 14.7, "action": "regrasping the plate by its rim", "object": "plate", "verb": "grasp", "contact": true },
      { "start_ts": 14.7, "end_ts": 19.8, "action": "placing the plate in a free slot", "object": "plate", "verb": "place", "contact": true }
    ]
  },
  "mistake_type": "slip_during_place",
  "recovery_type": "regrasp",
  "failure_stage": "place",
  "autonomy_level": "human_demonstration",
  "model_version": "",
  "evaluation_membership": "train",
  "leakage_groups": {
    "location": "loc_8f2a",
    "object": "obj_set_31",
    "mechanism": "dishwasher_pull_rack_v2",
    "operator": "op_5c1e",
    "environment_family": "kitchen_galley"
  },
  "source": "human",
  "source_metadata": {},
  "labeled_by": "<user-uid>",
  "confidence": null,
  "created_at": "2026-09-30T17:42:11.204000Z",
  "updated_at": "2026-09-30T17:42:11.204000Z"
}

The values illustrate the shape of the label. They do not describe a real delivery.

Fields

All timestamps are seconds from the start of the sequence.

Field	Type	Meaning
`uid`	string	ID of this label version. Read-only.
`sequence_uid`	string	The sequence the label belongs to. Read-only.
`version`	integer	Label version, starting at 1. Each write adds one. Read-only.
`is_current`	boolean	`true` for the current version. Read-only.
`outcome`	string	The behavioral outcome. Required. See Outcome values.
`progress`	number or null	Task progress from 0 to 1.
`quality`	integer or null	Execution quality from 1 to 5.
`speed`	integer or null	Execution speed from 1 to 5.
`subtasks`	list	Labeled time ranges. See Subtasks.
`hand_actions`	object	Per-hand action streams, `left` and `right`. See Hand actions.
`mistake_type`	string	Your tag for the kind of mistake. Up to 128 characters. Empty when unset.
`recovery_type`	string	Your tag for the kind of recovery. Up to 128 characters. Empty when unset.
`failure_stage`	string	Your tag for the stage where the failure happened. Up to 128 characters. Empty when unset.
`autonomy_level`	string	Who was in control. See Autonomy level. Empty when unset.
`model_version`	string	The policy or model version that drove the robot. Up to 255 characters. Empty for human demonstrations.
`evaluation_membership`	string	`train`, `held_out_eval` or `none`. Empty when unset. See Evaluation membership.
`leakage_groups`	object	Group IDs for split hygiene. See Leakage groups.
`source`	string	Who wrote the label: `human`, `model` or `imported`. Default `human`.
`source_metadata`	object	Import provenance. See Source metadata.
`labeled_by`	string or null	UID of the user who wrote the version. Returned only to callers who can edit the dataset. Read-only.
`confidence`	number or null	Model confidence from 0 to 1. Accepted only when `source` is `model`.
`created_at`, `updated_at`	string	Timestamps of the version. Read-only.

mistake_type, recovery_type and failure_stage are free text. Use the tags from your program’s failure taxonomy so that sequences group cleanly.

Outcome values

Outcome	Meaning
`expert_success`	The task is complete. Execution is fluent and at expert pace.
`slow_success`	The task is complete. Execution is correct and clearly slower than expert pace.
`partial_success`	Some of the task is complete. The final goal state is not reached.
`mistake_and_recovery`	An error occurs, such as a slip, a miss or a wrong placement. The actor recovers without outside help and completes the task.
`intervention`	A human steps in to correct or finish the task.
`unsafe`	The sequence contains an unsafe action or state.
`aborted`	The attempt stops before the task is complete, with no recovery.
`failure`	The attempt ends and the task is not achieved.
`novel_strategy`	The task is solved in a valid way that differs from the usual strategy.
`inefficient_strategy`	The task is solved with avoidable extra motion, repeated attempts or detours.

Each subtask can carry its own outcome from the same list.

Autonomy level

Value	Meaning
`human_demonstration`	A person performs the task.
`teleoperation`	A person drives the robot remotely.
`autonomous`	The robot runs on its policy.
`shared_autonomy`	The policy and a person share control.
`intervention`	A person takes over from the policy during the sequence.

Subtasks

Each subtask is {label, start_ts, end_ts, outcome}, with an optional reward.

Field	Rule
`label`	Natural-language description of the subtask. Required. 1 to 500 characters.
`start_ts`, `end_ts`	Finite numbers, 0 or more. `end_ts` is at least `start_ts`.
`outcome`	An outcome value, or `null`.
`reward`	Optional. A finite number from -1 to 1. `NaN`, infinity, booleans, strings and values outside the range are rejected.

Leave reward out when you did not label one. Avala stores it only when you send it, so a missing reward is “not labeled” and a 0 is a reward of zero. Sending null is the same as leaving it out. reward does not change the ordering or nesting rules below. A label holds at most 256 subtasks. The list must be ordered by start_ts. Any two subtasks are either disjoint or one fully contains the other, such as a phase and its steps. Touching ends count as disjoint. A partial overlap is rejected. When two subtasks start at the same time, list the enclosing one first.

Hand actions

hand_actions holds two independent streams, left and right. Each item describes what one hand does over a time range.

Field	Rule
`start_ts`, `end_ts`	Finite numbers, 0 or more. `end_ts` is at least `start_ts`.
`action`	Natural-language description of the action. Required. 1 to 500 characters.
`object`	The object the hand works on. Optional. Up to 255 characters.
`verb`	A short verb for the action. Optional. Up to 64 characters.
`contact`	`true` when the hand touches the object, `false` when it does not, `null` when unknown. Only JSON booleans are accepted.

Within one hand, items are ordered by start_ts and must not overlap. An item starts at or after the end of the previous one. Touching is fine.
The two hands overlap freely.
Each hand holds at most 3,600 items. That is one hour at one label per second.
left and right are the only keys. You can omit either one. An omitted hand is stored as an empty list.
An empty object or verb is stored as null. Every stored item has all six fields.

Sequence duration

When Avala knows the duration of the sequence, every subtask end_ts and every hand-action end_ts must be at most the duration plus 1 second. The extra second allows for labels written at one per second. Avala knows the duration when the sequence has exactly one extracted MCAP file with a recorded length. For other sequences the check is skipped.

Evaluation membership

evaluation_membership places the sequence in a split:

Value	Meaning
`train`	The sequence may be used for training.
`held_out_eval`	The sequence belongs to a held-out evaluation set. Keep it out of training.
`none`	The sequence is in neither split.

Avala sets held_out_eval on the sequences of every evaluation cohort. Those sequences never appear in a training release. See Evaluation.

Leakage groups

Leakage groups stop near-duplicates from landing on both sides of a train and evaluation split. leakage_groups accepts these keys, each with a group ID of up to 255 characters:

Key	Groups sequences that share
`location`	The same home, site or room.
`object`	The same physical object set.
`mechanism`	The same appliance or mechanism unit.
`operator`	The same collector or teleoperator.
`environment_family`	The same kind of environment and layout, such as a galley kitchen.

Any other key is rejected. Each key is optional. Avala builds evaluation cohorts so that no group value in a cohort appears in any training release. You can split your own data the same way: hold out by group, never by random sequence. Avala’s group IDs are pseudonymous, so you can group and split without learning who or where.

Source metadata

source_metadata records where an imported label came from, for example {"importer": "...", "run_id": "...", "task": "...", "log_path": "..."}.

It is a flat object with at most 20 keys.
Each key is 1 to 64 characters.
Each value is a string of up to 512 characters, a finite number or a boolean.
Nested objects, lists and null values are rejected.

How writes work

A write replaces the current label as a whole and records a new version. Any optional field you leave out is stored as unset. It is not copied from the previous version.
Writing needs edit access to the dataset. Reading needs the same access as reading the sequence, including public datasets.
confidence is rejected unless source is model.
Reading a sequence that has no label returns 404.

Technical validity

Technical validity is not part of the outcome label. Avala reports it from the acceptance engine and from quality targets, and it has no JSON schema of its own. Acceptance criteria. Every capture in a capture campaign gets a machine verdict: accept, quarantine or reject. The verdict comes from these criteria:

Criterion	What it checks
`capture_health`	The MCAP file validates and the channels the campaign requires are present.
`frame_rate`	The camera frame rate holds up across the clip, without a sustained collapse.
`completeness`	The length is close to the length the campaign asked for.
`framing`	The hands are visible for enough of the time the detector watched.
`shake`	The camera is not shaking for most of the clip.

Each criterion reports pass, fail, insufficient_evidence or skipped. A failed criterion records a reason code, such as acceptance_frame_rate_drop or hands_not_visible. A reviewer then confirms or overrides the verdict. The reason codes are listed in Accepted hour. Read acceptance with the MCP tools get_result_acceptance for one capture and get_campaign_acceptance_summary for a campaign. list_capture_submissions includes the machine acceptance summary of each capture. Quality targets. A project can set thresholds on quality metrics, such as acceptance rate, rejection rate and issue rate. Avala alerts when a metric crosses its threshold. Read them with the MCP tool list_quality_targets. Program checks that are not machine criteria, such as calibration, consent and timestamp sync, are reported as rejection reasons in the coverage report. See Accepted hour.

Read and write outcome labels

The REST endpoints live under https://api.avala.ai/api/v1:

Method and path	What it does
`GET /datasets/{owner}/{slug}/sequences/{sequence_uid}/outcome/`	Current label of one sequence. `404` when unlabeled.
`PUT /datasets/{owner}/{slug}/sequences/{sequence_uid}/outcome/`	Record a new current label.
`GET /datasets/{owner}/{slug}/sequences/{sequence_uid}/outcome/history/`	Every version of the label, newest first.
`GET /datasets/{owner}/{slug}/sequence-outcomes/`	Current labels in a dataset, paginated.

The dataset list accepts the filters outcome, evaluation_membership, source and autonomy_level. Each takes one value or a comma-separated list, and a sequence matches any value in the list. model_version matches one exact version. An unknown filter value returns 400. The sequence list, GET /datasets/{owner}/{slug}/sequences/, accepts outcome and evaluation_membership in the same form. It returns the sequences whose current label matches.

Write a label

curl -X PUT \
  "https://api.avala.ai/api/v1/datasets/your-org/kitchen-rollouts/sequences/<sequence-uid>/outcome/" \
  -H "X-Avala-Api-Key: $AVALA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "outcome": "failure",
    "progress": 0.4,
    "subtasks": [
      { "label": "Place the plate in the lower rack", "start_ts": 0.0, "end_ts": 21.5, "outcome": "failure" }
    ],
    "mistake_type": "slip_during_place",
    "failure_stage": "place",
    "autonomy_level": "autonomous",
    "model_version": "your-policy@2026-09-20",
    "evaluation_membership": "held_out_eval",
    "source": "imported",
    "source_metadata": { "importer": "your-eval-harness", "run_id": "r42", "task": "load_dishwasher" }
  }'

from avala import Client

client = Client()  # reads AVALA_API_KEY

label = client.sequence_outcomes.set(
    "your-org",
    "kitchen-rollouts",
    "<sequence-uid>",
    outcome="failure",
    progress=0.4,
    subtasks=[
        {"label": "Place the plate in the lower rack", "start_ts": 0.0, "end_ts": 21.5, "outcome": "failure"},
    ],
    mistake_type="slip_during_place",
    failure_stage="place",
    autonomy_level="autonomous",
    model_version="your-policy@2026-09-20",
    evaluation_membership="held_out_eval",
    source="imported",
    source_metadata={"importer": "your-eval-harness", "run_id": "r42", "task": "load_dishwasher"},
)
print(label.version, label.outcome)

import Avala from "@avala-ai/sdk";

const avala = new Avala(); // reads AVALA_API_KEY

const label = await avala.sequenceOutcomes.set("your-org", "kitchen-rollouts", "<sequence-uid>", {
  outcome: "failure",
  progress: 0.4,
  subtasks: [{ label: "Place the plate in the lower rack", startTs: 0.0, endTs: 21.5, outcome: "failure" }],
  mistakeType: "slip_during_place",
  failureStage: "place",
  autonomyLevel: "autonomous",
  modelVersion: "your-policy@2026-09-20",
  evaluationMembership: "held_out_eval",
  source: "imported",
  sourceMetadata: { importer: "your-eval-harness", run_id: "r42", task: "load_dishwasher" },
});
console.log(label.version, label.outcome);

avala sequence-outcomes set your-org kitchen-rollouts <sequence-uid> \
  --outcome failure \
  --progress 0.4 \
  --subtasks-file subtasks.json \
  --mistake-type slip_during_place \
  --failure-stage place \
  --autonomy-level autonomous \
  --model-version your-policy@2026-09-20 \
  --evaluation-membership held_out_eval \
  --source imported \
  --source-metadata '{"importer": "your-eval-harness", "run_id": "r42", "task": "load_dishwasher"}'

The TypeScript SDK takes camelCase option names and sends the API’s snake_case fields. Keys inside sourceMetadata are sent as written. In the CLI, --hand-actions-file takes a JSON file with {"left": [...], "right": [...]}, and --leakage-group KEY=ID can be repeated.

Read labels

# Current label of one sequence
curl "https://api.avala.ai/api/v1/datasets/your-org/kitchen-rollouts/sequences/<sequence-uid>/outcome/" \
  -H "X-Avala-Api-Key: $AVALA_API_KEY"

# Every version, newest first
curl "https://api.avala.ai/api/v1/datasets/your-org/kitchen-rollouts/sequences/<sequence-uid>/outcome/history/" \
  -H "X-Avala-Api-Key: $AVALA_API_KEY"

# Failed or aborted held-out sequences
curl "https://api.avala.ai/api/v1/datasets/your-org/kitchen-rollouts/sequence-outcomes/?outcome=failure,aborted&evaluation_membership=held_out_eval" \
  -H "X-Avala-Api-Key: $AVALA_API_KEY"

# The sequences themselves, filtered by their current label
curl "https://api.avala.ai/api/v1/datasets/your-org/kitchen-rollouts/sequences/?outcome=mistake_and_recovery,intervention" \
  -H "X-Avala-Api-Key: $AVALA_API_KEY"

from avala import Client

client = Client()

label = client.sequence_outcomes.get("your-org", "kitchen-rollouts", "<sequence-uid>")
versions = client.sequence_outcomes.history("your-org", "kitchen-rollouts", "<sequence-uid>")

page = client.sequence_outcomes.list(
    "your-org",
    "kitchen-rollouts",
    outcome=["failure", "aborted"],
    evaluation_membership="held_out_eval",
    model_version="your-policy@2026-09-20",
)
for item in page:
    print(item.sequence_uid, item.outcome, item.progress)

import Avala from "@avala-ai/sdk";

const avala = new Avala();

const label = await avala.sequenceOutcomes.get("your-org", "kitchen-rollouts", "<sequence-uid>");
const versions = await avala.sequenceOutcomes.history("your-org", "kitchen-rollouts", "<sequence-uid>");

const page = await avala.sequenceOutcomes.list("your-org", "kitchen-rollouts", {
  outcome: ["failure", "aborted"],
  evaluationMembership: "held_out_eval",
  modelVersion: "your-policy@2026-09-20",
});
for (const item of page.items) {
  console.log(item.sequenceUid, item.outcome, item.progress);
}

avala sequence-outcomes get your-org kitchen-rollouts <sequence-uid>
avala sequence-outcomes history your-org kitchen-rollouts <sequence-uid>
avala sequence-outcomes list your-org kitchen-rollouts \
  --outcome failure,aborted \
  --evaluation-membership held_out_eval

The dataset list returns up to 100 labels per page. Pass the cursor from the previous page to continue.

Read labels with the MCP server

The MCP server reads outcome labels. It does not write them.

get_sequence_outcome takes owner, slug and sequenceUid. It returns the current label.
list_sequence_outcomes takes owner and slug, plus the filters outcome, evaluationMembership, source, autonomyLevel and modelVersion, and limit and cursor.

Both tools return a short form by default. Pass detail: "full" to include subtasks, hand actions, the mistake and recovery tags, leakage groups, source metadata and the labeler. See Use Avala with AI to connect.

Export outcome labels with LeRobot

The LeRobot exporter can write each sequence’s current outcome label next to the dataset. One Avala sequence becomes one LeRobot episode.

avala lerobot export your-org/kitchen-rollouts \
  --repo-id your-hf-user/kitchen-rollouts \
  --output ./lerobot-out \
  --include-outcomes

from avala import Client
from avala.lerobot import export_dataset

export_dataset(
    Client(),
    "your-org",
    "kitchen-rollouts",
    repo_id="your-hf-user/kitchen-rollouts",
    output_dir="./lerobot-out",
    include_outcomes=True,
)

The labels are written to meta/avala_sequence_outcomes.jsonl, one JSON object per line and one line per saved LeRobot episode. LeRobot’s own episode table has no place for this metadata, so it is a separate file. Each line has these keys:

Key	Value
`episode_index`	The LeRobot episode index.
`avala_sequence_uid`	The Avala sequence the episode came from.
`outcome`	The outcome value, or `null` when the sequence has no label.
`outcome_version`	The label version that was exported.
`progress`, `quality`, `speed`	As in the label.
`subtasks`, `hand_actions`	As in the label. Timestamps are seconds from the start of the episode.
`mistake_type`, `recovery_type`, `failure_stage`, `autonomy_level`, `model_version`, `evaluation_membership`	As in the label, with `null` in place of an empty value.
`leakage_groups`	As in the label.
`outcome_source`, `outcome_source_metadata`, `outcome_confidence`	The label’s `source`, `source_metadata` and `confidence`.

A sequence without a label gets a line with only episode_index, avala_sequence_uid and "outcome": null.

The failure-and-recovery lane

Recoverable failures are delivered as their own lane: sequences whose outcome is mistake_and_recovery or intervention. The subtasks and hand actions of each sequence mark where the failure and the recovery happen. The lane ships as a separate slice in the release. You can train on it, weight it or leave it out. You can also build the same set yourself with ?outcome=mistake_and_recovery,intervention. Unsafe sequences are not part of the lane.

Getting Started

Fleet Learning

Platform

Use Cases

The outcome label

Example

Fields

Outcome values

Autonomy level

Subtasks

Hand actions

Sequence duration

Evaluation membership

Leakage groups

Source metadata

How writes work

Technical validity

Read and write outcome labels

Write a label

Read labels

Read labels with the MCP server

Export outcome labels with LeRobot

The failure-and-recovery lane

​The outcome label

​Example

​Fields

​Outcome values

​Autonomy level

​Subtasks

​Hand actions

​Sequence duration

​Evaluation membership

​Leakage groups

​Source metadata

​How writes work

​Technical validity

​Read and write outcome labels

​Write a label

​Read labels

​Read labels with the MCP server

​Export outcome labels with LeRobot

​The failure-and-recovery lane

The outcome label

Example

Fields

Outcome values

Autonomy level

Subtasks

Hand actions

Sequence duration

Evaluation membership

Leakage groups

Source metadata

How writes work

Technical validity

Read and write outcome labels

Write a label

Read labels

Read labels with the MCP server

Export outcome labels with LeRobot

The failure-and-recovery lane