Capability experiments

A capability experiment is one turn of the fleet learning loop, written down. It starts from a failure your model shows on a specific model version. It states a hypothesis about which data will fix it. It ends with a measured outcome and a decision about the next allocation. Avala opens a capability experiment for each failure condition in a gap report. Avala keeps the record current through collection, release and re-evaluation. You receive it with each capability report.

Fields

Field	What it holds
`model_version`	The model version that showed the failure. You supply this identifier.
`failure_condition`	The failure and the conditions where it appears, taken from the failure taxonomy. `mistake_type` and `failure_stage` use the same tags as the outcome labels of the failing sequences.
`hypothesis`	A testable statement: which data should change which metric, and in which direction.
`coverage_cells`	The coverage cells the experiment targets.
`capture_campaign`	The capture campaign that collects the data.
`dataset_release`	The versioned dataset release that carries the new sequences.
`evaluation_cohort`	The held-out evaluation scenes used to measure the outcome. Their sequences carry `evaluation_membership: "held_out_eval"`.
`outcome`	The measured result on the evaluation cohort for each model version, with confidence intervals.
`next_allocation`	What the next cycle collects, and why.

Example record

{
  "experiment_id": "cx_dishwasher_rack_regrasp",
  "status": "validating",
  "model_version": "your-policy@2026-09-20",
  "failure_condition": {
    "mistake_type": "slip_during_place",
    "failure_stage": "place",
    "taxonomy_version": "3",
    "conditions": ["lower_rack_pulled_out", "wet_ceramic", "rack_obstacle_present"]
  },
  "hypothesis": "Adding expert sequences with slip and recovery on lower dishwasher racks raises success on the rack-loading cohort and lowers the intervention rate.",
  "coverage_cells": [
    {
      "environment": "kitchen",
      "mechanism": "dishwasher_pull_rack",
      "material": "ceramic_wet",
      "interaction_primitive": "place",
      "failure_mode": "regrasp_after_slip",
      "embodiment": "human_egocentric"
    }
  ],
  "capture_campaign": { "project_uid": "<project-uid>" },
  "dataset_release": { "dataset": "your-org/kitchen-manipulation", "release": "r4" },
  "evaluation_cohort": { "cohort_id": "eval_kitchen_racks", "version": "2" },
  "outcome": null,
  "next_allocation": null
}

outcome and next_allocation stay empty until Avala re-evaluates the next model version.

How the record links to existing objects

A capability experiment holds references. Each field points at an object you can already open in Mission Control, the API, the SDKs and the MCP server.

Experiment field	Existing object	Where to read it
`failure_condition`	Fleet recordings and fleet events that show the failure, and the outcome labels of the failing sequences	`client.fleet.recordings.get(...)`, `client.fleet.events.list(recording_id=...)`, `client.sequence_outcomes.list(owner, slug, outcome=...)`, MCP `fleet_get_recording`, `fleet_list_events`, `list_sequence_outcomes`
`capture_campaign`	A project that runs the capture campaign	`client.projects.get(uid)`, MCP `list_capture_campaigns`, `get_campaign_acceptance_summary`
`coverage_cells`	Acceptance coverage on the campaign project	MCP `get_campaign_acceptance_coverage`
`dataset_release`	A dataset and its slices	`client.datasets.get_by_slug(owner, slug)`, `client.slices.get(owner, slug)`, MCP `get_dataset`, `get_slice`
`evaluation_cohort`	A slice of held-out evaluation scenes, kept out of every training release	`client.slices.get(owner, slug)`, `client.sequence_outcomes.list(owner, slug, evaluation_membership="held_out_eval")`, MCP `get_slice`, `list_sequence_outcomes`

The failure-and-recovery lane and each evaluation cohort are delivered as slices. You can list their items with client.slices.list_items(owner, slug).

Lifecycle

Status	Meaning
`proposed`	Avala drafted the hypothesis and the target coverage cells from the gap report.
`approved`	You approved the brief. The capture campaign is open.
`collecting`	Sequences are being captured, reviewed and accepted.
`released`	The dataset release is published. You are training on it.
`validating`	Avala is scoring the new model version on the evaluation cohort.
`closed`	The outcome is recorded and the next allocation is set.

A closed experiment stays in the record. The next experiment on the same failure condition links back to it, so you can follow a capability across model versions.

Getting Started

Fleet Learning

Platform

Use Cases

Fields

Example record

How the record links to existing objects

Lifecycle

​Fields

​Example record

​How the record links to existing objects

​Lifecycle

Fields

Example record

How the record links to existing objects

Lifecycle