Skip to main content

Why Physical AI needs to learn why, not just what

Research

Annotation pipelines keep the label and delete the judgment. Capturing that reasoning may be the most valuable training data in Physical AI.

By Avala Research

11 min read
Share

Consider what happens when a skilled annotator labels a complex driving scene.

They're looking at a cluster of LiDAR points near the road edge. The shape suggests a person, but the lower body is hidden behind a parked car. They check the camera feed: yes, a visible head and shoulders, with posture suggesting forward motion. They scrub adjacent frames to confirm temporal consistency. They weigh alternatives. A cyclist? No helmet visible, no handlebar signature in the point cloud. Someone standing still? The frame-to-frame motion says otherwise.

After this synthesis, perhaps a few seconds of work for an experienced annotator, they draw a bounding box and select "pedestrian" from a dropdown menu.

The dropdown selection is what persists. The reasoning chain that produced it is gone: the cues that mattered, the alternatives considered, the confidence level, the similar cases that informed the judgment. All of it evaporates at "submit."

We believe the judgment behind a label can be useful training and review data. Recording the cues considered, uncertainty, and applicable guideline makes a decision easier to inspect. Its value depends on the task and the quality of the record; it does not establish why a trained model later made a prediction.

Labels are the residue of reasoning

An annotator classifying a partially occluded object is not pattern-matching. They're synthesizing information across sensor modalities, reasoning about physics and geometry, applying learned heuristics about how the world works, and making probabilistic judgments under uncertainty. That is exactly the cognitive work we want AI systems to learn. Yet our pipelines store only its residue: bounding box coordinates and a class assignment. The perceptual strategies, the considered alternatives, the confidence calibration, the edge case reasoning: all of it is treated as ephemeral context rather than data.

The loss compounds. When a trained model later misclassifies a similar edge case, there's no way to diagnose whether the problem was insufficient training examples, ambiguous guidelines, or flawed reasoning in the original labels. When guidelines change, perhaps after an incident reveals a gap, previously labeled data can't be systematically re-evaluated, because no one recorded how annotators interpreted the old guidelines. When two annotators disagree on a difficult case, we capture the disagreement but not the reasoning divergence that caused it.

Annotation is only the sharpest instance of a general pattern. A support ticket shows "escalated to Tier 3" but not the synthesis of conversation history and customer context behind the escalation. An engineering change shows the code diff but not the architectural reasoning that shaped it. In each case the outcome survives and the intelligence that produced it dies.

Context graphs treat reasoning as data

The emerging fix goes by several names: decision traces, reasoning graphs, context graphs. We'll use "context graphs" here, but the idea matters more than the label: treat the reasoning behind decisions as data worth storing, not context to be discarded.

A context graph differs from a traditional knowledge graph in what it prioritizes. Knowledge graphs model entities and relationships: this person works at this company, this vehicle passed this location at this time. Context graphs add a decision layer on top: who decided what, based on what information, referencing what precedents, with what confidence and caveats.

For annotation, that means structured reasoning captured alongside each label. Rather than just "pedestrian at coordinates (x, y)," an illustrative trace might read: classified as pedestrian with reviewer-estimated confidence based on visible head and shoulders despite lower-body occlusion; considered cyclist but rejected it for absence of helmet or handlebar signature; similar to case #4521, where partial occlusion was resolved through temporal consistency; flagged as an edge case requiring expert review.

That adds a record of how annotators interpreted the scene: which cues they used, which alternatives they considered, and where they were uncertain. Foundation Capital’s Jaya Gupta and Ashu Garg describe context graphs as connected decision traces, explicitly distinguishing them from a model’s chain of thought. Our application to annotation is a design thesis: preserve reviewable human decisions alongside labels, then test whether that context improves review or learning.

Systems of record were built to forget

Reasoning has been lost for architectural reasons, not careless ones. Data warehouses like Snowflake and Databricks receive data via ETL after decisions are made; by the time data lands, the context is gone. CRM and ERP systems store current state: what an opportunity looks like now, not what it looked like when the decision was made. Ticketing systems capture outcomes, not the cross-system synthesis that happened in someone's head. Reasoning was never treated as data, so nothing was built to hold it. An annotator's judgment that a LiDAR cluster is a pedestrian rather than a cyclist was considered part of the process, not part of the output.

Four capabilities now make reasoning storable. Event sourcing preserves complete decision histories rather than current state. Graph databases model the relationships between decisions, precedents, and outcomes. Vector embeddings make reasoning traces semantically searchable: "find decisions similar to this one" becomes a queryable operation. Agents can help structure recorded actions, evidence, and explanations. Generated rationales still need checking against those records; plausible prose is not proof of the process that produced a decision.

That last point is the design principle companies implementing context graphs converge on: reasoning capture must emerge as exhaust from valuable work, not as a separate documentation burden. Annotators shouldn't write explanations. They should make decisions through interfaces that record the reasoning as structured data.

Decision records support review, not automatic explanations

Model development and assurance create related but distinct reasons to preserve annotation context.

For model development, an annotation such as pedestrian “preparing to cross” can carry assumptions about motion, visibility, and intent. Recording those assumptions helps a reviewer inspect the label and its alternatives. Whether that record improves a learned policy requires evaluation; annotation rationale is not automatically a causal explanation of the policy’s output.

For assurance, the EU AI Act’s Article 13 addresses transparency and information for deployers of high-risk systems. Applicability and obligations depend on the system and its legal context. Keeping annotation decisions can support technical review, but an annotation trace alone does not establish compliance, explain every model decision, or replace a system-specific safety assessment.

Agreement metrics measure the wrong thing

Agreement metrics ask whether annotators labeled the same things the same way. They are useful, but they do not show whether the annotators used the same evidence or interpreted the guideline consistently.

Agreement is the wrong proxy for the failures that matter. Two annotators might both label something "pedestrian" while holding completely different mental models of what distinguished it from a cyclist or a shadow. They agree on the output and diverge on the reasoning. When the trained model later encounters a case that falls between their mental models, it may fail in ways neither annotator's labels would predict.

Reasoning traces surface this hidden divergence. If annotators make their reasoning explicit, even through structured interface choices rather than free-form writing, systematic differences in interpretation become visible. Guidelines get refined to close ambiguities. Training targets specific reasoning gaps. The labels themselves become more reliable, because the reasoning behind them is queryable and auditable.

A related example comes from customer support, not robotics annotation. A 2024 LinkedIn research paper combines retrieval-augmented generation with a knowledge graph built from historical issues. The authors report a 28.6% reduction in median per-issue resolution time over approximately six months of deployment. This supports investigating structured history for retrieval; it does not establish that capturing annotator reasoning produces the same benefit or improves a robot policy.

Edge cases are where deleted reasoning costs the most

The long-tail distribution of driving scenarios makes rare events disproportionately important: a model that handles 99% of situations correctly but fails catastrophically on the remaining 1% is not safe to deploy. Edge cases, the ambiguous and never-before-seen situations, are where human judgment works hardest and where discarding it costs most.

Watch how edge cases are handled today. An annotator encounters a person on an electric unicycle, partially occluded, moving erratically. The guidelines don't address it. They make a judgment call, perhaps "cyclist" as the closest category, and move on. Another annotator classifies a similar case as "pedestrian," since there's no traditional bicycle. A third flags it for review. The disagreement is noted; the reasoning behind each classification is not. When guidelines are eventually updated for personal electric vehicles, there's no systematic way to identify which previous annotations the gap affected, how they were handled, or whether the annotators' reasoning matches the new policy. The institutional learning that should flow from edge case resolution leaks away.

With reasoning captured, each edge case compounds instead. Similar cases cluster by reasoning pattern, not just label outcome. Guidelines evolve from how skilled annotators actually reason through ambiguity. New annotators see relevant precedent automatically: here's how senior annotators handled this situation, and why. Better edge case handling produces more consistent labels, which improve model performance on difficult cases, which surface new edge cases, which enrich the reasoning corpus. The annotation operation becomes smarter over time rather than merely larger.

Sensor fusion is the same gap in concentrated form. When a camera detects an obstacle that LiDAR misses, or radar returns are ambiguous, current annotations capture the fused output but lose the per-sensor assessments and the logic that resolved the conflict. When the system errs, debugging means reconstructing what each sensor saw and guessing why fusion went wrong. If the reasoning were captured (LiDAR downweighted because rain degraded its returns, camera trusted because the object sat within its high-confidence range), diagnosis becomes tractable. With more sensor streams, preserve the source, timing, calibration assumptions, and uncertainty behind a review decision so disagreements can be investigated.

Capture reasoning as exhaust, not homework

The obvious objection: won't this slow annotation to a crawl? Only if it's designed as documentation. The principle that makes it practical is that reasoning capture should scale with decision difficulty, not apply uniformly to every label.

For routine cases where confidence is high and guidelines are clear, capture almost nothing: a confidence indicator, a scenario classification. The system learns to identify routine cases from annotator behavior and stays out of the way for the bulk of the work. For edge cases (ambiguous scenarios, guideline gaps, annotator disagreement), the annotator is already thinking carefully; capturing that thinking adds modest effort and generates disproportionate value.

Structured interfaces replace free-form writing. Rather than "explain your reasoning," the system prompts: which visual cues were most relevant? What alternatives did you consider? How confident are you? Does this relate to a previous case? The answers are structured, queryable, and comparable across annotators. And cross-references to similar past decisions turn the graph into searchable precedent, so consistent edge case handling no longer depends on every annotator independently rediscovering policy. The context graph becomes institutional memory that outlives individual contributors.

Volume or judgment

Companies building physical AI face a fork in how they treat training data. The conventional path optimizes volume and throughput: maximum labeled examples at minimum cost per label. That path scaled the labeling industry, and it systematically discards the one thing that makes human judgment worth paying for. The other path treats labels with reasoning traces as a different product from labels alone, because in safety-critical applications, where edge cases determine deployment readiness, the reasoning behind annotations decides whether the labels can be trusted, defended, and audited at all.

The practical question is how to preserve useful judgment without turning every annotation into a writing task. Context graphs are one possible design: connect decisions, source evidence, guidelines, and review outcomes so they can be inspected together. We expect that record to be most useful in ambiguous cases. Its effect on review effort, label quality, and downstream performance should be measured rather than assumed.


At Avala, we're building data infrastructure that captures not just labels but the reasoning behind them. If you're thinking about how annotation quality shapes deployment safety, we'd like to hear what you're seeing.

  • Data Infrastructure
  • Physical AI
  • Autonomous Vehicles
  • Decision Intelligence
  • AI Safety