> ## Documentation Index
> Fetch the complete documentation index at: https://avala.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Alerts & Notifications

> Alert routing to Slack, email, and webhooks

Get notified when fleet conditions change. Configure alert channels, define routing rules, and manage the alert lifecycle -- from detection through acknowledgment to resolution -- so issues get surfaced to the right people at the right time.

<Info>
  Fleet Management is in preview. APIs and features described on this page may change.
</Info>

## Alert Channels

An alert channel is a configured destination for notifications. Create channels for each communication path your team uses, then reference them in [recording rules](/docs/visualization/fleet/recording-rules) and alert policies.

| Channel     | Configuration                 | Use Case                                    |
| ----------- | ----------------------------- | ------------------------------------------- |
| **Slack**   | Webhook URL + channel name    | Team-wide notifications, on-call channels   |
| **Email**   | Recipient email addresses     | Individual alerts, management summaries     |
| **Webhook** | Endpoint URL + signing secret | Custom integrations, PagerDuty, Opsgenie    |
| **In-App**  | Automatic (no configuration)  | Dashboard notifications for all org members |

<Tip>
  In-app notifications are always enabled. Every alert that fires appears in the notification bell in Mission Control, regardless of other channel configurations.
</Tip>

### Create a Slack Channel

<CodeGroup>
  ```python Python theme={null}
  from avala import Client

  client = Client()

  channel = client.fleet.alerts.channels.create(
      type="slack",
      name="Robotics Alerts",
      config={
          "webhook_url": "https://hooks.slack.com/services/T00/B00/xxxx",
          "channel": "#robotics-alerts",
          "username": "Avala Fleet Bot",
          "icon_emoji": ":robot_face:"
      }
  )

  print(f"Channel created: {channel.name} ({channel.uid})")
  ```

  ```typescript TypeScript theme={null}
  import Avala from "@avala-ai/sdk";

  const avala = new Avala();

  const channel = await avala.fleet.alerts.channels.create({
    type: "slack",
    name: "Robotics Alerts",
    config: {
      webhook_url: "https://hooks.slack.com/services/T00/B00/xxxx",
      channel: "#robotics-alerts",
      username: "Avala Fleet Bot",
      icon_emoji: ":robot_face:",
    },
  });

  console.log(`Channel created: ${channel.name} (${channel.uid})`);
  ```

  ```bash CLI theme={null}
  avala fleet alerts channels create \
    --type slack \
    --name "Robotics Alerts" \
    --config '{"webhook_url": "https://hooks.slack.com/services/T00/B00/xxxx", "channel": "#robotics-alerts"}'
  ```
</CodeGroup>

### Create an Email Channel

<CodeGroup>
  ```python Python theme={null}
  channel = client.fleet.alerts.channels.create(
      type="email",
      name="Engineering Leads",
      config={
          "recipients": ["lead@example.com", "oncall@example.com"],
          "subject_prefix": "[Fleet Alert]"
      }
  )
  ```

  ```typescript TypeScript theme={null}
  const channel = await avala.fleet.alerts.channels.create({
    type: "email",
    name: "Engineering Leads",
    config: {
      recipients: ["lead@example.com", "oncall@example.com"],
      subject_prefix: "[Fleet Alert]",
    },
  });
  ```

  ```bash CLI theme={null}
  avala fleet alerts channels create \
    --type email \
    --name "Engineering Leads" \
    --config '{"recipients": ["lead@example.com", "oncall@example.com"]}'
  ```
</CodeGroup>

### Create a Webhook Channel

Webhook channels send an HTTP POST to your endpoint with the alert payload. Use a signing secret to verify that requests originate from Avala.

<CodeGroup>
  ```python Python theme={null}
  channel = client.fleet.alerts.channels.create(
      type="webhook",
      name="PagerDuty Integration",
      config={
          "url": "https://events.pagerduty.com/v2/enqueue",
          "signing_secret": "whsec_your_signing_secret",
          "headers": {
              "Content-Type": "application/json"
          }
      }
  )
  ```

  ```typescript TypeScript theme={null}
  const channel = await avala.fleet.alerts.channels.create({
    type: "webhook",
    name: "PagerDuty Integration",
    config: {
      url: "https://events.pagerduty.com/v2/enqueue",
      signing_secret: "whsec_your_signing_secret",
      headers: {
        "Content-Type": "application/json",
      },
    },
  });
  ```
</CodeGroup>

### Webhook Payload

Webhook channels deliver a JSON payload with the following structure:

```json theme={null}
{
  "uid": "alt_abc123",
  "rule_id": "rul_def456",
  "rule_name": "High Latency Alert",
  "severity": "warning",
  "status": "open",
  "device": {
    "uid": "dev_ghi789",
    "name": "robot-arm-01"
  },
  "recording": {
    "uid": "rec_jkl012",
    "url": "https://avala.ai/recordings/rec_jkl012"
  },
  "message": "High latency detected: 142ms (threshold: 100ms)",
  "metadata": {
    "latency_ms": 142,
    "threshold_ms": 100,
    "topic": "/diagnostics/latency"
  },
  "triggered_at": "2026-01-15T10:30:00Z"
}
```

<Tip>
  To verify webhook authenticity, compute an HMAC-SHA256 of the request body using your signing secret and compare it to the `X-Avala-Signature` header. See the [webhooks integration guide](/docs/integrations/webhooks) for implementation examples.
</Tip>

### Test a Channel

Send a test notification to verify your channel configuration before wiring it up to rules.

<CodeGroup>
  ```python Python theme={null}
  result = client.fleet.alerts.channels.test(channel_id="ch_abc123")

  if result.success:
      print("Test notification sent successfully")
  else:
      print(f"Test failed: {result.error}")
  ```

  ```typescript TypeScript theme={null}
  const result = await avala.fleet.alerts.channels.test({ channelId: "ch_abc123" });

  if (result.success) {
    console.log("Test notification sent successfully");
  } else {
    console.log(`Test failed: ${result.error}`);
  }
  ```

  ```bash CLI theme={null}
  avala fleet alerts channels test ch_abc123
  ```
</CodeGroup>

### List and Manage Channels

<CodeGroup>
  ```python Python theme={null}
  # List all channels
  channels = client.fleet.alerts.channels.list()
  for ch in channels:
      print(f"{ch.name} ({ch.type}) -- {ch.uid}")

  # Update a channel
  client.fleet.alerts.channels.update(
      channel_id="ch_abc123",
      name="Robotics Alerts (Updated)",
      config={"channel": "#robotics-alerts-v2"}
  )

  # Delete a channel
  client.fleet.alerts.channels.delete(channel_id="ch_abc123")
  ```

  ```typescript TypeScript theme={null}
  const channels = await avala.fleet.alerts.channels.list();
  for (const ch of channels.items) {
    console.log(`${ch.name} (${ch.type}) -- ${ch.uid}`);
  }

  await avala.fleet.alerts.channels.update({
    channelId: "ch_abc123",
    name: "Robotics Alerts (Updated)",
    config: { channel: "#robotics-alerts-v2" },
  });

  await avala.fleet.alerts.channels.delete({ channelId: "ch_abc123" });
  ```

  ```bash CLI theme={null}
  avala fleet alerts channels list
  avala fleet alerts channels update ch_abc123 --name "Robotics Alerts (Updated)"
  avala fleet alerts channels delete ch_abc123
  ```
</CodeGroup>

<Warning>
  Deleting a channel removes it from all rules and alert policies that reference it. Those rules will continue to evaluate, but the `notify` action targeting the deleted channel will be skipped. Update your rules to reference a different channel before deleting.
</Warning>

## Alert Rules

Alerts are linked to [recording rules](/docs/visualization/fleet/recording-rules) through the `notify` action. When a recording rule's condition matches, its notify action fires an alert to the specified channel.

### Severity Levels

Configure severity on the recording rule's notify action to control how alerts are displayed and routed.

| Severity   | Description                       | Default Behavior                                                      |
| ---------- | --------------------------------- | --------------------------------------------------------------------- |
| `critical` | Requires immediate attention      | All channels notified, repeated every 5 minutes until acknowledged    |
| `warning`  | Needs investigation               | Configured channels notified once                                     |
| `info`     | Informational, no action required | In-app notification only (unless explicitly routed to other channels) |

<CodeGroup>
  ```python Python theme={null}
  # Recording rule with severity-based alert routing
  rule = client.fleet.rules.create(
      name="Motor Overheat Critical",
      condition={
          "type": "threshold",
          "topic": "/sensors/temperature",
          "field": "temp_c",
          "operator": "gt",
          "value": 90
      },
      actions=[
          {"type": "create_event", "event_type": "error", "label": "Motor overheat"},
          {
              "type": "notify",
              "channel_id": "ch_slack_oncall",
              "severity": "critical",
              "message": "Motor temperature exceeded 90C -- immediate shutdown may be required"
          }
      ]
  )
  ```

  ```typescript TypeScript theme={null}
  const rule = await avala.fleet.rules.create({
    name: "Motor Overheat Critical",
    condition: {
      type: "threshold",
      topic: "/sensors/temperature",
      field: "temp_c",
      operator: "gt",
      value: 90,
    },
    actions: [
      { type: "create_event", eventType: "error", label: "Motor overheat" },
      {
        type: "notify",
        channelId: "ch_slack_oncall",
        severity: "critical",
        message: "Motor temperature exceeded 90C -- immediate shutdown may be required",
      },
    ],
  });
  ```
</CodeGroup>

### Escalation Policies

For critical alerts, define an escalation policy that notifies additional channels if the alert is not acknowledged within a specified time.

<CodeGroup>
  ```python Python theme={null}
  policy = client.fleet.alerts.policies.create(
      name="Critical Escalation",
      steps=[
          {
              "channel_id": "ch_slack_oncall",
              "wait": "5m"
          },
          {
              "channel_id": "ch_email_leads",
              "wait": "15m"
          },
          {
              "channel_id": "ch_webhook_pagerduty",
              "wait": None  # Final step, no further escalation
          }
      ]
  )

  # Attach policy to a rule's notify action
  rule = client.fleet.rules.create(
      name="Critical System Failure",
      condition={
          "type": "absence",
          "topic": "/system/heartbeat",
          "timeout": "60s"
      },
      actions=[
          {
              "type": "notify",
              "policy_id": policy.uid,
              "severity": "critical"
          }
      ]
  )
  ```

  ```typescript TypeScript theme={null}
  const policy = await avala.fleet.alerts.policies.create({
    name: "Critical Escalation",
    steps: [
      { channelId: "ch_slack_oncall", wait: "5m" },
      { channelId: "ch_email_leads", wait: "15m" },
      { channelId: "ch_webhook_pagerduty", wait: null },
    ],
  });

  const rule = await avala.fleet.rules.create({
    name: "Critical System Failure",
    condition: {
      type: "absence",
      topic: "/system/heartbeat",
      timeout: "60s",
    },
    actions: [
      {
        type: "notify",
        policyId: policy.uid,
        severity: "critical",
      },
    ],
  });
  ```
</CodeGroup>

## Alert Lifecycle

Every alert moves through a defined set of states. You can transition alerts manually via the SDK, CLI, or dashboard.

```
open  -->  acknowledged  -->  resolved
  |                              ^
  +------------------------------+
          (auto-resolve)
```

| State          | Description                                                               |
| -------------- | ------------------------------------------------------------------------- |
| `open`         | Condition matched, notifications sent. Alert is active and may escalate.  |
| `acknowledged` | A team member has seen the alert and is investigating. Escalation pauses. |
| `resolved`     | The issue has been addressed. Alert is closed.                            |

### Acknowledge an Alert

Acknowledging an alert stops escalation and signals to the team that someone is investigating.

<CodeGroup>
  ```python Python theme={null}
  client.fleet.alerts.acknowledge(
      alert_id="alt_abc123",
      note="Investigating -- checking motor temperature logs"
  )
  ```

  ```typescript TypeScript theme={null}
  await avala.fleet.alerts.acknowledge({
    alertId: "alt_abc123",
    note: "Investigating -- checking motor temperature logs",
  });
  ```

  ```bash CLI theme={null}
  avala fleet alerts acknowledge alt_abc123 --note "Investigating -- checking motor temperature logs"
  ```
</CodeGroup>

### Resolve an Alert

Mark an alert as resolved when the underlying issue has been fixed.

<CodeGroup>
  ```python Python theme={null}
  client.fleet.alerts.resolve(
      alert_id="alt_abc123",
      note="Fixed: firmware update applied to motor controller, temperature nominal"
  )
  ```

  ```typescript TypeScript theme={null}
  await avala.fleet.alerts.resolve({
    alertId: "alt_abc123",
    note: "Fixed: firmware update applied to motor controller, temperature nominal",
  });
  ```

  ```bash CLI theme={null}
  avala fleet alerts resolve alt_abc123 --note "Fixed: firmware update applied to motor controller"
  ```
</CodeGroup>

### Auto-Resolve

Alerts can auto-resolve when their triggering condition clears. Enable auto-resolve on the recording rule's notify action:

<CodeGroup>
  ```python Python theme={null}
  rule = client.fleet.rules.create(
      name="High CPU Usage",
      condition={
          "type": "threshold",
          "topic": "/system/diagnostics",
          "field": "cpu_percent",
          "operator": "gt",
          "value": 90,
          "window": "5m"
      },
      actions=[
          {
              "type": "notify",
              "channel_id": "ch_abc123",
              "severity": "warning",
              "auto_resolve": True,
              "resolve_after": "10m"
          }
      ]
  )
  ```

  ```typescript TypeScript theme={null}
  const rule = await avala.fleet.rules.create({
    name: "High CPU Usage",
    condition: {
      type: "threshold",
      topic: "/system/diagnostics",
      field: "cpu_percent",
      operator: "gt",
      value: 90,
      window: "5m",
    },
    actions: [
      {
        type: "notify",
        channelId: "ch_abc123",
        severity: "warning",
        autoResolve: true,
        resolveAfter: "10m",
      },
    ],
  });
  ```
</CodeGroup>

<Tip>
  When `auto_resolve` is enabled, the alert transitions to `resolved` after the `resolve_after` duration if the condition no longer matches. If the condition re-triggers during the resolve window, the alert stays in its current state.
</Tip>

## Alert History

View past alerts with their full lifecycle -- when they fired, who acknowledged them, and how they were resolved.

<CodeGroup>
  ```python Python theme={null}
  # List recent alerts
  alerts = client.fleet.alerts.list(
      status="resolved",
      since="2026-01-01T00:00:00Z",
      until="2026-02-01T00:00:00Z"
  )

  for alert in alerts:
      print(f"[{alert.severity}] {alert.rule_name}")
      print(f"  Triggered: {alert.triggered_at}")
      print(f"  Acknowledged: {alert.acknowledged_at} by {alert.acknowledged_by}")
      print(f"  Resolved: {alert.resolved_at}")
      print(f"  Duration: {alert.duration}")
      print()

  # Filter by severity
  critical_alerts = client.fleet.alerts.list(severity="critical")

  # Filter by device
  device_alerts = client.fleet.alerts.list(device_id="dev_abc123")

  # Filter by channel
  slack_alerts = client.fleet.alerts.list(channel_id="ch_abc123")
  ```

  ```typescript TypeScript theme={null}
  const alerts = await avala.fleet.alerts.list({
    status: "resolved",
    since: "2026-01-01T00:00:00Z",
    until: "2026-02-01T00:00:00Z",
  });

  for (const alert of alerts.items) {
    console.log(`[${alert.severity}] ${alert.ruleName}`);
    console.log(`  Triggered: ${alert.triggeredAt}`);
    console.log(`  Acknowledged: ${alert.acknowledgedAt} by ${alert.acknowledgedBy}`);
    console.log(`  Resolved: ${alert.resolvedAt}`);
    console.log(`  Duration: ${alert.duration}`);
  }

  const criticalAlerts = await avala.fleet.alerts.list({ severity: "critical" });
  const deviceAlerts = await avala.fleet.alerts.list({ deviceId: "dev_abc123" });
  ```

  ```bash CLI theme={null}
  avala fleet alerts list --status resolved --since 2026-01-01 --until 2026-02-01
  avala fleet alerts list --severity critical
  avala fleet alerts list --device dev_abc123
  ```
</CodeGroup>

### Alert Metrics

Aggregate alert statistics for reporting and SLA tracking.

<CodeGroup>
  ```python Python theme={null}
  metrics = client.fleet.alerts.metrics(
      since="2026-01-01T00:00:00Z",
      until="2026-02-01T00:00:00Z"
  )

  print(f"Total alerts: {metrics.total}")
  print(f"  Critical: {metrics.by_severity['critical']}")
  print(f"  Warning: {metrics.by_severity['warning']}")
  print(f"  Info: {metrics.by_severity['info']}")
  print(f"Mean time to acknowledge: {metrics.mean_time_to_acknowledge}")
  print(f"Mean time to resolve: {metrics.mean_time_to_resolve}")
  ```

  ```typescript TypeScript theme={null}
  const metrics = await avala.fleet.alerts.metrics({
    since: "2026-01-01T00:00:00Z",
    until: "2026-02-01T00:00:00Z",
  });

  console.log(`Total alerts: ${metrics.total}`);
  console.log(`  Critical: ${metrics.bySeverity.critical}`);
  console.log(`  Warning: ${metrics.bySeverity.warning}`);
  console.log(`  Info: ${metrics.bySeverity.info}`);
  console.log(`Mean time to acknowledge: ${metrics.meanTimeToAcknowledge}`);
  console.log(`Mean time to resolve: ${metrics.meanTimeToResolve}`);
  ```
</CodeGroup>

## Muting and Snoozing

Temporarily silence alerts during planned maintenance, known outages, or testing periods. Muted alerts still evaluate and log, but notifications are suppressed.

### Mute a Rule

Suppress all notifications from a specific rule for a set duration.

<CodeGroup>
  ```python Python theme={null}
  client.fleet.alerts.mute(
      rule_id="rul_abc123",
      duration="2h",
      reason="Scheduled maintenance on warehouse B robots"
  )
  ```

  ```typescript TypeScript theme={null}
  await avala.fleet.alerts.mute({
    ruleId: "rul_abc123",
    duration: "2h",
    reason: "Scheduled maintenance on warehouse B robots",
  });
  ```

  ```bash CLI theme={null}
  avala fleet alerts mute rul_abc123 --duration 2h --reason "Scheduled maintenance"
  ```
</CodeGroup>

### Mute a Device

Suppress all alerts for a specific device. Useful when taking a device offline for servicing.

<CodeGroup>
  ```python Python theme={null}
  client.fleet.alerts.mute_device(
      device_id="dev_abc123",
      duration="4h",
      reason="Firmware upgrade in progress"
  )
  ```

  ```typescript TypeScript theme={null}
  await avala.fleet.alerts.muteDevice({
    deviceId: "dev_abc123",
    duration: "4h",
    reason: "Firmware upgrade in progress",
  });
  ```

  ```bash CLI theme={null}
  avala fleet alerts mute-device dev_abc123 --duration 4h --reason "Firmware upgrade"
  ```
</CodeGroup>

### Snooze an Alert

Snooze a specific open alert to temporarily suppress its notifications. The alert returns to `open` state when the snooze expires.

<CodeGroup>
  ```python Python theme={null}
  client.fleet.alerts.snooze(
      alert_id="alt_abc123",
      duration="30m",
      reason="Known issue, fix deploying in next release"
  )
  ```

  ```typescript TypeScript theme={null}
  await avala.fleet.alerts.snooze({
    alertId: "alt_abc123",
    duration: "30m",
    reason: "Known issue, fix deploying in next release",
  });
  ```

  ```bash CLI theme={null}
  avala fleet alerts snooze alt_abc123 --duration 30m --reason "Known issue, fix deploying"
  ```
</CodeGroup>

### View Active Mutes

<CodeGroup>
  ```python Python theme={null}
  mutes = client.fleet.alerts.mutes.list()
  for mute in mutes:
      print(f"{mute.target_type}: {mute.target_id}")
      print(f"  Reason: {mute.reason}")
      print(f"  Expires: {mute.expires_at}")

  # Unmute early
  client.fleet.alerts.unmute(mute_id="mute_abc123")
  ```

  ```typescript TypeScript theme={null}
  const mutes = await avala.fleet.alerts.mutes.list();
  for (const mute of mutes.items) {
    console.log(`${mute.targetType}: ${mute.targetId}`);
    console.log(`  Reason: ${mute.reason}`);
    console.log(`  Expires: ${mute.expiresAt}`);
  }

  await avala.fleet.alerts.unmute({ muteId: "mute_abc123" });
  ```

  ```bash CLI theme={null}
  avala fleet alerts mutes list
  avala fleet alerts unmute mute_abc123
  ```
</CodeGroup>

<Tip>
  Schedule mutes in advance for recurring maintenance windows. Use the `starts_at` parameter to create mutes that activate at a future time:

  ```python theme={null}
  client.fleet.alerts.mute(
      rule_id="rul_abc123",
      starts_at="2026-02-01T02:00:00Z",
      duration="2h",
      reason="Weekly maintenance window"
  )
  ```
</Tip>

## Next Steps

<CardGroup cols={2}>
  <Card title="" icon="gears" href="/docs/visualization/fleet/recording-rules">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Recording Rules</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Define the conditions that trigger alerts across your fleet.</p>
  </Card>

  <Card title="" icon="gauge-high" href="/docs/visualization/fleet/fleet-dashboard">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Fleet Dashboard</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Monitor devices, recordings, and fleet health from the central dashboard.</p>
  </Card>

  <Card title="" icon="flag" href="/docs/visualization/fleet/events-and-markers">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Events & Markers</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Annotate recordings with timestamped events for detailed analysis.</p>
  </Card>

  <Card title="" icon="webhook" href="/docs/integrations/webhooks">
    <p style={{fontWeight: 600, fontSize: '18px', marginBottom: '4px', marginTop: '8px', color: 'inherit'}}>Webhooks</p>
    <p style={{fontSize: '14px', marginTop: '0px', opacity: 0.6}}>Receive real-time notifications for dataset, export, and task events.</p>
  </Card>
</CardGroup>