Data Lifecycle

​

Lifecycle Stages

​

1. Ingest

• Direct upload: Web UI or API
• Pipeline: Automated cloud storage sync
• Streaming: Real-time from vehicles

​

2. Process

• Validation: Schema and format checks
• Indexing: Full-text search indexing
• Thumbnails: Preview generation for visual data
• Metadata extraction: Automatic tag inference

​

3. Active Storage

• Immediately accessible via API and UI
• Stored on high-performance SSDs
• Replicated across availability zones
• Included in search results

​

4. Archive

• Lower storage costs
• Retrieval requires restore request
• Maintain full metadata and searchability
• Configurable retention policies

​

5. Delete

• Removes all data and metadata
• Cannot be undone
• Required for compliance (GDPR, etc.)

​

Retention Policies

retention:
  active_days: 90
  archive_days: 365
  delete_after: 730

exceptions:
  - tag: "golden-set"
    action: keep_forever
  - tag: "temporary"
    active_days: 7

​

Storage Tiers