Best Practices

​

Data Management

​

Organize Datasets by Purpose

​

Optimize File Sizes

​

Use Cloud Storage for Large Datasets

• No data transfer: Avala reads directly from your S3 or GCS bucket
• Your encryption: Data stays encrypted with your KMS keys
• Your retention: Control lifecycle policies independently
• Faster onboarding: No upload step — just point Avala to your bucket

​

API Usage

​

Paginate Large Result Sets

from avala import Client

client = Client()

# Iterate through all datasets automatically
page = client.datasets.list()
for dataset in page:
    print(dataset.name)

# The SDK handles pagination internally via CursorPage

import Avala from "@avala-ai/sdk";

const avala = new Avala();

// Iterate through all datasets
const page = await avala.datasets.list();
for (const dataset of page.items) {
  console.log(dataset.name);
}

// Fetch next page if available
if (page.hasMore) {
  const next = await avala.datasets.list({ cursor: page.nextCursor });
}

# First page
curl -s "$BASE_URL/datasets/" \
  -H "X-Avala-Api-Key: $AVALA_API_KEY" | jq '.next'

# Follow the 'next' URL for subsequent pages
curl -s "$BASE_URL/datasets/?cursor=abc123" \
  -H "X-Avala-Api-Key: $AVALA_API_KEY"

​

Respect Rate Limits

import time
from avala import Client
from avala.errors import RateLimitError

client = Client()

def fetch_with_retry(func, max_retries=3):
    for attempt in range(max_retries):
        try:
            return func()
        except RateLimitError as e:
            wait = e.retry_after or (2 ** attempt)
            time.sleep(wait)
    raise Exception("Max retries exceeded")

# Usage
dataset = fetch_with_retry(lambda: client.datasets.get("dataset-uid"))

import Avala from "@avala-ai/sdk";
import { RateLimitError } from "@avala-ai/sdk";

const avala = new Avala();

async function fetchWithRetry<T>(fn: () => Promise<T>, maxRetries = 3): Promise<T> {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await fn();
    } catch (e) {
      if (e instanceof RateLimitError) {
        const wait = e.retryAfter ?? 2 ** attempt;
        await new Promise((r) => setTimeout(r, wait * 1000));
      } else throw e;
    }
  }
  throw new Error("Max retries exceeded");
}

​

Use Exports for Bulk Data Retrieval

from avala import Client

client = Client()

# Create an export (much faster than fetching items individually)
export = client.exports.create(dataset="your-dataset-uid")

print(f"Export status: {export.status}")
print(f"Download URL: {export.download_url}")

import Avala from "@avala-ai/sdk";

const avala = new Avala();

const exportResult = await avala.exports.create({
  dataset: "your-dataset-uid",
});

console.log(`Download: ${exportResult.downloadUrl}`);

​

Annotation Workflows

​

Design Projects with Clear Instructions

• Label taxonomy: Define all labels before annotating. Adding labels mid-project creates inconsistency.
• Examples: Provide 5-10 annotated examples for each label class, covering edge cases.
• Edge case rules: Document what to do with partially occluded objects, truncated objects at image boundaries, and ambiguous cases.
• Quality bar: Define what “good enough” looks like — perfect pixel-level accuracy is not always necessary.

​

Use Multi-Stage Review Pipelines

Annotation → Spot Check (10-20%) → Targeted Review → Approved

• Spot check: Randomly review 10-20% of submissions to identify systemic issues
• Targeted review: Focus reviews on annotations flagged by AutoTag or low-confidence predictions
• Full review: Reserve for high-value or safety-critical datasets

​

Leverage Consensus for Validation

• Items where annotators disagree (review these first)
• Annotators who consistently deviate from the group
• Label classes that are ambiguously defined

​

Batch Work Effectively

​

Performance Optimization

​

Optimize Upload Throughput

​

Optimize Export Performance

• Export by dataset, not by individual items
• Use COCO format for the fastest export generation
• For very large datasets (100K+ items), exports run asynchronously — poll the export status instead of waiting synchronously

​

Monitor with the MCP Server

"List my datasets and their item counts"
"Show recent exports for project X"
"What's the task completion rate for project Y?"

​

Cost Management

​

Right-Size Your Data

1. Remove duplicates: Deduplicate images/frames before uploading
2. Sample strategically: For video, annotate every Nth frame instead of every frame (common: every 5th or 10th frame)
3. Use active learning: Prioritize items where the model is least confident, not random sampling
4. Pre-filter with models: Use Batch Auto-Labeling to auto-label easy cases and focus human annotation on hard cases

​

Minimize API Calls

​

Security

​

Protect Your API Keys

• Store keys in environment variables, never in source code
• Rotate keys periodically (generate new key, update integrations, delete old key)
• Use separate keys for development and production
• Keys are only displayed once at creation — store them securely

​

Use Cloud Storage with Least Privilege

• Read-only for datasets: s3:GetObject, s3:ListBucket
• Read-write for exports: Add s3:PutObject
• Never grant s3:DeleteObject unless absolutely necessary

​

Next Steps

• Quickstart — Get up and running in under 60 seconds
• Examples — Code examples for common workflows
• Rate Limits — Understand API limits and retry strategies
• Cloud Storage — Connect your own S3 or GCS bucket