Build a Distill workflow end to end

Example architecture

This tutorial distills a support teacher into a smaller student. It uses captured traffic, an exact-match verifier, a deployable recipe, and the rollout alias support-v1. Save each returned ID so the integration can stop and resume without repeating work.

1
Capture or import
Create a source and proxy production-like calls, or import existing JSONL.
2
Freeze evidence
Materialize a mutable dataset as one immutable, ready dataset version.
3
Define success
Create and test rewards, then list their IDs in the recipe.
4
Rehearse and train
Dry-run the exact budget and recipe, approve the hash, then start live work.
5
Read the proof
Inspect the run, report, regressions, examples, and published student.
6
Roll out
Provision shadow serving, advance by percentage, then choose full traffic.

Set up one resumable workspace script

Install curl and jq, then export a live token withdistill:read and distill:write. Keep tokens and upstream credentials in environment variables, never in source control or a copilot transcript. A test token is useful for learning, but it forces every run to be dry and its resources cannot validate a live-mode run.

bashenvironment

set -euo pipefail


export LIZZY_API_TOKEN="<live-token>"
export LIZZY_UPSTREAM_KEY="<teacher-provider-key>"
export LIZZY_TEACHER_MODEL="<teacher-model>"


API_BASE="https://lizzy.albinilabs.com/v1"


LOOP_BODY='{"name":"Support distillation","description":"Preserve support answer quality at lower cost"}'
LOOP_JSON=$(curl --fail-with-body -sS -X POST "$API_BASE/distill/loops" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: support-loop-v1" \
  --data "$LOOP_BODY")
LOOP_ID=$(jq -er '.id' <<<"$LOOP_JSON")
printf 'loop=%s
' "$LOOP_ID"

A successful create returns a resource you can retrieve later:

jsonexample response

{
  "id": "dlp_...",
  "object": "distill_loop",
  "name": "Support distillation",
  "slug": "support-distillation",
  "status": "active",
  "counts": { "sources": 0, "datasets": 0, "runs": 0, "live_deployments": 0, "captured_calls": 0 }
}

Capture traffic, then freeze it

Live traffic

Source and proxy

Best when real teacher traffic, or a production-like sample, can flow through Lizzy.

Source credential is a human secret

Hosted corpus

Connector pull

Hugging Face and Braintrust pulls keep their progress and finish as succeeded or failed.

Human connects, integration polls

Local corpus

Multipart JSONL

Reserve an upload, PUT each part, complete it, then resolve the generated version.

Raw bytes stay outside Pilot

The runnable path below uses capture. Source creation contains a write-only provider key, so a person performs this call or completes the equivalent UI handoff. Pilot can resume from the returned source ID.

bashcapture and freeze

SOURCE_BODY=$(jq -nc \
  --arg loop "$LOOP_ID" \
  --arg key "$LIZZY_UPSTREAM_KEY" \
  '{loop:$loop,name:"support-teacher",base_url:"https://api.openai.com/v1",upstream_key:$key,default_capture:true,capture_sample_rate:1}')
SOURCE_JSON=$(curl --fail-with-body -sS -X POST "$API_BASE/distill/sources" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: support-source-v1" \
  --data "$SOURCE_BODY")
SOURCE_ID=$(jq -er '.id' <<<"$SOURCE_JSON")


# This shows one request. Capture or import a production-like corpus before freezing.
curl --fail-with-body -sS -X POST "$API_BASE/proxy/chat/completions" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "X-Lizzy-Source: $SOURCE_ID" \
  -H "X-Lizzy-Tags: support,baseline" \
  -H "X-Lizzy-External-Id: ticket-1001" \
  --data "$(jq -nc --arg model "$LIZZY_TEACHER_MODEL" \
    '{model:$model,messages:[{role:"user",content:"How do I update my billing address?"}]}')"


# Capture is asynchronous. Do not freeze the dataset until this request is queryable.
CAPTURE_DEADLINE=$((SECONDS + 60))
CAPTURED_CALL_ID=""
while (( SECONDS < CAPTURE_DEADLINE )); do
  CALLS_JSON=$(curl --fail-with-body -sS \
    -H "Authorization: Bearer $LIZZY_API_TOKEN" \
    "$API_BASE/distill/calls?source=$SOURCE_ID&external_id=ticket-1001&limit=1")
  CAPTURED_CALL_ID=$(jq -r '.data[0].id // empty' <<<"$CALLS_JSON")
  if [[ -n "$CAPTURED_CALL_ID" ]]; then
    break
  fi
  sleep 2
done
if [[ -z "$CAPTURED_CALL_ID" ]]; then
  printf >&2 'capture watch timed out; resume with source=%s external_id=%s
' \
    "$SOURCE_ID" "ticket-1001"
  exit 2
fi
printf 'captured_call=%s source=%s external_id=%s
' \
  "$CAPTURED_CALL_ID" "$SOURCE_ID" "ticket-1001"


DATASET_BODY=$(jq -nc --arg loop "$LOOP_ID" --arg source "$SOURCE_ID" \
  '{loop:$loop,name:"Resolved support calls",filter:{sources:[$source],tags_any:["support"],dedupe:"exact",pii_scrub:true},split_config:{holdout_pct:10,seed:42}}')
DATASET_JSON=$(curl --fail-with-body -sS -X POST "$API_BASE/distill/datasets" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: support-dataset-v1" \
  --data "$DATASET_BODY")
DATASET_ID=$(jq -er '.id' <<<"$DATASET_JSON")


# Version creation is asynchronous and currently has no idempotency contract.
VERSION_JSON=$(curl --fail-with-body -sS -X POST \
  "$API_BASE/distill/datasets/$DATASET_ID/versions" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{"note":"Support baseline reviewed 2026-08-15"}')
DATASET_VERSION_ID=$(jq -er '.id' <<<"$VERSION_JSON")


VERSION_DEADLINE=$((SECONDS + 120))
VERSION_STATUS=$(jq -r '.status' <<<"$VERSION_JSON")
while (( SECONDS < VERSION_DEADLINE )); do
  VERSION_JSON=$(curl --fail-with-body -sS \
    -H "Authorization: Bearer $LIZZY_API_TOKEN" \
    "$API_BASE/distill/dataset_versions/$DATASET_VERSION_ID")
  VERSION_STATUS=$(jq -r '.status' <<<"$VERSION_JSON")
  case "$VERSION_STATUS" in
    ready) break ;;
    building) sleep 2 ;;
    failed)
      jq -c '{id,status,error}' <<<"$VERSION_JSON" >&2
      printf >&2 'dataset version failed; inspect dataset_version=%s dataset=%s
' \
        "$DATASET_VERSION_ID" "$DATASET_ID"
      exit 1
      ;;
    *)
      printf >&2 'unexpected dataset-version status=%s; inspect dataset_version=%s
' \
        "$VERSION_STATUS" "$DATASET_VERSION_ID"
      exit 1
      ;;
  esac
done
if [[ "$VERSION_STATUS" != "ready" ]]; then
  printf >&2 'dataset-version watch timed out; resume with dataset_version=%s dataset=%s
' \
    "$DATASET_VERSION_ID" "$DATASET_ID"
  exit 2
fi
printf 'dataset_version=%s rows=%s
' \
  "$DATASET_VERSION_ID" "$(jq -r '.row_count' <<<"$VERSION_JSON")"

A version moves from building to ready or failed. If the version-create response is lost, list/distill/datasets/{dataset_id}/versions and reconcile by dataset and time before creating another version. See the dataset guide for connector and upload state machines.

Define success, rehearse, and train

A recipe proposal is free advice. Its objective must be cost,quality, or latency. Include the reward IDs; omitting the field can select active verifier and judge rewards for compatibility, whilerewards: [] deliberately selects none. The proposal cost is a planning hint. The run budget is the hard cap.

bashreward, proposal, dry run

REWARD_JSON=$(curl --fail-with-body -sS -X POST "$API_BASE/distill/rewards" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: support-exact-v1" \
  --data '{"name":"support-exact-answer","kind":"verifier","config":{"check":"exact_match","normalize_whitespace":true}}')
REWARD_ID=$(jq -er '.id' <<<"$REWARD_JSON")


PROPOSAL_BODY=$(jq -nc --arg version "$DATASET_VERSION_ID" --arg reward "$REWARD_ID" \
  '{dataset_version:$version,objective:"quality",max_cost_usd:25,rewards:[$reward],outcome:"deployable_student"}')
PROPOSAL_JSON=$(curl --fail-with-body -sS -X POST "$API_BASE/distill/recipes/propose" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" --data "$PROPOSAL_BODY")
RECIPE_JSON=$(jq -c '{student_model,stages,rewards}' <<<"$PROPOSAL_JSON")


DRY_BODY=$(jq -nc --arg version "$DATASET_VERSION_ID" --argjson recipe "$RECIPE_JSON" \
  '{dataset_version:$version,student_model_name:"support-student-v1",recipe:$recipe,budget:{max_cost_usd:25},dry_run:true}')
DRY_JSON=$(curl --fail-with-body -sS -X POST "$API_BASE/distill/runs" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: support-dry-v1" \
  --data "$DRY_BODY")
DRY_RUN_ID=$(jq -er '.id' <<<"$DRY_JSON")


DRY_DEADLINE=$((SECONDS + 600))
DRY_STATUS=$(jq -r '.status' <<<"$DRY_JSON")
while (( SECONDS < DRY_DEADLINE )); do
  DRY_JSON=$(curl --fail-with-body -sS \
    -H "Authorization: Bearer $LIZZY_API_TOKEN" \
    "$API_BASE/distill/runs/$DRY_RUN_ID")
  DRY_STATUS=$(jq -r '.status' <<<"$DRY_JSON")
  case "$DRY_STATUS" in
    succeeded) break ;;
    queued|validating|running|evaluating) sleep 5 ;;
    failed|canceled)
      jq -c '{id,status,error,stages}' <<<"$DRY_JSON" >&2
      printf >&2 'dry run ended terminally; inspect dry_run=%s dataset_version=%s
' \
        "$DRY_RUN_ID" "$DATASET_VERSION_ID"
      exit 1
      ;;
    *)
      printf >&2 'unexpected dry-run status=%s; inspect dry_run=%s
' \
        "$DRY_STATUS" "$DRY_RUN_ID"
      exit 1
      ;;
  esac
done
if [[ "$DRY_STATUS" != "succeeded" ]]; then
  printf >&2 'dry-run watch timed out; resume with dry_run=%s dataset_version=%s
' \
    "$DRY_RUN_ID" "$DATASET_VERSION_ID"
  exit 2
fi
PLAN_HASH=$(jq -er '.plan_hash' <<<"$DRY_JSON")
printf 'dry_run=%s plan_hash=%s
' "$DRY_RUN_ID" "$PLAN_HASH"

bashapproved live run

LIVE_BODY=$(jq -nc \
  --arg version "$DATASET_VERSION_ID" \
  --argjson recipe "$RECIPE_JSON" \
  --arg dry "$DRY_RUN_ID" \
  --arg hash "$PLAN_HASH" \
  '{dataset_version:$version,student_model_name:"support-student-v1",recipe:$recipe,budget:{max_cost_usd:25},dry_run:false,validated_dry_run:$dry,plan_hash:$hash}')
LIVE_JSON=$(curl --fail-with-body -sS -X POST "$API_BASE/distill/runs" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: support-live-v1" \
  --data "$LIVE_BODY")
LIVE_RUN_ID=$(jq -er '.id' <<<"$LIVE_JSON")
printf 'live_run=%s status=%s
' "$LIVE_RUN_ID" "$(jq -r '.status' <<<"$LIVE_JSON")"

The live request needs a fresh idempotency key, but its dataset version, recipe, budget, and compute must match the successful live-mode dry run. The service returnsqueued and advances durably through validating,running, and evaluating to succeeded,failed, or canceled.

Evaluate the proof, then deploy in shadow

Poll the live run to a terminal state. A recipe with eval creates a report; a live recipe with publish creates the deployable student. A dry run can create a stub report, but never a student. Read the report and examples before treating a ready report as a product decision.

bashinspect and deploy

LIVE_DEADLINE=$((SECONDS + 1800))
LIVE_STATUS="queued"
while (( SECONDS < LIVE_DEADLINE )); do
  LIVE_JSON=$(curl --fail-with-body -sS \
    -H "Authorization: Bearer $LIZZY_API_TOKEN" \
    "$API_BASE/distill/runs/$LIVE_RUN_ID")
  LIVE_STATUS=$(jq -r '.status' <<<"$LIVE_JSON")
  case "$LIVE_STATUS" in
    succeeded) break ;;
    queued|validating|running|evaluating) sleep 10 ;;
    failed|canceled)
      jq -c '{id,status,error,stages}' <<<"$LIVE_JSON" >&2
      printf >&2 'live run ended terminally; inspect live_run=%s dataset_version=%s
' \
        "$LIVE_RUN_ID" "$DATASET_VERSION_ID"
      exit 1
      ;;
    *)
      printf >&2 'unexpected live-run status=%s; inspect live_run=%s
' \
        "$LIVE_STATUS" "$LIVE_RUN_ID"
      exit 1
      ;;
  esac
done
if [[ "$LIVE_STATUS" != "succeeded" ]]; then
  printf >&2 'live-run watch timed out; resume with live_run=%s dataset_version=%s
' \
    "$LIVE_RUN_ID" "$DATASET_VERSION_ID"
  exit 2
fi


REPORT_ID=$(jq -er '.report' <<<"$LIVE_JSON")
STUDENT_MODEL_ID=$(jq -er '.student_model' <<<"$LIVE_JSON")
curl --fail-with-body -sS \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  "$API_BASE/distill/reports/$REPORT_ID" | jq '{status,verdict,sample_size,metrics}'


DEPLOY_BODY=$(jq -nc \
  --arg student "$STUDENT_MODEL_ID" \
  --arg source "$SOURCE_ID" \
  --arg model "$LIZZY_TEACHER_MODEL" \
  '{student_model:$student,alias:"support-v1",mode:"shadow",teacher_fallback:true,fallback_source:$source,fallback_model:$model}')
DEPLOY_JSON=$(curl --fail-with-body -sS -X POST "$API_BASE/distill/deployments" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: support-shadow-v1" \
  --data "$DEPLOY_BODY")
DEPLOYMENT_ID=$(jq -er '.id' <<<"$DEPLOY_JSON")


# Wait for provisioning to settle before sending serving traffic.
DEPLOY_DEADLINE=$((SECONDS + 600))
DEPLOY_STATUS=$(jq -r '.status' <<<"$DEPLOY_JSON")
while (( SECONDS < DEPLOY_DEADLINE )); do
  DEPLOY_JSON=$(curl --fail-with-body -sS \
    -H "Authorization: Bearer $LIZZY_API_TOKEN" \
    "$API_BASE/distill/deployments/$DEPLOYMENT_ID")
  DEPLOY_STATUS=$(jq -r '.status' <<<"$DEPLOY_JSON")
  case "$DEPLOY_STATUS" in
    live|degraded) break ;;
    provisioning) sleep 5 ;;
    retired)
      jq -c '{id,status,health}' <<<"$DEPLOY_JSON" >&2
      printf >&2 'deployment retired before serving; inspect deployment=%s student_model=%s
' \
        "$DEPLOYMENT_ID" "$STUDENT_MODEL_ID"
      exit 1
      ;;
    *)
      printf >&2 'unexpected deployment status=%s; inspect deployment=%s
' \
        "$DEPLOY_STATUS" "$DEPLOYMENT_ID"
      exit 1
      ;;
  esac
done
if [[ "$DEPLOY_STATUS" != "live" && "$DEPLOY_STATUS" != "degraded" ]]; then
  printf >&2 'deployment watch timed out; resume with deployment=%s student_model=%s
' \
    "$DEPLOYMENT_ID" "$STUDENT_MODEL_ID"
  exit 2
fi


# Shadow returns the teacher answer and evaluates the student off the request path.
curl --fail-with-body -sS -D - -X POST "$API_BASE/proxy/chat/completions" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "X-Lizzy-External-Id: account-42" \
  --data '{"model":"lz:support-v1","messages":[{"role":"user","content":"How do I update my billing address?"}]}'

A deployment starts provisioning, then becomes live ordegraded; retirement is terminal. After shadow evidence is acceptable, change one traffic boundary at a time withPATCH /distill/deployments/{deployment_id}: first{"mode":"percent","rollout_percent":10}, then{"mode":"full"}. Pilot proposes each transition. Direct REST applies it immediately.

Diagnose one resource at a time

Dataset versionPoll while building. On failed, read its error and fix the source rows or filter. A watch timeout is not a reason to create a replacement.

Dry rundataset_version_not_ready means wait. invalid_recipe names the stage or field to fix. Rehearse again after any execution-affecting change.

Live rundry_run_not_validated or plan_hash_mismatch means the proof is missing or stale. A failed run exposes error, per-stage errors, logs, and metrics. Retrying can spend again and is a new human decision.

ResultsA null report points to an omitted or failed eval stage. A nullstudent_model points to an omitted, skipped, or failed publish stage.

Deploymentreport_not_ready needs a ready real report. degraded calls for keeping fallback, lowering traffic, and inspecting health. Do not create a duplicate alias while the first resource still exists.

Persist these values with the experiment record:

Loop, source or connector, dataset, and immutable dataset version IDs.
Selected reward IDs and the exact recipe proposal reviewed by the operator.
Dry-run ID, dry-run plan hash, live-run ID, and each raw run request body.
Idempotency keys until every create has a known response.
Report ID, student-model ID, deployment ID, bare alias, and fallback source and model.
The last human-approved rollout boundary and the event or observation that justified it.

For stage-specific recovery, continue with training, results, and serving.