Roll out a student model

Serving changes affect live traffic

Deployment reads require distill:read. Creating, changing, swapping, or retiring a deployment requires distill:write. A direct REST mutation takes effect as soon as the service accepts it. Pilot classifies those same actions as proposals and waits for a person before execution.

A newly created deployment is asynchronous. Persist its ID and poll it fromprovisioning to live or degraded. Retirement is terminal. A webhook can wake the client on distill.deployment.updated, but the resource GET remains the source of truth.

Change one rollout boundary at a time

1. 1
   Shadow

   Return the teacher answer and record the student's parallel result for comparison.

2. 2
   Percent

   Route a stable cohort to the student and the rest to the teacher.

3. 3
   Full

   Route every eligible request to the student, retaining failure fallback.

4. 4
   Retire

   Stop the alias and release its name while preserving deployment history.

1. 1

   Approve one boundary

   Record the exact deployment ID, desired mode, percentage, and fallback configuration.

2. 2

   Observe a defined window

   Check health, served-by cohorts, latency, quality, fallback count, and drift events.

3. 3

   Advance, hold, or reverse

   Review each traffic change on its own. Keep the previous percentage as the rollback point.

Create and verify a shadow deployment

Deployment creation requires an idempotency key. Supply the student ID from the successful live run and the teacher source and model you already use through the proxy.

DEPLOY_BODY='{
  "student_model": "sm_...",
  "alias": "support-v1",
  "mode": "shadow",
  "teacher_fallback": true,
  "fallback_source": "dsc_...",
  "fallback_model": "gpt-4.1-mini"
}'


curl --fail-with-body -sS -X POST "https://lizzy.albinilabs.com/v1/distill/deployments" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: support-shadow-v1" \
  --data "$DEPLOY_BODY"

{
  "id": "dep_...",
  "object": "distill_deployment",
  "livemode": true,
  "alias": "support-v1",
  "model": "lz:support-v1",
  "student_model": "sm_...",
  "student_model_name": "support-student-v1",
  "base_model": "qwen3-8b",
  "mode": "shadow",
  "rollout_percent": 0,
  "teacher_fallback": true,
  "fallback_source": "dsc_...",
  "fallback_model": "gpt-4.1-mini",
  "status": "provisioning",
  "health": { "state": "unknown" },
  "served_count": null
}

DEPLOYMENT_ID="dep_..."


for attempt in $(seq 1 30); do
  DEPLOY_JSON=$(curl --fail-with-body -sS \
    -H "Authorization: Bearer $LIZZY_API_TOKEN" \
    "https://lizzy.albinilabs.com/v1/distill/deployments/$DEPLOYMENT_ID")
  STATUS=$(jq -r '.status' <<<"$DEPLOY_JSON")
  case "$STATUS" in live|degraded) break ;; retired) exit 1 ;; esac
  sleep 5
done
test "$STATUS" = live || jq '{status,health}' <<<"$DEPLOY_JSON"


curl --fail-with-body -sS -D - -X POST "https://lizzy.albinilabs.com/v1/proxy/chat/completions" \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "X-Lizzy-External-Id: account-42" \
  --data '{
    "model": "lz:support-v1",
    "messages": [{"role":"user","content":"How do I update my billing address?"}]
  }'

In shadow mode, X-Lizzy-Served-By: shadow means the returned answer came from the teacher while the student ran in the background. KeepX-Lizzy-External-Id stable for later percentage bucketing and retainX-Lizzy-Call-Id when investigating one request.

Promote a stable percentage cohort

After shadow quality and health meet your policy, patch the existing deployment. Percent mode requires an integer from 0 to 100. Start with a small value and preserve the same student and fallback while you evaluate the routing change.

curl --fail-with-body -sS -X PATCH \
  "https://lizzy.albinilabs.com/v1/distill/deployments/dep_..." \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{"mode":"percent","rollout_percent":10}'

{
  "id": "dep_...",
  "model": "lz:support-v1",
  "mode": "percent",
  "rollout_percent": 10,
  "teacher_fallback": true,
  "status": "live",
  "health": { "state": "healthy" }
}

The router hashes the stable external ID into the percentage cohort. Reusing it keeps one account on a consistent path. Without an external ID, routing falls back to request content, so semantically similar calls can land in different cohorts.

Move to full traffic without discarding fallback

Full mode routes all eligible requests to the student. It does not disable teacher fallback. Keep fallback enabled until you have a deliberate reason, separate approval, and another recovery path.

curl --fail-with-body -sS -X PATCH \
  "https://lizzy.albinilabs.com/v1/distill/deployments/dep_..." \
  -H "Authorization: Bearer $LIZZY_API_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{"mode":"full"}'

The response reports mode: full and rollout_percent: 0. Zero does not mean zero student traffic in this mode; it means the percentage dial is inactive. Continue monitoring health, served-by headers, latency, quality signals, fallback rate, spend, and distill.student.drift_detected events.

Understand exactly when fallback can help

When the router selects the student and it fails before returning the first byte, a deployment with teacher_fallback: true and a validfallback_source sends the request to that source usingfallback_model. The response header becomesupstream_fallback.

Fallback cannot replace bytes after a streamed student response has already started. It also does not apply after retirement or when no deployment resolves the alias. With fallback disabled or unresolved, a student outage returns503 student_unavailable.

Diagnose serving without creating a second deployment

Save the deployment ID, bare alias, student-model ID, report ID, fallback source and model, creation idempotency key and raw body, current resource status, health, and the last approved mode and percentage. Include X-Lizzy-Call-Id andX-Lizzy-Served-By when investigating one proxy call.

Return to the results guide when a quality signal changes, or to training when the evidence supports a new student rather than a routing adjustment.