Run an experiment with Pilot

Pilot uses dashboard authority

Pilot runs inside Lizzy’s backend and works on behalf of the signed-in owner or admin. The backend creates a short-lived, encrypted act credential for the session, then classifies every workspace API action through a deny-by-default mediation policy. The model never receives the credential itself and cannot widen its scopes.

Workflow phases and statuses

A workflow stores progress for one objective. Its phase says where the experiment is. Its status says why control currently belongs to Pilot, a person, an approval, or a terminal outcome. Store both; the same phase can pause for different reasons.

textworkflow phases

scope → data → rewards → plan → dry_run → approval
      → training → results → deployment → complete


Exceptional control/end phases: blocked, abandoned

textworkflow statuses

in_progress       Pilot may continue the current phase.
input_required     A person must answer a specific question.
handoff_required   A person must create a secret/file-backed resource in the UI.
approval_required  A proposal card is waiting for an owner or admin.
blocked            Repeated attempts made no progress; intervention is required.
failed             The workflow failed and its checkpoint should be inspected.
completed          The workflow reached a terminal outcome.

Active workflow summaries are returned when a Pilot session is created or read. The authoritative thread read contains messages, pending proposals, and workflows. WebSocket frames make the UI responsive, but they do not replace that reconciliation read.

GET/v1/pilot/sessions/{session_id}

Reconcile the session’s messages, pending proposals, and workflow summaries after a reconnect or when a streaming turn ends unexpectedly.

Create a session and workflow

Opening Pilot calls POST /v1/pilot/sessions. The operation is get-or-create per signed-in user, workspace, and mode: a page refresh or second tab reuses the active session instead of consuming another session or splitting the thread. A new session returns HTTP 201; a reused one returns HTTP 200.

typescriptsame-origin dashboard client

// This runs inside the authenticated Lizzy dashboard. It is not a bearer-token API.
async function lizzyDashboard(path: string, init: RequestInit = {}) {
  const response = await fetch("/v1" + path, {
    ...init,
    credentials: "include",
    headers: { "Content-Type": "application/json", ...init.headers },
  });
  const body = await response.json();
  if (!response.ok) throw new Error(JSON.stringify(body));
  return body;
}


const session = await lizzyDashboard("/pilot/sessions", { method: "POST" });


const workflow = await lizzyDashboard("/pilot/workflows", {
  method: "POST",
  body: JSON.stringify({
    session: session.id,
    name: "Support quality v1",
    goal: "Reduce support inference cost while preserving holdout resolution quality",
    phase: "scope",
    checkpoint: { refs: {} },
  }),
});


console.log(workflow.id, workflow.phase, workflow.status);

jsonexample session response

{
  "id": "pls_01K2ZXQ8M4Y7V6R5T3S9N2P1E4",
  "status": "active",
  "expires_at": "2026-08-15T12:30:00Z",
  "livemode": false,
  "turns_used": 0,
  "turn_running": false,
  "ws_url": "/v1/pilot/ws",
  "workflows": []
}

jsonexample workflow response

{
  "id": "plw_01K2ZXQ8M4Y7V6R5T3S9N2P1F5",
  "object": "pilot_workflow",
  "session": "pls_01K2ZXQ8M4Y7V6R5T3S9N2P1E4",
  "name": "Support quality v1",
  "goal": "Reduce support inference cost while preserving holdout resolution quality",
  "phase": "scope",
  "status": "in_progress",
  "checkpoint": { "refs": {} },
  "no_progress_count": 0,
  "created_at": "2026-08-15T10:30:00Z",
  "updated_at": "2026-08-15T10:30:00Z"
}

Stop after three repeated results

Pilot records a progress key derived from the tool, action, payload, and result. Seeing the same progress key three times moves the workflow to phase: "blocked"and status: "blocked". The previous phase is saved ascheckpoint.resume_phase. This is a terminal point for autonomous work, not an invitation to vary arguments at random.

1
Attempt once
Read the typed response and update resource refs or the plan checkpoint.
2
Classify the result
Progress, a timed-out watch, human input, approval, terminal failure, and unknown endpoints need different next actions.
3
Repeat detected
Identical action and result increment no_progress_count rather than being narrated as new work.
4
Third repeat blocks
Pilot stops, persists resume_phase, and returns control to a person.
5
Resume after a change
Fix the prerequisite or replace the checkpoint, then resume the same plw_ ID.

jsonblocked workflow checkpoint

{
  "id": "plw_01K2ZXQ8M4Y7V6R5T3S9N2P1F5",
  "phase": "blocked",
  "status": "blocked",
  "checkpoint": {
    "refs": { "dataset_version": "dsv_01K2ZXQ8M4Y7V6R5T3S9N2P1G6" },
    "resume_phase": "dry_run"
  },
  "no_progress_count": 3
}

Hand secrets and files back to a person

When work needs a credential or a local file, Pilot creates one pending handoff and pauses the workflow. The person follows its dashboard destination, creates the resource, then completes the handoff with only the resulting resource ID. Pilot resumes from that reference without ever seeing the secret or bytes.

textimplemented handoff contract

source_credentials     destination /distill/loops
                       completion { "input": { "source": "dsc_…" } }


connector_credentials  destination /distill/connectors
                       completion { "input": { "connector": "conn_…" } }


dataset_upload         destination /distill/datasets
                       completion { "input": { "dataset": "ds_…" } }

1
Open the handoff destination
Read the handoff prompt, but enter upstream keys, connector tokens, and files only in the dedicated Lizzy form.
2
Verify the created resource
Confirm its mode, status, and identifier. A handoff does not accept arbitrary notes or a secret-shaped object.
3
Complete with the one expected reference
The input must contain only source, connector, ordataset with the matching ID prefix. A second completion is idempotent only when it resolves to the same terminal handoff status.

Review the exact action before approval

For proposal-tier work, Pilot prepares a closed, typed body and a human-readable summary. The owner or admin sees exactly what can execute. Approval uses the approver’s current authority, and the proposal ID doubles as the action’s idempotency key so a repeated click cannot start a second logical operation.

textproposal states

pending → executing → executed
pending → denied
pending → expired
executing → failed


approved can be observed as an open recovery/intermediate state.

jsonexample live-run proposal

{
  "id": "plp_01K2ZXQ8M4Y7V6R5T3S9N2P1H7",
  "session": "pls_01K2ZXQ8M4Y7V6R5T3S9N2P1E4",
  "intent": "run.create",
  "summary": "Start a training run: support-student-v1",
  "status": "pending",
  "dry_run": false,
  "body": {
    "dataset_version": "dsv_01K2ZXQ8M4Y7V6R5T3S9N2P1G6",
    "student_model_name": "support-student-v1",
    "recipe": { "stages": [{ "kind": "sft", "config": {} }] },
    "budget": { "max_cost_usd": 25 },
    "dry_run": false,
    "validated_dry_run": "run_01K2ZXQ8M4Y7V6R5T3S9N2P1J8",
    "plan_hash": "4f2a7d8c1b9e6a035d4c7f2180e9b6a34c5d7e8f1029384756abcdef01234567"
  },
  "result": null,
  "error": null,
  "created_at": "2026-08-15T11:00:00Z",
  "decided_at": null
}

Resume the existing workflow

Start with the workflow ID and session, phase and status, checkpoint refs, pending handoff or proposal, most recent resource state, structured error code, and request ID. A new chat or duplicate workflow discards that chain of evidence.

typescriptinspect and resume from the dashboard

const sessionId = "pls_01K2ZXQ8M4Y7V6R5T3S9N2P1E4";
const workflowId = "plw_01K2ZXQ8M4Y7V6R5T3S9N2P1F5";


const workflow = await lizzyDashboard(
  "/pilot/workflows/" + workflowId + "?session=" + encodeURIComponent(sessionId),
);


if (workflow.phase === "blocked") {
  // First fix the prerequisite named by the last result. Then resume this same ID.
  const resumed = await lizzyDashboard(
    "/pilot/workflows/" + workflowId + "/resume",
    { method: "POST", body: JSON.stringify({ session: sessionId }) },
  );
  console.log(resumed.phase, resumed.status, resumed.no_progress_count);
}

Resuming resets no_progress_count and returns a blocked workflow to its savedresume_phase. Supply a replacement checkpoint only when you intend to replace the stored plan or resource references. complete andabandoned workflows cannot be resumed. Abandoning a non-complete workflow sets its phase to abandoned and status to completed.

textoperator recovery decisions

input_required      Answer the specific question in the existing session.
handoff_required    Complete or cancel the named plh_ handoff in its UI destination.
approval_required   Approve or deny the exact plp_ proposal; do not create another card.
blocked             Fix the repeated prerequisite, then resume the same plw_ workflow.
failed              Inspect checkpoint refs and the underlying resource error before retrying.
turn disconnected   GET the session and workflow; a socket disconnect is not resource failure.
watch timed out     GET the saved run/proposal ID; do not submit a duplicate.