# Pilot execution playbook This is the agent-safe companion to the developer documentation. Pilot loads a version-pinned copy inside the API service, so it never depends on a website request while operating a workflow. The linked human pages add examples and API detail; this companion contains only typed-tool sequence, authority boundaries, stop conditions, and recovery. ## Remote MCP boundary Remote clients use `POST https://lizzy.albinilabs.com/v1/mcp` with a test or live `full_access` token; Pilot must be enabled. Start in test mode. Stateless Streamable HTTP returns JSON, not SSE, and no `Mcp-Session-Id`. This playbook is Pilot-only. Use `tools/list` and https://lizzy.albinilabs.com/docs/mcp for remote schemas; do not translate dotted actions into MCP arguments. MCP clients and dashboard share workflows. The MCP agent has the full authority of its token. ## Start and control one durable workflow Confirm goal, data, success, budget, and outcome before writing. Start one workflow with `distill_workflow.start`; its returned `workflow_id` scopes later phase tools. Existing workflows require the exact id for `inspect`, `switch`, `resume`, or `abandon`; never infer, repair, or replace it. Checkpoint refs authorize routing. `next_actions` permits a later action, not readiness. Stop on handoff, approval, input request, failure, block, or async watch. Never retry after a local wait. Answer with only explicitly requested fields. Omit unrequested names, ids, refs, and next-step interpretations. Copy a requested id exactly; never guess or correct it. Terminal phases `complete` and `abandoned` cannot `resume` despite saved refs. ## Wait at a human boundary An `approval_required` or `handoff_required` workflow already has a pending human step. Do not create another proposal, handoff, upload, or phase action, and do not try to complete the step with a generic tool. Tell the user what role or dashboard action is required and stop. On `handoff_required`, only the handoff is pending; setup is incomplete. Continue only after the durable workflow is woken with a changed status and safe checkpoint references. A verbal “yes,” “done,” or “approved” does not replace the recorded platform event. ## Capture traffic without handling a credential Review the source name and kind before `distill_data.configure_capture`. `kind: "managed"` needs a catalog `teacher_model` and no provider key; BYOK needs the location and credential. Never infer either choice. The tool returns a human handoff without accepting a BYOK credential. Stop, then resume only after the dashboard supplies the source reference. The customer must generate the traffic. `distill_data.wait_for_traffic` performs one bounded metadata read; it does not wait or poll. If no traffic is present, explain that the workflow is awaiting external traffic, stop on the returned `input_required` status, and preserve the workflow for a later turn. Captured bodies never belong in model context. ## Import through an existing connector If a connector is not configured, use `distill_data.prepare_connector_handoff` and let a human enter and verify its credential. Once the checkpoint contains the safe connector reference, call `distill_data.pull_connector` exactly once with the reviewed row limit and optional date bound. A pull is asynchronous. Persist the returned pull reference, rely on its registered event watch, and stop instead of listing or polling. On authentication failure, ask a human to repair the connector; another pull cannot repair a bad credential. A successful zero-row pull is still terminal and should be explained rather than retried. ## Hand off a file upload Use `distill_data.prepare_upload_handoff` with the filename. The human dashboard owns file bytes, upload parts, completion, and any short-lived storage capability. Stop on `handoff_required` and never ask for file contents, a signed location, or upload state in chat. Continue the same workflow only after the handoff supplies a safe dataset reference. Do not create a second upload or dataset merely because the human step takes longer than one turn. ## Freeze one immutable dataset version Use `distill_data.materialize` once the selected dataset is known. The operation creates one durable immutable version, so persist its reference and stop on the event watch while it builds. The ready version, not the mutable dataset definition, is the input to planning and training. If the version is empty or fails, inspect that same version once and explain its stable blocker. Correct the dataset selection or wait for missing traffic; do not materialize duplicates in a polling loop. Keep the frozen split and review note attached to the version used by the experiment. ## Continue from saved data references When the user explicitly asks to inspect a saved source, connector, dataset, or immutable version, use `distill_data.inspect` once with that exact reference. This is a read: report the returned state and do not turn it into materialization or another data-path write. For a short continuation such as “done” or “next,” follow the checkpoint instead of defaulting to a new capture. An incomplete handoff remains a human boundary. A saved source can support one bounded `distill_data.wait_for_traffic` read; a saved connector can support one reviewed `distill_data.pull_connector`; and a saved dataset can support one `distill_data.materialize`. If a connector pull or dataset version is already in progress, rely on its registered watch and stop. If the immutable version is ready, preserve it and follow the returned `next_actions` toward rewards. Never infer a different data path or create a duplicate resource. ## Choose and test rewards explicitly Use `distill_rewards.list` only when asked or no reviewed selection exists. Create or test confirmed criteria. Verifiers handle deterministic checks; subjective ones need a concrete rubric. Keep webhook secrets human-owned. Call `distill_rewards.select` with exact reviewed ordered ids; never guess or correct one. Explicit empty means supervised-only training; omission differs. Stop on invalid reward state instead of substituting another scorer. For reads, return only requested fields. Omit unrequested names, ids, configuration, counts, and next steps; `next_actions` never proves readiness. ## Plan and validate the exact recipe Use `distill_plan.propose` with one ready immutable version, an explicit reward selection, the objective, budget guidance, and either a report-only or deployable-student outcome. A serve-bound plan must include publication. `distill_plan.estimate` is a read and does not reserve compute. After the reviewed plan is saved, call `distill_plan.start_dry_run` exactly once. The validation is asynchronous, so persist the run reference, rely on its watch, and stop. The succeeded dry run and its plan hash prove that exact version, recipe, budget, and compute; any execution-affecting edit requires a new validation. ## Request and watch live training After the exact dry run succeeds in the same mode, call `distill_training.request_live`. This call creates the approval proposal; it does not spend money by itself. Stop on `approval_required` and never approve, repeat, or bypass the proposal. Approval creates the live run and wakes the same workflow with its safe run reference. Use `distill_training.inspect` once when the user asks for status or a terminal event wakes the workflow. A queued or running result already carries a watch, so stop instead of polling. Use `cancel` only when the user explicitly chooses to stop future work; a canceled or failed run is not permission to create another billable run. ## Read evidence and produced artifacts Use `distill_results.metrics`, `report`, and `student` by saved reference. Resolve the report and published student separately: a succeeded evaluation may produce a report without a deployable student, while a dry run never publishes one. Attach bounded artifacts rather than dumping logs or raw examples into chat. Synchronous reads return a bounded `evidence` envelope. Explain requested counts, progress, metrics, verdicts, health, and failure codes only when that envelope contains them. If a field is absent or the evidence is truncated, state that it is unavailable and offer the saved resource reference; never infer a result from the checkpoint alone. Report only requested fields. These reads do not prove run status; outer `completed` means the read. Never infer clean or healthy. For a multi-resource review, attach at most one report card. Its `ref` contains exactly the single saved report reference; never combine run, report, student, or deployment references in one card. Only a ready real report can support serving. Its verdict is evidence for the human decision, not automatic rollout approval. Explain regressions, missing measurements, skipped checks, cost, and the experiment objective before recommending iteration or deployment. ## Roll out one boundary at a time Use `distill_deployment.request_shadow` first, with a ready student and resolvable teacher fallback. It creates an approval proposal and routes no customer responses to the student. Stop for approval, then inspect once when the deployment event arrives. Use `request_percent`, `request_full`, or `request_retire` only for the single transition the user confirmed. Every transition creates its own approval proposal. Do not combine a model swap, fallback change, and traffic increase. A provisioning result is asynchronous; a degraded result is a blocker; full or retired is terminal for the requested boundary. ## Diagnose without a doom loop Start from the workflow phase, status, blockers, safe references, and the last returned action. Read at most one relevant durable resource when the current state is missing. Branch on the stable error code and preserve the request, workflow, and resource references for a later turn. A blocked workflow cannot run a phase tool. Correct its stable blocker, then call `distill_workflow.resume` once to return to the saved resume phase. If it remains blocked, stop and explain what must change instead of trying the phase action directly. An input request needs one corrected choice. A human handoff needs the dashboard. An approval needs an owner or admin. An asynchronous result needs its event watch. A repeated semantic result means stop and explain the blocker. Never retry a mutation, invent an id, create a duplicate resource, or turn documentation prose into an API command.