Skybert documentation

User Journey

1. Install Skybert in the Python project where the agent will live.
2. Prepare providers with Pi and the model provider you want to use.
3. Build locally from the fullscreen skybert workspace. Choose Build new agent or Add existing agent.
4. Run normally. Decorated functions capture observations in .skybert/. If you are logged in, runs will sync to Cloud for user-friendly annotation and dataset building.
5. Review and evaluate local observations or synced datasets. Pi edits code, prompts, contracts, validators, and versions.
6. Ship through git. Review and deploy changes like you normally would. Agents in production will automatically upload observations if configured with Cloud credentials.

Installation

Install Skybert in the agent project. The package includes the Python SDK, local workspace, fullscreen CLI, dataset tools, eval helpers, and Cloud login commands.

Install Pi and connect your model provider:

npm install -g --ignore-scripts @earendil-works/pi-coding-agent
pi
/login

New Agent

The guided path starts from a clean folder and the fullscreen workspace.

mkdir my-first-agent
cd my-first-agent
uv init
uv add skybert
skybert

Choose Build new agent. Skybert launches Pi, checks provider readiness, asks what useful task data you already have, generates or edits code locally, and writes evidence for the next step.

Add Existing Agent

Use this path when your repo already has agent, workflow, prompt, tool, API handler, or pipeline code and you want Skybert to add instrumentation around the right boundary.

cd existing-agent-repo
uv add skybert
skybert

• Choose Add existing agent in the fullscreen workspace.
• Pi inspects the active folder, finds likely agent entrypoints, and asks which path is the real agent if the code is ambiguous.
• Skybert should wrap the smallest stable task function, not app startup, server glue, or unrelated framework plumbing.
• Existing CSV, JSON, and JSONL files can be used as task examples or evaluation data.

Existing Agents

Agents show up in the Skybert TUI when Skybert detects decorated functions in the folder structure you launched from.

Select an existing agent from the Home screen to review runs, inspect datasets, monitor versions, or continue the same Pi build session with its prior context.

Skybert Framework

Skybert works at the Python function boundary. Decorate the function whose output you want to observe, review, evaluate, and improve.

• inputs=[...] names the function arguments that define a dataset row, such as question, topic, ticket_text, or document.
• Inputs are the case being evaluated. They are kept stable when Skybert reviews, compares, and reruns an example.
• Function parameters not listed as inputs can be useful tunable values, such as prompt, model, temperature, thresholds, or grader instructions.
• display_results=[...] can show extra returned fields beside the target.

import skybert

@skybert.optimize(
    agent="support-agent",
    version="0.1.0",
    inputs=["question"],
    target="answer",
)
def answer_question(question: str, prompt: str = "Answer briefly."):
    return {"answer": call_your_agent(question, prompt=prompt)}

Local Workspace

Skybert is local-first. Normal agent runs do not ask login questions or require Cloud. Observations, annotations, traces, datasets, sync state, and auth live under the project .skybert/ workspace.

Run skybert workspace when you need to see exactly which workspace is active.

Fullscreen CLI

Run skybert with no arguments to open the local workspace. It is the primary product surface for local work.

• Home shows Cloud login state, local counts, Pi readiness, and known agents.
• Build new agent launches the Pi builder for a new production-grade agent.
• Add existing agent asks Pi to inspect existing code and add instrumentation at the right function boundary.
• Continue with agent reopens the same Pi build session, so Pi keeps the prior build context when you resume work on an existing agent.
• View runs lets you review local observations as good, bad, partial, or unsure.
• Datasets previews local and Cloud datasets.
• Monitor summarizes run quality by decorator version when Cloud is connected.

Datasets and Eval

Datasets are the bridge between real observations and repeatable improvement. They can come from reviewed local runs, JSONL files, or production datasets created in Skybert Cloud and pulled locally.

python agent.py --interactive
skybert eval run --agent support-agent --function answer_question --dataset <dataset-id>

• Interactive mode is only for eval/control loops. Normal agent runs just record observations.
• SKYBERT_INTERACTIVE=1 python agent.py also starts interactive mode.
• skybert eval run-command ... -- python agent.py can launch the interactive process for a one-shot eval.
• Pi uses datasets and eval evidence to make local code improvements.

Cloud and Production

Cloud is optional for local builds. Use it when you want hosted observations, team review, production dataset creation, and version monitoring.

skybert login
skybert show-token

export SKYBERT_ENV=production
export SKYBERT_API_KEY=...
export SKYBERT_GIT_SHA=...

• Store the token from skybert show-token as SKYBERT_API_KEY in your deployment secret store.
• Set SKYBERT_ENV=production on deployed agents so production observations upload.
• Do not commit or log tokens. Use skybert show-token --json for automation.
• The TUI sync path is dataset-only. Runs sync from instrumented code when Cloud credentials are active.

Telemetry

Skybert records runs without telemetry extras. Add an extra only when you want Skybert spans in your existing OpenTelemetry or Logfire setup.

Annotations and Tracing

Use annotations when your product already knows whether an output was good, bad, incomplete, or preferred by a user. This is useful in user-facing products: thumbs up/down, edits, approvals, expert reviews, and support outcomes can become feedback on the exact run that produced the output.

That feedback gives human experts a direct way to steer the system. Skybert can turn annotated runs into datasets, so future build and optimization work is shaped by real user preferences and expert judgment. Skybert attaches feedback to the latest run unless you pass a specific run_id.

result = answer_question("How do I deploy?")

skybert.annotate(
    verdict="partial",
    corrected_output={"answer": "Deploy normally and set SKYBERT_API_KEY."},
    comment="The answer missed the token setup.",
)

For asynchronous product flows, store the run id with your own job, message, or review record, then annotate that run when feedback arrives later.

async def create_answer(question: str):
    result = await answer_question(question)
    return {
        "answer": result["answer"],
        "skybert_run_id": skybert.last_run_id(),
    }

async def handle_user_feedback(run_id: str, accepted: bool, edited_answer: str | None = None):
    skybert.annotate(
        run_id=run_id,
        verdict="good" if accepted else "partial",
        corrected_output={"answer": edited_answer} if edited_answer else None,
        comment="User feedback from the product UI.",
    )

Accepted verdict shortcuts are good, bad, partial, and unsure. Use skybert.Trace(...) to group larger work inside a run.

CLI

Environment