Build and optimize agent eval workflows with Kiln
Use Kiln to create eval datasets, rate outputs, optimize prompts and models, and ship the same AI task or agent definition from a local workbench into production code.
npx skills add agentskillexchange/skills --skill build-and-optimize-agent-eval-workflows-with-kiln
Use Kiln when an agent or AI product workflow needs a repeatable evaluation and optimization loop. The operator defines a task, collects or generates examples, builds evals, compares prompts and model settings, reviews outputs with teammates, and promotes the resulting task through Kiln’s Python library. Invoke this instead of normal ad hoc prompting when quality gates, datasets, preference ratings, RAG tests, or agent regressions need to be tracked over time. The scope is evaluation, optimization, and deployment handoff for AI tasks and agents, not a generic desktop product listing.
What this skill actually does
Inputs and prerequisites: Kiln Desktop or Python 3 with kiln-ai, model provider API keys or local Ollama.
Setup notes: Install Kiln Desktop from the official download page for a no-code eval workbench, or install the open Python library with pip install kiln-ai when the workflow needs to run from code.
Source and verification boundary: use https://docs.kiln.tech as the canonical reference before running the workflow; keep commands, API calls, CLI usage, and generated outputs reviewable against that upstream source.
Framework fit: publish this as a Multi-Framework workflow only when the operator can invoke the documented toolchain directly, rather than treating the upstream project as a generic product listing.