Trace and evaluate agent runs with Phoenix
Use Phoenix to trace LLM and agent calls, run evaluations, replay prompts, inspect datasets, and expose observability workflows through MCP.
npx skills add agentskillexchange/skills --skill trace-and-evaluate-agent-runs-with-phoenix
Use Phoenix when an operator needs an open-source observability workflow for agent or LLM applications. The workflow is to deploy Phoenix, instrument the target application with the relevant OpenTelemetry or framework integration, capture traces and datasets, run evaluations, and use the playground, CLI, or MCP surface to investigate failures and compare prompt, model, or tool changes.
What this skill actually does
Invoke this instead of ad hoc logs when multi-step agent behavior needs trace-level debugging, repeatable evaluations, prompt replay, retrieval checks, or production monitoring evidence. The scope boundary is AI observability and evaluation for agent workflows; it is not a generic analytics platform, hosted monitoring product, or replacement for the application framework.
Inputs are an LLM or agent application, Phoenix server, tracing integration, model provider credentials, evaluation datasets or rubrics, and optional MCP client configuration. Outputs are traces, spans, datasets, evaluation results, prompt-playground comparisons, and debugging evidence for release or regression decisions.
Inputs and prerequisites: Phoenix server, OpenTelemetry or supported framework integration, LLM or agent application, model provider credentials, optional MCP client.
Setup notes: Install or deploy Phoenix using the official Docker, Python, or cloud instructions, instrument the target LLM or agent app with a supported Phoenix tracing integration, then configure evaluations, datasets, playground replay, CLI access, or the Phoenix MCP server as needed.
Source and verification boundary: use https://arize.com/docs/phoenix as the canonical reference before running the workflow; keep commands, API calls, CLI usage, and generated outputs reviewable against that upstream source.
Framework fit: publish this as a Multi-Framework workflow only when the operator can invoke the documented toolchain directly, rather than treating the upstream project as a generic product listing.