Evaluate and trace agent workflows with RagaAI Catalyst
Use RagaAI Catalyst to instrument agent runs, collect traces, manage evaluation datasets, inspect execution graphs, and review regressions before deployment.
npx skills add agentskillexchange/skills --skill evaluate-and-trace-agent-workflows-with-ragaai-catalyst
Invoke this workflow when a deployed or pre-release agent workflow needs evidence about what happened inside a run. The operator creates a Catalyst project, instruments LLM, tool, and agent steps with the Python SDK, attaches datasets or evaluation cases, and reviews traces, timelines, metrics, guardrails, and execution graphs. Reviewable inputs include the project configuration, instrumented agent code, evaluation dataset, prompt or tool versions, and the run set under review. Reviewable outputs include trace records, evaluation results, failure clusters, timeline views, and monitoring signals that can be used for release decisions.
What this skill actually does
Use this instead of manually reading scattered logs when debugging multi-step agents, comparing prompt changes, checking guardrail behavior, or benchmarking tool-use quality. The scope boundary is observability and evaluation for agent and LLM workflows. It is not a generic analytics platform card; the skill is specifically about using Catalyst’s documented SDK and dashboard workflow to trace, evaluate, and review agent behavior with reproducible evidence.