Gate agent regressions from production traces with Tracely
Turn failing AI-agent production traces into hermetic regression cases and CI gates with Tracely.
npx skills add agentskillexchange/skills --skill gate-agent-regressions-from-production-traces-with-tracely
tracely-ai for Python agents or point an existing OTLP/OpenTelemetry exporter at Tracely. Configure an ingest key, tag runs with env=prod and env=ci, promote failures to cases, and run tracely gate, tracely replay, or the Tracely GitHub Action in CI.Use Tracely when an agent or operator needs to instrument AI-agent runs, detect failing traces, promote real failures into replayable regression cases, and block pull requests with the `tracely gate`, `tracely replay`, or Tracely GitHub Action workflow. Invoke this instead of using the product normally when the work is operationalizing trace-derived regression tests for an agent release process, not browsing an observability dashboard. The boundary is agent trace instrumentation, evaluator setup, failure-to-case promotion, and CI gating across Python SDK, OTLP, scenarios, MCP-assisted evaluator work, and GitHub Actions; it is not a generic SDK, observability platform, or dashboard listing.
What this skill actually does
Inputs and prerequisites: Tracely service, tracely-ai Python SDK or OTLP exporter, tracely CLI or Tracely GitHub Action.
Setup notes: Self-host Tracely with Docker Compose or Railway, then install tracely-ai for Python agents or point an existing OTLP/OpenTelemetry exporter at Tracely. Configure an ingest key, tag runs with env=prod and env=ci , promote failures to cases, and run tracely gate , tracely replay , or the Tracely GitHub Action in CI.
Source and verification boundary: use https://doc.tracely-studio.xyz as the canonical reference before running the workflow; keep commands, API calls, CLI usage, and generated outputs reviewable against that upstream source.
Framework fit: publish this as a Multi-Framework workflow only when the operator can invoke the documented toolchain directly, rather than treating the upstream project as a generic product listing.