Audit performance claims with Pstack benchmark-checklist
Check tuning, bottlenecks, correctness, repeatability, and end-to-end relevance before an agent reports a measured speedup or regression.
npx skills add agentskillexchange/skills --skill audit-performance-claims-with-pstack-benchmark-checklist
Use Pstack’s benchmark-checklist skill before reporting a measured speedup, regression, or performance-based tool choice. The agent audits the claim against executable evidence and returns a verdict with the measurement’s limits.
What this skill actually does
Installation and invocation
Run npx skills add backnotprop/pstack. In the interactive selector choose benchmark-checklist and your supported agent. Upstream documents this skill as usable independently; installing the entire Pstack suite is unnecessary. Explicitly invoke the installed skill with your claim, measurement script, before/after revisions, representative data, and existing run results. Command menus vary by host.
Workflow
- State the exact performance claim and inspect what the harness measures and omits. Record machine load and available cores.
- Identify the bottleneck in a separate profiling run. Check production-equivalent settings, versions, datasets, cache conditions, and hardware limits before comparing alternatives.
- Verify successful outputs and confirm the intended work occurred inside the timed region. Count errors and retries.
- For comparisons, alternate the two configurations for at least five runs each. Report median and range; investigate differences smaller than observed variation. Connect microbenchmarks to the end-to-end user path.
- Return a faster, slower, no-measurable-difference, or inconclusive verdict with units, run count, spread, bottleneck evidence, and unresolved gaps. Keep profiling overhead out of reported timings.
Scope
This workflow audits a specific performance claim. It does not supply a benchmark engine, CI service, or general coding framework. Use it with the project’s existing harness and suitable profiling tools. If comparative tuning, the bottleneck, correctness, or actual execution cannot be established, disclose the gap and retain an inconclusive verdict. A requested rough estimate may use one explicitly labelled run, while still checking errors and actual execution.
Source: the inspected Pstack benchmark-checklist instructions. Metadata and documented invocation were verified; no runtime or security review is implied.