Run TypeScript-native LLM eval suites with Evalite
Use Evalite when a TypeScript team needs repeatable Vitest-style eval runs for LLM apps before prompt, model, or workflow changes ship.
npx skills add agentskillexchange/skills --skill run-typescript-native-llm-eval-suites-with-evalite
Evalite is a TypeScript-native evaluation runner for LLM-powered apps. It is built around test-like eval files, datasets, scorers, and local reports so teams can treat model behavior changes like repeatable software tests.
What this skill actually does
Use this skill when an operator needs to define example cases, run an LLM or agent function against them, score outputs, inspect failures, and compare changes before altering prompts, models, retrieval behavior, or agent logic. It is especially useful when the surrounding application and CI workflow already live in TypeScript.
The boundary is repeatable eval execution for TypeScript LLM applications. It is not a general LLM observability platform, not a hosted product card, and not a generic testing library listing; the operator workflow is to install Evalite, write eval cases and scorers, run the eval suite, and review the report as a release gate.
Source evidence: upstream describes Evalite as a TypeScript-native, Vitest-based eval runner for LLM-powered apps, with exact npm package adoption mapped to the same GitHub repository.