Diagnose incomplete web extraction with PageSift MCP audit evidence
Triage missing web content across local agents using PageSift MCP audit records, HTML snapshots, and local extraction issue reports.
npx skills add agentskillexchange/skills --skill diagnose-incomplete-web-extraction-with-pagesift-mcp-audit-evidence
Use this workflow when an agent returns missing body text, empty tables, or lost headings and several local coding agents need a shared extraction service with evidence for diagnosing the failure. The operator configures PageSift once, inspects the extraction audit trail, and records a reproducible local issue before relying on the captured content.
What this skill actually does
Prepare the shared service
Install the standalone binary with cargo install pagesift --locked --version 0.81.2 and run pagesift doctor to check available engines. Configure cache, browser and per-host limits in ~/.config/pagesift/pagesift.toml. The upstream example caps cache at 2048 MB, uses two browsers, and sets a 1000 ms per-host interval. Start pagesift mcp http --port 3003 --audit-log ~/.local/state/pagesift/audit.jsonl. Connect an MCP client to http://127.0.0.1:3003/mcp; Claude Code can register it with claude mcp add --scope user --transport http pagesift http://127.0.0.1:3003/mcp.
Diagnose incomplete extraction
Ask the connected agent to retrieve the target with pagesift_scrape and check the returned content against the requested body, table or headings. When content is missing, inspect the matching audit record: URL, requested and actual engine, HTTP status, text character count, warnings and snapshot path. Open the saved HTML snapshot when present to distinguish source content from extraction loss. With auditing enabled, call pagesift_report_issue with the URL, expected content and observed result; this appends a local report rather than sending an upstream issue. Return the affected URL, missing material, warning/error codes, engine and evidence paths, and state whether the result is usable.
Scope and operating limits
This skill covers shared-service setup and extraction failure triage. It does not cover general crawling platform deployment, browser interaction, or automatic upstream issue submission. Keep the service on its documented loopback listener. Clients share concurrency and per-host limits; crawl jobs are held in memory and disappear on restart. A data directory already held by another PageSift process prevents shared caching. Respect ROBOTS_DISALLOWED results and use an appropriate permitted source. Audit records redact credential-like query values by default, but saved HTML snapshots contain page material and need local access controls. Repository adoption and package download counts establish metadata signals; no runtime or security certification is implied.