doc-scraper

SKILL.md

Single responsibility: Convert documentation websites into organized, categorized reference files suitable for Claude skills or offline archives. (BP-4)

| RELEVANT | Target URL, selectors, output path | Unrelated documentation | | PERIPHERAL | Similar site examples for selector hints | Historical scrape data | | DISTRACTOR | Other projects, unrelated URLs | Previous failed attempts |

| No content extracted | Selector mismatch | Inspect page, update maincontent selector | | Wrong pages scraped | URL pattern issue | Check include/exclude patterns | | Rate limited | Too aggressive | Increase ratelimit to 1.0+ seconds | | Memory issues | Too many pages | Add maxpages limit, enable checkpoints |

Scrape documentation websites into organized reference files. Use when converting docs sites to searchable references or building Claude skills. Source: jmagly/ai-writing-guide.

View raw