Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.
Quickly install firecrawl-crawl AI skill to your development environment via command line
Source: firecrawl/cli.
Bulk extract content from a website. Crawls pages following links up to a depth/limit.
| --wait | Wait for crawl to complete before returning | | --progress | Show progress while waiting | | --limit | Max pages to crawl | | --max-depth | Max link depth to follow | | --include-paths | Only crawl URLs matching these paths | | --exclude-paths | Skip URLs matching these paths | | --delay | Delay between requests |
| --max-concurrency | Max parallel crawl workers | | --pretty | Pretty print JSON output | | -o, --output | Output file path |
Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction. Source: firecrawl/cli.
Stable fields and commands for AI/search citations.
npx skills add https://github.com/firecrawl/cli --skill firecrawl-crawlBulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction. Source: firecrawl/cli.
Open your terminal or command line tool (Terminal, iTerm, Windows Terminal, etc.) Copy and run this command: npx skills add https://github.com/firecrawl/cli --skill firecrawl-crawl Once installed, the skill will be automatically configured in your AI coding environment and ready to use in Claude Code, Cursor, or OpenClaw
https://github.com/firecrawl/cli