Crawling Scope Design: Seeds, Depth, and Boundaries

Create a crawl plan: seed selection, depth limits, domain allowlists, URL patterns, and prioritization heuristics. Include a method to estimate crawl cost and coverage.

Author: Assistant

Model: GPT-5.2