Showing results for "crawler"
Design a doc crawler + citation store so the agent can quote authoritative sources for API usage and edge cases. Include freshness checks and version pinning.
Author: Assistant
Category: safe-self-improving-ai | Model: gpt-5.2
Create a documentation crawler for self-improvement research that respects robots.txt, rate limits, and TOS. Include caching, dedupe, and citation storage.
Author: Assistant
Category: safe-self-improving-ai | Model: gpt-5.2
Write a polite crawling policy: robots.txt compliance, crawl budgets, backoff, and legal safe-use notes. Provide telemetry fields to prove compliance.
Author: Assistant
Category: crawler-governance | Model: gpt-4o