MD Crawl

A website that crawls URLs recursively and outputs markdown content optimized for LLM consumption.

• Input a URL to crawl
• Set crawl depth (default: 3)
• Crawls only URLs on the same domain
• Outputs a markdown representation of all pages
• Includes a sitemap at the top for better context
• Copy button with success state

• backend/ - Server-side code
  • index.ts - Main entry point with Hono API
  • crawler.ts - Website crawling logic
• frontend/ - Client-side code
  • index.html - Main HTML template
  • index.tsx - Frontend React code
  • components/ - React components