Serve Markdown to LLMs via Accept Header

Overview / Description

Serve Markdown to LLMs via Accept Header is an AI developer tool that uses HTTP content negotiation to deliver clean Markdown to AI agents and LLMs while serving standard HTML to browsers — all from a single canonical URL. It is built for web developers and site owners who want their content consumed natively by AI agents, RAG pipelines, and LLM clients without scraping or preprocessing layers.

The core mechanism is the HTTP Accept header (Accept: text/markdown), as defined in RFC 9110 and RFC 7763. When an AI agent sends this header, the server returns Markdown stripped of navigation, styles, scripts, and layout wrappers — reducing token usage, lowering latency, and raising signal-to-noise for retrieval-augmented generation (RAG). When a browser makes the same request without that header, it receives the usual HTML response. No separate URL, no duplicate content, no scraping proxy required.

The tool ships a live AI-readiness checker that tests any URL for correct Vary: Accept headers, 406 error responses, q-value support, and actual Markdown serving from the edge. A copy-paste configuration library covers thirteen platforms and frameworks — Nginx, Caddy, WordPress, Discourse, Laravel, Rails, Cloudflare Workers, Next.js, Astro, Apache, SvelteKit, Nuxt/Nitro, Express, Go, and Django — so developers can implement content negotiation in minutes. An AI agent support matrix documents which agents already send Accept: text/markdown by default.

The tool appears to be a free, open developer resource with no pricing tiers, sign-up, or account requirements visible on the homepage.

Used For

Web developers and site owners who want existing pages to serve clean, token-efficient Markdown to AI agents and RAG pipelines via standard HTTP content negotiation without separate URLs or scraping.

Pricing

Plan

Free

Free — no sign-up required

View pricing

Pros & Cons

Pros

  • Content negotiation via Accept: text/markdown serves one canonical URL to both browsers (HTML) and AI agents (Markdown) with no duplicate pages
  • Token efficiency: strips nav, styles, scripts, and layout wrappers so LLMs and RAG pipelines consume only prose content
  • Live AI-readiness checker tests any URL for correct Vary: Accept headers, 406 responses, q-value support, and Markdown delivery from the edge
  • Copy-paste server config recipes for 14+ platforms including Nginx, Cloudflare Workers, Next.js, Django, and Rails
  • Standards-based implementation following RFC 9110, RFC 7763, and RFC 8288 — no proprietary protocol required

Cons

  • Requires server-side configuration changes on the origin — static site hosts with no header-routing support cannot implement content negotiation without a proxy layer
  • Not all AI agents send Accept: text/markdown by default; adoption depends on the agent support matrix which may lag behind new agent releases
  • No managed hosting or SaaS layer — implementation is entirely self-serve, which means debugging is on the developer
  • No documentation visible for edge cases like partial Markdown support or fallback behavior when q-values conflict

Questions & Answers

Alternatives

llms.txt, Firecrawl, Jina Reader, Markdownify, r.jina.ai

Serve Markdown to LLMs via Accept Header | AI Tools Directory