Serve Markdown to AI Agents with HTTP Accept Headers
Serve Markdown to AI Agents with Accept Headers
A new site, AcceptMarkdown.com, advocates serving Markdown versions of web pages to AI agents via HTTP content negotiation. By honoring the `Accept: text/markdown` header, sites can reduce token usage, improve retrieval quality, and cut latency for AI clients. The site offers quick-start guides, copy-paste configs for popular servers and frameworks, and a compatibility matrix showing which AI agents send the header.
Agents spend context on your prose, not your DOM.
- lo_fye
Please don’t tell me we are willing to make it easier for AI to read our content than we are to make it easier for humans to read our content.
Better idea: make your UI always prioritize the content.
- k1m
I agree with Roy Fielding on this:
> It is a bad design trade-off to send a bunch of header fields on every request just to tell the server all of the possible variations of preference held by the user, particularly when there is a very small chance that any of those dimensions are applicable to the target resource. It has been a bad design trade-off ever since the very brief period in 1993-94 when folks didn't know which image format would be usable on all UAs and there was no CSS or javascript to allow for client-side adaptation.
> ...The caching impact of proactive negotiation is far worse than the one extra round trip per site for reactive negotiation, and even that round-trip isn't necessary in formats that support client-side adaptation.
On the caching impact, Simon Willison wrote:
> ...you can’t deploy an application that uses content negotiation via the Accept header behind the Cloudflare CDN — for example serving JSON or HTML for the same URL depending on the incoming Accept header. If you do, Cloudflare may serve cached JSON to an HTML client or vice-versa.
Note: I posted this in another comment with links to those two quotes which I couldn't copy easily now - will add later.
- lekevicius
I'll do that once any of the top 4 AI chatbots says they'll start making requests with this header. Before that it's just a neat idea with no adoption.
I also think it's exceedingly unlikely that any of the top 4 chatbots would choose to load websites this way. Too many risks for a 0.01% adoption even years later.
- collimarco
What about clean, semantic HTML?
It was already optimized for bots and search engines (which are bots) and it has been used for decades. Why we need to serve in markdown now?
There are also many parts of the HTML, like navs, that are useful for bots and AI and may be removed in the markdown version.
- joshum97
I feel like I am going insane. Who in their right mind would feed an LLM raw HTML in the first place??
HTML is a markup language. User agents present it in a way that makes sense for the user—visually, or through assistive technology. Nothing about adding LLM “users” should change this—their user agent, the harness, should present HTML in a way they can natively understand, by translating it to Markdown.
We are not going to rewrite the entire web because harness developers are too lazy or stupid to pull an HTML to markdown package from npm. If some sites want to do so anyway, good for them, in many cases I’d love to skip the CSS/JS and read the markdown directly (or better, nicely formatted). But don’t blame website authors for your harness wasting your tokens.
- theletterf
Seeing all the questions in this thread, I strongly recommend Dachary Carey's post on the topic of agent-ready docs: https://dacharycarey.com/2026/02/18/agent-friendly-docs/
- kaangiray26
hoping for this to get mainstream so that I can just view the pages without any ads, js and bloat
- MiroslavPokorny
Why would any website want to put extra effort to "give" their content as markdown to AI and get nothing in return ?