Spokes.wiki Search About
Software Source Code source ↗ source url updated Wed Aug 05 2026 00:00:00 GMT+0000 (Coordinated Universal Time)

PullMD

A self-hosted URL- and file-to-Markdown service (aeterna-labs, AGPL-3.0, 365★/28 forks; created 2026-04-27, last pushed 2026-08-01 on the v3.x line) — “for humans and AI agents.” Web pages, documents, images, audio and YouTube in; clean Markdown out. Routed here 2026-08-05.

The extraction path is a cascade rather than one library: Mozilla Readability and Trafilatura for static HTML, Cloudflare’s native Markdown where the origin offers it, and headless Chromium via Playwright when the page needs JavaScript. v3 adds document conversion (MarkItDown), image captioning and audio transcription through an OpenAI-compatible endpoint, and YouTube transcripts. Node/Express with Python FastAPI sidecars, SQLite cache, Docker, a PWA front end, SSRF protection with a CIDR/hostname allowlist, and Reddit/Hacker News thread detection that preserves comment trees.

Tier: T2 — first-party README, primary on itself. No measurements of extraction quality against any alternative, which is the whole competitive claim in this category.

Strip-at-capture grows a server and an invocation surface

The spoke’s token-budget corner gained its fifth lever three days ago with md-this-page: strip the page before it becomes context, using Readability, upstream of any harness. That page recorded a specific limit — “Not an agent tool. Nothing autonomous consumes its output; there’s no MCP surface, no CLI, no way for a harness to invoke it.”

PullMD is that same lever with the limit removed. Its README carries a whole AI-agent integration section: a MCP server, a Claude Code skill that activates on web-reading requests, a universal prompt for other clients, and OAuth 2.1 for the claude.ai web connector. So the strip happens on a server an agent calls, not in a browser a person drives.

That is a real difference in kind, not just packaging. MD-This-Page is a human deciding what not to paste — the manual, pre-agentic form of the discipline. Here the agent decides what to fetch and the stripping is a service it invokes, which puts capture-time reduction inside the loop for the first time in this corner. It also inherits the corner’s standing weakness: nothing here is measured. The category’s claims about cleaner output and fewer tokens remain first-party across every entry.

The cascade is the design

Worth separating from the feature list, because it is the part other tools in this corner do not have. md-this-page is Readability alone. PullMD tries Readability, Trafilatura, an origin-provided Markdown, and a real browser, in order — an admission that no single extractor works across the web, and that the honest architecture is a fallback chain with a browser at the bottom. Whether the chain picks well is exactly what nobody has measured.

Adjacency: the parked scraping cluster

_inbox/firecrawl.md holds a closely related tool (URL → clean Markdown, MCP server, LLM-optimized output) parked on 2026-06-23 under web-data-extraction, on the reasoning that extraction infrastructure is not agent tooling. This ingest sits awkwardly against that decision and the park record now says so. The line drawn here: Firecrawl is a crawling-and-scraping platform that offers an agent interface, PullMD’s stated purpose and README structure put agent consumption first. That distinction is thin, and if a third such tool arrives the web-data-extraction cluster is the better home for all three.

md-this-page · text-first-agent-design · context-rot · model-context-protocol · headroom · safe-prompt-pruning · aeterna-labs