mmdingest

article → markdown api

Articles to clean Markdown.
For LLMs, RAG, or reading.

Paste a URL from Medium, Dev.to, or Substack. Get clean Markdown with YAML frontmatter: title, author, dates, tags, cover image.

Ingest an articleView source
curl
$ curl "https://mdingest.knightker.workers.dev/v1/medium?url=https://medium.com/@user/article"

---
title: How to Build APIs
author: Jane Doe
published: 2024-03-15
reading_time: 8 min
tags: [api, backend]
---

# How to Build APIs

Full article body in clean Markdown...

the problem

Web pages are noisy. LLMs need clean text.

No paywalls

Medium articles via Freedium mirror. Full content, not truncated previews.

No UI noise

Strips ads, nav, sidebars, popups, cookie banners. Just the article body.

Structured metadata

YAML frontmatter with title, author, dates, reading time, tags, cover image.

RAG-ready

Clean Markdown. Drop it into your vector DB, LLM context, or CI.

supported sources

Three sources, one API

Medium/v1/mediumFreedium mirrorDev.to/v1/devtoForem APISubstack/v1/substackPublic API

api

Three ways to ingest

Same logic, four entry points. HTTP API for your code, CLI for your terminal, MCP for your AI tools.

HTTP API

for your code, RAG pipeline, or CI
javascript
const res = await fetch(
  "https://mdingest.knightker.workers.dev/v1/devto?url=https://dev.to/user/post"
);
const { metadata, markdown } = await res.json();

CLI

pipe-friendly, markdown to stdout
terminal
$ bun run src/cli.ts https://dev.to/user/post > article.md
# --json for {metadata, markdown}
# --provider medium to override

MCP server

for Claude, Cursor, and other AI tools
mcp
# Remote (zero setup):
{ "url": "https://mdingest.knightker.workers.dev/v1/mcp" }
# Tools: ingest_article, list_providers

Read the API docs →

faq

Common questions

mdingest is an API that converts blog, article, and newsletter pages from Medium, Dev.to, and Substack into clean Markdown with YAML frontmatter. It strips ads, navigation, and UI noise so the output is ready for LLM ingestion, RAG pipelines, or reading.

Three sources: Medium (via Freedium mirror, 42 accepted domains including custom domains like itnext.io and towardsdatascience.com), Dev.to (via public Forem API), and Substack (via public API, free posts only).

Yes. The default output is clean Markdown with structured YAML frontmatter (title, author, dates, reading time, tags, cover image). Drop it directly into your vector database, LLM context window, or CI pipeline. Use format=json to get { metadata, markdown } as separate fields.

Yes. 30 requests per minute per IP. When exceeded, the API returns HTTP 429 with { code: "RATE_LIMITED", details: { retryAfter } } where retryAfter is seconds until the limit resets.

Yes. The entire codebase is MIT licensed and on GitHub. Deploy it on Cloudflare Containers (Worker + Docker container), or run it locally with bun run dev. The CLI and MCP server work standalone without the HTTP server.

Yes. Medium articles are fetched via the Freedium mirror, which provides full article content regardless of Medium's paywall. The dual-source approach combines the download endpoint (markdown body, tags, dates) with the __data.json endpoint (author, reading time, cover image).

No. Substack's public API only returns full content for free posts. Paid posts are server-side truncated, so the API returns SUBSTACK.PAID_POST (403) when a post is behind the paywall. Only free Substack posts can be converted.

source code

MIT licensed. Self-hostable.

The entire codebase is on GitHub. Deploy it on Cloudflare, run it locally, or just read how it works.

github.com/LOsioChico/mdingest →