- Date:
- 2025 – present
- Status:
- [live] indexfast.co
- Stack:
- Next.js 16, Cloudflare Workers (OpenNext), Neon Postgres, Drizzle, MCP
- Role:
- Founder; sole engineer
Subject: IndexFast: indexing work an AI agent can run end to end
New pages on fast-moving sites often sit undiscovered for weeks. IndexFast watches a site's sitemaps, finds the URLs search engines haven't picked up, and pushes them through the channels the engines officially accept. A person sets it up once; after that an agent in Claude Code or Cursor can run it.
The problem
Programmatic-SEO sites, directories and AI-built apps publish pages faster than crawlers visit them. Most "instant indexing" tools paper over this with grey-hat tricks. I wanted the honest version: send each URL to every engine that accepts submissions, and keep a record of what each engine answered.
What I built
- Ownership by key file: adding a domain generates an IndexNow key, and the site hosting
/<key>.txtis the proof. Nothing is submitted for an unverified site. - A Cloudflare Worker cron that syncs sitemaps and drains a submission queue, with retries that respect each engine's throttling.
- Four ways in, one core: a dashboard, a REST API with scoped
idxf_keys, a CLI published to npm, and an authenticated MCP server (Streamable HTTP). The MCP layer only translates calls; business logic lives in one operations module. - Prepaid plans on Razorpay, and free public SEO tools that need no sign-up.
Decisions
- Google's Indexing API is used only for the page types Google allows (job postings and livestreams). The product never promises Google indexing.
- The MCP server negotiates protocol versions from 2024-11-05 to 2025-11-25 and returns tool failures as
isErrorresults instead of transport errors, so an agent can read the failure and react. - Workers have 128 MB and short CPU budgets, so sitemap parsing and submission run in small batches.
What I took from it
Building for agents first changes the API. Every response has to explain the next step in plain words, because the caller is a model reading it, not a person looking at a screen.