Skip to content

← projects

Date:
2025 – present
Status:
[live] indexfast.co
Stack:
Next.js 16, Cloudflare Workers (OpenNext), Neon Postgres, Drizzle, MCP
Role:
Founder; sole engineer

Subject: IndexFast: indexing work an AI agent can run end to end

New pages on fast-moving sites often sit undiscovered for weeks. IndexFast watches a site's sitemaps, finds the URLs search engines haven't picked up, and pushes them through the channels the engines officially accept. A person sets it up once; after that an agent in Claude Code or Cursor can run it.

The problem

Programmatic-SEO sites, directories and AI-built apps publish pages faster than crawlers visit them. Most "instant indexing" tools paper over this with grey-hat tricks. I wanted the honest version: send each URL to every engine that accepts submissions, and keep a record of what each engine answered.

What I built

  • Ownership by key file: adding a domain generates an IndexNow key, and the site hosting /<key>.txt is the proof. Nothing is submitted for an unverified site.
  • A Cloudflare Worker cron that syncs sitemaps and drains a submission queue, with retries that respect each engine's throttling.
  • Four ways in, one core: a dashboard, a REST API with scoped idxf_ keys, a CLI published to npm, and an authenticated MCP server (Streamable HTTP). The MCP layer only translates calls; business logic lives in one operations module.
  • Prepaid plans on Razorpay, and free public SEO tools that need no sign-up.

Decisions

  • Google's Indexing API is used only for the page types Google allows (job postings and livestreams). The product never promises Google indexing.
  • The MCP server negotiates protocol versions from 2024-11-05 to 2025-11-25 and returns tool failures as isError results instead of transport errors, so an agent can read the failure and react.
  • Workers have 128 MB and short CPU budgets, so sitemap parsing and submission run in small batches.

What I took from it

Building for agents first changes the API. Every response has to explain the next step in plain words, because the caller is a model reading it, not a person looking at a screen.