DocupDocup
Esc

Architecture

How Docup is put together: a Cloudflare Worker for the app and docs sites, Trigger.dev for AI work, and Postgres, Redis and Qdrant for data.

Docup is two programs and five data stores. The first program is a Cloudflare Worker that serves the marketing site, the dashboard, the API and every docs site. The second is a set of background tasks on Trigger.dev that do the slow, AI-heavy work. This page shows how they fit together.

The big picture#

Text
Browser ──► Cloudflare Worker (worker.ts)
              ├─ www.docup.dev   → redirect to docup.dev
              ├─ /ingest/*       → PostHog proxy
              ├─ static files    → Cloudflare Assets (env.ASSETS)
              └─ page cache (Redis) ─► vinext app
                                         ├─ marketing site, login, dashboard, API
                                         └─ docs sites, rewritten by proxy.ts
                                               to /sites/<address>/...

The Worker#

One Worker named docup runs the whole product. It is built with vinext, which implements the Next.js APIs on Vite, and deployed to Cloudflare Workers. It is reachable on two kinds of address:

  • docup.dev, a custom domain, for the app.
  • */*, a wildcard route, for docs sites and custom domains. A custom domain cannot be a wildcard, so docs sites and custom domains come in on this wildcard route pattern.

The Worker entrypoint wraps vinext with several routing and proxy steps:

  1. Redirect www.docup.dev to docup.dev with a 301.
  2. Redirect to custom domains if a docs site has a custom domain configured. This sends search engines and browsers to the custom domain, except for Ask AI bot checks and non-read requests.
  3. Proxy /ingest/* on the app host to PostHog, so product analytics are not blocked by ad blockers. Cookies are stripped and visitor IP is forwarded.
  4. Static assets. Serves requests for static files directly from Cloudflare Assets (env.ASSETS) without invoking the app.
  5. Markdown negotiation for requests with an Accept: text/markdown header, rewriting the path to serve the raw Markdown file instead of HTML.
  6. Page cache and analytics. Pages that are the same for every visitor (docs pages, site files, and marketing pages) are answered from Redis without running the app. To prevent a slow Redis from blocking requests, cache reads have a deadline of 400 ms; if exceeded, the page is rendered from the app instead. It keys different payloads (like hover prefetches versus full page loads) using a SHA-256 hash of client-side navigation headers, and caches marketing pages separately for signed-in and signed-out visitors. It also intercepts POST requests to /docup-view (sent by the client-side navigation beacon) to record views directly to Cloudflare Analytics Engine, answering with a 204 without invoking the app. See Caching.

Everything else goes to vinext. For a docs host, the proxy rewrites the request to a route under app/sites/[subdomain]. See Docs hosting.

Background tasks#

Reading a repository and calling a model can take minutes, far beyond a request. Those jobs run as Trigger.dev tasks, started from the Worker with a scoped secret key.

Task Started by Does
sync-repo Creating a project, or retrying a failed first run Reads the repository, plans the docs, writes every page.
retry-page Retry on a failed page Writes one page again.
update-docs A GitHub push webhook Updates the pages affected by new commits.
index-site Publish, unpublish, page deletion, or a live page updated by a push Rebuilds the search index for Ask AI.
check-domains Cron schedule (every 10 minutes in production) Keeps custom domains in sync with Cloudflare for SaaS.

The browser follows a running sync live through a read-only token scoped to a single run. See Background tasks.

Data stores#

Store Used for
Postgres The source of truth: users, workspaces, projects, pages, runs and site settings. Reached from the Worker through Hyperdrive using a request-cached pool of up to 4 connections, and from tasks directly. See Database.
Redis (Upstash) The page cache (which keeps cached pages for 43,200 seconds), the live-site cache, rate limits, webhook de-duplication and the daily analytics salt.
Qdrant One vector collection holding the indexed sections of every live site, for Ask AI.
R2 Images owners upload: logo, dark logo, favicon and social image. Uses the docup-assets bucket.
Analytics Engine One data point per docs view, recorded during requests by the page cache wrapper and read back through the SQL API. Uses the docup-docs-analytics dataset.

External services#

  • GitHub for repository reads, the GitHub App and sign-in.
  • Google for sign-in, and Vertex AI for the planner, writer, chat and embedding models.
  • Cloudflare Turnstile for the Ask AI bot check.
  • PostHog for product analytics of Docup itself, not of docs sites.

Design principles visible in the code#

  • Postgres is the source of truth. Everything else is a cache or an index that can be rebuilt.
  • Drafts and live versions are separate columns, so publishing is a single transaction.
  • Anonymous endpoints fail closed. Ask AI refuses to run without Redis, because nothing else keeps its cost in check. Signed-in actions fail open on a Redis outage.
  • The filter is the security boundary. Secrets are dropped before any content reaches a model.
  • Tasks do not retry the AI. A retry would pay for every model call again, so a failed run is shown to the user.
Documentation written with Docup