Documentation that CI can fail

Your docs are lying to you.

The code changed. The docs didn’t. docattest fails the build at the exact line that stopped being true.

docs/retries.md

Retried `3` times. <!-- fact: RETRY.maxAttempts -->
Waits `200` ms. <!-- fact: RETRY.backoffMs -->

terminal

$ npx docattest check
docs/retries.md
  ✗ L1 fact: expected 3, got 5 from RETRY.maxAttempts
  ✓ L2 fact: RETRY.backoffMs is 200
results: 1 pass, 1 fail, 0 error, 0 skipped
FAILED
$ npx docattest check
docs/retries.md
  ✓ L1 fact: RETRY.maxAttempts is 5
  ✓ L2 fact: RETRY.backoffMs is 200
results: 2 pass, 0 fail, 0 error, 0 skipped
ok
Dale, a grumpy yellowed page, sweating Paige, a happy page with a green check
  • “Retries 3 times.” It’s 5.
  • “Two regions supported.” Count again.
  • “Returns null on failure.” It throws now.
  • “Default timeout is 30s.” Cute.
  • “This option is required.” Since when?
  • “See the full list below.” That list is missing three.

How a lie gets caught

  1. You write a sentence

    Plus a comment saying where the truth lives.

    Retried `3` times.
    <!-- fact: RETRY.maxAttempts -->
  2. The code changes

    Nobody opens the docs. Nobody ever does.

    maxAttempts: 3 5
  3. The build says no

    It names the page and the exact line that’s now wrong.

    ✗ L1 fact: expected 3, got 5

Five ways to prove a sentence

  • Example```ts checkfails if it throws
    ```ts check
    import { retry } from 'acme-client';
    assert.equal(await retry(async () => 'ok'), 'ok');
    ```
    Try it in the playground →
  • Fact<!-- fact: expr -->fails if page and code disagree
    Retried `3` times. <!-- fact: RETRY.maxAttempts -->
    Try it in the playground →
  • Cases<!-- cases: runner -->fails if a table row is wrong
    <!-- cases: backoff -->
    | case       | attempt | result |
    |------------|---------|--------|
    | first try  | 1       | 200    |
    | third try  | 3       | 800    |
    Try it in the playground →
  • Citation<!-- test: file > "name" -->fails if the test is gone
    Throws the last error. <!-- test: test/retry.test.ts > "throws the last error" -->
    Try it in the playground →
  • Snapshotsnapshot("name")fails if outside data drifts
    Regions: <!-- fact: snapshot("regions").map(r => r.code) -->
    - `eu`
    - `us`
    How snapshots work →

The crew

Gate guards the build

Run these in CI. Exit 1 turns the build red.

  • Runs every proof on every page. A failure names the page, the line, and what the code says instead.

    ✗ L10 fact: expected 3, got 5 from RETRY.maxAttempts
    FAILED
  • Also compares with main, so deleting a proof to get a green build fails while its sentence is still there.

    ✗ docs/retries.md:10 orphaned-claim: "Tried at most `3` times." is still on the page, but the fact that proved it … is gone

Snoop answers questions

Run these while you write, before you push.

  • The sentences your uncommitted change might have made untrue: re-read these.

    RETRY
      docs/retries.md:10 proven fact  Tried at most `3` times.
  • Every sentence that describes a symbol, before you rename or change it.

    docs/retries.md:10 fact  Tried at most `3` times.
  • Lists and tables copied from the code with no proof, and the proof to add.

    ? L31 list drifts from ERROR_CODES (missing: NO_SESSION) → <!-- fact: ERROR_CODES --> above the list
  • Exported code that no page mentions yet: your docs to-do list.

    ERROR_CODES

Bolt wires up your tools

Run these once. They keep working after that.

  • Hooks for Claude Code, Cursor or Gemini CLI, so the agent can’t finish while a proof fails. Where the agent allows it, it also hears which sentences to re-read after each edit.

    PostToolUse (Edit|Write): npx --no docattest impact --hook
    Stop: npx --no docattest check --hook
  • A pre-commit hook: a commit that makes the docs wrong never lands.

    pre-commit: npx --no docattest check --quiet || exit 1

Make your docs earn it

  1. Install
  2. Configure
    // docattest.config.json
    {
      "pages": ["docs/**/*.md"],
      "surfaces": { "default": "./src/index.ts" },
      "adapter": "typescript"
    }
  3. Prove a sentence
    Retried `5` times. <!-- fact: RETRY.maxAttempts -->
  4. Run it