Sanity Challenge · built on Sanity Context

Your apiVersion is a promise. Pinned reads the
fine print.

Paste a Sanity client config. See what it does today at the version you pinned, what silently changes when you bump it, and where Sanity’s own docs disagree with themselves, with every claim cited.

Run the demo opsWhy this exists
14
rules as content
15/15
eval cases exact
3
docs findings
pinned-snowy.vercel.app/analyze
1import {createClient} from 'next-sanity'
2 
3export const client = createClient({
4  projectId: process.env.NEXT_PUBLIC_SANITY_PROJECT_ID,
5  dataset: 'production',
6  apiVersion: '2024-01-01',
7  useCdn: false,
8  token: process.env.SANITY_API_READ_TOKEN,
9  // perspective: (not set)
10})
Parse Rules Evaluate Explain
criticalline 6Docs disagree here
Unpublished drafts are served to your site

Token + apiVersion before 2025-02-19 + no perspective → default is raw, so drafts.* reach production.

apiVersion time machine2025-02-19

Bump here → drafts-leak-raw-default goes away

14 version-change rules, stored as Sanity documents. Drag to browse.

warning
apiVersion is computed from the current date
apiVersion-dynamic
warning
No apiVersion: client falls back to v1
apiVersion-missing
warning
apiVersion comes from an env var with no fallback
apiVersion-unresolved-env
warning
Experimental API version vX
apiVersion-vX
warning2025-02-19
Queries for drafts.* stop returning drafts after a version bump
bump-hides-drafts
critical2025-02-19
Unpublished drafts are served to your site
drafts-leak-raw-default
warning
drafts perspective with the CDN not disabled
drafts-requires-no-cdn
info
listen() without includeAllVersions
listen-include-all-versions
warning
previewDrafts was renamed to drafts
previewDrafts-deprecated
warning2025-02-19
Empty-string projection key changes meaning
projection-empty-string
info2025-02-19
Bumping adds Content Release versions to raw results
raw-bump-adds-versions
warning2025-02-19
raw also returns Content Release versions
raw-now-includes-versions
warning2025-02-19
Studio useClient can’t see Content Releases
useClient-releases
warning2025-02-19
Release queries return nothing at this API version
versions-invisible-pre-2025

The 20-second problem

Three ordinary lines. Your unpublished drafts are live.

This is the client from countless 2024 tutorials. It sends a token, pins a version before 2025-02-19, and never sets a perspective. At that version the default perspective is raw, and authenticated raw queries return drafts.* documents next to published ones.

Bump the version and the default flips to published. The leak stops, and any preview code built on it silently goes blank. Nothing in your code changed.

sanity/lib/client.tscritical
1import {createClient} from 'next-sanity'
2 
3export const client = createClient({
4  projectId: process.env.NEXT_PUBLIC_SANITY_PROJECT_ID,
5  dataset: 'production',
6  apiVersion: '2024-01-01',
7  useCdn: false,
8  token: process.env.SANITY_API_READ_TOKEN,
9  // perspective: (not set)
10})
Now · 2024-01-01

Default raw. Drafts are served to your production site.

After bump · 2025-02-19

Default published. The leak stops, and drafts vanish from anything that relied on them.

The thesis

Some contradictions in the docs aren’t errors. They’re versions.

A Knowledge Base build flags conflicting claims and asks which one is ground truth. For a versioned API that’s the wrong question. Both of these are true, on either side of one date:

2025-02-19

“The default perspective is raw.”

True for every API version before 2025-02-19.

“The default perspective is published.”

True from 2025-02-19 onwards.

So Pinned models each boundary as structured data, uses Knowledge Base Instructions to keep the build from collapsing them, and reserves “wrong” for claims that are wrong at every version.

How it works

Rules are content. Findings are computed. The model only explains.

A keyword search can find the page about perspectives. It can’t evaluate this against your code:

callSite in createClient,withConfigANDtoken is presentANDapiVersion < 2025-02-19ANDperspective is unset

That rule is a document in the dataset. So is every source it quotes. See all rules.

  1. 1

    Parse

    Your code becomes a Babel AST, then typed facts: apiVersion, perspective, token, useCdn, call sites, queries, packages.

  2. 2

    Load rules

    14 version-change rules live in a public Sanity dataset as documents with typed conditions, boundaries, and quoted sources.

  3. 3

    Evaluate

    A deterministic engine fires a rule only when every condition holds. The model never decides what’s wrong.

  4. 4

    Explain

    An agent on Sanity Context MCP reads a Knowledge Base built from Sanity’s docs and changelog, and cites the entries it used.

Docs findings

Where Sanity’s docs disagree with themselves

Found while building the Knowledge Base and checked by hand on the live pages. Each one is a docsFinding document linked to the rules it affects. Version-scoped conflicts get an Instruction rather than a winner.

Stalestatus: triaged

Sanity Context + Vercel AI SDK guide uses the pre-v7 system option

Claim A
The guide installs the latest ai package and passes system: systemPrompt to generateText.Connect Sanity Context with Vercel AI SDK ↗
Claim B
AI SDK 7 renamed the top-level system option to instructions (a codemod rename-system-to-instructions exists).AI SDK: Migrate 6.x to 7.0 ↗

Resolution · Version-scoped by package, not API: system is correct for ai@6, instructions for ai@7. Suggested fix: pin the install to a major version in the guide, or update the example to instructions.

Version-scopedstatus: triaged· affects drafts-leak-raw-default, bump-hides-drafts, apiVersion-missing

Perspectives page calls raw the default after saying the default changed

Claim A
The raw example is annotated perspective: 'raw', // default value, optional, and the prose says the query uses “the default raw perspective (explicitly set in this example but can safely be omitted)”.Perspectives for Content Lake ↗
Claim B
A Gotcha at the top of the same page: “With the release of API version 2025-02-19, the default perspective changed from raw to published.”Perspectives for Content Lake ↗

Resolution · Claims about the default perspective are version-scoped. Before API version 2025-02-19 the default is raw; from 2025-02-19 it is published. Always state the boundary; never present either claim as universally true. Suggested docs fix: annotate the example // default before v2025-02-19 and note that omitting it now returns published content only.

Wrongstatus: triaged· affects versions-invisible-pre-2025

v2025-02-19 changelog example uses partOfReleases (and is missing a ])

Claim A
The function is listed as sanity::partOfRelease(releaseID), but its example is *[sanity::partOfReleases("rel-summer"), which has an extra “s” and no closing bracket.Content Lake v2025-02-19: Content Releases APIs and new perspective defaults ↗
Claim B
The GROQ functions reference documents the function as sanity::partOfRelease.GROQ functions reference ↗

Resolution · The function is sanity::partOfRelease(releaseId). Treat the changelog example as a typo; copying it fails to parse. Suggested fix: *[sanity::partOfRelease("rel-summer")].

Evaluation

15 configs, three arms

Five old tutorials, four Studio plugins, three correct modern configs, three tricky ones. The same review prompt is given to a model alone, to a model with the Knowledge Base, and to Pinned.

ArmPrecisionRecallExactBoundary date rightCitedFalse criticals
Model alone80%71%10/155/60/150
Model + Knowledge Base45%82%3/157/715/150
Pinned (rules + engine)100%100%15/1510/1015/150

Honest caveat: the cases were written alongside the rules, so Pinned’s row shows the engine matches its documented rules, not held-out accuracy. Boundary dates in Pinned’s row come from rule documents, so they’re correct by construction. Full per-case table in evals/results.md.

Check the client you shipped last year.

No login. Nothing stored. Findings in under a second, with explanations cited from Sanity’s docs.

Open PinnedQuery the public dataset ↗