Structured LLM extraction with Zod contracts

The pattern

Any LLM feature I ship follows a contract-first pipeline:

const Contract = z.object({
  title: z.string().min(1),
  dueDate: z.string().date().optional(),
  category: z.enum(['exam', 'assignment', 'meeting']),
  confidence: z.number().min(0).max(1),
});

const out = await llm.generate({
  system: 'Return ONLY JSON matching: ' + describe(Contract),
  json: true,
});
const parsed = Contract.safeParse(out);

safeParse fails → the feature degrades gracefully (retry once, then fall back to the human/editorial review queue — like cal-ucr’s manual parse mode), never crashes, never lies.

Rules I trust

1. Contract first. The schema is the product spec. The prompt is filler around it.
2. Refuse, don’t repair. Silent catch → default means a wrong date in a calendar. Loud failure means a ticket. Pick loud.
3. Confidence is a routing field, not a magic number — low-confidence entries go to a human queue, not into prod DB tables.
4. Store the raw output forever (or for the retention window). When the contract changes you’ll want to re-derive, not re-ask.
5. Don’t trust the model with file paths, SQL, or URLs — put them through a second validation layer that treats them like fs.realpath() hostile users.

What about agents?

Tool-calling agents are the same pattern with a loop on top. The loop is easy; the contracts are the work.