--- title: Schema Design for Accurate Extraction | Tabstack description: The schema you pass to /extract/json and /generate/json is the most important factor in extraction quality. Learn the patterns that produce reliable results. --- The schema you pass to `/extract/json` and `/generate/json` is the most important factor in extraction quality. It’s not just a shape definition; it’s a set of instructions for the AI doing the extraction. This guide covers the patterns that produce reliable results and the common mistakes that produce noise. --- ## Descriptions are instructions, not documentation Every property description you add tells the extraction AI what to look for and how to interpret it. Without descriptions, the AI has only the property name to go on, and names are ambiguous. - [TypeScript](#tab-panel-216) - [Python](#tab-panel-217) ``` // Ambiguous: AI guesses what 'price' means json_schema: { type: 'object', properties: { price: { type: 'number' } } } // Specific: AI knows exactly what to extract json_schema: { type: 'object', properties: { price: { type: 'number', description: 'Monthly price in USD as a number. Null if pricing requires contacting sales.' } } } ``` ``` # Ambiguous: AI guesses what 'price' means json_schema = { "type": "object", "properties": { "price": {"type": "number"}, }, } # Specific: AI knows exactly what to extract json_schema = { "type": "object", "properties": { "price": { "type": "number", "description": "Monthly price in USD as a number. Null if pricing requires contacting sales.", }, }, } ``` The second version extracts correctly even when the price is presented as `"$49/mo"`, a range, or missing entirely. The description tells the AI how to handle each case. --- ## Be explicit about null cases Pages don’t always have every field. Tell the AI what to return when data is absent. - [TypeScript](#tab-panel-218) - [Python](#tab-panel-219) ``` properties: { annual_discount: { type: ['number', 'null'], description: 'Percentage discount for annual billing (e.g. 20 for 20% off). Null if no annual option is offered.' }, trial_days: { type: ['number', 'null'], description: 'Length of free trial in days. Null if no trial is available.' } } ``` ``` "properties": { "annual_discount": { "type": ["number", "null"], "description": "Percentage discount for annual billing (e.g. 20 for 20% off). Null if no annual option is offered.", }, "trial_days": { "type": ["number", "null"], "description": "Length of free trial in days. Null if no trial is available.", }, } ``` Declaring `null` in the type and describing when it applies is the strongest signal you can give the extractor. It is not a guarantee. Read the next section before you write validation logic against it. --- ## What you get back when a field can’t be filled This is the part to understand before you put `/extract/json` in a pipeline. **A call never fails because the page was missing your data.** You get a `200` and an object shaped like your schema. There is no “no match” status, no partial-match flag, and no list of fields the extractor couldn’t fill. Marking a field `required` does not change that; it makes the extractor try harder, it does not make the call fail. What lands in an unfillable field depends on its type. Observed against the live API on 2026-07-27: | Schema type | Field absent from the page | Same field marked `required` | | ----------- | ------------------------------------------------ | ---------------------------- | | `string` | `null` | `""` | | `number` | `0` | `-1` | | `boolean` | `false` | `false` | | `array` | `[]` | `[]` | | `object` | Object present, leaves filled by the rules above | Same | **Absent strings are reliable. Absent numbers and booleans are not.** A missing number comes back as a real number and a missing boolean as `false`, so neither is distinguishable from a genuine `0` or a genuine `false`. That distinction is the whole risk. If your schema is mostly strings you will see clean `null`s and never notice. Add one number field and you have a silent data-quality bug: ``` // founded_year comes back as 0 for a page that never mentioned a founding year const age = 2026 - data.founded_year; // 2026, not "unknown" // employee_count comes back as 0, so this row silently fails the filter if (data.employee_count > 50) enrich(row); ``` ### Writing pipeline logic against this - **Validate values, don’t just null-check.** Treat `0`, `-1`, `false`, and `""` as suspect for any field that could legitimately be absent from the page. - **Prefer `['number', 'null']` over `number`,** and say in the description when `null` applies. It shifts the odds toward `null`; it does not remove the need to validate. - **Give absence its own field** when a distinction actually matters to you. A `has_pricing` boolean the extractor can affirmatively set is more trustworthy than inferring absence from a `0` in `price`. - **Sanity-check against a known-empty page** while building. Run your schema against a URL you know lacks the fields and record what each one returns, so your validation matches real behavior rather than assumed behavior. ### Malformed requests Schema problems and page problems fail differently: | Request | Response | | ---------------------------------------------- | ------------------------------------------------ | | `json_schema` omitted | `400` `{ "error": "json_schema is required" }` | | `json_schema` isn’t a JSON object | `400` `{ "error": "invalid JSON request body" }` | | `json_schema` contains an unknown type keyword | `200`, field returns `null` | The third row is worth noting: an invalid schema is **not** rejected. A typo like `{ "type": "banana" }` returns `200` with `null` rather than a validation error, so a broken schema fails quietly on every call instead of loudly on the first. Test a new schema against a page you know well before running it at volume. See [Error Reference](/production/error-reference/index.md) for the full error shape. --- ## Match your schema depth to the page structure If the page has a two-level hierarchy (categories containing products), your schema should reflect that: - [TypeScript](#tab-panel-220) - [Python](#tab-panel-221) ``` json_schema: { type: 'object', properties: { categories: { type: 'array', description: 'Top-level product categories on the page', items: { type: 'object', properties: { name: { type: 'string', description: 'Category heading' }, products: { type: 'array', items: { type: 'object', properties: { name: { type: 'string', description: 'Product name' }, price: { type: 'number', description: 'Price in USD' } } } } } } } } } ``` ``` json_schema = { "type": "object", "properties": { "categories": { "type": "array", "description": "Top-level product categories on the page", "items": { "type": "object", "properties": { "name": {"type": "string", "description": "Category heading"}, "products": { "type": "array", "items": { "type": "object", "properties": { "name": {"type": "string", "description": "Product name"}, "price": {"type": "number", "description": "Price in USD"}, }, }, }, }, }, }, }, } ``` A flat schema trying to capture a nested page will produce unreliable results: items get merged and hierarchy is lost. --- ## Use `enum` to constrain categorical fields For fields with a known set of values, `enum` dramatically improves consistency: - [TypeScript](#tab-panel-222) - [Python](#tab-panel-223) ``` properties: { billing_period: { type: 'string', enum: ['monthly', 'annual', 'one-time', 'unknown'], description: 'How often the plan is billed. Use unknown if unclear.' }, tier: { type: 'string', enum: ['free', 'starter', 'pro', 'enterprise'], description: 'Plan tier. Use enterprise for anything requiring sales contact.' } } ``` ``` "properties": { "billing_period": { "type": "string", "enum": ["monthly", "annual", "one-time", "unknown"], "description": "How often the plan is billed. Use unknown if unclear.", }, "tier": { "type": "string", "enum": ["free", "starter", "pro", "enterprise"], "description": "Plan tier. Use enterprise for anything requiring sales contact.", }, } ``` Without `enum`, you’ll get `"monthly"`, `"Monthly"`, `"per month"`, `"billed monthly"`: all meaning the same thing but inconsistent to process downstream. --- ## Keep array item schemas tight Broad array item schemas produce noisy results. The AI will include anything that loosely matches. - [TypeScript](#tab-panel-224) - [Python](#tab-panel-225) ``` // Too broad: captures navigation links, footer links, ads links: { type: 'array', items: { type: 'string' } } // Tight: captures only what you want documentation_links: { type: 'array', description: 'Links to API documentation pages only, not navigation, marketing, or footer links', items: { type: 'object', properties: { title: { type: 'string', description: 'Link text' }, url: { type: 'string', description: 'Absolute URL' } } } } ``` ``` # Too broad: captures navigation links, footer links, ads "links": { "type": "array", "items": {"type": "string"}, } # Tight: captures only what you want "documentation_links": { "type": "array", "description": "Links to API documentation pages only, not navigation, marketing, or footer links", "items": { "type": "object", "properties": { "title": {"type": "string", "description": "Link text"}, "url": {"type": "string", "description": "Absolute URL"}, }, }, } ``` --- ## Add context about the page in a top-level description You can add a `description` at the top level of your schema to give the AI page-level context: - [TypeScript](#tab-panel-226) - [Python](#tab-panel-227) ``` json_schema: { type: 'object', description: 'Pricing information from a SaaS product pricing page. Focus on subscription plans only. Ignore one-time add-ons and enterprise custom pricing blocks.', properties: { plans: { /* ... */ } } } ``` ``` json_schema = { "type": "object", "description": "Pricing information from a SaaS product pricing page. Focus on subscription plans only, ignore one-time add-ons and enterprise custom pricing blocks.", "properties": { "plans": {...}, }, } ``` This is especially useful when a page has multiple sections and you want to scope extraction to a specific one. --- ## Debug extraction issues If results are incomplete or inaccurate: 1. **Add more specific descriptions.** This is the most common fix. 2. **Upgrade `effort` to `max`.** Content may not be in the initial HTML. 3. **Simplify the schema.** Remove fields you don’t need; fewer fields means less for the AI to get wrong. 4. **Add `nocache: true`.** This confirms the issue isn’t a stale cached result. 5. **Check for JS rendering.** Open the URL and disable JavaScript in your browser to see what the extractor sees at `min`/`standard`.