4. Structured output
Make the model produce JSON that matches a JSON Schema. The backend is xgrammar 0.2.3, pinned for the whole engine; there is no fallback to another backend.
Generate
schema = { "type": "object", "required": ["city", "population"], "additionalProperties": False, "properties": { "city": {"type": "string", "maxLength": 40}, "population": {"type": "integer", "minimum": 0, "maximum": 100000000}, }, } with LLM("Qwen/Qwen2.5-0.5B-Instruct", config=StokastiskConfig(max_model_len=2048)) as llm: results = llm.generate_structured(["Describe Paris."], schema, max_tokens=96) print(results) # [{'city': 'Paris', 'population': 20000000}]
generate_structured returns one parsed object per prompt, already validated against the schema.
| Raises | When |
|---|---|
StructuredOutputError |
an output is not valid JSON, violates the schema, or was cut off at max_tokens |
EngineError |
the backend rejected the schema or request |
TypeError / ValueError |
prompts that are not non-empty strings |
Bound open-ended fields
A grammar keeps the model inside the schema but cannot make it stop. An unbounded integer or string can run to max_tokens, and the output is then reported as truncated. Always set maxLength, maximum, maxItems.
Validation helpers (no GPU needed)
from stokastisk import validate_json_output from stokastisk.structured import check_schema, json_schema_params check_schema(schema) # returns the schema or raises StructuredOutputError obj = validate_json_output('{"city": "Rome", "population": 2800000}', schema) params = json_schema_params(schema, max_tokens=96) # vllm SamplingParams for llm.generate(...)
validate_json_output never returns invalid data: it raises StructuredOutputError.
Next: OpenAI-compatible server.