Structured Output

Overview

Structured output allows you to enforce that the AI model's response follows a specific JSON schema. Instead of receiving freeform text, you can guarantee that responses conform to a predefined structure, making it easier to parse and integrate AI responses into your applications. LlmTornado supports both simple JSON mode and strict structured output with JSON schemas.

Quick Start

Here's a basic example of getting structured JSON output:

csharp

using LlmTornado;
using LlmTornado.Chat;
using LlmTornado.Chat.Models;

TornadoApi api = new TornadoApi("your-api-key");

Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.OMini,
    ResponseFormat = ChatRequestResponseFormats.Json
});

chat.AppendSystemMessage("Solve the math problem given by user, respond in JSON format.");
chat.AppendUserInput("2+2=?");

ChatRichResponse response = await chat.GetResponseRich();
Console.WriteLine(response.Content);
// Output: {"result": 4, "problem": "2+2"}

Prerequisites

Basic understanding of JSON and JSON schemas
Familiarity with chat basics from the previous documentation
A model that supports structured output (most modern models do)
Understanding of C# object serialization

Detailed Explanation

Structured Output Modes

LlmTornado offers several ways to control response formatting:

1. JSON Mode

Forces the model to respond with valid JSON, but without enforcing a specific structure:

csharp

ChatRequest request = new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O,
    ResponseFormat = ChatRequestResponseFormats.Json
};

2. Structured JSON with Schema

Enforces a specific JSON schema for the response:

csharp

ChatRequest request = new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O240806,
    ResponseFormat = ChatRequestResponseFormats.StructuredJson("weather_data", new
    {
        type = "object",
        properties = new
        {
            city = new { type = "string" },
            temperature = new { type = "number" },
            conditions = new { type = "string" }
        },
        required = new List<string> { "city", "temperature" },
        additionalProperties = false
    })
};

3. Text Mode (Default)

Standard text response without format restrictions:

csharp

ChatRequest request = new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O,
    ResponseFormat = ChatRequestResponseFormats.Text
};

How It Works

When you specify a structured output format:

Schema Definition: You provide a JSON schema defining the expected structure
Model Constraint: The AI model is constrained to generate only responses matching this schema
Validation: LlmTornado ensures the response conforms to the schema
Parsing: You can safely deserialize the response into your C# objects

Key Components

ChatRequestResponseFormats

Static class providing factory methods for response formats:

csharp

// Simple JSON mode
ChatRequestResponseFormat jsonFormat = ChatRequestResponseFormats.Json;

// Structured JSON with schema
ChatRequestResponseFormat structuredFormat = ChatRequestResponseFormats.StructuredJson(
    "schema_name",
    schemaObject
);

// Plain text (default)
ChatRequestResponseFormat textFormat = ChatRequestResponseFormats.Text;

Basic Usage

Simple JSON Response

csharp

TornadoApi api = new TornadoApi("your-api-key");

Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.OMini,
    ResponseFormat = ChatRequestResponseFormats.Json
});

chat.AppendSystemMessage("Extract the user's name and age from their message. Respond in JSON format.");
chat.AppendUserInput("Hi, I'm Alice and I'm 28 years old.");

ChatRichResponse response = await chat.GetResponseRich();
Console.WriteLine(response.Content);
// Output: {"name": "Alice", "age": 28}

Structured JSON with Schema

csharp

TornadoApi api = new TornadoApi("your-api-key");

// Define the schema
object weatherSchema = new
{
    type = "object",
    properties = new
    {
        city = new { type = "string", description = "The city name" },
        temperature = new { type = "number", description = "Temperature in Celsius" },
        conditions = new { type = "string", description = "Weather conditions" },
        forecast = new
        {
            type = "array",
            items = new
            {
                type = "object",
                properties = new
                {
                    day = new { type = "string" },
                    high = new { type = "number" },
                    low = new { type = "number" }
                }
            }
        }
    },
    required = new List<string> { "city", "temperature", "conditions" },
    additionalProperties = false
};

Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O240806,
    ResponseFormat = ChatRequestResponseFormats.StructuredJson("weather_response", weatherSchema)
});

chat.AppendUserInput("What's the weather like in Prague?");

ChatRichResponse response = await chat.GetResponseRich();
Console.WriteLine(response.Content);

Parsing the Response

csharp

using Newtonsoft.Json;

// Define a C# class matching your schema
public class WeatherData
{
    public string City { get; set; }
    public double Temperature { get; set; }
    public string Conditions { get; set; }
}

// Get structured response
Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O,
    ResponseFormat = ChatRequestResponseFormats.Json
});

chat.AppendSystemMessage("Provide weather data in JSON format with city, temperature, and conditions fields.");
chat.AppendUserInput("Weather in London?");

ChatRichResponse response = await chat.GetResponseRich();

// Parse the JSON response
WeatherData? weather = JsonConvert.DeserializeObject<WeatherData>(response.Content);
Console.WriteLine($"City: {weather?.City}, Temp: {weather?.Temperature}°C, Conditions: {weather?.Conditions}");

Advanced Usage

Complex Nested Structures

csharp

object complexSchema = new
{
    type = "object",
    properties = new
    {
        user = new
        {
            type = "object",
            properties = new
            {
                name = new { type = "string" },
                email = new { type = "string", format = "email" },
                age = new { type = "integer", minimum = 0, maximum = 150 }
            },
            required = new[] { "name", "email" }
        },
        preferences = new
        {
            type = "object",
            properties = new
            {
                theme = new { type = "string", @enum = new[] { "light", "dark", "auto" } },
                notifications = new { type = "boolean" },
                language = new { type = "string" }
            }
        },
        tags = new
        {
            type = "array",
            items = new { type = "string" },
            minItems = 1,
            maxItems = 10
        }
    },
    required = new[] { "user" },
    additionalProperties = false
};

Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O240806,
    ResponseFormat = ChatRequestResponseFormats.StructuredJson("user_profile", complexSchema)
});

chat.AppendUserInput("Create a user profile for John Doe, age 35, email john@example.com, who prefers dark theme.");
ChatRichResponse response = await chat.GetResponseRich();

Combining with O1 Reasoning Models

csharp

Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O1241217,
    ReasoningEffort = ChatReasoningEfforts.Low,
    ResponseFormat = ChatRequestResponseFormats.Json
});

chat.AppendSystemMessage("Analyze the problem and provide a solution in JSON format.");
chat.AppendUserInput("How can I optimize my database queries?");

ChatRichResponse response = await chat.GetResponseRich();
Console.WriteLine(response.Content);

Multi-Provider Support

Different providers may have varying levels of support for structured output:

csharp

// OpenAI - Full support for structured output
Conversation openAiChat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O,
    ResponseFormat = ChatRequestResponseFormats.StructuredJson("data", schema)
});

// Groq - Supports JSON mode
Conversation groqChat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.Groq.Meta.Llama3370BVersatile,
    ResponseFormat = ChatRequestResponseFormats.Json
});

// For providers without native support, use system messages
Conversation genericChat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.Anthropic.Claude3.Sonnet
});
genericChat.AppendSystemMessage("Always respond in valid JSON format matching this structure: {city: string, temp: number}");

Best Practices

Schema Design

Be Specific: Clearly define property types and constraints
Use Descriptions: Add descriptions to help the model understand field purposes
Set Required Fields: Mark essential fields as required
Limit Flexibility: Use additionalProperties: false to prevent unexpected fields
Validate Ranges: Use min/max for numbers, minItems/maxItems for arrays

Error Handling

csharp

try
{
    ChatRichResponse response = await chat.GetResponseRich();
    
    if (string.IsNullOrEmpty(response.Content))
    {
        Console.WriteLine("No content in response");
        return;
    }

    WeatherData? data = JsonConvert.DeserializeObject<WeatherData>(response.Content);
    
    if (data == null)
    {
        Console.WriteLine("Failed to parse response");
        return;
    }
    
    // Use the data
    Console.WriteLine($"Temperature: {data.Temperature}");
}
catch (JsonException ex)
{
    Console.WriteLine($"JSON parsing error: {ex.Message}");
}
catch (Exception ex)
{
    Console.WriteLine($"API error: {ex.Message}");
}

Performance Considerations

Schema Complexity: Simpler schemas process faster
Token Usage: Structured output may use slightly more tokens
Model Selection: Choose models with native structured output support for best results
Caching: Consider caching responses for repeated queries

Prompt Engineering

csharp

// Good: Clear instructions with examples
chat.AppendSystemMessage(@"
Extract contact information in JSON format.
Example: {""name"": ""John"", ""email"": ""john@example.com"", ""phone"": ""+1234567890""}
");

// Better with structured output: Let the schema do the work
Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
    Model = ChatModel.OpenAi.Gpt4.O,
    ResponseFormat = ChatRequestResponseFormats.StructuredJson("contact", contactSchema)
});
// No need for example in system message - schema is self-documenting

Common Issues

Issue: Model Not Respecting Schema

Problem: Response doesn't match the expected structure Solutions:

Ensure the model supports structured output
Verify your schema is valid JSON Schema
Check that additionalProperties is set correctly
Use a more capable model (e.g., GPT-4 instead of GPT-3.5)

Issue: JSON Parsing Errors

Problem: Unable to deserialize the response Solutions:

csharp

// Use error-tolerant parsing
try
{
    WeatherData? data = JsonConvert.DeserializeObject<WeatherData>(
        response.Content,
        new JsonSerializerSettings
        {
            MissingMemberHandling = MissingMemberHandling.Ignore,
            NullValueHandling = NullValueHandling.Ignore
        }
    );
}
catch (JsonException ex)
{
    Console.WriteLine($"Parse error: {ex.Message}");
    Console.WriteLine($"Response was: {response.Content}");
}

Issue: Empty or Missing Fields

Problem: Required fields are missing or null Solutions:

Mark fields as required in your schema
Add validation after parsing
Provide clear field descriptions
Include examples in your prompt

Issue: Provider Compatibility

Problem: Feature not available for your chosen provider Solutions:

Check provider capabilities in the Feature Matrix
Fall back to JSON mode with prompt instructions
Use a different provider that supports the feature
Consider OpenAI or Google Gemini for best structured output support

API Reference

ChatRequestResponseFormats

Static factory for response format configurations:

Methods

ChatRequestResponseFormat Json - Enables JSON mode
ChatRequestResponseFormat StructuredJson(string name, object schema) - Creates structured JSON format with schema
ChatRequestResponseFormat Text - Default text format (no structure enforcement)

ChatRequest Properties

ResponseFormat - Specifies the desired response format
ReasoningEffort - For O1 models, controls reasoning depth (Low, Medium, High)

ChatRichResponse Properties

Content - The structured response content as a string
Blocks - Rich response blocks (for multi-part responses)
Usage - Token usage information

Chat Basics - Learn the fundamentals of chat conversations
Function Calling - Combine structured output with function calling
Models - See which models support structured output
Streaming - Stream structured output in real-time

Structured Output ​

Overview ​

Quick Start ​

Prerequisites ​

Detailed Explanation ​

Structured Output Modes ​

1. JSON Mode ​

2. Structured JSON with Schema ​

3. Text Mode (Default) ​

How It Works ​

Key Components ​

ChatRequestResponseFormats ​

Basic Usage ​

Simple JSON Response ​

Structured JSON with Schema ​

Parsing the Response ​

Advanced Usage ​

Complex Nested Structures ​

Combining with O1 Reasoning Models ​

Multi-Provider Support ​

Best Practices ​

Schema Design ​

Error Handling ​

Performance Considerations ​

Prompt Engineering ​

Common Issues ​

Issue: Model Not Respecting Schema ​

Issue: JSON Parsing Errors ​

Issue: Empty or Missing Fields ​

Issue: Provider Compatibility ​

API Reference ​

ChatRequestResponseFormats ​

Methods ​

ChatRequest Properties ​

ChatRichResponse Properties ​

Related Topics ​

Structured Output

Overview

Quick Start

Prerequisites

Detailed Explanation

Structured Output Modes

1. JSON Mode

2. Structured JSON with Schema

3. Text Mode (Default)

How It Works

Key Components

ChatRequestResponseFormats

Basic Usage

Simple JSON Response

Structured JSON with Schema

Parsing the Response

Advanced Usage

Complex Nested Structures

Combining with O1 Reasoning Models

Multi-Provider Support

Best Practices

Schema Design

Error Handling

Performance Considerations

Prompt Engineering

Common Issues

Issue: Model Not Respecting Schema

Issue: JSON Parsing Errors

Issue: Empty or Missing Fields

Issue: Provider Compatibility

API Reference

ChatRequestResponseFormats

Methods

ChatRequest Properties

ChatRichResponse Properties

Related Topics