LlmTornado

Agent Streaming ​

Overview ​

Streaming enables real-time response generation from agents, providing immediate feedback to users as the AI processes and generates content. This is especially valuable for long-form responses or interactive applications.

Quick Start ​

csharp
using LlmTornado.Agents;
using LlmTornado.Agents.DataModels;

TornadoAgent agent = new TornadoAgent(
    client: api,
    model: ChatModel.OpenAi.Gpt41.V41Mini,
    instructions: "You are a helpful assistant.",
    streaming: true
);

// Define streaming handler
ValueTask streamHandler(AgentRunnerEvents runEvent)
{
    if (runEvent is AgentRunnerStreamingEvent streamingEvent)
    {
        if (streamingEvent.ModelStreamingEvent is ModelStreamingOutputTextDeltaEvent deltaTextEvent)
        {
            Console.Write(deltaTextEvent.DeltaText);
        }
    }
    return ValueTask.CompletedTask;
}

// Run with streaming
Conversation result = await agent.RunAsync(
    "Tell me a story about AI",
    onAgentRunnerEvent: streamHandler
);

Event Types ​

AgentRunnerEvents ​

Base event class for all agent events:

  • AgentRunnerStreamingEvent - Streaming text deltas
  • AgentRunnerToolCallEvent - Tool execution events
  • AgentRunnerCompletionEvent - Response completion

Model Streaming Events ​

  • ModelStreamingOutputTextDeltaEvent - Text chunks
  • ModelStreamingOutputToolCallDeltaEvent - Tool call updates
  • ModelStreamingOutputCompletionEvent - Stream completion

Basic Usage ​

csharp
ValueTask HandleEvents(AgentRunnerEvents runEvent)
{
    switch (runEvent.EventType)
    {
        case AgentRunnerEventTypes.Streaming:
            if (runEvent is AgentRunnerStreamingEvent se)
            {
                if (se.ModelStreamingEvent is ModelStreamingOutputTextDeltaEvent delta)
                {
                    Console.Write(delta.DeltaText);
                }
            }
            break;
            
        case AgentRunnerEventTypes.ToolCall:
            if (runEvent is AgentRunnerToolCallEvent toolEvent)
            {
                Console.WriteLine($"\nCalling tool: {toolEvent.ToolName}");
            }
            break;
    }
    return ValueTask.CompletedTask;
}

TornadoAgent agent = new TornadoAgent(api, model, streaming: true);
await agent.RunAsync("Your prompt", onAgentRunnerEvent: HandleEvents);

Best Practices ​

  • Flush console output for immediate display
  • Handle all event types for robust applications
  • Consider buffer for better performance
  • Implement error handling in event handlers