Rerank

Overview

Reranking improves search result relevance by re-scoring and reordering documents based on their semantic relevance to a query. LlmTornado provides reranking capabilities through providers like Cohere and Voyage AI, which are essential for building high-quality retrieval systems and improving search accuracy in RAG applications.

Quick Start

csharp

using LlmTornado;
using LlmTornado.Rerank;
using LlmTornado.Rerank.Models;

TornadoApi api = new TornadoApi("your-api-key");

string query = "What is machine learning?";
List<string> documents = [
    "Machine learning is a subset of artificial intelligence.",
    "Python is a popular programming language.",
    "Deep learning uses neural networks.",
    "JavaScript is used for web development."
];

RerankResult? result = await api.Rerank.CreateRerank(
    RerankModel.Cohere.Gen3.Multilingual,
    query,
    documents);

foreach (RerankEntry entry in result.Results)
{
    Console.WriteLine($"Rank {entry.Index}: Score {entry.RelevanceScore:F4}");
    Console.WriteLine($"Document: {documents[entry.Index]}");
}

Prerequisites

The LlmTornado package installed
A valid API key for reranking providers (Cohere, Voyage)
Understanding of Embeddings
Basic knowledge of search and retrieval concepts

Detailed Explanation

What is Reranking?

Reranking is a two-stage retrieval process:

Initial Retrieval - Use fast methods (embeddings, BM25) to get candidate documents
Reranking - Apply more sophisticated models to reorder results by relevance

Why Use Reranking?

Better Accuracy - More precise relevance scoring
Context-Aware - Understands query-document relationships
Improved User Experience - Most relevant results appear first
Cost-Effective - Only rerank top candidates, not entire corpus

Supported Models

Cohere

rerank-multilingual-v3.0 - Supports 100+ languages
rerank-english-v3.0 - Optimized for English

Voyage AI

rerank-2 - High-quality reranking model

Basic Usage

Simple Reranking

csharp

string query = "best practices for API design";
List<string> documents = [
    "REST APIs should use HTTP methods correctly",
    "Database optimization techniques",
    "API versioning strategies are important",
    "Frontend design patterns"
];

RerankResult? result = await api.Rerank.CreateRerank(
    RerankModel.Cohere.Gen3.Multilingual,
    query,
    documents);

Console.WriteLine("Top results:");
foreach (RerankEntry entry in result.Results.Take(3))
{
    Console.WriteLine($"{entry.Index}: {documents[entry.Index]}");
    Console.WriteLine($"Score: {entry.RelevanceScore:F4}\n");
}

Top-K Results

Get only the most relevant results:

csharp

RerankResult? result = await api.Rerank.CreateRerank(
    new RerankRequest
    {
        Model = RerankModel.Cohere.Gen3.English,
        Query = "python programming tutorials",
        Documents = documents,
        TopN = 5  // Return only top 5 results
    });

With Return Documents

Include document text in response:

csharp

RerankResult? result = await api.Rerank.CreateRerank(
    new RerankRequest
    {
        Model = RerankModel.Cohere.Gen3.Multilingual,
        Query = query,
        Documents = documents,
        ReturnDocuments = true
    });

foreach (RerankEntry entry in result.Results)
{
    Console.WriteLine($"Score: {entry.RelevanceScore:F4}");
    Console.WriteLine($"Document: {entry.Document?.Text}");
}

Advanced Usage

Reranking with Structured Documents

Rerank complex document structures:

csharp

public class SearchResult
{
    public string Title { get; set; }
    public string Content { get; set; }
    public string Url { get; set; }
}

List<SearchResult> results = GetSearchResults();
List<string> documents = results.Select(r => $"{r.Title}: {r.Content}").ToList();

RerankResult? reranked = await api.Rerank.CreateRerank(
    RerankModel.Cohere.Gen3.English,
    "artificial intelligence applications",
    documents,
    topN: 10);

// Map back to original results
List<SearchResult> topResults = reranked.Results
    .Select(entry => results[entry.Index])
    .ToList();

foreach (SearchResult result in topResults)
{
    Console.WriteLine($"{result.Title}: {result.Url}");
}

Combining with Vector Search

Two-stage retrieval pipeline:

csharp

async Task<List<string>> SearchWithRerank(string query, int finalCount = 5)
{
    // Stage 1: Vector search to get candidates
    EmbeddingResult? queryEmbedding = await api.Embeddings.CreateEmbedding(
        EmbeddingModel.OpenAi.Gen2.Ada,
        query);
    
    // Get top 50 candidates from vector database
    List<string> candidates = await VectorSearch(queryEmbedding, topK: 50);
    
    // Stage 2: Rerank candidates
    RerankResult? reranked = await api.Rerank.CreateRerank(
        RerankModel.Cohere.Gen3.Multilingual,
        query,
        candidates,
        topN: finalCount);
    
    // Return reranked results
    return reranked.Results
        .Select(entry => candidates[entry.Index])
        .ToList();
}

// Usage
List<string> results = await SearchWithRerank("machine learning basics", 10);

Filtering by Relevance Score

Only return results above a threshold:

csharp

RerankResult? result = await api.Rerank.CreateRerank(
    RerankModel.Cohere.Gen3.English,
    query,
    documents);

double threshold = 0.5;
List<RerankEntry> relevant = result.Results
    .Where(entry => entry.RelevanceScore >= threshold)
    .ToList();

Console.WriteLine($"Found {relevant.Count} documents above threshold");

Multi-Language Support

Rerank documents in multiple languages:

csharp

string query = "machine learning";
List<string> multilingualDocs = [
    "Machine learning is a subset of AI",  // English
    "機械学習は人工知能の一部です",              // Japanese
    "El aprendizaje automático es parte de la IA",  // Spanish
    "Maschinelles Lernen ist Teil der KI"  // German
];

RerankResult? result = await api.Rerank.CreateRerank(
    RerankModel.Cohere.Gen3.Multilingual,
    query,
    multilingualDocs);

// All languages ranked by relevance
foreach (RerankEntry entry in result.Results)
{
    Console.WriteLine($"Score: {entry.RelevanceScore:F4}");
    Console.WriteLine($"Text: {multilingualDocs[entry.Index]}\n");
}

RAG Pipeline Integration

Complete RAG implementation:

csharp

async Task<string> AnswerQuestion(string question)
{
    // 1. Get initial candidates from vector store
    List<string> candidates = await GetVectorSearchResults(question, topK: 20);
    
    // 2. Rerank for best matches
    RerankResult? reranked = await api.Rerank.CreateRerank(
        RerankModel.Cohere.Gen3.English,
        question,
        candidates,
        topN: 5);
    
    // 3. Use top results as context
    List<string> context = reranked.Results
        .Select(entry => candidates[entry.Index])
        .ToList();
    
    // 4. Generate answer with context
    Conversation conversation = api.Chat.CreateConversation(new ChatRequest
    {
        Model = ChatModel.OpenAi.Gpt4.Turbo,
        SystemMessage = $"Answer based on this context:\n{string.Join("\n\n", context)}"
    });
    
    conversation.AddUserMessage(question);
    ChatRichResponse answer = await conversation.GetResponseRich();
    
    return answer.Content;
}

Best Practices

Two-Stage Retrieval - Use fast search first, then rerank
Appropriate Top-K - Rerank 20-100 candidates for best results
Score Thresholds - Filter out low-relevance results
Cost Management - Rerank only when necessary
Language Selection - Use multilingual model for non-English content

Common Issues

Poor Reranking Results

Solution: Increase candidate pool size
Prevention: Ensure initial retrieval gets diverse results

Slow Performance

Solution: Reduce number of documents to rerank
Prevention: Use caching for repeated queries

Cost Concerns

Solution: Limit reranking to top candidates only
Prevention: Cache reranking results

API Reference

RerankRequest

RerankModel Model - Model to use
string Query - Search query
List<string> Documents - Documents to rerank
int? TopN - Number of results to return
bool? ReturnDocuments - Include document text

RerankResult

List<RerankEntry> Results - Ranked results
RerankMeta Meta - Metadata about the request

RerankEntry

int Index - Original document index
double RelevanceScore - Relevance score (0-1)
RerankDocument Document - Document (if requested)

RerankModel

Cohere.Gen3.Multilingual - Cohere multilingual v3
Cohere.Gen3.English - Cohere English v3
Voyage.Gen2.Default - Voyage rerank-2

Embeddings - Creating embeddings
Vector Stores - Storing embeddings
Vector Databases - External vector DBs
Chat Basics - Building RAG systems

Rerank ​

Overview ​

Quick Start ​

Prerequisites ​

Detailed Explanation ​

What is Reranking? ​

Why Use Reranking? ​

Supported Models ​

Cohere ​

Voyage AI ​

Basic Usage ​

Simple Reranking ​

Top-K Results ​

With Return Documents ​

Advanced Usage ​

Reranking with Structured Documents ​

Combining with Vector Search ​

Filtering by Relevance Score ​

Multi-Language Support ​

RAG Pipeline Integration ​

Best Practices ​

Common Issues ​

Poor Reranking Results ​

Slow Performance ​

Cost Concerns ​

API Reference ​

RerankRequest ​

RerankResult ​

RerankEntry ​

RerankModel ​

Related Topics ​

Rerank

Overview

Quick Start

Prerequisites

Detailed Explanation

What is Reranking?

Why Use Reranking?

Supported Models

Cohere

Voyage AI

Basic Usage

Simple Reranking

Top-K Results

With Return Documents

Advanced Usage

Reranking with Structured Documents

Combining with Vector Search

Filtering by Relevance Score

Multi-Language Support

RAG Pipeline Integration

Best Practices

Common Issues

Poor Reranking Results

Slow Performance

Cost Concerns

API Reference

RerankRequest

RerankResult

RerankEntry

RerankModel

Related Topics