Reranking

Score and reorder documents by relevance to a query to improve search and RAG results

The Reranking endpoint (POST /v1/rerank) scores a list of documents against a query and returns them in relevance order. Use it as the second stage of search or retrieval-augmented generation: retrieve a wide candidate set with embeddings, then rerank the top of it for the final context. The request and response follow the Cohere rerank API, so existing Cohere-style clients adapt with a base URL and model change.

Every example here reads your key from NEBUL_API_KEY; export it first. The Quick Start covers key setup.

Request

POST https://api.inference.nebul.io/v1/rerank

POST https://api.inference.nebul.io/rerank behaves identically.

Parameters

Parameter	Type	Required	Description
`model`	String	Yes	The model ID, e.g. `BAAI/bge-reranker-v2-m3`.
`query`	String	Yes	The query to rank documents against.
`documents`	Array[String]	Yes	Documents to score.
`top_n`	Integer	No	Number of results to return. Defaults to all documents.
`return_documents`	Boolean	No	Include each document's text in the response. Defaults to `false`.
`raw_scores`	Boolean	No	Return raw logits instead of normalized scores. Defaults to `false`.

Example

Rank four documents against one query:

import os

import requests

response = requests.post(
    "https://api.inference.nebul.io/v1/rerank",
    headers={
        "Authorization": f"Bearer {os.environ['NEBUL_API_KEY']}",
        "Content-Type": "application/json",
    },
    json={
        "model": "BAAI/bge-reranker-v2-m3",
        "query": "What is machine learning?",
        "documents": [
            "Machine learning is a subset of artificial intelligence.",
            "The weather today is sunny and warm.",
            "Deep learning uses neural networks with multiple layers.",
            "Python is a popular programming language.",
        ],
        "top_n": 3,
        "return_documents": True,
    },
)

for result in response.json()["results"]:
    print(f"{result['relevance_score']:.4f}  {result['document']}")

The Python tab prints the machine learning document at about 0.999, the deep learning document far behind, and the weather document last. Rank by relevance_score, not by position in your input array: index tells you where each result came from.

Response

Results are sorted by score, highest first:

{
  "object": "rerank",
  "results": [
    {
      "relevance_score": 0.9994296431541443,
      "index": 0,
      "document": "Machine learning is a subset of artificial intelligence."
    },
    {
      "relevance_score": 0.001597845577634871,
      "index": 2,
      "document": "Deep learning uses neural networks with multiple layers."
    },
    {
      "relevance_score": 0.00003120191104244441,
      "index": 3,
      "document": "Python is a popular programming language."
    }
  ],
  "model": "BAAI/bge-reranker-v2-m3",
  "usage": {
    "prompt_tokens": 289,
    "total_tokens": 289
  },
  "id": "infinity-d348afd1-b785-4f8a-8c63-b926b7c77dea",
  "created": 1792150157
}

Field	Type	Description
`object`	String	Always `"rerank"`.
`results`	Array	Documents in relevance order, highest score first.
`results[].index`	Integer	Position of the document in the input array.
`results[].relevance_score`	Number	Relevance of the document to the query. Normalized to 0-1 unless `raw_scores` is set.
`results[].document`	String	The document text, present only when `return_documents` is `true`.
`model`	String	The model ID that scored the documents.
`usage.prompt_tokens`	Integer	Tokens counted across the query and all documents.
`id`	String	Request identifier, prefixed `infinity-`.
`created`	Integer	Unix timestamp of the response.

Raw scores

Normalized scores cluster hard against the ends of the 0-1 range: a clear match lands near 1.0 and everything else falls away fast. Set raw_scores: true for the model's unnormalized logits when you want to calibrate your own threshold:

"query": "Python programming",
"documents": [
    "Python is a high-level programming language.",
    "Java is another programming language.",
],
"raw_scores": True,

Those two documents score 6.0 and -3.375 as raw logits. A cutoff tuned on logits is stable for a given model; renormalize your calibration if you switch models.

RAG pattern

Retrieve wide with embeddings, then rerank narrow. The reranker reads the query and documents together, so it judges fine distinctions that vector distance misses:

# Step 1: retrieve a wide candidate set (vector search, keyword search, or both)
candidates = [
    "Machine learning algorithms can be supervised or unsupervised.",
    "The capital of France is Paris.",
    "Neural networks consist of layers of interconnected nodes.",
    "Python has a large ecosystem of libraries for data science.",
    "Deep learning is a subset of machine learning.",
]

# Step 2: rerank against the user's question
response = requests.post(
    "https://api.inference.nebul.io/v1/rerank",
    headers={"Authorization": f"Bearer {os.environ['NEBUL_API_KEY']}"},
    json={
        "model": "BAAI/bge-reranker-v2-m3",
        "query": "How do neural networks work?",
        "documents": candidates,
        "top_n": 2,
        "return_documents": True,
    },
)

# Step 3: feed the survivors to the model as context
context = "\n\n".join(r["document"] for r in response.json()["results"])
print(context)

The neural networks document scores 0.93; the next candidate sits four orders of magnitude lower. A candidate pool of 20 to 100 documents is the usual sweet spot: large enough for recall, small enough to keep the rerank request cheap and fast.

The model also handles mixed languages. A Dutch query against English, Dutch, French, and weather documents returns the Dutch document first (0.94) with the English one well behind (0.12).

Model

Examples on this page use BAAI/bge-reranker-v2-m3, the default reranking model. It scores cross-lingual relevance: query in one language, documents in another. The Model Catalog lists any other reranking models with their specs. For the first retrieval stage, see Embeddings.