mxbai-rerank

Open weights · mixedbread-ai · Updated 12 Oct 2026

mixedbread-ai ships two generations of reranker. v1 (xsmall/base/large) is a DeBERTa-v3 cross-encoder family, English-only, and its xsmall variant is compact enough to run in the browser — which is why it's one of the options in this site's live demo. v2 (base/large) is a newer, Qwen2.5-based, RL-trained generation that adds support for 100+ languages, including Chinese. Both generations ship under Apache 2.0.

Model variants

The v1 family are DeBERTa-v3 cross-encoders trained on MS MARCO passage ranking; v2 is a Qwen2.5-based generation trained with a three-step reinforcement-learning process (GRPO, contrastive learning, preference learning). Start with mxbai-rerank-base-v2 for new production use, especially with any non-English content — it beats v1's large on every published benchmark below. Keep xsmall-v1 only for browser or edge deployments, since v2 has no comparably small variant.

On speed, mixedbread’s own table gives A100 latency of 2.24 s for large-v1, 0.89 s for large-v2 and 0.67 s for base-v2. It doesn’t say for what batch, so read it as relative speed — the v2 models are faster despite being larger — not as a per-query figure.

Benchmarks

Both runs are mixedbread’s own. The first column is their cross-generation comparison table (mxbai-rerank on GitHub); the other two come from the v1 model cards: 11 BEIR datasets, reranking the top 100 from Lucene keyword search, where keyword search alone scores 38.0. Until October 2026 this table showed xsmall-v1 and base-v1 at ~55.5 and ~59.8 on BEIR, and an MS MARCO MRR@10 column from ~38.0 to ~42.3, none of which we could trace; mixedbread publish 43.9 and 46.9 on their 11-dataset run and no MS MARCO figures for these models. In September the page had already corrected large-v1 from an unsourced ~62.1.

Multilingual (why v2 exists)

v1 was never a multilingual model — it just hadn't been benchmarked on non-English data until mixedbread's own 2026 retrospective. If your documents aren't all English, v2 is the generation to use; v1 is not a substitute.

Quick start

v2, self-hosted

pip install -U mxbai-rerank

from mxbai_rerank import MxbaiRerankV2

reranker = MxbaiRerankV2("mixedbread-ai/mxbai-rerank-base-v2")  # or large-v2

query = "How do I add reranking to my RAG pipeline?"
documents = [
    "Rerankers score each query-passage pair with a cross-encoder.",
    "BM25 is a classical keyword-based retrieval method.",
    "London is the capital of the United Kingdom.",
    "Two-stage retrieval: retrieve 50 candidates, rerank to top 5.",
]

results = reranker.rank(query=query, documents=documents)
for r in results:
    print(f"{r.score:.4f}  {r.document[:80]}")

v1, self-hosted (sentence-transformers)

pip install sentence-transformers

from sentence_transformers import CrossEncoder

model = CrossEncoder("mixedbread-ai/mxbai-rerank-base-v1")

query = "How do I add reranking to my RAG pipeline?"
passages = [
    "Rerankers score each query-passage pair with a cross-encoder.",
    "BM25 is a classical keyword-based retrieval method.",
    "London is the capital of the United Kingdom.",
    "Two-stage retrieval: retrieve 50 candidates, rerank to top 5.",
]

scores = model.predict([(query, p) for p in passages])
ranked = sorted(zip(scores, passages), reverse=True)

for score, text in ranked:
    print(f"{score:.4f}  {text[:80]}")

In a RAG pipeline (v2)

from mxbai_rerank import MxbaiRerankV2

reranker = MxbaiRerankV2("mixedbread-ai/mxbai-rerank-large-v2")

def rag_answer(query: str, vector_db, llm) -> str:
    # Stage 1: retrieve wide
    candidates = vector_db.search(query, top_k=50)
    # Stage 2: rerank tight
    results = reranker.rank(query=query, documents=candidates, top_k=5)
    top5 = [r.document for r in results]
    # Stage 3: generate
    return llm.complete(f"Context:\n" + "\n\n".join(top5) + f"\n\nQ: {query}")

Hosted API

Mixedbread’s current hosted reranker is not v1 or v2. Its documentation now files both generations under legacy models: the weights stay on Hugging Face, but the direct rerank API is no longer documented, and existing integrations are asked to contact Mixedbread’s support about migrating. Hosted reranking now runs inside Mixedbread Search, on documents you have uploaded to a Mixedbread store, with mxbai-rerank-v3.1-listwise as the default (July 2026). It is a listwise model — it reads the candidate set as a whole and follows natural-language instructions such as “the most recent booking wins” — and its weights are not published.

In Mixedbread’s own 900-example instruction-following evaluation, the v3 listwise model reached an MRR of 0.93*, against 0.84* for Voyage rerank-2.5, 0.77* for Cohere Rerank 4 Pro and 0.71* for zerank-2 (announcement); v3.1, which replaced it in September, is described as faster and better (v3.1 post). No independent run exists yet.

The direct endpoint below, with the open v2 weights, is what Mixedbread documented until 2026. The SDK is mixedbread (the older mixedbread_ai package was v1-era):

pip install mixedbread

from mixedbread import Mixedbread

mxbai = Mixedbread(api_key="YOUR_API_KEY")

result = mxbai.rerank(
    model="mixedbread-ai/mxbai-rerank-large-v2",
    query="How do I add reranking to my RAG pipeline?",
    input=[
        "Rerankers score each query-passage pair jointly.",
        "BM25 is a keyword-based retrieval method.",
        "London is the capital of the United Kingdom.",
        "Two-stage retrieval: retrieve wide, rerank tight.",
    ],
    top_k=3,
)

for item in result.data:
    print(f"{item.score:.4f}  rank {item.index + 1}")

Browser use

The v1 xsmall variant is compact enough to run in the browser via transformers.js — this is exactly what our demo uses. v2's smallest variant (0.5B) has no equivalent browser-sized build yet:

// transformers.js (ES module)
import { AutoTokenizer, AutoModelForSequenceClassification }
  from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@4";

const tokenizer = await AutoTokenizer.from_pretrained(
  "mixedbread-ai/mxbai-rerank-xsmall-v1", { dtype: "q8" }
);
const model = await AutoModelForSequenceClassification.from_pretrained(
  "mixedbread-ai/mxbai-rerank-xsmall-v1", { dtype: "q8" }
);

const inputs = tokenizer(
  [query, query],
  { text_pair: [doc1, doc2], padding: true, truncation: true }
);
const { logits } = await model(inputs);
const scores = logits.sigmoid().tolist();

With dtype: "q8" the quantized weights are roughly 35 MB — fast to download and cached in IndexedDB after the first run.

Pros and cons

Other models