Rerank endpoints lack the unified standard that Chat Completions has, so the same code may return 404 or 422 after you change vendors. Troubleshooting should start by separating errors by HTTP status code, then move on to soft failures where the response is 200 but the result is wrong. This article covers five categories: authentication, candidate sets, rate limits, timeouts, and empty results. It closes with reusable retry code and a logging checklist. Models and prices are governed by the in-site model pricing.

Note: Cohere, Jina, Voyage, and other names are trademarks of their respective owners and are used here for technical discussion only. Endpoint paths, quotas, and error codes may change between versions; refer to the relevant vendor documentation.

1. Separate the five problem classes first

Symptom Common status codes Typical cause Order of handling
Authentication failure 401, 403 Wrong key, missing header, model not enabled Fix credentials and permissions first
Request rejected 400, 422 Candidate set too large, field mismatch, empty document Inspect request body and quotas
Wrong path 404 Misspelled endpoint, outdated model name, bad base URL concatenation Compare with vendor documentation
Rate limit or quota 429 Sudden concurrency spike, shared key, retry storm Back off, reduce concurrency
Timeout or upstream error 504, 5xx Too many candidates, documents too long, upstream overloaded Shrink the request, retry a limited number of times
Empty response 200 Invalid top_n, wrong field read, documents filtered out Add assertions and alerts

Log at least the timestamp, endpoint, model, document count, total character count, top_n, status code, elapsed time, retry count, and request ID. Never log the full key, and never print entire user documents.

2. 401 and 403: check authentication, then permissions

A 401 means the identity was not verified and usually has nothing to do with model capability or balance. Troubleshoot in this order:

  1. Confirm the header uses Authorization: Bearer plus the key; some gateways require x-api-key instead.
  2. Check the key for leading or trailing spaces, line breaks, or stray quotes. Copying from a chat window is the most common source.
  3. Check whether an environment variable is overridden by local configuration or the deployment platform, and whether production and local use different keys.
  4. Confirm the key is enabled for the rerank endpoint. Some accounts have chat permission but no rerank permission.
  5. Check the endpoint path. /v1/rerank and /v2/rerank may use different formats, and appending /rerank to a chat base URL often produces 404 or 401.

A 403 means the identity is valid but permissions are insufficient. Focus on whether the model is enabled, whether the IP is allowlisted, project permissions, and any overdue balance. Do not retry a 401; retries only multiply failure logs and may trigger risk controls. Get a minimal curl request working first, then return to business code.

3. 400 and 422: oversized candidate sets and field mismatches

An oversized candidate set is the most common source of 400. Hosted endpoints usually impose limits on documents per request, tokens per document, and request body size. Cohere, for example, recommends no more than one thousand documents per request and truncates each document to 4096 tokens by default. Other vendors use different values that change with model versions, so check the relevant documentation before integrating. When limits are exceeded, the right response is not blind retrying:

  • Shard the candidate set, rerank each shard separately, then merge by score.
  • Control the candidate count during recall, for example fifty to two hundred documents.
  • Chunk or truncate very long documents first; do not push a whole PDF into documents.
  • Remove empty strings, null values, whitespace-only entries, and duplicate documents.

Field mismatches also produce 400 or 422. top_n may be written as top_k, or the reverse; Jina and Cohere mostly use top_n, while Voyage uses top_k. documents may be passed as an array of objects rather than an array of strings, or id and text may sit at the wrong level. When top_n exceeds the document count, some endpoints return an error and others truncate automatically. Parameters such as return_documents, truncation, and max_tokens_per_doc may not be supported by the current provider. NaN, Infinity, and unescaped newlines in JSON also cause parse failures.

Troubleshooting method: keep a redacted copy of the original request body, start from a minimal example, and add parameters back one at a time, changing only one variable per attempt.

4. 429: rate limits, quotas, and retry storms

A 429 means the request was rate-limited or the quota was exhausted. Common causes include requests per second or concurrency above the plan, a token quota exhausted during peak hours, multiple services sharing one key, and client retries without backoff. Read the Retry-After and quota fields in the response headers before deciding whether to wait or reduce concurrency.

Handling order:

  1. Determine whether the limit is per key or at the account level.
  2. Change retries to exponential backoff with jitter, and respect Retry-After.
  3. Cap concurrency and absorb peaks with a queue instead of retrying indefinitely.
  4. Move batch jobs to off-peak hours and separate offline evaluation from production traffic.
  5. Prepare a degradation path for critical workloads: skip reranking and keep the recall order.

Do not retry on a fixed one-second interval until the limit clears. When multiple instances retry at once, they create a retry storm that turns a small rate limit into a sustained outage.

5. Timeouts, 504, and 5xx

Timeouts divide into connection timeouts and read timeouts. A connection timeout usually points to network, DNS, proxy, or TLS problems. A read timeout more often means the candidate set is too large, documents are too long, the upstream is queued, or the model is cold-starting. A 504 generally means the gateway did not receive an upstream response in time, which is a separate issue from the client's own timeout.

Troubleshooting order:

  1. Test with a minimal request: one query and five short documents. Confirm the network is not completely blocked.
  2. Increase the document count step by step to find the latency inflection point.
  3. Check per-document length and total tokens, and truncate where necessary.
  4. Check whether the client timeout is shorter than the server's processing time.
  5. Retry 5xx and timeouts a limited number of times, then degrade once the threshold is exceeded.
  6. Record the request ID and any upstream trace ID to simplify reconciliation with the vendor.

Reranking is generally not a streaming endpoint, so long processing time cannot be hidden behind a stream. For latency-sensitive workloads, move reranking into an asynchronous pipeline or rerank only the top few dozen candidates.

6. Response is 200 but the result is empty or wrong

These soft failures are harder to spot. documents may be an empty array and the endpoint returns an empty result. top_n may be zero or negative. Business filters may exclude every document. The response field may be read incorrectly; some APIs return results and others return data. The index refers to the position in the original document array, but after sharding the offset was forgotten. Score semantics may differ: some normalize to zero-to-one, others output logits, and sort direction may vary. The response may return document text rather than indices, and deduplication may have merged identical text.

Add client-side assertions: the result count must not exceed top_n, every index must be within range, and scores must be monotonic according to the vendor's semantics. When an anomaly appears, log a summary of the raw response rather than silently returning an empty list.

7. Reusable retry code

The example below divides errors into three classes: authentication and parameter errors are not retried; 429 and 5xx are retried with backoff; timeouts are retried a limited number of times. When retries are exhausted, it falls back to the original recall order.

import random
import time
import requests

def rerank(query, documents, top_n=10, model="<rerank-model>",
           url="<base-url>/v1/rerank", api_key="<key>"):
    payload = {"model": model, "query": query,
               "documents": documents, "top_n": top_n}
    headers = {"Authorization": f"Bearer {api_key}"}
    last_error = None
    for attempt in range(5):
        try:
            resp = requests.post(url, json=payload, headers=headers,
                                 timeout=(3.05, 30))
        except requests.Timeout as exc:
            last_error = exc
            time.sleep(min(2 ** attempt + random.random(), 20))
            continue

        if resp.status_code in (400, 401, 403, 404, 422):
            raise RuntimeError(f"non-retryable {resp.status_code}: {resp.text[:300]}")
        if resp.status_code == 429 or resp.status_code >= 500:
            retry_after = resp.headers.get("Retry-After")
            wait = float(retry_after) if retry_after else min(
                2 ** attempt + random.random(), 20)
            time.sleep(wait)
            last_error = RuntimeError(f"retryable {resp.status_code}")
            continue

        data = resp.json()
        results = data.get("results") or data.get("data") or []
        if not results:
            return list(range(min(top_n, len(documents))))
        return [item["index"] for item in results[:top_n]]

    raise RuntimeError(f"retries exhausted: {last_error}")

# Callers still need to adapt to vendor fields; Voyage, for example, uses data and top_k.

If documents are sharded for reranking, add the shard offset to each index before merging and sorting globally.

8. Logging and monitoring checklist

Record at least the following for every call: timestamp, endpoint, model, document count, total characters or estimated tokens, top_n, status code, error type, elapsed time, retry count, and request ID. Never log the full key, and never write entire user documents into logs; when investigation is needed, log a hash, the length, and a redacted prefix.

Monitor four metrics: the 429 ratio, the timeout and 5xx ratio, the empty-result ratio, and p95 latency. Break them down by business line so that a high-frequency, low-value task cannot consume the quota and mask an anomaly in a core workload.

FAQ

What is the difference between 401 and 403?

A 401 means the identity was not verified, usually because the key is missing, malformed, or expired. A 403 means the identity is valid but permissions are insufficient: the model may not be enabled, the IP may not be allowlisted, or the project or balance may be restricted. Fix authentication first, then check permissions.

What should I do when the candidate set exceeds the endpoint limit?

Shard on the client, rerank each shard, merge by score, and add offsets to the indices. A better fix is to control the count during recall; a few dozen to a few hundred is usually enough. Do not send thousands of documents into reranking.

How many times should I retry a 429?

There is no universal answer. Use a bounded number of retries, for example three to five, with exponential backoff and jitter, respect Retry-After, and cap concurrency. Degrade after retries are exhausted instead of letting requests pile up indefinitely.

Can a Rerank endpoint stream like a chat endpoint?

Generally no. Reranking must see the whole candidate set before it can produce a stable order, and most endpoints return synchronously. When latency matters, reduce the candidate count, truncate document length, or move reranking into an asynchronous job.

Why does a 200 response contain no results?

Check whether documents is empty, whether top_n is valid, and whether business filters excluded every document. Then check whether the response field was read incorrectly and whether shard merging produced out-of-range indices. Assertions and empty-result alerts are more effective than after-the-fact investigation.

Next steps

Once troubleshooting is complete, freeze a minimal reproduction and a regression suite, then create a test key on ZSQH AI Gateway to verify the call path. Billing and quotas are governed by the model pricing page.

Related reading: