The most common mistake in enterprise Rerank API procurement is comparing only the price per call. Reranking touches queries and document text, call volume scales with the candidate count, and quality must be validated on your own data. Fixating on unit price often leads to rework at the quota, compliance, or acceptance stage.

This article provides a checklist you can take directly into a meeting, covering volume, quota, invoicing, privacy, evaluation, fallback, and acceptance. It deliberately avoids fixed unit prices and quotas; the in-site model pricing and the signed contract govern.

Note: Cohere, Jina, BAAI, BGE, OpenAI, and other names belong to their respective owners. This article has no affiliation with or agency relationship to those organizations. It is a procurement discussion, not legal advice. Quotas, prices, and data processing terms are governed by the signed contract and official pages.

1. Procurement checklist at a glance

Summarize the procurement items in a table first, then work through them one by one:

Item Questions to confirm Acceptance material
Usage and quota Monthly quota, QPS, candidates per request, behavior when exceeded Quota description, load-test record
Billing and invoicing Billing unit, settlement cycle, invoicing entity, invoice type Quotation, contract, sample invoice
Data and privacy Retention, training, storage location, deletion, subprocessors Data processing agreement, security overview
Quality Evaluation set, baseline, metrics, acceptance thresholds Evaluation report, regression record
Access and audit Sub-accounts, key rotation, call logs Permission matrix, sample logs
Fallback and SLA Incident notification, backup path, version changes SLA terms, drill record
Cost monitoring Usage dashboard, budget alerts, project allocation Dashboard screenshot, alert configuration

2. Calculate volume before discussing unit price

Rerank volume is not "number of user questions." It is queries multiplied by candidate documents per call, then multiplied by the retry factor:

daily calls ≈ daily queries × candidates per query × (1 + retry rate)

Ten thousand queries per day with thirty candidates each means three hundred thousand document scorings. Billing by search unit or token makes document length expand the bill further. Before procurement, prepare a daily average, a campaign peak, and a twelve-month growth forecast. Estimates based on demo traffic usually understate demand.

Confirm the billing granularity as well: per query, per document, per token, or packaged as "per search." Different definitions cannot be compared directly; convert them to the same business volume first.

3. Quota, rate limits, and peak behavior

Quota terms should be put in writing. Ask specifically:

  • Monthly and annual quotas, and whether trial and production quotas are separate.
  • Whether rate limits are measured in QPS, requests per minute, or concurrency.
  • The maximum candidates per request and maximum length per document.
  • Whether exceeding the quota produces an error, automatically converts to pay-as-you-go, or stops service.
  • Whether multiple projects can share quota, and whether a per-key cap can be set.

"Behavior when exceeded" is the most easily overlooked item. Some providers return an error; others automatically switch to postpaid billing, so the bill can rise before anyone approves it. Require budget alerts and hard caps.

4. Billing, contracts, and invoices

The web price is only a reference; obtain a formal quotation and contract. Confirm:

  • Billing unit, tiered pricing, minimum spend, or committed volume.
  • Prepaid or postpaid, settlement cycle, and payment terms.
  • Contracting entity, invoicing entity, invoice type, and tax rate.
  • Whether special VAT invoices are supported and whether the line item matches the service.
  • Notice period for price changes and whether already-purchased quota is affected.
  • Refund policy, treatment of unused quota, and data export after termination.

When using an API gateway, distinguish clearly who receives payment, who issues the invoice, and who bears responsibility for incidents. Cross-border services also involve currency, foreign exchange, and taxation, so finance should be involved early.

5. Data privacy and processing boundaries

A reranking request contains both the query and candidate documents, so its information density may exceed an ordinary chat. Contracts and policy documents often contain customer names, prices, and personal information. Once that data crosses a border or is retained for a long period, the risk propagates to the business side.

Confirm each item during procurement:

  • Whether request content is retained, for how long, and for what purpose.
  • Whether it is used for training or service improvement, and whether this can be disabled in writing.
  • Storage and backup locations, and cross-border transfer paths.
  • Deletion mechanisms: whether deletion can be done in bulk per tenant and how quickly it takes effect.
  • The subprocessor list and notification of changes.
  • Encryption, access control, incident notification, and a data processing agreement.

Teams with especially sensitive data can evaluate open-source self-hosting: weights run inside their own VPC and data does not leave the internal network, at the cost of GPUs, operations, and upgrades. For hosted services, write "no retention, no training, deletion on request" into the contract.

6. Quality evaluation: evaluation sets and NDCG

Draw 100 to 300 queries from real logs for the evaluation set, covering common phrasings, typos, multiple languages, long documents, and cases where "the material contains no answer." Pair each query with the top 20 to 50 candidates from production recall, labeled 0 (irrelevant), 1 (relevant), or 2 (directly answers).

Use NDCG@10 as the primary metric, supported by MRR, Top-3 hit rate, p95 latency, and cost per query. Also record Recall@k of the candidate set; it is the ceiling for reranking quality.

import math

def dcg(rels, k):
    return sum((2 ** r - 1) / math.log2(i + 2) for i, r in enumerate(rels[:k]))

def ndcg_at_k(ranked_rels, k=10):
    ideal = sorted(ranked_rels, reverse=True)
    idcg = dcg(ideal, k)
    return dcg(ranked_rels, k) / idcg if idcg else 0.0

# Fill in relevance grades in reranked order: 2=direct answer, 1=relevant, 0=irrelevant
ranked = [2, 0, 1, 0, 0, 1, 0, 0, 0, 0]
print(round(ndcg_at_k(ranked), 4))

Do not copy acceptance thresholds from public leaderboards, and do not accept a verbal claim that "quality is good." Before the pilot, the business owner and the provider should agree in writing on the NDCG@10 improvement relative to baseline, the minimum absolute value, the p95 latency ceiling, and the error-rate ceiling, and ensure the result is reproducible and regression-testable.

7. Access, audit, and key management

  • Assign separate keys by environment and project; sharing is not acceptable.
  • Give keys the minimum necessary permissions, support rotation and revocation, and never place them in front-end code or public repositories.
  • Where supported, configure IP allowlists and per-key rate caps so one project cannot exhaust the global quota.
  • Retain call-log duration, model version, elapsed time, status code, and usage, and redact content fields according to compliance requirements.
  • The console should support member roles and operation auditing; revoke access promptly when staff leave.

When using a unified gateway, also confirm its logging policy and sub-account capabilities. Every additional relay layer adds a data processor, and both the contract and the review must cover it.

8. Fallback and disaster recovery

Reranking is an enhancement step and should not become a single point of failure. The procurement plan should specify:

  • Error codes and notification mechanisms when the service is unavailable.
  • Whether there are backup paths across regions or providers.
  • The timeout budget and the logic for reverting to vector similarity or keyword ranking after a timeout.
  • Queuing, retry, and circuit-breaking behavior under rate limits, so retry storms do not amplify an incident.
  • The migration window when a model is retired or a version changes.

Engineer the reranker as an independent interface with a switch that can turn reranking off, and rehearse the fallback regularly. Being able to revert with one switch is more useful than any promise.

9. Pilot acceptance and cost monitoring

Proceed in order rather than signing a large long-term contract immediately:

  1. Define the pilot scope: business line, query types, daily call ceiling, and success criteria.
  2. Prepare the evaluation set and baselines: no reranking, the current solution, and the candidate solution should use the same candidates.
  3. Connect a small traffic share, run side by side, and record quality, latency, error rate, and actual usage.
  4. Organize manual review, sample differences before and after ranking, and confirm that the metric improvement is not a labeling artifact.
  5. After acceptance, sign the formal contract or expand capacity, and configure budget alerts and monthly reviews.

After launch, monitor four numbers: daily calls, average candidate count, cache hit rate, and cost per business line. Optimizations include removing useless candidates, disabling reranking for low-value queries, caching frequent results, and setting budgets by project. Reducing candidates from fifty to twenty lowers both bill and latency; confirm the quality impact with the evaluation set.

FAQ

How is a Rerank API usually billed?

Common definitions include per call, per search unit, per document, and per token. Vendors define them differently, so convert to the same business volume before comparing, and rely on the contract and the in-site model pricing page.

Can a trial quota be used for a formal launch?

Most trial quotas are intended for evaluation, may restrict commercial use, and carry no production SLA. Production workloads should use paid quota or a private deployment, with confirmed invoicing, data processing terms, and incident responsibility.

What NDCG score counts as acceptable?

There is no universal number. It depends on candidate quality, labeling standards, and business tolerance. A more reasonable approach is to set an improvement target relative to baseline, then confirm absolute quality through manual review.

If data is sensitive, is private deployment the only option?

Not necessarily. You can evaluate a hosted service that commits to no retention, no training, deletion on request, and a signed data agreement. If that still fails compliance requirements, choose open-source self-hosting. Both paths should be evaluated on total cost.

Can a vendor model upgrade affect quality?

Possibly. Ranking behavior, score scale, and latency can all change. Procurement should require advance notice of version changes, pin versions where possible, and keep a regression evaluation set to run before and after any upgrade.

Next steps

To connect a reranker to an existing RAG pipeline, sign up for ZSQH AI Gateway with a small budget to manage keys, quota, and usage logs in one place, then run the pilot acceptance process from this checklist. Models and prices are governed by the model pricing page; procurement terms are governed by the contract and official documentation.

Related reading: