Embedding APIs convert text into vectors for semantic search, RAG, clustering, deduplication, and recommendations. They do not generate text, but they are the foundation of almost every knowledge-base application.

This guide explains how to call an embedding API, how to choose a model, and how to evaluate cost and retrieval quality before you build a production pipeline.

Note: Embedding model interfaces and dimensions differ by vendor. This article uses the OpenAI-compatible API as an example; refer to official or in-site documentation for specifics.

1. What Embeddings Can Do

  • Semantic search: find content by meaning instead of keyword matching.
  • RAG: chunk documents, embed them, retrieve relevant pieces, and pass them to an LLM for an answer.
  • Clustering and classification: group similar texts together.
  • Deduplication: find repeated or near-duplicate content.
  • Recommendations: surface similar articles or products.

The common thread is similarity. An embedding model maps text to a point in a high-dimensional space, and nearby points are expected to mean similar things. That property is useful even when no generative model is involved.

2. How to Call It

The OpenAI-compatible endpoint is usually:

POST /v1/embeddings

Request:

{
  "model": "text-embedding-3-small",
  "input": ["First piece of text", "Second piece of text"]
}

The response contains one vector per input. Model names and dimensions follow the model pricing page. Most providers accept batches, which is cheaper and faster than sending one request per chunk, but every model has a maximum input length. Truncate or split long text deliberately rather than letting the provider cut it silently. If your vector database expects normalized vectors, apply the same normalization to documents and queries; mixing conventions reduces similarity quality.

3. Python Example

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-key",
    base_url="https://ai.zhongshangqihang.com/v1",
)

resp = client.embeddings.create(
    model="text-embedding-3-small",
    input=["What is an API gateway", "What is an API Gateway"],
)
print(len(resp.data[0].embedding))

The two inputs above are nearly identical, so their vectors should be close. Running small checks like this is a quick way to confirm that the endpoint, key, and model name are correct before embedding a full corpus.

4. The Basic RAG Flow

  1. Clean documents and split them into chunks by paragraph or heading.
  2. Call the Embedding API for every chunk.
  3. Store the vectors in a vector database such as pgvector, Milvus, or Qdrant.
  4. When a user asks a question, embed the question as well.
  5. Retrieve the most similar chunks.
  6. Send the retrieved context and the question to an LLM.

Chunk size, overlap, and metadata all affect the result. Store the source, section title, and access permissions alongside each vector; without them, you can retrieve the right text but cannot show where it came from or whether the user is allowed to see it.

5. How to Choose a Model

Criterion What to confirm
Dimension Higher is not always better; it also raises storage and cost
Language Quality in Chinese, English, and other target languages
Maximum input The length of a single chunk
Price Billing per token
Consistency Vectors from different models cannot be mixed
Deployment Whether private deployment or data compliance is supported

Evaluate on your own queries. A model that performs well on general benchmarks may miss domain terms, product names, or abbreviations that matter to your users. If you need multilingual retrieval, test each language separately rather than assuming the average score applies everywhere.

6. How to Calculate Cost

Monthly cost =
  total document tokens × unit price
+ query tokens × unit price
+ vector storage and bandwidth

Embedding a document collection is usually a one-time cost, while queries are a recurring cost. Changing the embedding model means re-embedding the entire collection, so include that migration cost in the decision. Also account for storage growth: vectors are small individually but add up when you keep several versions of a corpus.

7. Common Pitfalls

  • Mixing old and new vectors after changing the embedding model, which corrupts retrieval.
  • Chunks that are too large to retrieve precisely, or too small to keep context.
  • Not storing source and permission metadata, so answers cannot be traced.
  • Relying on vector search alone without keyword hybrid retrieval, which makes recall unstable.
  • Ignoring multilingual quality differences.

FAQ

What is the difference between embeddings and an LLM?

Embeddings output vectors for retrieval and clustering. A large language model outputs text for generation and reasoning.

Can embeddings be deployed locally?

Yes. Open-source embedding models can run on your own infrastructure, but they require GPU capacity and ongoing operations.

Can one key call both embedding and chat models?

On a gateway, this is usually possible. The model is selected in the request parameters, so the same key can serve both endpoints.

Next Steps

If you need a unified endpoint, pay-as-you-go billing, and access to multiple models, you can register for the ZSQH AI Gateway and run a small test with both embedding and chat models. Model names and prices are listed on the model pricing page.

Related reading: