An Embedding API does not generate text; it maps text into a sequence of numbers. Semantic search, RAG, clustering, deduplication, and recommendations all work from that sequence. Integration itself is not hard. The difficult parts are dimensions, batching, normalization, and indexing. Get those details wrong and retrieval quality degrades quietly.

Using the OpenAI-compatible interface as an example, this article covers requests and responses, Python and curl calls, batching strategy, vector storage, RAG wiring, common errors, and migration checks.

Note: OpenAI, PostgreSQL, and other brand names mentioned here are trademarks of their respective owners. This article is for technical explanation only and implies no affiliation with those vendors. Interfaces, models, and prices are subject to official documentation and on-site pages.

1. Endpoint and authentication

The OpenAI-compatible embedding endpoint is:

POST https://ai.zhongshangqihang.com/v1/embeddings
Authorization: Bearer sk-your-key
Content-Type: application/json

Write the Base URL only up to /v1. Do not append /embeddings yourself and then hand it to an SDK. The SDK adds the path automatically based on the method; duplicate segments produce a 404.

The request body needs only two fields at minimum:

{
  "model": "text-embedding-3-small",
  "input": ["Who pays for return shipping", "Who covers the logistics cost of a return"]
}

input can be a single string or an array of strings. An array returns multiple vectors in order, which suits batch processing. Empty strings are usually rejected, so filter them out during cleaning.

2. Request and response fields

Field Location Description
model Request Required. Model names follow platform documentation and model pricing
input Request String or array of strings; keep each item under the model limit
dimensions Request Supported by some models only; shortens output dimensions
encoding_format Request Usually float; base64 can reduce transfer size
data[].embedding Response Array of vector values, ordered to match the input
data[].index Response Position of the input within the array
usage Response prompt_tokens and total_tokens, for reconciliation

The index field matters. On retries or concurrent merges, do not assume the response order always matches the input order. Aligning by index is more reliable.

3. Python call and similarity

After installing the SDK, a minimal example:

import os
from math import sqrt
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ZSQH_API_KEY"],
    base_url="https://ai.zhongshangqihang.com/v1",
)

resp = client.embeddings.create(
    model="text-embedding-3-small",
    input=["Who pays for return shipping", "Who covers the logistics cost of a return"],
)
a, b = resp.data[0].embedding, resp.data[1].embedding
cos = sum(x * y for x, y in zip(a, b)) / (
    sqrt(sum(x * x for x in a)) * sqrt(sum(y * y for y in b))
)
print(len(a), round(cos, 4), resp.usage.total_tokens)

This snippet verifies three things at once: the key works, the dimension matches expectations, and two similar texts produce a sensible similarity score. In production, use NumPy or the distance functions built into your vector store instead of running pure Python over large volumes.

4. Dimensions, normalization, and model consistency

Dimensions of common OpenAI embedding models:

  • text-embedding-3-small: 1536 dimensions.
  • text-embedding-3-large: 3072 dimensions.
  • text-embedding-ada-002: 1536 dimensions.

The dimensions parameter can shorten the output of the text-embedding-3 family, for example from 3072 to 1024, saving storage and bandwidth. Shortening loses some information. Whether that trade-off is worth it should be validated on your own evaluation set, not decided from the dimension number alone.

OpenAI's documentation states that its embeddings are normalized to length 1, so cosine similarity can be computed directly with a dot product. If you use a self-hosted model, a third-party model, or manually truncated dimensions, re-apply L2 normalization. The safest approach is to use one normalization routine for both writes and queries and to document it in code.

There is also a hard constraint: every vector in a vector store must come from the same model, the same dimension, and the same normalization method. After switching models, old vectors cannot be mixed with new ones, or similarity scores become meaningless.

5. How to split batch requests

Batching reduces HTTP overhead, but bigger is not always better. OpenAI's current documentation sets these boundaries: each input item may not exceed roughly 8K tokens, and all inputs in one request may not exceed 300,000 tokens in total. Specific limits follow the provider's documentation.

In practice, split by token budget rather than a fixed number of items:

  1. Clean the text first, removing whitespace, duplicates, and overly long passages.
  2. Estimate tokens per item with a tokenizer and split long text into chunks beforehand.
  3. Group items by total token budget, for example a few tens of thousands of tokens per batch, leaving room for retries.
  4. Use exponential backoff for 429 and 5xx responses. Do not blindly retry 4xx parameter errors.
  5. Record the model, dimension, token count, and latency for each batch to support reconciliation and reproduction.

Retries should be idempotent in design. Use "text hash + model name + dimension" as a cache key to avoid paying twice for the same text. When a batch job fails partway, first determine which index values succeeded, then fill in only the missing parts.

6. Storing vectors

Taking PostgreSQL with pgvector as an example, the table can be simplified to:

CREATE TABLE chunks (
  id bigserial PRIMARY KEY,
  content text NOT NULL,
  source text,
  embedding vector(1536)
);
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);

Query with cosine distance:

SELECT id, content
FROM chunks
ORDER BY embedding <=> $1
LIMIT 5;

Beyond vectors, store at least the original text, source, tenant or access scope, chunk position, model name, and dimension. Without this metadata, answers cannot cite sources or filter by permission. Production deployments also need to consider index type, recall parameters, backups, and incremental updates. Do not wait until the data volume grows to add them.

7. RAG wiring order

A working RAG pipeline usually looks like this:

  1. Parse and clean documents, removing headers, footers, and repeated content.
  2. Chunk by heading or paragraph, keeping a small overlap between chunks.
  3. Call embedding for each chunk and write vectors to the store in batches.
  4. When a user asks a question, embed the question with the same model.
  5. Retrieve Top-K by cosine similarity, then filter by permission and source.
  6. Add keyword search for hybrid recall, or add a reranking model, when needed.
  7. Pass the retrieved results and the question to a chat model and require cited answers.

There is no universal chunk size. For Chinese documents, a common starting point is 300 to 800 characters, adjusted based on Q&A results. Chunks that are too large retrieve imprecisely; chunks that are too small lack context. Rather than agonizing over parameters, build a small evaluation set with fixed questions and reference answers, and run it every time you change the pipeline.

8. Common errors and troubleshooting

  • 401 Unauthorized: wrong key, deleted key, missing Bearer prefix, or an empty environment variable.
  • 404 Not Found: Base URL missing /v1, or a duplicated path segment.
  • model not found: wrong model name, or the account lacks access to that model.
  • 400 input too long: a single item exceeds the model limit; chunk it before requesting.
  • 400 empty input: the array contains an empty string; filter it during cleaning.
  • 429: rate or concurrency limit exceeded; lower concurrency and apply exponential backoff.
  • Dimension mismatch: writes and queries use different models or different dimensions.
  • Similarity all zeros or unusually close: check normalization, whether the wrong field is read, and whether zero vectors were written.

When debugging, print the model name, number of inputs, length of each item, and returned dimension first. Most problems are not "the model performs poorly" but some inconsistency between the request and the storage layer.

9. Migration and launch checks

Migrating from an official API or another gateway to an OpenAI-compatible gateway usually requires changing only two things: base_url and api_key. Embedding has one extra cost, though: if the model or dimension changes, historical vectors must be rebuilt.

Before launch, check at least:

  • Whether the new platform supports the model names and dimensions you use.
  • Whether the whole store must be re-embedded, and whether the rebuild window and cost are acceptable.
  • Whether the query side and write side switch at the same time, avoiding a mix of old and new vectors.
  • Whether there is a canary plan to shift part of the traffic and compare recall first.
  • Whether monitoring covers error rate, latency, token consumption, and empty-result rate.
  • Whether there is a rollback path, keeping old keys and old indexes for a period after the switch.

If you only change the Base URL and keep the model and dimensions, migration is much simpler. If the model changes too, schedule it as a data rebuild project. For current models and prices, refer to model pricing.

FAQ

How is the Embedding endpoint different from Chat Completions?

Embedding returns fixed-length vectors for retrieval, clustering, and similarity computation; Chat Completions returns generated text. Authentication is usually the same, but request bodies and response structures are entirely different.

How many texts can I send at once?

The endpoint accepts an array of strings, but the real limits are total tokens and per-item length. OpenAI's current documentation lists 300,000 tokens per request and about 8K tokens per item. In practice, split by token budget rather than running right at the limit.

Do vectors need to be normalized manually?

If the platform explicitly states that vectors are normalized, cosine similarity can be computed with a dot product. For self-hosted models, manually truncated dimensions, or uncertain sources, apply the same L2 normalization to both writes and queries.

After changing the embedding model, can old vectors still be used?

They cannot be mixed directly. Vector spaces from different models are not comparable, and identical dimensions do not imply semantic alignment. Rebuild the index after switching models and record the model name and dimension in metadata.

Do Chinese texts need word segmentation first?

Usually not. Modern embedding models process raw text directly; segmentation may even lose context. What needs work is cleaning, chunking, deduplication, and length control.

Next steps

To get one request working first, create a low-limit key from the sign-up page and adapt the curl or Python example above to your own text. When choosing a production model, check model pricing and compare recall on a small set of real business texts. Prices and models change, so rely on what the site currently shows.

Related reading: