"OpenAI-compatible" has become a near-universal selling point for LLM services: change the Base URL and your existing OpenAI SDK can call a different provider's model. It sounds like a one-line configuration change, but real migrations often run into 404s, parameter errors, interrupted streaming, and inconsistent tool-call formats. The reason is simple: compatibility usually covers the most common endpoints, not every capability.

This article explains where compatibility ends, how to write the Base URL, SDK configuration, parameter differences, and migration steps, so you can switch models or providers without rewriting business code.

Note: OpenAI is a trademark of its owner, and this article is not affiliated with the vendor. Each provider's compatibility scope is defined by its documentation; models and prices are subject to the in-site model pricing.

1. What "compatible" actually covers

An OpenAI-compatible API generally reuses the same HTTP paths, request body fields, and response structures. Compatibility varies widely by endpoint:

Endpoint Purpose Typical compatibility
/v1/chat/completions Chat and text generation Most widespread; basic fields usually supported
/v1/embeddings Text vectors Common, but verify vector dimensions
/v1/models Model listing Some return only a subset or nothing
/v1/images/generations Image generation Depends on the provider
/v1/audio/transcriptions Speech transcription Depends on the provider
/v1/responses The newer Responses API Varies; confirm before migrating

So "compatible" does not mean "equivalent." The minimum bar for a service is that /v1/chat/completions returns correctly. If your workload depends on tool calling, structured output, or multimodality, verify each capability; do not rely on the marketing page.

2. How to write the Base URL

The SDK appends paths such as /chat/completions to the Base URL automatically. So the Base URL usually ends at /v1, not at a specific endpoint. Common forms:

Form Common? Notes
https://api.example.com/v1 Most common SDK appends /chat/completions
https://api.example.com/openai/v1 Some gateways Path prefix defined by the provider
https://api.example.com/v1/chat/completions Wrong Produces a duplicated path and a 404
https://api.example.com Depends on implementation Some services add /v1 automatically; do not assume

Three details:

  1. Do not add extra trailing slashes, which can produce //chat/completions.
  2. Distinguish the "console URL" from the "API URL"; the former usually cannot serve as a Base URL.
  3. If the provider's example works as written, copy it rather than rewriting the path yourself.

3. SDK setup: three equivalent forms

The official Python SDK:

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-key",
    base_url="https://api.example.com/v1",
)

resp = client.chat.completions.create(
    model="your-model-name",
    messages=[{"role": "user", "content": "Reply with OK only"}],
)
print(resp.choices[0].message.content)

The Node SDK:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: "https://api.example.com/v1",
});

const resp = await client.chat.completions.create({
  model: "your-model-name",
  messages: [{ role: "user", content: "Reply with OK only" }],
});
console.log(resp.choices[0].message.content);

You can also use environment variables to keep the URL and key out of code:

export OPENAI_API_KEY="sk-your-key"
export OPENAI_BASE_URL="https://api.example.com/v1"

The official SDK usually reads OPENAI_BASE_URL, though variable names can differ by language and version; check the SDK documentation you use. In production, put the Base URL in a configuration service and the key in a secrets manager.

4. Parameter compatibility: where migrations break

A basic chat request succeeding does not mean every parameter works. Check these before migrating:

Parameter Common difference
model Must use the provider's registered name, not the official model name
max_tokens Some newer models use max_completion_tokens; the old field may be rejected or ignored
temperature / top_p Reasoning models may fix or ignore these
stream Mostly supported, but streaming event structures may differ
tools / tool_choice Support varies widely; parallel_tool_calls may not be available
response_format json_object is common; json_schema support varies
seed / logprobs / n Many compatible services do not support these and may error if sent
reasoning_effort Only some reasoning models support it, and the values may mean something different

A safe approach is to maintain a "minimum parameter set" with only the fields the business truly needs. Disable unsupported parameters through configuration rather than hard-coding them into requests.

5. Migrating from the official API

Follow this order to keep risk contained:

  1. Inventory call sites: List every service, script, and scheduled job that uses the OpenAI SDK.
  2. Record required capabilities: Note where chat, streaming, tool calling, structured output, embeddings, and multimodality are used.
  3. Create a new key with a budget: Give the migration its own key, spend limit, and alerts.
  4. Switch the Base URL: Prefer environment variables or configuration, not business-code changes.
  5. Run smoke tests: Send one request with curl and one with the SDK to confirm the URL, key, and model name.
  6. Run a regression sample: Prepare 30–100 real requests and compare output quality, latency, error codes, and token usage.
  7. Roll out gradually: Move one non-critical workload first, observe for a week, then expand. Keep a one-step rollback.

Migration is not "swap the URL and done." Changing models can change output style, context length, and tool-call success rates. An evaluation set is more reliable than a feeling.

6. Common errors and troubleshooting

Error Likely cause Fix
401 Unauthorized Wrong, expired, or missing key Check request headers and for stray characters in the key
403 Forbidden Key lacks model permission or risk controls triggered Verify model permissions and account status
404 Not Found Wrong Base URL path Remove /chat/completions and confirm whether /v1 is needed
400 Bad Request Unsupported parameter or malformed format Eliminate parameters one by one; fall back to a minimal body
429 Too Many Requests Concurrency or quota exceeded Reduce concurrency and retry with exponential backoff
500 / 502 / 503 Upstream or gateway error Retry and log the request ID; switch to a fallback if it persists
Streaming interrupted Proxy buffering, timeout, or network jitter Disable proxy buffering and set reasonable timeouts and heartbeats

When troubleshooting, preserve the original request: URL, headers (with the key redacted), body, status code, and response body. "The call failed" alone is rarely enough to locate the problem.

7. Streaming, tool calls, and multimodality: the real differences

  • Streaming: Compatible APIs usually return Server-Sent Events starting with data: and ending with [DONE]. But incremental tool-call fields and whether usage statistics are included in the stream differ by provider. Confirm support before enabling usage tracking.
  • Tool calling: Function names and argument JSON structure are broadly similar, but parallel calls, strict mode, and reasoning fields may be missing. For complex agents, run a real tool-chain test first.
  • Structured output: json_object is usually available; JSON Schema constraints may not be. Keep parsing and validation downstream; never assume the model returns valid JSON.
  • Multimodality: Whether images use URLs or Base64, and which formats and sizes are supported, varies considerably. Validate size and format before upload.
  • Responses API: The newer endpoint is less widely covered by compatible services than Chat Completions. Confirm separately before migrating an app that depends on it.

8. Cost and billing

Billing rules can differ on compatible APIs too:

  1. Confirm whether input and output are priced separately or combined.
  2. Confirm whether cache hits and reasoning tokens are billed separately.
  3. Streaming responses do not always return usage; estimate tokens locally when needed.
  4. Split keys by project or business line so month-end reconciliation is possible.
  5. Set daily budgets and anomaly alerts to prevent retry loops from multiplying costs.

For actual rates, see model pricing. Do not extrapolate total cost from one model's rate; models, context lengths, and billing rules all change.

FAQ

Is an OpenAI-compatible API the same as the official OpenAI API?

No. Compatible means the interface format is similar; the provider, models, billing, and data terms may all differ. The same call pattern does not imply the same capabilities or compliance.

Should the Base URL include /v1?

Check the provider's documentation. Most require /v1, and the SDK appends /chat/completions; a few gateways use their own path prefix. The test is whether Base URL plus /chat/completions reaches the correct endpoint.

Why does curl work but the SDK returns 404?

A common cause is that the SDK appends a path while the Base URL already contains /chat/completions; another is an environment variable overriding the in-code configuration. Print the actual request URL and compare.

Do compatible APIs support tool calling and structured output?

Partially, depending on the provider and model. Test both with a real tool chain and a JSON Schema before migrating; do not test only plain chat.

What if answer quality drops after migration?

First confirm that the model name, temperature, and prompt have not changed, then compare on the same sample set. If needed, adjust prompts, switch models, or keep two providers with traffic splitting. Do not judge from one or two subjective impressions.

Next steps

The value of an OpenAI-compatible API is lower switching cost, but what is compatible is the protocol, not the capability. Start with a minimal request body, then enable streaming, tool calling, and structured output one by one. If you need a unified OpenAI-compatible entry point, key management, and usage statistics, register for the ZSQH AI Gateway; models and prices are listed at model pricing.

Related reading: