Dify brings prompt orchestration, knowledge-base retrieval, workflows, and agents into one visual interface, but the platform's built-in model list may not cover the models you actually have. As long as the target service exposes an OpenAI-compatible endpoint, you can add it as a "custom model" and run your apps with your own key and your own billing.

In practice, most people get stuck not on which button to click but on a few details: a Base URL missing /v1, a model name that does not match what the gateway lists, a function-calling toggle left off, or Dify's network being unable to reach your internal address. This article follows the order "prepare, add, bind, troubleshoot." Interface names refer to the current version.

Note: Dify is a trademark of its owner, and this article is not affiliated with the vendor. Model names, context lengths, and prices are subject to the in-site model pricing; examples use common versions as of October 2026.

1. Confirm four things before you start

Do not open Dify yet. First verify the upstream with a minimal request. You need to confirm:

  1. Protocol: Does the API speak OpenAI-compatible, with at least /v1/chat/completions?
  2. Base URL: Most services expect the URL to end at /v1, not at the full /v1/chat/completions.
  3. Model name: It must match exactly what the gateway returns from /v1/models; watch case, date suffixes, and aliases.
  4. Capabilities: Streaming, function calling, and vision input; for a knowledge base, also confirm the embedding dimensions.

Run a quick preflight with curl:

curl https://your-gateway.example.com/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-name",
    "messages": [{"role": "user", "content": "Reply with OK only"}]
  }'

If the response includes choices, your URL, key, and model name are all correct. Getting this working before touching Dify saves half the debugging.

2. Add a model provider in Dify

Dify manages model integrations as plugins, and entry points and field names may change between versions. The typical process looks like this:

  1. Open the model provider settings page.
  2. Find the OpenAI-API-compatible plugin and install or enable it; the exact name depends on your version.
  3. Click Add Model and choose the model type.
  4. Enter the model name, API key, and Base URL.
  5. Set context length, max output, and other parameters based on the model's real capabilities.
  6. Save and click Test to confirm that it returns content.

Field-by-field guidance:

Field What to do Common mistake
Base URL Usually ends at /v1, e.g. https://api.example.com/v1 Adding /chat/completions causes 404
Model name Match the /v1/models response exactly Wrong case or version suffix returns "model not found"
API key Paste the whole string; do not add Bearer manually Extra spaces or quotes return 401
Model type Choose LLM, Embedding, Rerank, speech-to-text, or TTS by purpose The wrong type hides the model from apps
Context length Use the provider's actual supported value; err on the smaller side Setting the theoretical maximum causes upstream rejections
Capability toggles Enable streaming, function calling, and vision only if supported Agent tool calls fail or image uploads are rejected

3. Do not copy parameters from the marketing page

A model page's "maximum context" is usually a theoretical ceiling. The usable value also depends on output length, billing rules, and gateway limits. Leave some headroom:

  • Set context length to a value the provider explicitly supports; do not max it out to "look stronger."
  • Set max output separately so long generations are not truncated and costs do not run away.
  • If you are unsure about function calling, leave it off, validate with plain chat, then enable features one by one.
  • For vision models, confirm whether images are passed by URL or Base64; some compatible endpoints support only one.
  • Once an embedding model is used in a knowledge base, its dimensions are fixed; switching models usually requires rebuilding the index.

If you need several models under one provider, add them one at a time. The externally exposed model name can differ from the upstream name, but keep it readable and traceable, for example by including the purpose or version in the name.

4. Bind the model to your app

Adding a model successfully does not mean the app is using it. Check at least three places in Dify:

  • Chat assistant / text generation apps: Select the model on the app orchestration page, and make sure prompt variables do not hard-code a model name.
  • Workflows / agents: Each LLM or agent node selects its own model; different nodes can mix models.
  • Knowledge base: Set the embedding and rerank models in system model settings, and use the same pair for indexing and querying.

A practical split is to use small models for classification, rewriting, and extraction, and a strong model for final generation. This controls cost and prevents the strong model from being saturated by large numbers of short requests.

5. Common errors and troubleshooting order

Symptom Likely cause Fix
401 Unauthorized Wrong, expired, or malformed key Retest with curl and paste the key again
404 Not Found Wrong Base URL path Confirm it ends at /v1, without the full endpoint path
model not found Model name does not match the gateway listing Check /v1/models and mind the case
Request timeout Dify cannot reach upstream, or upstream is slow Check the network, DNS, firewall, and proxy
400 context length Context or output limit set higher than actual Lower the values; split long text first
429 Too Many Requests Upstream rate limit Reduce concurrency; add retries with backoff
Tool call failed Model lacks function calling or parameters are malformed Disable tools and validate plain chat first
Knowledge-base retrieval issues Embedding dimensions do not match Switch back to the original model or rebuild the index

One easily missed point: if Dify runs in the cloud or in a container, localhost, 127.0.0.1, and private network addresses are usually unreachable. Some versions also enable SSRF protection that blocks internal requests outright. Expose the model service at a public HTTPS address, or configure proxying and allowlists according to the documentation in a self-hosted environment.

6. Stability and cost

After integration, add three things:

  1. Set timeouts and retries: Base timeouts on your slowest business path. Retry only 429 and 5xx responses to avoid double billing.
  2. Log calls: Keep at least the model name, latency, token usage, and error code so you can trace a problem to a specific node.
  3. Control context: Retrieving more chunks is not necessarily better. Irrelevant passages raise cost and interfere with answers.

On cost, Dify itself only orchestrates; charges come from upstream model calls. Moving simple nodes to small models, compressing prompts, and cutting useless retries usually beats switching providers. For actual rates, see model pricing; do not build a long-term budget on fixed numbers.

FAQ

Do I have to install a plugin to use custom models in Dify?

Newer versions manage integrations through model provider plugins; names and entry points depend on your version. If your version lacks the plugin, search the plugin marketplace for OpenAI-compatible options or upgrade to a version that supports it.

Why does curl work but Dify fails?

Check three things first: whether Dify's network can reach the address; whether the UI is appending a path to your Base URL; whether the key picked up spaces or newlines. Cloud-hosted Dify being unable to reach your internal address is the most common cause.

Can I connect a non-OpenAI-compatible API directly?

Not directly. Convert it to OpenAI format through a gateway or adapter layer first. The adapter must handle message structure, tool calls, and streaming events; a URL rewrite alone is not enough.

Can one provider have multiple models?

Yes. Configure each model's name, context, and capability toggles separately, then choose per app. Use naming that reflects purpose so team members do not select the wrong one.

Can I keep using a knowledge base after changing the embedding model?

Usually not without rebuilding. Vector dimensions or semantic spaces change, so old indexes no longer match new queries. Rebuild and reindex the knowledge base, and validate retrieval quality on a small dataset before switching.

Next steps

Dify integration comes down to three points: validate the upstream with curl, set parameters to real capabilities, then confirm bindings node by node. If you want a unified OpenAI-compatible entry point and usage management, register for the ZSQH AI Gateway; models and prices are listed at model pricing.

Related reading: