LangChain is not a model service. It connects prompts, models, tools, and output parsers into an application. As long as the target service offers an OpenAI-compatible endpoint, you can connect it with ChatOpenAI from langchain-openai and point requests at your own base URL. What usually blocks people is not the framework but three fields: which path level the address should include, where the key is read from, and whether the model name matches what the gateway actually hosts.

This article follows the practical order: install, minimal example, environment variables, chains, streaming, agents, troubleshooting, and migration. LangChain releases new versions quickly, so class names and parameters should be checked against the documentation for the version you installed.

Note: LangChain is a trademark of LangChain, Inc.; OpenAI is a trademark of its respective owner. This article is not affiliated with these vendors. Model names and prices follow the model pricing page, and interface details follow vendor documentation.

1. Install: two core packages

After creating a virtual environment, install:

python -m venv .venv
source .venv/bin/activate
pip install -U langchain langchain-openai

langchain-openai is the integration package for OpenAI and compatible endpoints, langchain provides higher-level capabilities such as chains and agents, and langchain-core is installed as a dependency. In production, pin versions in requirements.txt so different machines do not end up with inconsistent APIs. The old import path langchain.chat_models.ChatOpenAI has moved; the current path is langchain_openai.

2. Minimal example: ChatOpenAI pointed at a custom address

The core is three parameters: model, base_url, and api_key. base_url usually ends at /v1; do not append /chat/completions, because the client adds the path itself.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="your-model-name",
    base_url="https://your-gateway.example.com/v1",
    api_key="sk-your-key",
    temperature=0.3,
    timeout=60,
)

resp = llm.invoke("Explain in one sentence what an OpenAI-compatible API is.")
print(resp.content)

Get this block working before adding prompts and tools. A sensible verification order:

  1. Use curl to request base_url plus /chat/completions and confirm the address and key work.
  2. Create ChatOpenAI in Python and call invoke once.
  3. Print resp.content and resp.response_metadata to check the model name and usage fields.
  4. Only then add prompt templates, output parsers, and tools.

If you get a 404 here, check the path level first. A 401 points to the key or the authentication header. A model-not-found error means you should compare names against /v1/models. The model name must exactly match what the gateway returns, including case, date suffixes, and aliases.

3. Managing parameters and environment variables

Do not hardcode keys in source code. The OpenAI SDK reads OPENAI_API_KEY, and it can also read OPENAI_BASE_URL; some older code and documentation use OPENAI_API_BASE. Behavior differs slightly across versions. The safest approach is to pass parameters explicitly and treat environment variables only as defaults.

Parameter Purpose Common form
model Model identifier Exactly what /v1/models returns
base_url API root address https://host/v1
api_key Authentication key Pass explicitly or read OPENAI_API_KEY
temperature Sampling randomness Within the range the model supports
timeout Per-request timeout Adjust to task length
max_retries Client retry count Usually around 2

For local development you can use a .env file with python-dotenv, but .env must be listed in .gitignore. In team settings, split keys by project and environment, with separate limits for production and testing, so a problematic key can be disabled on its own.

4. Chains: connecting prompts and models with LCEL

The LangChain Expression Language (LCEL) composes components with the pipe operator. The most basic chain is "prompt → model → output parser":

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a technical support assistant. Be specific and lead with the conclusion."),
    ("user", "{question}"),
])

chain = prompt | llm | StrOutputParser()
print(chain.invoke({"question": "Which path level should the base URL include?"}))

invoke takes one input and returns one result, batch takes a list, and stream returns chunks. The output parser converts the model message into a string or a structured object. When you need JSON, prefer the structured-output feature the model supports, then add parsing and validation; do not rely on prompt wording alone.

5. Streaming and batch calls

Streaming suits chat interfaces and long answers:

for chunk in chain.stream({"question": "Explain what tool calling is"}):
    print(chunk, end="", flush=True)

If invoke works but stream does not produce incremental text, a proxy in the path may be buffering, or the upstream may not support streaming. Check whether the gateway returns text/event-stream and whether the reverse proxy has buffering disabled. For batch calls, watch concurrency: batch may send requests in parallel by default. On 429 responses, lower concurrency or split the batch, and keep failed items so they can be retried.

6. Agents: model plus tools plus a loop

An agent is essentially a loop: the model decides whether to call a tool, the runtime executes the tool and returns the result, and the model decides the next step until it produces a final answer. LangChain 1.x provides high-level interfaces such as create_agent; version 0.3 and earlier commonly used AgentExecutor. The two styles differ, and copying older tutorials often leads to import failures.

Tools can be defined with a decorator. The function name, parameter types, and docstring all influence how the model selects the tool:

from langchain_core.tools import tool

@tool
def get_order_status(order_id: str) -> str:
    """Look up shipping status by order number."""
    return "Shipped, expected to arrive within three days"

With a custom gateway, note that tool calling depends on full upstream support for tools, tool_choice, and the result format. Some compatible services implement only basic chat and will ignore parameters or fail when a tool call appears. Verify with one tool and one simple question before adding more tools and parallel calls.

7. Common errors and troubleshooting

Symptom Likely cause Action
401 authentication error Invalid key, missing key, or stray spaces Pass api_key explicitly; check environment variables
404 model_not_found Model name mismatch or extra path in the address Query /v1/models; keep base_url at /v1
429 Too Many Requests Too much concurrency or quota exhausted Reduce concurrency, split batches, retry with backoff
400 context_length_exceeded Input exceeds the context limit Trim history, split documents, summarize first
OutputParserException Output does not match the expected format Constrain the format, validate, retry once
APIConnectionError / timeout Network, proxy, certificate, or upstream failure Check proxy variables, DNS, and timeout settings
Tool calls do nothing Upstream does not support the tools protocol Use a model that supports tool calling or a plain chain

When troubleshooting, bypass LangChain first and call the same address with curl or the OpenAI SDK. If the direct call works and LangChain fails, the problem is usually in parameter names, environment variables, or versions. If both fail, the issue is in the gateway, the key, or the network.

8. Migrating from older patterns

  • Import path: change langchain.chat_models.ChatOpenAI to langchain_openai.ChatOpenAI.
  • Parameter names: model_name becomes model, and openai_api_base becomes base_url.
  • The legacy OpenAI text-completion class is giving way to chat models; new projects should use ChatOpenAI directly.
  • Some legacy chains and agents moved into compatibility packages in 1.x. Read the vendor migration notes before upgrading, and do not mix major versions.
  • Dependency management: pin versions before upgrading and run the core path in a test environment, with emphasis on streaming, tool calling, and structured output.

FAQ

Should the base_url include /v1?

Most OpenAI-compatible services expect /v1, and the client appends /chat/completions. Some gateways place the version prefix elsewhere, so follow the provider's documentation. A simple test: a curl request to base_url + /chat/completions should succeed.

Can I use the OPENAI_BASE_URL environment variable?

Yes, the OpenAI SDK reads it; some older documentation calls it OPENAI_API_BASE. Support varies across versions, so production code should pass base_url explicitly to avoid sending requests to the wrong address.

Why does invoke work while stream returns nothing?

Common causes are proxy buffering, an upstream that does not support streaming, or code that treats the stream as a complete string. Confirm that the response type is an event stream, then check the reverse-proxy configuration and how the stream is read.

Can LangChain connect to non-OpenAI models such as Claude or DeepSeek?

Yes. If the service offers an OpenAI-compatible endpoint, use ChatOpenAI with a different address and model name. If you use a vendor's native protocol, use the matching integration package. The parameters and tool-calling capabilities differ, so verify each path separately.

What should I do when an agent reports a parameter error on a tool call?

First check whether the tool-call structure returned by the model follows the protocol, then check the tool function's parameter types and required fields. Some compatible gateways have incomplete support for parallel tool calls; disable parallel calls and give the agent one tool at a time to narrow down the cause.

Next steps

The key to connecting LangChain to a custom API is to lock down three things: the address, the key, and the model name. Verify them with a minimal request, then add chains and tools. If you need a unified OpenAI-compatible entry point, virtual keys, and usage records, you can sign up for ZSQH AI Gateway; model and pricing details are on the model pricing page.

Related reading: