Integrating an OCR API looks like nothing more than "send an image, get text back", but what decides whether a project reaches production is everything else: image format and size, synchronous versus asynchronous calls, how coordinates and fields are defined, and how to recover when results are inconsistent. These issues are invisible in a demo, but real invoices and scans expose them.

Using OpenAI-compatible interfaces and common OCR services as examples, this article explains the path from upload to recognition to structured output, with examples, preprocessing, error troubleshooting, and a migration checklist.

Note: OpenAI, Google, Microsoft, Alibaba Cloud, Tencent Cloud, Baidu, and other brands mentioned in this article are trademarks of their respective owners. This article is for technical explanation only and has no affiliation with the vendors; interface, model, image-limit, and billing rules are subject to each platform's official documentation and on-site pages.

1. Interface and authentication

OCR integration comes in two common forms. First confirm which one you are using:

  • Dedicated OCR endpoint: a standalone recognition endpoint that accepts images or PDFs and returns text blocks, coordinates, and confidence scores; some support invoice, ID, and table templates.
  • Multimodal chat endpoint: goes through /v1/chat/completions, places the image in the message, and asks a vision model to output text or JSON. It suits complex layouts, handwriting, and understanding tasks.

Authentication for OpenAI-compatible interfaces is usually:

POST https://ai.zhongshangqihang.com/v1/chat/completions
Authorization: Bearer sk-your-key
Content-Type: application/json

Set the base URL only up to /v1; do not append paths yourself before passing it to an SDK, or you will often get a 404. Some cloud providers use Access Key + Secret Key signing or exchange a token first, so the Bearer pattern does not apply directly. Follow the official documentation.

2. How to send an image: URL, Base64, and file upload

There are three common upload methods; the choice depends on where the image lives and what the endpoint supports:

Method Best for Notes
Public URL Images already in object storage or a CDN The server must be able to reach it; the link must stay valid long enough; avoid exposing sensitive images
Base64 data URL Local files, intranet images, quick tests Size grows by roughly one third; a wrong prefix or encoding causes 400
Multipart file upload Vendor-specific OCR endpoints Field names, encoding, and size limits follow the documentation

A Base64 value usually looks like data:image/jpeg;base64,<encoded content>. Common mistakes include omitting the prefix, leaving line breaks inside the string, or using URL-safe encoding without converting it back.

JPEG, PNG, and WebP are the most widely supported formats; GIF, TIFF, BMP, and PDF depend on the specific endpoint. A single image is commonly limited to a few MB up to 20 MB, and exceeding the limit usually returns 413 or 400. Higher resolution is not always better, because the server may downscale very large images. Scans are commonly 200 to 300 DPI, and phone photos often start at 1600 to 2400 pixels on the long edge; verify with real samples.

3. Synchronous or asynchronous

Synchronous endpoints suit single images and interactive scenarios: send a request and wait for the result. The risk is timeout, especially with large images, multi-page PDFs, or server-side queues, where the connection may be closed early.

Asynchronous endpoints suit batches and multi-page documents. The flow is usually:

  1. Submit the image or file and receive a task ID.
  2. Poll the status or wait for a callback.
  3. Download or query the recognition result when the task completes.
  4. Results usually have a retention period, so persist them promptly.

Do not choose based only on interface simplicity. When a request may take tens of seconds, synchronous calls occupy connections and threads, and retries amplify the pressure. Asynchronous designs must handle idempotency, duplicate submissions, polling backoff, and result expiry. In production, create a table for task IDs and record status and result location.

4. curl example

The following uses the OpenAI-compatible interface for one image recognition call. Use the model names shown on model pricing, and choose one that accepts image input:

export ZSQH_API_KEY="sk-your-key"

curl "https://ai.zhongshangqihang.com/v1/chat/completions" \
  -H "Authorization: Bearer $ZSQH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "Read the text in the image and output it line by line, with no explanation."},
          {"type": "image_url", "image_url": {"url": "https://example.com/invoice.jpg"}}
        ]
      }
    ]
  }'

For a 401, check the key and request header first; for a 400, check whether the image URL is reachable and the format is supported; for a 404, check the base URL and whether the path is duplicated.

5. Python calls and structured output

Real projects rarely need plain text alone; they need fields. Ask the model to output JSON according to a schema, then validate it locally:

import base64
import json
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ZSQH_API_KEY"],
    base_url="https://ai.zhongshangqihang.com/v1",
)

with open("invoice.jpg", "rb") as f:
    image_b64 = base64.b64encode(f.read()).decode()

schema_hint = '{"invoice_no": "", "date": "", "amount": "", "seller": ""}'
resp = client.chat.completions.create(
    model=os.environ["OCR_MODEL"],  # a model name from the pricing page that accepts image input
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": f"Read the invoice and output this JSON, using null for fields you cannot find: {schema_hint}"},
            {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_b64}"}},
        ],
    }],
)
data = json.loads(resp.choices[0].message.content)
print(data)

This code runs, but that does not mean it is production-ready. Model output may include code fences, explanatory text, or extra fields, and json.loads will fail immediately. In practice: prefer the platform's JSON mode or structured output; strip code fences before parsing; validate types with Pydantic or jsonschema; on failure, send the error back to the model for one retry, then route to a human. Cross-check amounts, dates, and tax IDs as well.

6. Coordinates, tables, and structured fields

Traditional OCR returns text-block content, coordinates, and confidence; multimodal models follow the prompt, and their coordinates are not always reliable. Confirm the coordinate format first: is it a four-point polygon, [x1, y1, x2, y2], or [x, y, w, h]; are units pixels or normalized to 0 through 1000; is the origin at the top left; and how must coordinates be converted after rotation and cropping?

If you resized the image before recognition, map coordinates back to the original image by the same ratio, or highlight boxes will be offset. For tables, define row, column, merged-cell, and empty-value rules explicitly; for complex tables, use a dedicated table-recognition endpoint and then let a model organize the result.

Define the schema for structured fields in advance: field names, types, required status, null representation, and how dates and amounts are normalized. Invoices also need currency, tax amount, and the arithmetic relationship between subtotal, tax, and total. Confidence can drive routing: send low-confidence fields to human review instead of discarding them.

7. Preprocessing: make recognition more stable

Most recognition error is decided before the request is sent. Process in this order:

  1. Read the EXIF orientation and rotate the image so it is not sideways.
  2. Crop to the document area and remove large backgrounds and unrelated objects.
  3. Correct small skew angles; detect edges first when the tilt is obvious.
  4. Control resolution: 200 to 300 DPI for scans, and 1600 to 2400 pixels on the long edge for phone photos as a starting point.
  5. For faded or low-contrast documents, convert to grayscale and increase contrast moderately.
  6. Split multi-page PDFs or long screenshots by page, keeping a small overlap at split points.

Sharpening cannot rescue a blurry image, and enlarging low-resolution text does not create detail. The goal of preprocessing is to reduce interference, not to make the image look attractive. Whenever you change parameters, compare field accuracy on the same batch of real samples.

8. Common errors and troubleshooting

Error Common cause What to do
401 authentication error Wrong, expired, or badly formed key Check the Bearer format, spacing, and environment variables
403 Forbidden No model permission, region restriction, insufficient sub-account rights Check model authorization and account policy
400 image error Wrong Base64 prefix, corrupted file, unsupported format Decode and inspect the file header; switch to JPEG or PNG
413 Payload Too Large Image or request body exceeds the limit Compress, resize, crop, or use file upload or async processing
429 Too Many Requests Rate or concurrency limit exceeded Reduce concurrency and retry with exponential backoff
Timeout or 502 Large image, long document, server-side queue Switch to an async task or shrink the request
Empty or disordered result Wrong orientation, low resolution, complex layout Rotate, increase clarity, and split columns into blocks

During troubleshooting, record the request ID, model name, image dimensions, and latency. Many cases of "the model performs poorly" are actually a wrong image, unconverted coordinates, or an inconsistent schema.

9. Migration and launch checklist

Moving from one OCR service to another usually involves more than changing base_url and api_key. Before launch, confirm:

  • Whether the new endpoint is dedicated OCR or conversational vision, and whether the request body is compatible.
  • Which image formats, single-image size, page count, and resolution limits are supported.
  • Whether it is synchronous or asynchronous, how long results are retained, and whether callbacks need a public address.
  • Whether the coordinate format, units, and origin match the old service.
  • Whether the field schema, language support, and invoice types cover existing business.
  • Whether there is a canary plan to shift part of the traffic and compare accuracy.
  • Whether there is a rollback path, keeping the old key and endpoint for a period.

Encapsulate OCR in a dedicated client, such as recognize(image, schema) -> Result, and keep vendor differences inside an adapter layer so business code does not need rewriting when you switch services. Models, limits, and prices are subject to model pricing.

FAQ

Does the OCR API support PDF?

It depends on the endpoint. Dedicated OCR services usually support PDF, and multi-page documents usually go through async processing; OpenAI-compatible chat endpoints generally treat input as images, so PDFs must be split into pages and converted, or handled through a file endpoint. Follow the documentation.

What is the maximum image size?

It varies. Common limits are a few MB to 20 MB per image, plus pixel and page-count limits. Compress and resize before upload to reduce timeouts and transfer cost.

Why is the recognition result out of order?

Common causes are multi-column layouts, tables, and mixed images. Traditional OCR sorts by coordinates, while multimodal models generate in reading order; both can make mistakes. Split by column, request output by region, or switch to an endpoint with layout analysis.

What is the most reliable way to get structured fields?

Define the schema first, have the model output JSON, and validate types, patterns, and business rules locally. Prefer structured output where the platform supports it; otherwise add one retry with the error message, then route to a human.

What must change when migrating from one OCR service to another?

Besides the base URL and key, check the request structure, sync versus async behavior, image limits, coordinate format, schema, and rate-limit rules. Start with a canary, compare field accuracy on real samples, then switch all traffic.

Next steps

To get one request working first, create a low-quota key on the sign-up page and test upload, recognition, and JSON output with a real invoice. When choosing a production service, check model pricing and compare field accuracy on a small labeled image set. Prices change, so use the on-site page as the source of truth.

Related reading: