GeniusRelay
Sign upConsole

Blog

OpenAI-compatible API with 599 models

Point your existing OpenAI SDK at one base URL and switch between 599 models without rewriting client code.

An OpenAI-compatible API uses the same request and response shape as OpenAI's chat completions endpoint. Your existing SDK, prompt structure, streaming loop and tool-calling code keep working; only the base URL, key and model name change.

GeniusRelay provides 599 models from 37 providers through one such endpoint, including GLM, DeepSeek, Qwen, Claude, Gemini, GPT, Mistral, Meta, Anthropic and Xiaomi. You keep one account, one bill, and one integration to maintain.

What compatibility covers

The endpoint accepts the standard fields:

  • model
  • messages
  • temperature
  • top_p
  • max_tokens
  • stream
  • stop
  • presence_penalty
  • frequency_penalty
  • seed
  • tools and tool_choice
  • response_format

Responses keep the choices[].message shape, including tool calls and streaming deltas. Error bodies follow the OpenAI envelope too, with error.message, error.type and error.code, so existing retry handlers can read them. The full field list is in the API reference.

Migration in four lines

Most OpenAI SDK code moves with a base URL and key swap:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_KEY",
    base_url="https://api.geniusrelaygpt.com/v1",
)

resp = client.chat.completions.create(
    model="deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Hello"}],
)

Node looks the same:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.geniusrelaygpt.com/v1",
  apiKey: process.env.API_KEY,
});

const resp = await client.chat.completions.create({
  model: "deepseek-v4.1-flash",
  messages: [{ role: "user", content: "Hello" }],
});

If your client reads a config file instead of constructing the SDK, point it at the same URL. Cursor, Cline and SillyTavern all have guides: Cursor, Cline, SillyTavern.

Model selection

One endpoint is only useful if the models behind it are the right ones. The catalogue splits into clear groups:

  • Uncensored: 43 variants with fewer refusals — browse
  • Reasoning: 404 thinking-capable models — browse
  • Vision: 334 image-input models — browse
  • Tool calling: 430 models that accept the tools schema — browse
  • Long context: 384 models at 128K or more — browse

Cheap starting points for a smoke test:

Every model page lists input, output and cached-input rates, so you can size a run before sending it. The comparison page puts up to four side by side.

Things that differ from OpenAI, and how to handle them

Compatibility is a format contract, not a behavior guarantee.

  • Model names are not OpenAI names. gpt-4o may or may not exist; check the model page and copy the exact ID.
  • Capabilities vary. A model can ignore response_format or reject images. Confirm modalities and tags before wiring it in.
  • Context limits are per model. Sending 300K tokens to an 8K model is a 400, not a silent truncation.
  • Pricing is per model, not per account tier. Switching models changes the bill immediately.

Treat model as a deployment parameter, not a constant. Keep it in config, validate it at boot, and log it with every request so cost regressions are traceable.

Streaming and tool calling

Streaming is standard SSE. Read the delta, not the full message:

stream = client.chat.completions.create(
    model="deepseek-v4.1-flash",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

Tool calling follows the usual two-pass loop: send tools, receive tool_calls, execute them, send tool messages back. Models tagged tools are the ones to try. See the streaming guide for timeouts and keep-alives.

Cost and reliability

  • Prepaid balance. Top up first; requests fail with insufficient_quota when the balance is empty instead of billing you later.
  • Per-key limits. Set a spend cap and expiry per key so one runaway script cannot consume the account.
  • Cached input pricing. Repeated prefixes bill at the cached rate on supported models. Keep the stable part of the prompt first.
  • Failover. If a route is unavailable, traffic can move to another route. Your retry code should still handle 429 and 5xx.

Read error codes and billing rules before going to production, and start with the quickstart.

FAQ

Do I need to rewrite my app?

Usually no. If the app already uses the OpenAI SDK, change the base URL, key and model.

Can I use streaming?

Yes. Standard SSE responses are supported.

Can I use tool calling?

Yes, on models tagged tools. The schema and loop are unchanged.

Does compatibility mean identical output?

No. Different weights produce different answers. Run your own evaluation set before switching a production prompt.

Which models are cheapest?

DeepSeek V4.1 Flash and Claude Haiku 5.5 are the inexpensive mainstream options; the cheap LLM API guide covers smaller models too.

Is there an OpenAI-compatible embeddings endpoint?

This guide covers chat completions. Check the API reference for the endpoints currently exposed.

Error handling that survives production

The error envelope is compatible too, which means the usual handling ports over:

import time

def call(model, messages, tries=3):
    for attempt in range(tries):
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except Exception as exc:
            status = getattr(exc, "status_code", None)
            if status == 429 or (status and status >= 500):
                time.sleep(2 ** attempt)
                continue
            raise
    raise RuntimeError("retries exhausted")

Retry 429 and 5xx with exponential backoff. Do not retry 400 or 401: a wrong model ID or missing key will fail the same way every time. Log the request ID and model with every failure.

Concurrency and batching

The endpoint is HTTP; the limits are per key. Start with four to eight concurrent requests and raise until 429s appear, then back off. For bulk work, prefer one request with many items over many one-item requests — the repeated instructions are cheaper to send once.

from concurrent.futures import ThreadPoolExecutor

def run(row):
    return call("deepseek-v4.1-flash", [{"role": "user", "content": row}])

with ThreadPoolExecutor(max_workers=6) as pool:
    results = list(pool.map(run, rows))

Set a per-key cap before a batch. A pool of 20 workers with no cap can empty a balance in minutes.

Configuration checklist

  • Base URL in env, never hard-coded in more than one place.
  • Model ID validated at boot against the model list.
  • Timeout of 60–120 seconds, shorter for streamed calls.
  • Fallback model in the same price class.
  • max_tokens set explicitly on every call path.
  • Request ID and model logged on failure.

Client-specific setup lives in the Cursor and Cline guides. Streaming timeouts are covered in the streaming guide.

Related reading