GLM API Python: Chat, Streaming and Tools Examples

Install and first request

Create a virtual environment and install the SDK. Version 1 or later of the openai package is assumed throughout.

Copypython -m venv .venv
source .venv/bin/activate
pip install --upgrade openai

Get a key from the API key page; new users can claim a $0.50 trial with one click, valid for 7 days and no card needed. Keep the key in an environment variable, then make the first call:

Copyimport os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.glmapi.cc/v1",
    api_key=os.environ["GLM_API_KEY"],
)

resp = client.chat.completions.create(
    model="uncensored",
    messages=[
        {"role": "system", "content": "You are a concise writing assistant."},
        {"role": "user", "content": "Suggest five titles for a space-western novel."},
    ],
    temperature=0.8,
)

print(resp.choices[0].message.content)
print(resp.usage)

That is the whole integration. The service is an independent, OpenAI-compatible endpoint for one open-weight model, so it is not the official API of the vendor behind GLM or of anyone else. Only POST /v1/chat/completions and GET /v1/models exist. The quickstart lists the supported request fields.

Two small habits help from the start. First, call client.models.list() once in a smoke test: it hits GET /v1/models, so it proves that the key, the base URL and the network path all work before you spend a single token on a completion. Second, print resp.usage during development. Prompt and completion token counts show you early whether a system prompt is larger than you thought, and at $0.15 per 1M input and $0.50 per 1M output you can price a call in your head.

Multi-turn chat and sampling settings

The API is stateless, so you keep the conversation list yourself and send it each time. A small class is enough:

Copyclass Chat:
    def __init__(self, client, system, max_turns=12):
        self.client = client
        self.system = system
        self.max_turns = max_turns
        self.history = []

    def say(self, text):
        self.history.append({"role": "user", "content": text})
        window = self.history[-self.max_turns * 2:]
        resp = self.client.chat.completions.create(
            model="uncensored",
            messages=[{"role": "system", "content": self.system}] + window,
            temperature=0.9,
            top_p=0.95,
            frequency_penalty=0.3,
        )
        reply = resp.choices[0].message.content
        self.history.append({"role": "assistant", "content": reply})
        return reply

The slice self.history[-self.max_turns * 2:] keeps the last twelve user and assistant pairs. A real application might also pin a few early messages, such as a character sheet or key facts, and summarise the middle. Trimming to the last 12 turns keeps input cost flat as a conversation grows. The context window is 100,000 tokens for prompt and completion together, so this is a cost choice more than a necessity. The parameters temperature, top_p, stop, seed and the penalties are passed through. Set seed when you want repeatable runs while testing a prompt, and leave max_tokens out unless you want to cap the length.

Streaming responses

Streaming uses server-sent events and gives you tokens as they are generated, which matters for any interactive tool. Pass stream=True and iterate:

Copystream = client.chat.completions.create(
    model="uncensored",
    messages=[{"role": "user", "content": "Describe a harbour at dawn in 120 words."}],
    stream=True,
)

parts = []
for chunk in stream:
    if not chunk.choices:
        continue
    delta = chunk.choices[0].delta.content
    if delta:
        parts.append(delta)
        print(delta, end="", flush=True)

text = "".join(parts)

The guard on chunk.choices matters because the final chunk carries usage and may have an empty choices list. One operational point: a streamed request holds its concurrency slot until the stream ends. Wrap the loop in a try block with a finally clause that closes the stream if your code raises halfway through, for example when a client disconnects. Always finish or close the stream, otherwise a loop that abandons streams will hit the concurrency limit while doing no useful work.

JSON mode

When the output feeds other code, ask for a JSON object. Mention JSON in the prompt as well, so the model knows what shape you expect:

Copyimport json

resp = client.chat.completions.create(
    model="uncensored",
    response_format={"type": "json_object"},
    messages=[
        {"role": "system", "content": "Reply with a JSON object only."},
        {"role": "user", "content": "Give a character with keys name, age, motto."},
    ],
)

data = json.loads(resp.choices[0].message.content)
print(data["name"])

Still wrap json.loads in a try block in production code. JSON mode makes valid output likely, and a request cut off by a stop sequence or your own limit can still leave a truncated object. Retrying once with the same input is a cheap fix, and since failed requests are not charged, the cost of a retry is only the call that succeeds.

Tool calling

Tool calling follows the OpenAI shape: you describe functions in tools, the model returns tool_calls, you run them and send results back with the role tool. A complete round trip:

Copyimport json

tools = [{
    "type": "function",
    "function": {
        "name": "roll_dice",
        "description": "Roll a die with the given number of sides.",
        "parameters": {
            "type": "object",
            "properties": {"sides": {"type": "integer"}},
            "required": ["sides"],
        },
    },
}]

messages = [{"role": "user", "content": "Roll a twenty-sided die for me."}]
first = client.chat.completions.create(
    model="uncensored", messages=messages, tools=tools, tool_choice="auto"
)
msg = first.choices[0].message

if msg.tool_calls:
    messages.append(msg)
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = {"value": 17, "sides": args["sides"]}  # replace with real logic
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })
    final = client.chat.completions.create(
        model="uncensored", messages=messages, tools=tools
    )
    print(final.choices[0].message.content)
else:
    print(msg.content)

The loop matters more than the details of the dice. The model asks for a call, your code runs it, and you append both the assistant message and the tool result before asking again. Skip the assistant message and the tool_call_id in the result will have nothing to match, which is the usual cause of a confused second reply. With tool_choice set to auto the model decides whether to call a tool at all, so always handle the branch where it answers directly, as the code above does. Streamed responses can carry tool_calls as well, delivered in pieces that you join by index before parsing the arguments.

Validate the arguments before acting on them. The model can produce a malformed call, and your code should treat the arguments as untrusted input, the same as anything from a user.

Async batches within the limits

For many prompts at once, use AsyncOpenAI with a semaphore matching your concurrency limit. A full key allows 8 concurrent requests, a trial key 2:

Copyimport asyncio, os
from openai import AsyncOpenAI

client = AsyncOpenAI(
    base_url="https://api.glmapi.cc/v1",
    api_key=os.environ["GLM_API_KEY"],
)
gate = asyncio.Semaphore(8)  # use 2 on a trial key

async def one(prompt):
    async with gate:
        r = await client.chat.completions.create(
            model="uncensored",
            messages=[{"role": "user", "content": prompt}],
        )
        return r.choices[0].message.content

async def main():
    prompts = [f"Write a one-line tagline for product {i}." for i in range(40)]
    results = await asyncio.gather(*(one(p) for p in prompts))
    print(len(results))

asyncio.run(main())

The semaphore handles concurrency but not the per-minute cap of 300 requests. For long batches add a small delay or a token-bucket limiter. At $0.15 per 1M input and $0.50 per 1M output tokens, 40 tagline requests of about 60 tokens in and 25 out cost $0.00036 + $0.0005 = $0.00086 in total, so the limits, not the price, shape batch design. See the pricing page for top-ups.

Rate limits, retries and headers

Over a limit, the API returns HTTP 429. The error code is rate_limited for too many requests per minute and concurrency for too many at once. Temporary overload also returns 429 with a Retry-After: 5 header. The SDK retries some failures on its own and you can set max_retries on the client, but explicit handling is clearer:

Copyimport time
from openai import RateLimitError

def ask(prompt, tries=4):
    for attempt in range(tries):
        try:
            raw = client.chat.completions.with_raw_response.create(
                model="uncensored",
                messages=[{"role": "user", "content": prompt}],
            )
            print("balance:", raw.headers.get("X-Balance-USD"))
            return raw.parse().choices[0].message.content
        except RateLimitError as e:
            wait = float(e.response.headers.get("Retry-After", 2 ** attempt))
            time.sleep(wait)
    raise RuntimeError("still rate limited")

Here with_raw_response gives access to X-Balance-USD, X-Request-Id and the rate-limit headers. Log the request ID with any error you want to report. Never put the key in a notebook you commit or in front-end code.

Testing without spending credit, and common mistakes

Most of a Python integration is plumbing, and plumbing does not need a live model. Keep the network call behind one function, and your tests can replace it. With the standard library only:

Copyfrom unittest.mock import MagicMock

def reply_text(client, prompt):
    r = client.chat.completions.create(
        model="uncensored",
        messages=[{"role": "user", "content": prompt}],
    )
    return r.choices[0].message.content.strip()

def test_reply_text_strips_whitespace():
    fake = MagicMock()
    fake.chat.completions.create.return_value.choices = [
        MagicMock(message=MagicMock(content="  hello  "))
    ]
    assert reply_text(fake, "hi") == "hello"

Run a handful of real calls only at the edges: one smoke test per deploy, plus an occasional run of your saved prompt set. A single smoke test with about 100 tokens in and 20 out costs $0.000015 + $0.00001 = $0.000025, which is nothing, and failed requests are not charged at all. The $0.50 trial credit is far more than a test suite needs.

For prompt work, fix seed and temperature so you can compare two versions of a system prompt on equal terms. Changing both the prompt and the sampling settings between runs makes the results impossible to read. Store each run's output with the request ID from the X-Request-Id header, and you can trace a surprising answer later.

Common Python mistakes

These cause most of the support questions that come from Python users:

• Passing the key as OPENAI_API_KEY only. If you set base_url in code but forget the key variable, the SDK sends an empty or wrong key. Pass api_key explicitly.
• Forgetting /v1. The base URL ends in /v1. The SDK adds /chat/completions itself.
• Using a model name from another service. Only uncensored is valid here.
• Unbounded concurrency. asyncio.gather over a thousand coroutines with no semaphore produces a wall of concurrency errors.
• Treating a streamed reply as one string. Join the deltas, and skip chunks with an empty choices list.
• Ignoring the context limit. History plus completion must fit in 100,000 tokens. Trim old turns before you reach it.

For top-ups and the current rate card, see the pricing page. The service is text only, so there are no image, audio or embedding calls to wire up, and prompts are not used for training.

More guides

• GLM API Base URL: Connect Any OpenAI Client
• GLM API Pricing: $0.15 In, $0.50 Out per 1M Tokens
• Gemma API: Myths vs Facts