vLLM 0.32 Breaking Changes Explained: Locked-Down CORS and Template Limits

Things that worked yesterday stop working, and most of them fail without a clear message. A web page that called vLLM straight from the browser gets a CORS error, even though curl still works.

vLLM is open-source software that runs a large language model on your own server, usually a GPU server, and answers chat requests over HTTP, much like a hosted AI API. These vLLM breaking changes are not bugs. Version 0.32.0 changed several defaults to close security gaps, and each one can break a setup that relied on the old, open behaviour.

Picture a common setup. A small team runs vLLM on one GPU server inside the office network. An internal chat page at chat.example.com calls the server's chat endpoint directly from the browser with fetch. The setup is made up, but every break below is real on 0.32.0.

After the upgrade, the chat page shows a network error. The server's access log shows OPTIONS requests answered with 400. A teammate's Python script, which calls the same endpoint, still works. This guide explains why, and what to change.

• Origin: the scheme, host and port of a web page, such as https://chat.example.com. Two pages share an origin only if all three match.
• CORS: a browser rule. The server says, through response headers, which other origins may read its replies. The browser enforces it, not the server.
• Preflight: a short OPTIONS request the browser sends first, asking whether a real request is allowed. A cross-origin JSON POST always triggers one.
• Chat template: a small Jinja program stored with the model. It turns a list of chat messages into the single text string the model reads.
• max_tokens: the most tokens (word pieces) the model may write in one reply. The context window, max_model_len in vLLM, is the limit on prompt plus reply together.

Four changes in vLLM 0.32.0 hit the most deployments without a clear error: the CORS default, chat template limits, the image format allowlist and the max_tokens rule. The table shows each one.

The release notes list more breaking changes. Python 3.11 as the minimum and newer transformers and tokenizers ranges fail at install time. Removed options, such as --mamba-cache-mode all, fail at startup.

A few narrower changes alter API output quietly. Responses API streams now mark reasoning with content_part events instead of reasoning_part events. Two logprob-related fields changed format, and watermarking, if configured, now applies when a request omits the field. Read the full list if you use those features.

Because vLLM 0.32.0 no longer tells browsers that every website may read its replies. In 0.31.0 the --allowed-origins flag defaulted to ["*"], a wildcard. In 0.32.0 it defaults to an empty list.

vLLM passes this list straight to the CORS middleware from Starlette, the web toolkit under FastAPI. With an empty list, no reply carries the Access-Control-Allow-Origin header. Browsers then refuse to hand the reply to the page's JavaScript.

Think of the browser as a careful courier between a shop, the web page, and a warehouse, vLLM. The warehouse labels each parcel with the one shop allowed to receive it, or any shop, and the courier checks the label. Where the picture breaks: a real warehouse would refuse the wrong shop, but vLLM relies on the courier alone. A script or curl comes without a courier and still gets the parcel.

The pull request explains why. With the old default, any website a user visited could read responses from a vLLM server that the user's browser could reach, such as one on the office network.

CORS is not a lock on the server

Pause and predict: a Python script sends the same POST with an Origin header vLLM does not allow. Does the server refuse it? Decide, then read on.

CORS only changes what a browser lets a page read. It does not stop requests from scripts, servers or curl. Our test client, which is not a browser, sent a JSON POST with an unlisted Origin header. The server still answered 200.

This is why the teammate's Python script kept working. It is also why the new default is not authentication. Anyone who can reach the port can still call the model, so keep --api-key or a gateway in front.

The 0.32.0 code checks the key only on paths under /v1, /v2, /inference and /cohere. The vLLM security page lists open paths such as /invocations, /pooling and /score. It also lists control paths such as /pause, though the 0.32.0 code serves /pause only when VLLM_SERVER_DEV_MODE=1.

Beginners can stop here. You know why the chat page broke while the script still worked, and that CORS is a browser rule, not a lock. The rest is the exact fix and the other three changes.

Start vLLM with --allowed-origins and a JSON list of the exact origins your pages use. The value must be JSON, because vLLM parses it with json.loads. A bare URL without brackets and quotes fails at startup.

vllm serve MODEL_NAME \
  --api-key "$VLLM_API_KEY" \
  --allowed-origins '["https://chat.example.com", "http://localhost:5173"]'

Write each origin exactly as the browser sends it: scheme, host and port, with no path and no trailing slash. The methods and headers flags keep their old wildcard defaults, so you usually need only this one flag.

You can pass ["*"] to get the old behaviour back. The pull request keeps that option on purpose, but it reopens the gap the change closed. Prefer a short list, or put your own backend between the page and vLLM. If the page itself sends the API key, every user of the page can read it, so that suits only a trusted internal page.

1. 1

   Find the origins

   Open the page in the browser and read the Origin header of its failed request in the network tab. Write down each distinct value.

2. 2

   Add the flag

   Restart vLLM with --allowed-origins set to a JSON list of those values, quoted for your shell. Keep --api-key on.

3. 3

   Check the preflight with curl

   Send the OPTIONS request shown below with your page's origin. On the Python server, expect HTTP 200 and an access-control-allow-origin header naming that origin.

4. 4

   Check an unlisted origin

   Repeat with a made-up origin such as https://evil.example.net. The Python server should answer 400 with the text Disallowed CORS origin.

5. 5

   Reload the page

   The browser should now show the model's reply. If not, compare the origin character by character, including http versus https and the port. Then check the key: vLLM's 401 reply carries no CORS header, so a wrong key also looks like a CORS error.

curl -i -X OPTIONS http://localhost:8000/v1/chat/completions \
  -H "Origin: https://chat.example.com" \
  -H "Access-Control-Request-Method: POST" \
  -H "Access-Control-Request-Headers: content-type,authorization"

We ran it against a stand-in: vLLM's CORS settings in a small FastAPI app served by uvicorn. With the allowlist set, it returned HTTP/1.1 200 OK and access-control-allow-origin: https://chat.example.com. With the 0.32 default it returned 400 Bad Request and the body Disallowed CORS origin.

Rebuild vLLM's CORS wiring in a tiny FastAPI app and send it a preflight. vLLM's own middleware code passes the four CORS flags to Starlette's CORSMiddleware, so this shows the same decision without loading a model.

# pip install fastapi==0.136.3 starlette==1.9.0 httpx==0.28.1
from fastapi import FastAPI
from fastapi.testclient import TestClient
from starlette.middleware.cors import CORSMiddleware


def server(allowed_origins):
    app = FastAPI()
    app.add_middleware(CORSMiddleware, allow_origins=allowed_origins,
                       allow_credentials=False, allow_methods=["*"], allow_headers=["*"])

    @app.post("/v1/chat/completions")
    def chat(body: dict):
        return {"ok": True}
    return app


preflight = {"Origin": "https://chat.example.com",
             "Access-Control-Request-Method": "POST",
             "Access-Control-Request-Headers": "content-type"}

for label, origins in [("0.31 default", ["*"]), ("0.32 default", []),
                       ("0.32 + allowlist", ["https://chat.example.com"])]:
    r = TestClient(server(origins)).options("/v1/chat/completions", headers=preflight)
    print(label, r.status_code, r.headers.get("access-control-allow-origin"))

We ran this with fastapi 0.136.3, starlette 1.9.0 and httpx 0.28.1. Besides a deprecation warning about httpx, it printed three lines: 0.31 default 200 *, then 0.32 default 400 None, then 0.32 + allowlist 200 https://chat.example.com.

The None in the middle line is the whole story. With no allow-origin header, the browser treats the reply as unreadable, whatever the status code.

What just happened? The server made its decision in one header, access-control-allow-origin. With it, a browser hands the reply to the page; without it, the browser hides the reply.

vLLM 0.32.0 puts a time limit on chat template rendering for chat and Responses requests. It closes a denial-of-service hole: one slow template could tie up the renderer for everyone.

A chat template is a Jinja program, and Jinja has loops. A template with nested loops can run for a very long time on a single request. Jinja's sandbox stops a single range() above 100,000, but two nested loops of 100,000 still mean ten billion steps.

The 30-second render limit

The Python server now wraps the chat render for generation requests in a timeout. The setting is the environment variable VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT, in seconds, with a default of 30. Setting it to 0 turns the limit off.

When a render takes longer, the request fails with a message saying rendering timed out and naming that variable. In the 0.32.0 source this maps to HTTP 500, so do not auto-retry it. vLLM then swaps in a fresh pool of render threads, so later requests are not stuck behind the slow one.

We checked this behaviour with our own harmless toy template and a 2-second limit. The slow render timed out at 2.01 seconds. A normal render on the same one-thread pool then also timed out, waiting behind it.

A normal render on a new pool finished at once, and the slow thread was still alive afterwards. That is exactly why vLLM replaces the pool, and why a restart is still worth doing after repeated timeouts.

Scope matters here. In the 0.32.0 source, the limit wraps the Jinja chat render used by chat completions and the Responses API. The synchronous render has no limit, and that path serves offline Python calls and chat-style input to pooling endpoints such as embeddings. The Rust frontend instead counts template steps, with a budget of 500,000 per render.

Templates sent in requests

The release notes also say 0.32.0 rejects a chat_template key inside chat_template_kwargs on the Responses API. In the 0.30.0 and 0.31.0 packages, the Responses renderer already refused it with the same error.

The refusal applies unless the server sets --trust-request-chat-template, which defaults to off. Version 0.32.0 only adds a second, earlier copy of that check, so clients that worked on 0.31 see no change.

What the release notes say versus the code

The release notes say templates are now validated statically and rendered under a timeout. In the 0.32.0 package we found the timeout and the Rust step budget, but no separate static check in the Python renderer.

The notes also tag this fix with advisory GHSA-4hhp-h66f-j5j7. As of 23 October 2026, that ID opens an older, unrelated advisory about SSRF and local file reads in one multimodal processor, fixed in 0.26.0. Pull request 50300 carries the same ID in its title. Do not cite it as the template advisory.

Because some formats Pillow can open hand the file to outside programs. EPS is PostScript, and Pillow may call Ghostscript to draw it. That is a large attack surface for a file a stranger uploaded.

vLLM 0.32.0 now opens untrusted image bytes with an allowlist named ALLOWED_IMAGE_FORMATS. It lists 37 formats, including PNG, JPEG, WEBP, GIF, TIFF, BMP and AVIF. The shared image loader uses it for URLs, base64 data and local files alike.

We tested the list with Pillow 12.3.0, the new minimum version. Formats Pillow could open but the list refuses were BUFR, EPS, GRIB, HDF5, MPEG and WMF. An EPS file opened with plain Image.open and failed with the allowlist.

vLLM reports the failure as Failed to load image. There is no flag to change the list in 0.32.0; it is a constant in the source. If you send scanned documents as EPS, convert them to PNG before the request.

One quieter case: if your deployment registered an extra Pillow plugin, such as pillow-heif for phone photos, those formats are refused too. In our test, a HEIF image opened normally without the allowlist and failed with it.

In vLLM 0.32.0, max_tokens is an upper bound on the reply, not space reserved in advance. A long prompt is accepted as long as it leaves room for at least one reply token, and the reply is capped to the space left. One rule stays: max_tokens itself may not exceed max_model_len.

Before, vLLM subtracted max_tokens from the context window to get the largest allowed prompt. A client that always sent a large max_tokens would see long prompts rejected, even when a shorter reply would have fit.

We ran the real functions from both source packages with a 32,768-token context window. The table shows the results. For chat requests, max_completion_tokens takes the place of max_tokens when both are set.

This removes errors, so why is it a break? Some clients used the 400 error as a signal to shorten the prompt or switch models. Now they get a shorter reply instead. On the internal chat page, a user who pastes a very long document may see an answer that stops mid-sentence, with no error.

Check the finish_reason field of each reply. It is length when the reply hit the cap, and stop when the model ended on its own. Compare usage.completion_tokens with what you asked for to catch silent capping.

Upgrade in a staging copy first and test each of the four changes on purpose. Most breaks show up within minutes if you know where to look.

1. 1

   Check Python and pins

   vLLM 0.32.0 needs Python 3.11 or newer, transformers from 5.16.1 to below 5.20.0 and tokenizers 0.23.2 or newer. Run pip install vllm==0.32.0 in a fresh environment.

2. 2

   List browser clients

   Find every web page that calls vLLM directly. Add their origins with --allowed-origins and run the curl preflight check for each one.

3. 3

   Render your templates

   Send a long, realistic conversation through each custom chat template and time it. Raise VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT only for a template you wrote or reviewed.

4. 4

   Test image inputs

   Send one sample of each image format your users upload. Anything outside the allowlist fails with Failed to load image.

5. 5

   Watch reply lengths

   For a day after release, alert when finish_reason is length on long prompts. That shows where capped replies replaced old errors.

Frame it as defence in depth for a model server. Three of the four changes remove a way for a stranger to read data or waste compute, and each has a clear opt-in for trusted cases.

1. 1

   Name the trust boundary

   The model server sits behind an API gateway. Browsers, scripts and uploads are untrusted input until checked.

2. 2

   Separate CORS from auth

   CORS limits which websites a browser lets read replies. Authentication limits who may call at all. You need both.

3. 3

   Bound every user-driven cost

   Templates get a time budget, prompts get a length check and replies get a token cap. One request should not be able to block other users for long.

4. 4

   Shrink the parser surface

   Accept only the file formats you need, decoded in-process. Refuse formats that call outside programs.

5. 5

   Make defaults closed

   Ship closed and let operators open only what they need, with explicit flags kept in config.

An interviewer listens for the difference between browser-enforced rules and server-enforced ones. Saying CORS protects the API is the classic mistake.