WebGPU Inference Explained: Running ML Models in the Browser

Most first attempts fail on the user's device, not in your code. WebGPU inference in the browser works well on a recent laptop, then breaks on an old phone, a locked-down office PC or a Linux desktop. The model also has to be downloaded before it can answer anything.

Picture a common setup. A support portal wants to tag each complaint by category and mask phone numbers before the text leaves the customer's laptop. The team picks a small text classifier of about 10 crore (100 million) parameters, roughly 100 MB as 8-bit numbers. The setup is made up, but each problem below is real.

Three things then go wrong. Some visitors have no WebGPU at all, so the feature silently does nothing. Others wait a minute on mobile data for the model to download, every visit. A few find the cached model gone after a week, because the browser deleted it.

This guide walks through each one with the official specs and tests we ran. It stays on the browser runtime. Choosing the model itself is covered in our small language models guide.

You need seven words. Each gets a plain meaning here, tied to the complaint tagger.

• Model weights: the numbers a model learned during training. The tagger's 100 MB file is almost entirely weights.
• Inference: using those weights to turn an input into an output, such as complaint text into the label Billing.
• GPU: the graphics chip. It does thousands of small multiplications at once, which is what model inference mostly is.
• WebGPU: the browser API that lets a web page send work to the GPU, including general maths through compute shaders, small programs that run on the GPU.
• WebAssembly (Wasm): a low-level code format that MDN says brings near-native performance to the web. It runs on the CPU. It is the usual fallback when WebGPU is missing.
• Adapter and device: in WebGPU, an adapter represents one GPU the browser offers you. A device is your private connection to it, with its own limits.
• Quantisation: storing weights with fewer bits, such as 8 or 4 instead of 32, so the file is smaller. It can cost some accuracy.

WebGPU inference is running a model's maths on the user's GPU from a web page, with no server doing the work. The page downloads the weights once, copies them into GPU memory and runs compute shaders over each input.

WebGPU is a W3C specification. As of 26 October 2026 it is a Candidate Recommendation Draft dated 15 September 2026. MDN, the Mozilla developer docs, describes it as the successor to WebGL with support for general-purpose GPU computation.

The flow has five steps. The page checks that navigator.gpu exists, which is the entry point to WebGPU. It asks for an adapter, then a device. It loads the weights from a local cache or the network into GPU buffers, which are blocks of GPU memory. Then each input runs through the model.

An everyday picture helps. Think of a restaurant kitchen. The adapter is the kitchen and the device is your booked counter in it. Buffers are your storage bins, and a compute shader is one recipe many cooks follow at once.

Where the picture breaks: a real kitchen lets you bring any bin size. WebGPU gives your counter strict default limits, and you must ask for bigger bins up front, before you start cooking.

Chrome, Edge and Safari ship WebGPU on their main desktop platforms, and Firefox ships it on Windows and Apple Silicon Macs. Linux and Android support is partial. MDN still marks WebGPU as not Baseline, its label for features that work across all major browsers.

The practical reading is simple. Many of your users will have WebGPU, but not all, and the browser name alone will not tell you. The spec also lets a supporting browser return no adapter at its own discretion.

WebGPU only works in secure contexts, meaning pages served over HTTPS or from localhost. MDN also confirms it is available inside web workers, background threads that keep heavy work off the page's main thread.

Check the API, then ask for an adapter, then look at what kind of adapter you got. Each step can fail on its own, so test all three before you load any weights.

// Pick a backend: WebGPU if a real adapter exists, otherwise WebAssembly.
async function pickBackend() {
  if (!("gpu" in navigator)) return { backend: "wasm", why: "no navigator.gpu" };
  const adapter = await navigator.gpu.requestAdapter();
  if (adapter === null) return { backend: "wasm", why: "no adapter" };
  if (adapter.info?.isFallbackAdapter) return { backend: "wasm", why: "software adapter" };
  return { backend: "webgpu", adapter };
}

The spec says requestAdapter() resolves to null when the browser cannot or chooses not to return an adapter. An adapter can also be a fallback adapter. The spec defines that as one with significant performance caveats, traded for wider compatibility or privacy.

What our headless Chromium test showed

We ran this check with Playwright in headless Chromium 153 on a Linux container with no GPU. navigator.gpu existed, but requestAdapter() returned null, so pickBackend chose wasm with the reason no adapter.

With the flags enable-unsafe-webgpu and use-webgpu-adapter=swiftshader, Chromium offered SwiftShader, a software GPU that runs on the CPU. Its info.isFallbackAdapter was true, so the check chose wasm again. A small compute shader still ran on it and doubled 1,024 numbers correctly.

That is the lesson. A software adapter works for testing, but it is a CPU in disguise. For real users, treat it like no GPU and use your WebAssembly path, which is built for the CPU.

A WebGPU device starts with fixed default limits from the spec, whatever the GPU can really do. The two that matter for weights are maxBufferSize, 268,435,456 bytes (256 MiB), and maxStorageBufferBindingSize, 134,217,728 bytes (128 MiB).

maxBufferSize caps one block of GPU memory. maxStorageBufferBindingSize caps one storage binding, the window onto a buffer that a shader reads through one slot. A weight tensor, one named array of weights, bigger than these must be split, or you must request higher limits.

Requesting the limits the GPU really has

The adapter reports its true limits, and you can ask for them when you create the device. If you ask for more than the adapter supports, the spec says requestDevice rejects with an OperationError.

// Ask for the adapter's real limits instead of the defaults.
const adapter = await navigator.gpu.requestAdapter();
const device = await adapter.requestDevice({
  requiredLimits: {
    maxBufferSize: adapter.limits.maxBufferSize,
    maxStorageBufferBindingSize: adapter.limits.maxStorageBufferBindingSize,
  },
});
console.log(device.limits.maxBufferSize); // 1073741824 in our test

We tested this on the SwiftShader adapter in Chromium 153. A device created with no requested limits refused a 300 MiB buffer with a validation error. That error helpfully said the adapter supports 1,073,741,824 bytes (1 GiB).

A device created with the code above accepted the same 300 MiB buffer. Asking for twice the adapter's limit was rejected with an OperationError, as the spec says. We also found that an adapter can create only one device; a second request failed with adapter is consumed.

How big are real model files?

File size is roughly parameters times bits per weight, divided by 8. The table shows what that means. Real files add a little for metadata, and some layers are often kept at higher precision.

The complaint tagger at 8-bit is 100 MB, well inside the defaults. A 1-billion-parameter model at 4-bit is 500 MB. That needs at least two 256 MiB buffers, or four 128 MiB bindings, unless the device gets higher limits.

Half precision also needs care. WebGPU lists shader-f16, 16-bit float support in shaders, as an optional feature. Our SwiftShader adapter did not offer it, so a 16-bit model there would need converting to 32-bit floats or another code path.

Fall back whenever there is no real GPU adapter, and keep the fallback model small. WebAssembly runs on the CPU in every major browser, so it reaches users WebGPU misses, at a lower speed.

Two limits shape the WebAssembly path. The first is memory. A standard 32-bit WebAssembly memory tops out at 65,536 pages of 64 KiB, which is 4 GiB. In Chromium 153, asking for 65,537 pages failed with a RangeError.

Memory64, 64-bit WebAssembly memory, raises that. The SpiderMonkey team, who build Firefox's JavaScript engine, report it shipped in Chrome 133 and Firefox 134. Chrome's engine, V8, caps it at 262,144 pages (16 GiB), and our test hit that bound.

The same SpiderMonkey post warns that Memory64 is slower, by 10 percent to over 100 percent depending on the workload. The reason is bounds checks: 32-bit memory can skip them, 64-bit cannot. So prefer a model that fits in 4 GiB.

The second limit is threads. Multi-threaded WebAssembly shares memory through SharedArrayBuffer, which MDN says needs a cross-origin isolated page. That means sending two HTTP headers from your server.

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

In our test, a page without these headers had crossOriginIsolated false and no SharedArrayBuffer at all. With them, both were available. MDN notes that require-corp blocks cross-origin resources that do not opt in, so test your embeds first.

Store the weights after the first download in the Cache API or the origin private file system, and read them from there on every later visit. Both count against the same per-site storage quota.

The Cache API stores request and response pairs. MDN notes it works in normal pages and workers, not only in service workers, and its items do not expire unless deleted, though the browser may still evict them.

// Download a weights file once, keep it in the Cache API, reuse it later.
async function getWeights(url) {
  const cache = await caches.open("model-weights-v1");
  let res = await cache.match(url);
  const fromCache = Boolean(res);
  if (!res) {
    res = await fetch(url);
    if (!res.ok) throw new Error("download failed: " + res.status);
    await cache.put(url, res.clone());
  }
  return { bytes: await res.arrayBuffer(), fromCache };
}

We ran this with a 40 MiB test file in Chromium 153. The first call downloaded it and returned fromCache false. The second call returned fromCache true, and navigator.storage.estimate() reported 41,943,399 bytes under caches.

OPFS, the origin private file system, is a private folder for your site that the user does not see. MDN says it offers byte-level file access, with faster synchronous reads and writes inside web workers. It suits weights you want to read in pieces.

We streamed the same 40 MiB file into OPFS. The estimate then showed 41,943,208 bytes under fileSystem, alongside the cached copy. So do not keep the same weights in both, or you pay twice.

How much space do you get, and when is it deleted?

Quotas differ by browser, according to MDN's storage quotas page. Chromium lets one origin use up to 60 percent of the disk. Firefox's default best-effort mode allows the smaller of 10 percent of the disk or 10 GiB per site.

Safari allows around 60 percent of the disk per origin in browser apps. But with cross-site tracking prevention on, Safari deletes script-created data for a site the user has not interacted with in the last seven days of browser use.

Eviction is all or nothing. MDN says that when a browser evicts an origin, all its data goes at once, Cache API and IndexedDB together. Under storage pressure, browsers evict the least recently used origin first.

navigator.storage.persist() asks to exempt your site from that eviction. Firefox shows the user a prompt. Safari and most Chromium-based browsers decide silently, from the user's history of interaction with the site.

// Check how much space you have, and ask the browser not to evict it.
const { usage, quota } = await navigator.storage.estimate();
console.log((usage / 1048576).toFixed(1), "MiB used of", (quota / 1048576).toFixed(0));
const kept = await navigator.storage.persist();
console.log(kept ? "persistent" : "best-effort, may be evicted");

In our headless test, persist() returned false, which makes sense for a fresh profile with no engagement. Your code must handle a missing cache on every visit, not just the first. A write past the quota fails with a QuotaExceededError.

The main privacy gain is that the input can stay on the device. The main cost is that the model weights do not, and the first answer waits for a download.

Privacy: what stays private and what does not

When inference runs in the tab, the complaint text never has to reach your server. That helps with sensitive input such as phone numbers or health details. It only holds if your code does not also send the text elsewhere, such as to analytics.

The other side is that your weights become public. Anyone can open the network tab and save the file. If the model is a trade secret, or was trained on data you must not expose, keep it on the server.

Latency: slow first answer, fast later answers

There is no network round trip per request, so later answers depend only on the user's device. The first answer waits for the download, then for the weights to reach GPU memory.

Download time is size in megabits divided by connection speed. A 500 MB model takes about 80 seconds at 50 Mbps and nearly 7 minutes at 10 Mbps. The 100 MB tagger takes about 16 seconds at 50 Mbps.

So start the download early, show real progress, and run the model in a web worker. A slow phone may still answer more slowly than your server would, because its GPU is far weaker.

Use the browser when the model is small, the input is sensitive or the feature must work offline. Use a server when the model is large, must stay secret or must give the same answer on every device.

In our judgement, tasks where a small model is often enough are good candidates. Examples are text classification, spotting personal data, short-text similarity search, background removal in photos and simple speech-to-text.

A hybrid is common and sensible. Run the small model in the browser when WebGPU is available, and send the request to your server when it is not. The complaint tagger could do exactly that.

WebNN, the Web Neural Network API, is a separate W3C draft. It describes a low-level API for hardware-accelerated neural network inference. As of 26 October 2026 it is a Candidate Recommendation Draft dated 21 October 2026.

It is not a released standard, so build on WebGPU and WebAssembly today. Watch WebNN as a possible third backend once browsers ship it on by default.

Lead with the trade-off: you move compute and cost from your servers to users' devices, and pay with downloads, device variety and storage you do not control. Then walk through the design in order.

1. 1

   State the goal

   Run a small model close to the user for privacy, offline use or zero per-request cost.

2. 2

   Pick backends

   WebGPU when a real adapter exists, WebAssembly on the CPU otherwise, and a server fallback for very weak devices.

3. 3

   Size the model

   Estimate parameters times bits divided by 8. Check it against the 256 MiB default buffer limit and request higher limits when needed.

4. 4

   Plan delivery and caching

   Serve weights from a CDN, cache them with the Cache API or OPFS, call persist() and expect eviction anyway.

5. 5

   Keep the page responsive

   Run inference in a web worker and show download progress. Send cross-origin isolation headers if you use WebAssembly threads.

6. 6

   Measure

   Log backend choice, download time, time to first answer and cache hit rate per device class.