# GLM 5.3 how to run locally: 753B weights, 8 GPUs, vLLM or SGLang

> The model cards give no hardware list. The vLLM and SGLang recipes start at eight H200 GPUs for GLM 5.3, and Flash is smaller but still a server job.

URL: https://claudegpt.si/news/glm-5-3-how-to-run-locally  
Published: 2026-10-30T14:52:25.036Z  
Author: ClaudeGPT Si Newsroom  
Section: Guides  
Topics: GLM, Z.ai, Hugging Face, vLLM, SGLang, Open-weight models  
Publisher: ClaudeGPT Si

## Key takeaways

- GLM-5.3 has 753B parameters, and the vLLM recipe names 8xH200 or 8xH20 for single-node FP8.
- GLM-5.3-Flash has 321B parameters in its Hugging Face listing and its vLLM recipe defaults to 8x H200 with FP8.
- Neither model card lists hardware, and neither names a GGUF repository.

GLM 5.3 how to run locally comes down to one fact: you can run it locally only if "locally" means a GPU server. The [GLM-5.3 model card](https://huggingface.co/zai-org/GLM-5.3) lists 753 billion parameters, and the [vLLM recipe](https://recipes.vllm.ai/zai-org/GLM-5.3) names eight 141 GB GPUs (H200 or H20) as the single-node setup for the default FP8 weights. GLM 5.3 Flash is the smaller sibling, at 321 billion parameters, and it still lands in server territory.

The commands below come from the model cards, the vLLM recipes and the SGLang cookbook pages. We could not run these models in our test environment, so we did not run any of these commands. Treat the hardware lines as what the vendors' pages say, not as something we measured.

## How big is GLM 5.3?

| Model | Parameters | Tensor types | Hardware on the model card |
| --- | --- | --- | --- |
| [GLM-5.3](https://huggingface.co/zai-org/GLM-5.3) | 753B | BF16, F8_E4M3, F32 | Not stated |
| [GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash) | 321B in the Hugging Face listing; the introduction says 320B total, 18B active | BF16, F8_E4M3, F32 | Not stated |

Neither model card lists hardware requirements. The numbers that exist are on the recipe pages that the cards link to. GLM-5.3 is a mixture-of-experts model, which means only part of the network runs for each token, but all the weights still have to sit in memory.

The size of the files on disk, from the recipes:

- GLM-5.3 BF16 weights are about 1.5 TB, according to the SGLang cookbook.
- The NVFP4 build of GLM-5.3 is about 465 GB in the vLLM recipe and about 0.45 TB in the SGLang cookbook. It needs Blackwell GPUs and the SGLang page calls it experimental.
- For GLM-5.3-Flash, the vLLM recipe gives about 306 GiB for the default FP8 checkpoint, before runtime and KV-cache overhead, and says BF16 needs roughly twice that.

## What hardware does GLM 5.3 need?

The vLLM recipe for GLM-5.3 says: 8xH200 or 8xH20 (141 GB each) for single-node FP8, and 8xB200 (180 GB each) for the full 1M-token context. AMD MI300X, MI325X, MI350X and MI355X are covered with ROCm. The BF16 weights need a multi-node setup.

The SGLang cookbook gives the same starting point, 8x H200 with tensor parallelism and FP8, and adds that BF16 needs an 8x B300 node or multiple nodes. It also lists Ascend NPUs.

The vLLM recipe also lists these software requirements: a vLLM build with native GLM5Next support, Transformers 5.15.0 or newer, and DeepGEMM for FP8 speed.

## How to run GLM 5.3 with vLLM

Install vLLM, then serve the model. The model card's two lines are the short version:

```bash
pip install vllm
vllm serve "zai-org/GLM-5.3"
```

The vLLM recipe gives the full command for 8xH200:

```bash
VLLM_USE_RUST_FRONTEND=1 vllm serve zai-org/GLM-5.3 \
  --tokenizer-mode glm5_next \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 5 \
  --tool-call-parser glm47 \
  --reasoning-parser glm47 \
  --enable-auto-tool-choice \
  --served-model-name glm-5.3
```

`--tensor-parallel-size 8` splits the model across eight GPUs. `--tokenizer-mode glm5_next` uses the model's own chat rendering; the recipe says to use `auto` on builds that don't recognize it. The speculative-config lines turn on multi-token prediction, which guesses several tokens ahead to speed up decoding.

For the full 1M-token context on 8xB200, the recipe says to start from the same command with `--max-num-seqs 32` and tune to your VRAM.

## How to run GLM 5.3 with SGLang

From the model card:

```bash
pip install sglang
python3 -m sglang.launch_server \
    --model-path "zai-org/GLM-5.3" \
    --host 0.0.0.0 \
    --port 30000
```

The card also gives a Docker version that mounts your Hugging Face cache and passes `HF_TOKEN`. For tuned flags, the SGLang cookbook page says to start around `--mem-fraction-static 0.8` on H200 with TP8 and tune up. The cookbook's Deploy panel generates the other commands, and the only full commands printed on the page are for AMD MI355X.

## How to run GLM 5.3 Flash locally

Flash uses the same tools. From its model card:

```bash
pip install vllm
vllm serve "zai-org/GLM-5.3-Flash"
```

```bash
pip install sglang
python3 -m sglang.launch_server --model-path "zai-org/GLM-5.3-Flash" --host 0.0.0.0 --port 30000
```

The [vLLM recipe for Flash](https://recipes.vllm.ai/zai-org/GLM-5.3-Flash) lists 8x H200 with FP8 as the default setup. It also tests one GB200 tray at tensor parallel 4, and 4x H200 at tensor parallel 4 for KV-cache offloading. Its default command:

```bash
export VLLM_ENGINE_READY_TIMEOUT_S=3600
export VLLM_USE_RUST_FRONTEND=1
vllm serve zai-org/GLM-5.3-Flash \
  --tokenizer-mode glm5_next \
  --tensor-parallel-size 8 \
  --disable-flashinfer-autotune \
  --tool-call-parser glm47 \
  --enable-auto-tool-choice \
  --reasoning-parser glm47
```

It needs vLLM 0.29.0 or newer and FlashInfer 0.6.17 or newer. One note on Hopper GPUs: the recipe says FP8 KV cache is not supported for this model there, so the KV cache stays in BF16.

The [SGLang cookbook for Flash](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.3-Flash) lists H100, H200, B200, B300, GB200 and GB300 on NVIDIA, MI355X on AMD, TPU7x, and Ascend, and asks for SGLang v0.5.20 or later. It does not give a GPU count for most NVIDIA setups.

Flash is the first natively multimodal model in the GLM-5 series, according to its card. Through Transformers it loads with `AutoModelForMultimodalLM`, not the text-only class that GLM-5.3 uses.

## Call the server

Both vLLM and SGLang expose an OpenAI-compatible `/v1/chat/completions` endpoint. The model cards give `http://localhost:8000/v1/chat/completions` for vLLM and `http://localhost:30000/v1/chat/completions` for SGLang. Any client that speaks the OpenAI API can point at it.

Two settings from the cards matter:

- `reasoning_effort` accepts `low`, `high` or `max`, and defaults to `max`.
- `clear_thinking` defaults to `false`. For chat use, the cards recommend passing `clear_thinking=true`.

## Can you run it on a laptop with GGUF?

Both cards link to community quantizations for llama.cpp, Ollama, LM Studio and Jan. The GLM-5.3 card says there are about 100 of them. Neither card names a specific GGUF repository, and we did not open one, so we can't give you a file size or a tested command. Check the quantization's own page for its size before downloading anything, and expect a 753B-parameter model to stay large even when compressed.

For model-specific hardware, the card also links guides for KTransformers and Unsloth, which we did not follow.

## Common problems

- The server fails on an old install. Both recipes require recent builds with native GLM5Next support, so upgrade vLLM or SGLang first.
- The tokenizer mode is rejected. Switch `--tokenizer-mode glm5_next` to `auto`, as the vLLM recipe suggests.
- Out of memory at long context. The vLLM recipe names `--max-num-seqs` as the main lever for 1M-token context.
- SGLang with host-memory caching fails to start. On MI355X with `--hicache-size 180`, the Flash cookbook says the server doesn't start if host memory is insufficient.

## Frequently asked questions

### How big is GLM 5.3?

The model card lists 753B parameters. The SGLang cookbook puts the BF16 weights at about 1.5 TB, and the vLLM recipe puts the NVFP4 build at about 465 GB.

### How do I run GLM 5.3 Flash locally?

Install vLLM or SGLang and serve zai-org/GLM-5.3-Flash with the commands on its model card. The vLLM recipe defaults to 8x H200 with FP8, and it also lists one GB200 tray at tensor parallel 4.

### Can I run GLM 5.3 on one consumer GPU?

None of the pages we read describe a single consumer GPU setup. The recipes start at four to eight data center GPUs. Community GGUF quantizations exist, but we did not test or size any.

### Does GLM 5.3 have an OpenAI-compatible API when self-hosted?

Yes. Both vLLM and SGLang expose a /v1/chat/completions endpoint, on port 8000 for vLLM and port 30000 for SGLang in the model card commands.

## Sources

- [zai-org/GLM-5.3](https://huggingface.co/zai-org/GLM-5.3), Hugging Face (Z.ai)
- [zai-org/GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash), Hugging Face (Z.ai)
- [GLM-5.3 on vLLM recipes](https://recipes.vllm.ai/zai-org/GLM-5.3), vLLM
- [GLM-5.3-Flash on vLLM recipes](https://recipes.vllm.ai/zai-org/GLM-5.3-Flash), vLLM
- [GLM-5.3 SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.3), SGLang
- [GLM-5.3-Flash SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.3-Flash), SGLang