GLM 5.3 how to run locally: 753B weights, 8 GPUs, vLLM or SGLang

GLM 5.3 how to run locally comes down to one fact: you can run it locally only if "locally" means a GPU server. The GLM-5.3 model card lists 753 billion parameters, and the vLLM recipe names eight 141 GB GPUs (H200 or H20) as the single-node setup for the default FP8 weights. GLM 5.3 Flash is the smaller sibling, at 321 billion parameters, and it still lands in server territory.

The commands below come from the model cards, the vLLM recipes and the SGLang cookbook pages. We could not run these models in our test environment, so we did not run any of these commands. Treat the hardware lines as what the vendors' pages say, not as something we measured.

How big is GLM 5.3?

Neither model card lists hardware requirements. The numbers that exist are on the recipe pages that the cards link to. GLM-5.3 is a mixture-of-experts model, which means only part of the network runs for each token, but all the weights still have to sit in memory.

The size of the files on disk, from the recipes:

• GLM-5.3 BF16 weights are about 1.5 TB, according to the SGLang cookbook.
• The NVFP4 build of GLM-5.3 is about 465 GB in the vLLM recipe and about 0.45 TB in the SGLang cookbook. It needs Blackwell GPUs and the SGLang page calls it experimental.
• For GLM-5.3-Flash, the vLLM recipe gives about 306 GiB for the default FP8 checkpoint, before runtime and KV-cache overhead, and says BF16 needs roughly twice that.

What hardware does GLM 5.3 need?

The vLLM recipe for GLM-5.3 says: 8xH200 or 8xH20 (141 GB each) for single-node FP8, and 8xB200 (180 GB each) for the full 1M-token context. AMD MI300X, MI325X, MI350X and MI355X are covered with ROCm. The BF16 weights need a multi-node setup.

The SGLang cookbook gives the same starting point, 8x H200 with tensor parallelism and FP8, and adds that BF16 needs an 8x B300 node or multiple nodes. It also lists Ascend NPUs.

The vLLM recipe also lists these software requirements: a vLLM build with native GLM5Next support, Transformers 5.15.0 or newer, and DeepGEMM for FP8 speed.

How to run GLM 5.3 with vLLM

Install vLLM, then serve the model. The model card's two lines are the short version:

pip install vllm
vllm serve "zai-org/GLM-5.3"

The vLLM recipe gives the full command for 8xH200:

VLLM_USE_RUST_FRONTEND=1 vllm serve zai-org/GLM-5.3 \
  --tokenizer-mode glm5_next \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 5 \
  --tool-call-parser glm47 \
  --reasoning-parser glm47 \
  --enable-auto-tool-choice \
  --served-model-name glm-5.3

--tensor-parallel-size 8 splits the model across eight GPUs. --tokenizer-mode glm5_next uses the model's own chat rendering; the recipe says to use auto on builds that don't recognize it. The speculative-config lines turn on multi-token prediction, which guesses several tokens ahead to speed up decoding.

For the full 1M-token context on 8xB200, the recipe says to start from the same command with --max-num-seqs 32 and tune to your VRAM.

How to run GLM 5.3 with SGLang

From the model card:

pip install sglang
python3 -m sglang.launch_server \
    --model-path "zai-org/GLM-5.3" \
    --host 0.0.0.0 \
    --port 30000

The card also gives a Docker version that mounts your Hugging Face cache and passes HF_TOKEN. For tuned flags, the SGLang cookbook page says to start around --mem-fraction-static 0.8 on H200 with TP8 and tune up. The cookbook's Deploy panel generates the other commands, and the only full commands printed on the page are for AMD MI355X.

How to run GLM 5.3 Flash locally

Flash uses the same tools. From its model card:

pip install vllm
vllm serve "zai-org/GLM-5.3-Flash"

pip install sglang
python3 -m sglang.launch_server --model-path "zai-org/GLM-5.3-Flash" --host 0.0.0.0 --port 30000

The vLLM recipe for Flash lists 8x H200 with FP8 as the default setup. It also tests one GB200 tray at tensor parallel 4, and 4x H200 at tensor parallel 4 for KV-cache offloading. Its default command:

export VLLM_ENGINE_READY_TIMEOUT_S=3600
export VLLM_USE_RUST_FRONTEND=1
vllm serve zai-org/GLM-5.3-Flash \
  --tokenizer-mode glm5_next \
  --tensor-parallel-size 8 \
  --disable-flashinfer-autotune \
  --tool-call-parser glm47 \
  --enable-auto-tool-choice \
  --reasoning-parser glm47

It needs vLLM 0.29.0 or newer and FlashInfer 0.6.17 or newer. One note on Hopper GPUs: the recipe says FP8 KV cache is not supported for this model there, so the KV cache stays in BF16.

The SGLang cookbook for Flash lists H100, H200, B200, B300, GB200 and GB300 on NVIDIA, MI355X on AMD, TPU7x, and Ascend, and asks for SGLang v0.5.20 or later. It does not give a GPU count for most NVIDIA setups.

Flash is the first natively multimodal model in the GLM-5 series, according to its card. Through Transformers it loads with AutoModelForMultimodalLM, not the text-only class that GLM-5.3 uses.

Call the server

Both vLLM and SGLang expose an OpenAI-compatible /v1/chat/completions endpoint. The model cards give http://localhost:8000/v1/chat/completions for vLLM and http://localhost:30000/v1/chat/completions for SGLang. Any client that speaks the OpenAI API can point at it.

Two settings from the cards matter:

• reasoning_effort accepts low, high or max, and defaults to max.
• clear_thinking defaults to false. For chat use, the cards recommend passing clear_thinking=true.

Can you run it on a laptop with GGUF?

Both cards link to community quantizations for llama.cpp, Ollama, LM Studio and Jan. The GLM-5.3 card says there are about 100 of them. Neither card names a specific GGUF repository, and we did not open one, so we can't give you a file size or a tested command. Check the quantization's own page for its size before downloading anything, and expect a 753B-parameter model to stay large even when compressed.

For model-specific hardware, the card also links guides for KTransformers and Unsloth, which we did not follow.

Common problems

• The server fails on an old install. Both recipes require recent builds with native GLM5Next support, so upgrade vLLM or SGLang first.
• The tokenizer mode is rejected. Switch --tokenizer-mode glm5_next to auto, as the vLLM recipe suggests.
• Out of memory at long context. The vLLM recipe names --max-num-seqs as the main lever for 1M-token context.
• SGLang with host-memory caching fails to start. On MI355X with --hicache-size 180, the Flash cookbook says the server doesn't start if host memory is insufficient.