Discrete visual tokens can match continuous ones, a new paper says

Researchers at Huazhong University of Science and Technology and Meituan LongCat say discrete visual tokens can match continuous image features at understanding images. In a paper posted on October 27, they introduce Factorized Vector Quantization (FVQ), a way to turn image features into discrete tokens that keeps improving as you give it more bits. Their 8B model is the first discrete-token model they know of that competes with Qwen3-VL on text-heavy tasks.

The paper has 60 upvotes on Hugging Face daily papers. The idea behind it matters for anyone following "native" multimodal models, which treat images and text as the same kind of token.

What a discrete visual token is

Most vision-language models run an image through a vision encoder and hand the language model a list of continuous vectors. A discrete approach instead snaps each patch of the image to entries in a fixed codebook, the way text is split into tokens from a vocabulary. The image becomes a sequence of integer IDs, and one next-token objective can train on both text and images.

The cost is quantization error: the vector you get back is not exactly the original. The common assumption, which the authors cite, is that this error destroys meaning, so continuous features set a ceiling that discrete ones cannot pass.

Error is the wrong thing to measure

The team tested the assumption with eight-group residual vector quantization (RVQ), where each codebook encodes what the previous ones missed. They kept only the odd groups or only the even groups of the Qwen3 vision encoder's features. The odd-group version had a reconstruction error of 0.246, ten times the 0.027 of plain single-codebook quantization. It still scored 60.7 on their eight-benchmark average, against 51.2 for the single codebook (OCRBench divided by 10 in the average).

So reconstruction error did not predict understanding. What did was the bit budget per patch: the number of codebooks times log2 of the codebook size. With 16,384 entries per codebook, each codebook adds 14 bits, and 16 codebooks give 224 bits per patch. More bits helped on everything, including OCR, where discrete tokens usually fail because fine print is lost.

Where existing methods stall and what FVQ changes

The problem is that the extra bits stop paying off. Going from 8 to 16 codebooks doubles the nominal budget, but RVQ stays at 64.0 on the average and product quantization (PQ) moves from 64.1 to 64.3. In RVQ, later codebooks only see small leftovers. In PQ, the feature is split into narrower and narrower slices.

FVQ gives every codebook a full view instead. Each codebook receives a dense projection of the whole feature, the projections are built so that together they lose nothing before quantization, and the codebooks work in parallel with equal roles. Rounding to the nearest codeword is the only source of error. In the authors' controlled setup, FVQ with 16 codebooks scores 65.3 on the average, against 64.6 for the original continuous Qwen3 vision encoder.

The 8B model

For the main model the team paired Qwen3-8B with a 16-codebook FVQ tokenizer on the Qwen3 vision encoder, trained on 100B, 10B, 60B and 30B tokens across four stages (tokenizer, alignment, pre-training, fine-tuning). The authors report these results against continuous 8B-class models:

Against other discrete models, the paper reports gains of 22.8% over UniAR and 59.2% over Emu3-Chat on a nine-metric average. Some of those baseline numbers were re-run by the authors under their own evaluation setup.

What to be careful about

"Competitive" is the paper's word, and the table shows gaps: about 5 points on MMMU and ChartQA. The model matches or beats Qwen3-VL on some tests and trails on others. Rows marked in the paper were re-run by the authors, and the comparison with their own continuous baseline comes from a smaller setup built on Qwen3-1.7B and LLaVA-OneVision data.

The bit budget is also large. At 224 bits per patch, each patch needs 16 indices instead of one. The paper does not report what that costs at inference time. The result also comes from one group, and the authors write that FVQ is "one approach" to the saturation problem, not the only one.

The tokenizer gains did carry over to two other vision encoders. Going from one codebook to eight FVQ codebooks raised the eight-benchmark average from 38.7 to 61.1 on MoonViT-v2 and from 44.7 to 53.9 on SigLIP2.

Why it matters

If discrete tokens can hold document-level detail, a single model can read and write images and text in one stream. That is a simpler path than bolting an encoder to a language model. The paper suggests the limit was never discretization itself but how many informative bits each patch got.

Code is listed on GitHub with a project page. We did not check what the repository contains, so whether model weights are released is not confirmed here. Watch for independent reproductions of the OCR and document results, which are the claim most likely to be tested.