# Platonic representation hypothesis: LeCun's JEPA proof and results > Balestriero and LeCun prove a version of the hypothesis for JEPAs. A 109M-parameter text model beat language models of up to 72B on alignment with vision models. URL: https://claudegpt.si/news/platonic-representation-hypothesis-jepa Published: 2026-10-31T02:50:49.740Z Author: ClaudeGPT Si Newsroom Section: Research Topics: Yann LeCun, JEPA, Representation learning, Research Publisher: ClaudeGPT Si ## Key takeaways - Balestriero and LeCun prove that JEPAs trained on different modalities learn the same representation up to one rotation, if their training pairs are two observations of one scene. - A 109M-parameter good-view text JEPA scored .184 alignment with DINOv2, ahead of Qwen2.5-72B at .154, and used 42 times fewer samples than next-token prediction for similar alignment. - The theory is idealized and the headline result varies by run: a fourth seed scored .121 and the four-seed average is .168. The Platonic representation hypothesis says that AI models trained on different kinds of data, such as text and images, end up building the same picture of the world as they get bigger. In a [paper posted to arXiv on October 29, 2026](https://arxiv.org/abs/2610.39302), Randall Balestriero and Yann LeCun claim to prove a version of it for models called JEPAs, and show that training text models the JEPA way makes them line up with vision models better than much larger language models do. The numbers are the interesting part. A 109M-parameter text model beat language models of up to 72B parameters on this measure, and it needed far less data to do it. ## What the Platonic representation hypothesis says Every neural network turns its input into a list of numbers, called an embedding or representation. The hypothesis, first put forward by Huh and colleagues, observes that as vision models and language models grow, the way they arrange things in that number space becomes more alike. A photo of a dog and the sentence "a dog on a lawn" land in similar places relative to other photos and sentences. Until now this was an observation. The paper tries to explain it, and says when it should hold and when it should not. ## What a JEPA is JEPA stands for joint-embedding predictive architecture. A JEPA learns by taking two views of the same thing and training the model so their embeddings agree. It also includes a term that stops it from collapsing to one answer for everything. The paper uses a recent variant called LeJEPA, which relies on a regularizer named SIGReg. The paper separates two kinds of views: - **Good views** are two different observations of the same scene, such as two captions of one photo, or two frames of one video clip. - **Bad views** are pieces of the same document, such as a masked span of text and the text around it. ## What the paper proves The main theorem says that JEPAs trained separately on any number of modalities learn the same representation up to one rotation. Two conditions apply: the training pairs must be good views, and any remaining mismatch between the modalities must be bounded. In practice that means you can line up two separately trained encoders by fitting one rotation, and the paper says that takes roughly k paired examples for k embedding dimensions. The authors also argue that next-token prediction, the way language models are trained, is a JEPA with bad views. The positives are a prefix and its continuation. That would explain why language models line up with vision models mostly through scale. The paper also shows that next-token models are recovered only up to an invertible linear map, not a rotation, which is a weaker guarantee. Another result says that detail specific to one modality, such as the style of a text, costs alignment. In a test on 300 random worlds, the measured error stayed below the theoretical bound every time, at 18% of it at most. ## Results with numbers **Small scale.** On Flickr30k (29k images, five captions each), the authors trained a 33M-parameter text encoder three ways and matched it to an image JEPA with one fitted rotation. Good-view training scored a mutual 10-nearest-neighbor overlap of .062, against .047 for bad views and .042 for next-token prediction. Image-to-caption recall at rank 1 was .044, .027 and .021. **Sample efficiency.** On web text with 109M parameters, good views reached an alignment score of .109 with 21.6M samples. Next-token prediction needed 900M samples to reach .111. That is 42 times fewer samples, and 14 times fewer text tokens. **Against big language models.** Trained on 900M samples from the texts of 940M web images, the good-view text model scored .184 against DINOv2, a vision model. Qwen2.5-72B scored .154 and Llama-3-70B scored .151. The same text model also beat them against CLIP's image encoder (.230, against .210 and .206). **A rotation makes a captioner.** The authors trained a decoder on text only, then fed it DINOv2 image embeddings passed through one fitted rotation. It reached a CIDEr captioning score of .838 on COCO in the best setup. Earlier methods that use encoders trained on image-text pairs score higher in the paper's comparison: CapDec .918 and DeCap .912. **No pairs at all.** On a benchmark for aligning models without paired data, the text JEPA reached 64% top-5 accuracy on ImageNet-100. Language models up to 8B parameters reached at most 56%. **A scaling law.** The authors fit a curve for how alignment grows with data and the share of good views. It predicted 85 unseen runs with an error of .0093, close to the .0096 seed-to-seed noise. Without good views, alignment on web text stayed flat at .054 to .057 from 0.41M to 50.9M images. ## Limits The authors list several. The theory is idealized: finite worlds, perfect optimization, unique solutions, and no learned predictor, so it does not cover world models that predict forward in time. Good image views need several observations of one scene, such as video, and an image JEPA trained on 7k clips is much weaker than DINOv2. Most experiments therefore fix DINOv2 as the vision side. The headline result is also less firm than it looks. A fourth training run with twice the SIGReg weight reached only .121 on the same test, and the four-run average is .168. Training can collapse without gradient clipping: 3 of 36 default runs did. The scaling law describes one recipe, its exponents are poorly pinned down, and it overestimated good-view runs at 320M samples. Some comparisons also change the data along with the views, for example recaptions made by a model that sees images through CLIP. The paper has not been peer reviewed, and the results come from the authors' own experiments. ## What to watch The authors released [code](https://github.com/galilai-group/prh-proof) under the MIT license. It includes 345 training runs and tests that check the theory numerically. Full reproduction took about 5,500 GPU-hours. The open question the paper leaves is whether good views can be scaled for images the way they were for text. Read the [full paper on arXiv](https://arxiv.org/html/2610.39302) or its [Hugging Face page](https://huggingface.co/papers/2610.39302). ## Frequently asked questions ### What is the Platonic representation hypothesis? It says that models trained on different data types, such as images and text, converge toward a shared representation of the world as they scale. The new paper tries to turn it into a theorem for JEPAs. ### What is a JEPA? A joint-embedding predictive architecture is a self-supervised model trained so that the embeddings of two views of the same thing agree, with a term that prevents collapse. The paper uses the LeJEPA variant. ### What are good views and bad views? Good views are two observations of the same scene, such as two captions of one photo. Bad views are parts of one document, such as a masked span and its context. The paper says next-token prediction is a JEPA with bad views. ### Is the code available? Yes. The authors released it under the MIT license on GitHub, with 345 training runs and tests that check the theory. Full reproduction took about 5,500 GPU-hours. ## Sources - [Proving and Improving the Platonic Representation Hypothesis with JEPAs](https://arxiv.org/abs/2610.39302), arXiv - [Proving and Improving the Platonic Representation Hypothesis with JEPAs (full text)](https://arxiv.org/html/2610.39302), arXiv - [prh-proof code repository](https://github.com/galilai-group/prh-proof), GitHub (galilai-group) - [Paper page](https://huggingface.co/papers/2610.39302), Hugging Face