# What are embeddings? How AI turns meaning into numbers

> An embedding is a list of numbers that places text near similar text. Here is how they work and why search and RAG depend on them.

URL: https://claudegpt.si/news/what-are-embeddings  
Published: 2026-11-09T06:56:02.583Z  
Author: ClaudeGPT Si Newsroom  
Section: Research  
Topics: Embeddings, RAG, OpenAI, Voyage AI, Vector search  
Publisher: ClaudeGPT Si

## Key takeaways

- An embedding is a vector of numbers whose distance to other vectors shows how related two texts are.
- The word2vec paper of January 2013 showed word vectors could be learned from 1.6 billion words in under a day.
- Embeddings power semantic search and the retrieval step of RAG, and vectors from different models cannot be mixed.

An embedding is a list of numbers that stands for the meaning of a piece of text, so a computer can measure how close two pieces of text are in meaning. OpenAI's docs put it plainly: ["An embedding is a vector (list) of floating point numbers"](https://developers.openai.com/api/docs/guides/embeddings), and the distance between two vectors tells you how related the texts are. Small distance means related, large distance means not. That one idea sits under semantic search, recommendations and RAG.

## What embeddings are

Take the sentence "When is Apple's conference call scheduled?" An embedding model turns it into a vector, for example 1,024 numbers. Take a paragraph that announces the date of that call and turn it into a vector too. The two vectors point in nearly the same direction, even though the paragraph never uses the word "scheduled" the way the question does. A paragraph about photosynthesis lands far away.

This is the example Anthropic's embeddings guide uses: six short documents, one query, and the nearest vector is the right answer. A keyword search would have to match words. An embedding search matches meaning.

The numbers themselves mean nothing to a human. No single position says "this is about finance". The meaning lives in the geometry: texts about similar things end up near each other.

## Where the idea comes from

The modern version starts with word vectors. In January 2013, Tomas Mikolov, Kai Chen, Greg Corrado and Jeffrey Dean posted ["Efficient Estimation of Word Representations in Vector Space"](https://arxiv.org/abs/1301.3781), the word2vec paper. It proposed two new model designs for learning continuous vector representations of words from very large datasets. The authors say it takes "less than a day to learn high quality word vectors" from a 1.6 billion word dataset. They also report the best results they knew of on their test of syntactic and semantic word similarity.

The point of the paper was cost. Earlier neural methods produced good vectors but were too slow to train on that much text. Cheap training meant everyone could have word vectors.

Today's models embed whole sentences, pages, code, images and video, not single words. The principle is the same: train a neural network so that things with similar meaning get similar vectors.

## How embeddings are used

Embeddings measure closeness, so any job that needs "find things like this" can use them. OpenAI's guide lists search, clustering, recommendations, anomaly detection, diversity measurement and classification. Google's Gemini docs list RAG, information retrieval, search reranking, anomaly detection, classification and clustering.

**Search.** Embed every document once and store the vectors. At query time, embed the query and return the documents with the nearest vectors.

**RAG.** Retrieval-augmented generation fetches relevant passages and puts them in the prompt before the model answers. Embeddings are the fetching step. Our [explainer on RAG](https://claudegpt.si/news/what-is-rag-retrieval-augmented-generation) covers the full pipeline: chunk the documents, embed the chunks, retrieve the closest ones, and add them to the prompt.

**Classification and clustering.** Feed embeddings to a simple classifier, or group vectors that sit near each other to find themes in a pile of support tickets.

## The models and their numbers

Every provider's docs describe the same few knobs. Figures below are from each provider's documentation as of November 9, 2026.

| Provider | Model | Default dimensions | Max input |
| --- | --- | --- | --- |
| OpenAI | text-embedding-3-small | 1,536 | 8,192 tokens |
| OpenAI | text-embedding-3-large | 3,072 | 8,192 tokens |
| Google | gemini-embedding-2 | 3,072 | 8,192 tokens |
| Google | gemini-embedding-001 | 3,072 | 2,048 tokens |
| Voyage AI | voyage-4, voyage-4-large, voyage-4-lite | 1,024 | 32,000 tokens |

Anthropic does not make its own embedding model. Its [docs](https://platform.claude.com/docs/en/build-with-claude/embeddings) point to Voyage AI, now part of MongoDB, and say to assess several vendors.

On quality, OpenAI lists MTEB scores of 62.3% for text-embedding-3-small and 64.6% for text-embedding-3-large. MTEB is a public benchmark of embedding tasks. Google reports 68.17 for gemini-embedding-2 at 1,536 dimensions. These are vendor-reported numbers, and the two companies' figures may not be directly comparable.

## Settings that matter in practice

**Shorter vectors.** Longer vectors cost more to store and search. OpenAI's `dimensions` parameter and Google's `output_dimensionality` (128 to 3,072) let you shrink them. OpenAI says a text-embedding-3-large vector cut to 256 dimensions still beats an unshortened text-embedding-ada-002 vector on MTEB. Google's own numbers for gemini-embedding-2 show 63.31 at 128 dimensions against 68.17 at 1,536. If you cut vectors yourself, normalize them again.

**Cosine similarity.** OpenAI recommends it. OpenAI and Voyage both return vectors normalized to length 1, so a plain dot product gives the same ranking, and faster.

**Query versus document.** Retrieval models treat the two differently. Voyage's docs say to always set `input_type` to "query" or "document" for retrieval, and explain that this prepends a short instruction to the text. Gemini's older model has a `task_type` setting, and gemini-embedding-2 uses text prefixes instead.

**Compression.** Voyage's docs describe storing vectors as 8-bit integers or single bits, which cuts storage and cost by 4x and 32x.

**Never mix models.** Google notes the embedding spaces of its two models are incompatible, so upgrading means re-embedding all your data. The same holds across providers.

## What embeddings do not do

An embedding captures similarity, not truth. A passage that is close to a question is not necessarily correct. Closeness is a signal, not a check. That is why RAG systems often add a reranker, a second model that reads the query and each candidate and orders them by relevance. Voyage sells both.

Embeddings also depend on the model's training data. Voyage's docs list models tuned for code, finance and law, because a general model can miss the vocabulary of a specialist field.

## What to watch

The trend in the docs is toward embeddings of more than text. Google's gemini-embedding-2 takes text, images, video, audio and PDFs, and Voyage has a multimodal model for interleaved text, images and video. One search index across all of them is now something you can buy off a price list.

## Frequently asked questions

### What is an embedding in AI?

An embedding is a list of floating point numbers that represents a piece of text, an image or other data. Texts with similar meaning get vectors that are close together, which lets software measure relatedness by distance.

### How are embeddings used in RAG?

In RAG, documents are split into chunks and embedded ahead of time. At question time the query is embedded, the nearest chunks are retrieved, and they are added to the model's prompt.

### Does Anthropic have an embedding model?

No. Anthropic's docs say it does not offer its own embedding model and point to Voyage AI as one provider, while advising readers to compare several vendors.

### Which similarity measure should I use for embeddings?

OpenAI recommends cosine similarity. OpenAI and Voyage AI vectors are normalized to length 1, so a dot product gives the same ranking as cosine similarity and is faster to compute.

## Sources

- [Efficient Estimation of Word Representations in Vector Space](https://arxiv.org/abs/1301.3781), arXiv
- [Vector embeddings](https://developers.openai.com/api/docs/guides/embeddings), OpenAI
- [Embeddings](https://platform.claude.com/docs/en/build-with-claude/embeddings), Anthropic
- [Gemini API embeddings](https://aistudio.google.com/docs/embeddings), Google
- [What is RAG? Retrieval-augmented generation explained](https://claudegpt.si/news/what-is-rag-retrieval-augmented-generation), ClaudeGPT Si
