# What is fine-tuning? How to train a model on your own examples

> Fine-tuning shapes how a model behaves using your examples. OpenAI's docs say to try prompting first, and its platform is closing to new users.

URL: https://claudegpt.si/news/what-is-fine-tuning  
Published: 2026-11-08T14:56:44.817Z  
Author: ClaudeGPT Si Newsroom  
Section: Research  
Topics: Fine-tuning, OpenAI, LoRA, Language models, Research  
Publisher: ClaudeGPT Si

## Key takeaways

- Fine-tuning trains a base model on your input and output examples; OpenAI's docs list supervised, vision and DPO methods.
- OpenAI recommends trying prompt engineering first, and says its fine-tuning platform is no longer open to new users.
- The LoRA paper reports about 10,000 times fewer trainable parameters and about 3 times less GPU memory than full fine-tuning of GPT-3 175B.

Fine-tuning means training an existing AI model further on your own examples, so it does one job better than the base model does out of the box. You show it inputs and the outputs you want, it adjusts its weights, and you get a new version of the model. Before you do it, try a better prompt, because that is what OpenAI's own docs tell you to do first.

## What fine-tuning is

[OpenAI's fine-tuning guide](https://developers.openai.com/api/docs/guides/fine-tuning) describes it as training a base model on example inputs and outputs so it performs well on your task. The steps it lists are simple: collect a dataset of examples, upload it as a JSONL file, create a fine-tuning job in the dashboard or the API, and evaluate the result.

The guide names three methods:

- **Supervised fine-tuning:** train on examples of correct responses. It suits classification, specific output formats and fixing instruction-following failures.
- **Vision fine-tuning:** supervised training with image inputs, for tasks such as image classification.
- **Direct preference optimization (DPO):** give a good and a bad response for each prompt and mark which is better. It suits matching tone and style, or summaries that focus on the right content.

## When to fine-tune, and when not to

OpenAI recommends prompt engineering first, since it may be enough. Fine-tuning adds four things on top of prompting, according to the same guide:

1. It can use more examples than fit in one prompt.
2. It allows shorter prompts with fewer examples, which can cut cost and delay per request.
3. It lets you train on proprietary data without sending it in every request.
4. It can produce a smaller, cheaper, faster model for one task.

For facts that change, such as a product catalog or a policy document, the guide says to put the relevant context in the prompt. Pulling that context in at request time is what [retrieval-augmented generation](https://claudegpt.si/news/what-is-rag-retrieval-augmented-generation) does. In short, fine-tuning shapes how a model behaves, and retrieval supplies what it should know right now.

## How LoRA makes fine-tuning cheaper

Training every weight in a large model needs a lot of memory. Low-Rank Adaptation, or LoRA, freezes the original weights and trains small added matrices in each layer. The [LoRA paper](https://arxiv.org/abs/2106.09685) reports that, compared with fully fine-tuning GPT-3 175B with Adam, it cuts trainable parameters by about 10,000 times and GPU memory by about 3 times.

The authors report that it matched or beat full fine-tuning on RoBERTa, DeBERTa, GPT-2 and GPT-3, and that unlike adapter methods it adds no inference delay. Their motivation was cost: serving a separate fully tuned 175B-parameter copy for every task is too expensive.

The numbers come from the models the authors tested, so treat them as an illustration of the idea, not a promise for every model.

## What changed at OpenAI

The fine-tuning page now carries a status note: OpenAI is winding down the platform. It is no longer open to new users. Existing users can still create training jobs for the coming months, and fine-tuned models stay available for inference until their base models are deprecated.

As of November 8, 2026, the supported models listed are `gpt-4.1`, `gpt-4.1-mini` and `gpt-4.1-nano` for supervised fine-tuning and DPO, and `gpt-4o-2024-08-06` for vision. The page does not say how many examples you need, and it sends you to the pricing page for costs.

That means a new project on OpenAI's platform cannot start fine-tuning there. Options that remain are better prompts, retrieval, or tuning an open-weight model yourself with a method such as LoRA.

## A short checklist before you fine-tune

- Write down one task and a way to score it. Without a test set, you cannot tell if tuning helped.
- Try a stronger prompt and a few examples in the prompt first.
- If the model lacks facts, add retrieval instead.
- Start with a small dataset, evaluate, then grow it.

For how the cost of a longer prompt works, see [what prompt caching is](https://claudegpt.si/news/what-is-prompt-caching).

## Frequently asked questions

### Is fine-tuning the same as RAG?

No. Fine-tuning changes how a model behaves by training it on examples. Retrieval-augmented generation puts relevant documents into the prompt at request time, which suits facts that change.

### How many examples do I need to fine-tune a model?

OpenAI's guide does not give a number. Start small, evaluate against a test set, then add more.

### What is LoRA?

Low-Rank Adaptation freezes a model's weights and trains small added matrices instead. The paper reports about 10,000 times fewer trainable parameters than full fine-tuning of GPT-3 175B.

### Can I still fine-tune on OpenAI?

OpenAI says the platform is no longer accessible to new users. Existing users can still create jobs for the coming months.

## Sources

- [Fine-tuning guide](https://developers.openai.com/api/docs/guides/fine-tuning), OpenAI
- [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685), arXiv