Skip to content
Devgo

Fine-tuning

Taking a pre-trained model and training it further on your specific data to specialize it. Cheaper than training from scratch. Fine-tuning adapts a general model to your domain, style, or task. Used to make LLMs follow your format, tone, or know your domain better.

When to use
  • Specializing a model to your domain/style.
  • When RAG isn't enough (format, tone, behavior).
  • Reducing reliance on API models.
When not to
  • When the base model is already good enough.
  • When you need updatable knowledge (use RAG).
  • Small datasets (risk overfitting).

Definition

Fine-tuning continues training a pre-trained model on a smaller, task-specific dataset. The model's weights are updated to specialize. Types: full fine-tuning (all weights), LoRA/QLoRA (efficient, only adapter weights), instruction tuning (SFT), preference tuning (RLHF/DPO).

Why it exists

Training a model from scratch costs millions. Fine-tuning a pre-trained model on your data is much cheaper and adapts it to your needs (domain, style, format).

How it works

Start with a pre-trained model (e.g., Llama). Prepare a dataset of examples (input → output). Run training: the model's weights update to fit your data. LoRA freezes most weights and trains small adapters (cheaper). Evaluate on held-out data. Deploy the fine-tuned model.

Scaling limits

Needs quality data (garbage in, garbage out). Risk of catastrophic forgetting (loses general ability). Compute cost even if less than pre-training. Hard to evaluate quality. Overfitting on small datasets.

Cost shape

Much cheaper than pre-training. LoRA: tens to hundreds of dollars. Full fine-tuning: more. Needs GPUs.

Alternatives

  • RAG — Retrieve external knowledge at inference, no training, easy to update.
  • Prompt engineering — No training, but limited control.

Failure modes

  • Catastrophic forgetting.
  • Overfitting.
  • Poor quality data.
  • Hard to evaluate.

,