Do we need fine-tuning or better prompts?

Most tasks need better prompts and retrieval first. If those reach the accuracy you need, fine-tuning adds cost and maintenance for no gain. It earns its place when a narrow task runs at high volume and the prompt-only version stops improving on your test set.

How much data does it take?

It depends on the task, and we answer it with your own data in the suitability check. A classification task with clear categories needs fewer examples than a task that writes in your house style. The labels must be correct, because the model learns the mistakes along with everything else.

Does our data leave our environment during training?

No. Training runs on your own servers or in your own cloud account, and the training data, the test set and the tuned model stay there. The open-weight model is downloaded from Hugging Face into your environment and needs no public AI service after that.

What happens when a better base model comes out?

The training data and the test set are kept. A new base model is then one training run and one score rather than a new project. We tell you to switch if the new base model with prompting alone beats the tuned model on your test set.

Model fine-tuning

A tuned model ships only if it beats the base model

The test set is built from your own past work.

01Examples from past work
02Fine-tune an open-weight model
03Score on a held-out set
04Compare with the base model

Better than the base model on your set?

Yes

Ship it, versioned in MLflow

Served with vLLM on your GPUs

No

Keep the base model

Improve prompts and retrieval instead

You leave knowing whether fine-tuning pays for this task

01
A written answer on whether fine-tuning is worth it
02
A tuned model scored against two baselines
03
The model running on your own infrastructure

What it is

Fine-tuning means training an existing model further on your own examples so it does one task better. We use open-weight models such as Llama and Mistral, which are models whose files you download from Hugging Face and run yourself. The result is a model your business owns, running on your own servers or in your own cloud account.

The tasks are narrow:

Classification, such as sorting a claim, an email or a transaction into the right category
Extraction, such as reading the fields from a loan application, a policy schedule or a supplier invoice
House style, such as writing a file note, a credit memo or a client letter the way your team writes them

When it is not worth it

Most tasks start with prompting and retrieval, and many end there. A clear prompt with a few worked examples handles a lot of classification and extraction. Retrieval gives the model facts from your documents, and fine-tuning is a poor way to teach a model facts.

Fine-tuning is not worth it when:

The model lacks facts rather than skill, which retrieval fixes and fine-tuning does not
The prompt-only version still improves each time someone works on the prompt
Volume is low, and a larger model with a longer prompt costs less than a training pipeline
The task changes often, and each change needs a new training set

We put the answer in writing at the end of the suitability check. The engagement stops there if the answer is no, and you keep the test set for the prompt-only version.

When it is worth it

Fine-tuning is worth it when a narrow task runs at high volume and its output follows a fixed format or a house style. The prompt-only version also has to stop improving on your test set. A smaller tuned model can also replace a large general model on that one task. The smaller model needs less hardware, which matters when it runs on your own servers.

Evaluation

We build the test set first, from your own records, and hold it back from training. The model never sees it, and it catches a model that memorised its training examples. Every version of the model is scored on it against two baselines. The first is the base model before tuning. The second is the best prompt-only version we can build on the same model.

The tuned model goes live only if it beats both on the measures you agree at the start. Examples are accuracy per category or the share of fields extracted correctly. The scores come to you as a table with the errors listed, and a person can read what each version gets wrong.

Training and serving

Training uses LoRA (low-rank adaptation), a method that trains a small set of added weights rather than the whole model. That keeps a training run within hardware a business can own or rent, such as one or more NVIDIA GPU servers. The training data, the test set and the tuned model stay in your environment.

MLflow records each training run with the version of the training data, the settings and the scores. The tuned weights go into the MLflow model registry with a version number. vLLM serves the tuned model inside your environment, and on-prem AI deployment covers how we host it, log requests and control access.

You roll back to the previous version in the registry if a new one performs worse in production.

Where it fits

A tuned model is usually one step inside a larger process. An extraction model reads the fields from a document. A workflow automation build then routes them, checks them and puts a person where an error costs money. A classification model can sort the documents that LLM integration and RAG then searches. The labelled records it trains on often come out of data engineering work already done.

Handover

Your team gets the training code, the training set, the test set and a retraining procedure in your own repositories. The procedure covers how to add new labelled records, run a training job, score it against the test set and promote it in the registry.

model-fine-tuning.stack.yaml

# Tools on this service. Same stack across every Zinto build.
service: Fine-tuning
runs_on: your servers, on premises
stack:
  - Hugging Face
  - Llama
  - Mistral
  - MLflow
  - vLLM
  - On-prem NVIDIA GPUs

Other services

All seven services →

U1Strategy and readiness auditA ranked list of what to build, what to buy and what to leave, checked against your data, systems and security rules U2On-prem AI deploymentCustom GPU servers in your building or data centre, running open-weight models on bare metal, with every request logged in your environment U3Forward deployed engineersSenior engineers inside your team and your systems until the work is live and handed over U4Agents and workflow automationAgents that read, check and post work inside your systems of record, with a person where an error costs money U5Data engineeringThe data layer your agents and reports read from U6LLM integration and RAGAnswers from your own documents, each with its source