Fine-tuning

BETA

This is a beta feature. For more information and access requests, please contact us through the Support Portal.

Fine-tuning trains a LoRA adapter on your own examples. The base model's weights are untouched; a small adapter learns your domain, wording and answer format. When the job succeeds, the adapter appears in the model catalog. Serve it before you can query it like any other project model.

Tailored AI overview — start a fine-tuning job, open models, or jump to Data Designer.

Use it when a general model is fluent but wrong about your specifics: internal procedures, legal wording, writing style.

Supported for text LLMs only, see Supported base models.

Need examples to train on? Data Designer is optimized to produce inputs that are ready for these runs.

---

How it works

1. Pick a training dataset. Upload JSONL, or choose a file already in the project (including one from Data Designer).

Select training dataset — upload JSONL or reuse a file, including one from Data Designer.

2. Optionally add a validation dataset in the same format, so training can report eval loss. If you skip this and the training file has at least 50 rows, 10% is held out automatically. Under 50 rows with no validation file, the run reports no eval loss.
3. Configure the job. Choose a base model, a suffix for the adapter name, and hyperparameters (or keep the model defaults). Tick auto-serve to have the adapter served automatically once training succeeds. Click Create Job.
4. Training runs. The dataset(s) are downloaded and validated, then the LoRA adapter is trained. You can watch live progress, loss, and cancel at any point.
5. Serve the adapter. When the job succeeds, it shows up with your other models (tagged as fine-tuned). It stays Not serving until you load it from the job or the model catalog, unless you ticked auto-serve.

Supported base models

Fine-tuning currently covers text LLMs only, and trains LoRA adapters. Embedding, speech, vision and reranking models cannot be fine-tuned and no full-weight fine-tuning is offered.

Base models with a training recipe:

• NVIDIA Nemotron-3-Nano-30B-A3B
• Qwen3-30B-A3B-Instruct-2507

---

Dataset format

Training data must be JSONL. Each line is one example with an id and a messages list of role / content objects.

json

1

{"id": "1", "messages": [{"role": "user", "content": "What is our return window?"}, {"role": "assistant", "content": "Customers can return unused items within 30 days of delivery."}]}

Each conversation must be trainable: an optional system message first, then alternating user / assistant turns, last message assistant, and no empty content.

Validation happens before any model loads and is all-or-nothing: a single invalid row fails the job, naming the line and the reason. No rows are dropped.

Examples are truncated at 2048 tokens. Longer conversations lose their tail, so split them before uploading.

If you generated the file in Data Designer, it is already in this format. If you have source material (policies, manuals, tickets) but not a training file yet, start there.

---

Wizard defaults

Hyperparameters are pre-filled from the selected model. Use Reset to defaults to restore them.

Setting	Typical default	Why you might change it
Epochs	1 (range 1 to 50)	More passes can help on a smaller file. Too many can memorize the data.
Batch size	1, 2, 4, or 8	Examples per GPU per step. The effective batch is fixed at 32 either way, so this changes neither training results nor how many loss points you get. It trades GPU memory against the number of accumulation steps. Lower it if a run hits out-of-memory.
Learning rate multiplier	1 (range 0.1 to 10)	1 keeps the recommended rate for that model. Lower if training looks unstable; higher learns faster but can overshoot.
LoRA rank	16 (8, 16, 32, 64, or 128)	Higher rank can capture more detail. Lower stays lighter and is less likely to overfit. Rank cannot exceed 128.
Suffix	None	Label (max 18 characters) appended to the fine-tuned model name, which reads ft:{base-model}:{id}:{suffix}
Validation file	None	Provide your own eval split, or skip and let a 10% holdout be taken when the training file has 50+ rows.
Serve automatically after training	Off	Loads the adapter as soon as the job succeeds, instead of leaving it Not serving until you serve it. See Serving a fine-tuned model.

Leave the defaults unless you have a reason to change them.

Configure — base model, suffix, hyperparameters, and optional auto-serve.

---

In the product

Feature	What you get
Wizard	Training dataset, optional validation dataset, then configure and start.
Live run tracking	Phase progress (download dataset(s), validate, train, upload adapter), loss charts, and cancel.
All runs	A list of fine-tuning jobs for the project, with status (queued, running, succeeded, failed, cancelled).
Finished adapters	Successful jobs appear in Inference → Models. Serve or unload them, or tick Serve automatically after training.

Job details — Succeeded, Adapter Not serving, and Serve model.

Training metrics — loss, learning rate, and gradient norm while the job runs.

Job details — Succeeded, Adapter Not serving, and Serve model.

Training metrics — loss, learning rate, and gradient norm while the job runs.