Data Designer

BETA

This is a beta feature. For more information and access requests, please contact us through the Support Portal.

Data Designer turns the documents you already have into a training dataset. It processes your files (extracts text, cleans it, and chunks it into passages), then uses that as input to a synthetic dataset generation pipeline. The output is supervised fine-tuning (SFT) JSONL, optimized as input for fine-tuning runs.

You need an API key for the active project. Generation and the optional quality judge call models through that key, using the same keys and model access as the rest of Studio. Create one under API keys if you do not have a key yet; see Projects.

Data Designer overview — create a dataset, track workflows, and open files.

What this enables

Use Data Designer when you want a model that answers in your domain (company support, internal policies, legal reference, product knowledge) without assembling thousands of training rows by hand.

From a guided wizard you can:

• Start from raw documents, or reuse files you already processed
• Generate supervised fine-tuning (SFT) examples: a user instruction paired with a grounded assistant response
• Review a small preview before you commit to a full run
• Apply quality checks so weak or duplicate examples can be dropped
• Download the finished dataset as JSONL, or send it on to fine-tune a model in AI Studio

How it works

1. Upload documents (or reuse data from a previous run). Keep OCR enabled for PDFs, scans, and images. PII redaction always runs: names, emails, phone numbers, and similar details are removed before generation. You cannot turn this off.
2. Configure the job. Name the dataset, set how many examples to create, and pick models. The generation model writes the synthetic instruction and response columns. You can pick a different model for the optional LLM judge (for example a larger model to write the pairs and a smaller, cheaper one to score them). Paste a project API key so those calls can run.
3. Preview, then generate. Data Designer extracts text, cleans it, and chunks it into passages. Those passages feed a synthetic dataset pipeline (NVIDIA NeMo Data Designer): an LLM produces SFT instruction and response pairs. You get a 5-row preview first, then the full run.
4. Quality checks (optional). The LLM judge scores each pair so weak or duplicate examples can be dropped. Instruction, response, and the judge all run as inference on the API key you provided.
5. Download the finished SFT JSONL, or continue to LoRA fine-tuning. You can track live progress and cancel a run while it is going.

Powered by NVIDIA NeMo Data Designer

Dataset generation uses NVIDIA NeMo Data Designer, NVIDIA’s framework for building high-quality synthetic datasets from seed data.

Instead of a single prompt that dumps rows, NeMo Data Designer orchestrates generation from your document passages. It produces related instruction and response columns with an LLM, supports a preview step before a full run, and can score examples with an optional LLM judge. This runs against Nebul-hosted models in your project.

Learn more in the NeMo Data Designer documentation.

---

Supported files and size limits

Upload documents that contain the knowledge or style you want the model to learn. For example, internal docs for a company assistant, or reference material for a domain-specific agent.

File types

Type	Extensions
PDF	.pdf
Word	.docx
PowerPoint	.pptx
Images	.png, .jpg, .jpeg
Plain text	.txt, .md

Keep OCR enabled when you upload files whose text has to be read from the page, such as PDFs, scans, and images. It is on by default in the recommended profile.

How much you can upload

Limit	Value
Per file	512 MB
Total per upload	5 GB

---

How many examples to generate

You choose the record count in the wizard. During beta, a run can create 1 to 10,000 examples. The default is 1,000. Preview always generates 5 sample rows first.

Goal	Suggested size
Focused assistant (support, policies, a single domain)	500 to 1,000 records
Broader topics or more response variety	5,000+ records

Quality matters more than a high count. Use Preview to inspect samples before you start the full generation.

Generated instructions and responses default to English. You can also generate the dataset in Dutch, German, or Polish, independent of the language of the source documents.

---

Wizard defaults

The wizard starts from the Recommended for Fine-tuning profile. Most of these sit under Advanced settings. PII redaction is always on and cannot be disabled.

Setting	Default	What you can do
OCR for scanned pages	On	Leave on for PDFs, scans, and images. Turn off for plain .txt / .md if you want a faster extract.
Normalize text	On	Cleans spacing and encoding. Rarely needs to be off.
Redact PII	Always on	Finds and redacts names, emails, phone numbers, and similar. Not a toggle.
Chunking strategy	Recursive (recommended)	Prefers natural breaks (paragraphs, then sentences) before cutting by length. Switch to paragraph or sentence if your docs are already well structured; character windows if you need a fixed size.
Chunk size / overlap	800 / 200	Smaller chunks give tighter questions; larger chunks keep more surrounding context. Raise overlap if important sentences were getting split across chunks.
LLM judge	On	Scores each instruction and response (1 to 5) and records reasoning. Turn it off to save inference cost if you will not score rows. You can pick a different (often cheaper) model for the judge than for generation.
Drop failed validations	On	Removes pairs that did not pass automated checks. Turn this off if you want every generated row kept so you can curate keep/drop yourself from the judge report.
Remove exact duplicates	On	Keeps one copy of identical pairs. Turn off if repeated examples are useful for your task.
Minimum judge score	4 (of 5)	Rows below this are dropped. Lower it to keep more data, or turn dropping off and review scores and reasoning yourself.
Records	1,000 (preview: 5)	See How many examples to generate.
Generation language	English	Set Dutch, German, or Polish if you want generated pairs in that language even when the source docs are not.
Temperature	0.7	Lower for more consistent wording; higher for more variety.

A Fast profile exists (OCR off, looser quality gates) and a Thorough profile (stricter gates). Use Custom when you mix settings.

Custom prompts

Under Advanced settings you can override how pairs are written and how they are scored. Leave a field empty to keep Data Designer's default.

Prompt	What the default does	Why you might edit it
Instruction prompt	Tells the generation model how to write the user-side question from the source passage.	Change the question style (for example shorter, more like support tickets, or always in a given format). Keep {{ content }} so the model still sees the passage.
Response prompt	Tells the generation model how to write the assistant answer.	Change answer length, tone, or structure. Keep {{ instruction }} and {{ content }} so the answer stays tied to the question and the source.
Judge prompt	Tells the judge model how to score each pair (accuracy, completeness, whether it is good training data).	Raise or lower the bar, or score for something specific (for example "must cite a policy name"). Variables: {{ instruction }}, {{ response }}, {{ content }}.

---

In the product

Feature	What you get
Wizard	Guided flow: pick a source, configure, preview a few rows, then generate the full SFT dataset.
Live run tracking	Watch extraction, generation, and quality checks as they happen. Cancel a run you no longer need.
Runs and files	See every run, then browse cleaned and chunked data as well as the finished SFT dataset.
LLM judge review	If the judge is on, open the generated rows to inspect scores, pass/fail, and the model's reasoning. Download the judge file if you want to keep or drop examples yourself.

Workflow detail — pipeline steps from upload through finished.

File preview — inspect chunked seed data before you generate or download.

When the dataset is ready, continue in Fine-tuning.