Fine-Tuning · 52 topics
How do you decide between prompt engineering, RAG and fine-tuning?
Short version: prompting and retrieval change what the model sees; fine-tuning changes how the model responds to what it sees.
Start with prompting (cheapest, fastest). Add RAG when the gap is knowledge: private, large or fast-changing facts, citations, access control.
Fine-tune when the gap is behavior: consistent format or tone, domain-specific reasoning, tool-use patterns, lower latency and cost by moving to a smaller model, or prompts that have become too long and brittle.
Do not fine-tune to inject facts that change often, or when you have no reliable evaluation set and clean data.
Common production answer is a hybrid: a small tuned model or adapter, retrieval for facts, and a tight prompt. Decide with an eval set, trying the cheapest option first.
What is the difference between continued pretraining, supervised fine-tuning, instruction tuning and preference tuning?
Continued (domain-adaptive) pretraining: more next-token training on raw domain text (legal, medical, code) to add vocabulary and knowledge; needs lots of data and compute and usually hurts instruction following, so SFT follows it.
Supervised fine-tuning (SFT): train on prompt and response pairs where the response is the exact target.
Instruction tuning: SFT on a broad, diverse set of instructions so the model follows arbitrary tasks and formats.
Preference tuning (RLHF, DPO and relatives): optimize toward responses humans or judges prefer, to shape helpfulness, safety and style.
Should you start from a base model or an instruction-tuned (chat) model?
For most application tasks, start from the instruction-tuned variant: it already follows instructions, handles multi-turn chat formatting and carries the provider's safety tuning, so a small SFT set only has to teach your task.
Start from a base model when you need full control over the output distribution or chat format, are doing heavy domain adaptation or research, or will run your own complete post-training pipeline.
Always use the same chat template the model was trained with.
Walk me through how you fine-tune an LLM end to end.
Define the task and success metrics; build a baseline with prompting and measure it.
Collect and clean data, split into train, validation and test sets, and check for leakage and duplicates.
Choose the base model and method (full, LoRA or QLoRA), apply the chat template, mask the loss to the response tokens, and set hyperparameters.
Train with logging, watch training and validation loss, save checkpoints, then evaluate on the held-out test set and a regression suite for general abilities and safety.
Deploy behind a feature flag or canary, monitor quality, and iterate by adding failure cases to the data.
How do you prepare a high-quality fine-tuning dataset?
Gather real or realistic prompts that match production inputs, and write or select correct, consistent target responses (domain experts for specialized tasks).
Clean and normalize: remove duplicates and near-duplicates, broken or truncated examples, wrong labels, and boilerplate; filter by length and quality heuristics.
Cover edge cases, refusals and diverse phrasings; balance the mix of tasks so one type does not dominate.
Version the dataset with its provenance, so any model can be traced back to the exact data it was trained on.
What is a chat template, and what goes wrong if you get it wrong?
A chat template formats system, user and assistant messages into the exact token sequence (special role and end-of-turn tokens) the model was trained on.
Training or serving with a different template shifts the input distribution: quality drops, the model may not stop generating, or it may echo role tokens.
Use the tokenizer's built-in chat template for both training and inference, and test a few formatted examples by decoding them back to text.
What is loss masking, and why train only on the response tokens?
In SFT, labels for prompt tokens are set to an ignore value so the loss is computed only on the assistant's response (completion-only loss).
Otherwise the model also spends capacity learning to predict user prompts and system text, which wastes signal and can bias the model toward imitating the prompt style.
Packing multiple short examples into one sequence improves GPU efficiency, but needs correct attention masks so examples do not attend to each other.
How much data do you need for supervised fine-tuning?
Quality and diversity beat quantity: a few hundred to a few thousand clean, representative examples often suffice for style, format and narrow tasks; complex skills or new domains need more.
The effective approach is to start small, train, inspect failures, and add targeted data where the model is weak, rather than collecting a huge noisy set.
Plot performance against dataset size (a learning curve); if it plateaus, more of the same data will not help.
Can you use synthetic data for fine-tuning? What are the risks?
Yes: a stronger model can generate instructions and responses, rewrite or augment existing data, or create hard cases and preference pairs; it is common when labeled data is scarce.
Risks: errors and hallucinations copied into training data, low diversity and style collapse, model-collapse effects when repeatedly training on model output, and licensing limits on using another provider's outputs.
Mitigate with filtering (rule checks, LLM judges, deduplication, human review of a sample), mixing in real data, and always evaluating on real, human-verified test cases.
How do you split data and prevent contamination between training and evaluation sets?
Hold out a validation set for tuning and early stopping, and a separate test set touched only for final reporting.
Deduplicate across splits (exact and near-duplicate matching), because paraphrased copies of test items in training inflate scores.
Split by group when examples are related (same document, user or conversation), and by time when the data is time-ordered.
Check that public benchmarks you report on are not in your training data.
How do you govern training data for fine-tuning (licensing, PII, toxicity, consent)?
Track provenance and licensing for every source, and exclude data you have no right to use for training.
Detect and remove or mask personal and confidential data, secrets and credentials, since models can memorize and regurgitate rare strings.
Filter toxic or biased content as appropriate, respect consent and retention policies, and keep audit logs of which dataset version produced which model.
What should you consider when choosing the maximum sequence length for training?
Cover the lengths your real inputs and outputs need; truncating examples can cut off the answer and teach the model to stop early.
Memory and compute grow with sequence length (attention cost, activation memory), so choose the smallest length that covers most examples and handle outliers by filtering or splitting.
Use packing or length-grouped batching to reduce padding waste, and make sure the length matches the context you will use at inference.
What are the trade-offs between full fine-tuning, LoRA and QLoRA?
Full fine-tuning updates every weight: highest capacity and best for large domain shifts, but needs the most GPU memory, produces a full-size checkpoint per task, and forgets more.
LoRA trains small low-rank matrices (often 0.1% to 2% of parameters): far less memory, small swappable adapters, less forgetting, and quality close to full fine-tuning for most tasks.
QLoRA does LoRA on a 4-bit quantized frozen base: lowest memory (fits large models on one GPU) at the cost of slower training steps and a small possible quality drop, so watch evaluation regressions.
Choose by budget, model size and how far the target behavior is from the base model.
How does LoRA work?
For a frozen weight W0 of shape d by k, LoRA learns two small matrices: B (d by r) and A (r by k), with rank r much smaller than d and k.
The forward pass becomes h = W0 x + (alpha / r) x B(A x). A is initialized randomly and B to zero, so the adapter starts as a no-op and training begins from the base model's behavior.
Only A and B receive gradients, which cuts trainable parameters and optimizer memory sharply.
After training you can merge B times A into W0 for zero extra inference latency, or keep adapters separate and swap them per task.
How do you choose LoRA rank, alpha and dropout?
Rank r sets adapter capacity: small ranks (8 to 16) suit style and format tasks; larger ranks (32 to 128) help with harder skills, at the cost of more parameters and overfitting risk. Beyond some rank extra capacity stops helping.
Alpha is a scaling factor: dividing by r keeps the effective scale stable when you change rank. A common setting is alpha equal to r or twice r; sweep it with the learning rate.
Dropout (for example 0.05) adds light regularization; skip it if your kernels or libraries require zero dropout.
Tune rank and learning rate against validation metrics and a regression suite, not only training loss.
Which modules should LoRA target?
Early practice attached LoRA only to the attention query and value projections.
Current practice often targets all linear layers (query, key, value, output projections plus the MLP gate, up and down projections), which usually gives better quality for the same rank at modest extra cost.
More targeted modules means more trainable parameters, so adjust rank down if memory is tight.
What is QLoRA, and what makes it memory-efficient?
The frozen base model is stored in 4-bit NormalFloat (NF4), a data type designed for normally distributed weights; LoRA adapters train in higher precision (BF16).
Double quantization quantizes the quantization constants themselves, saving additional memory.
Paged optimizers use unified memory to absorb memory spikes (for example from long sequences) and avoid out-of-memory crashes.
During the forward and backward pass, weights are dequantized on the fly to compute in BF16; gradients flow only into the adapters.
What other parameter-efficient fine-tuning methods exist?
Adapters: small bottleneck layers inserted inside each transformer block.
Prefix tuning and prompt tuning: learn continuous virtual tokens prepended to the input or hidden states; very few parameters but usually weaker on harder tasks.
IA3: learned scaling vectors on activations. DoRA: separates weight magnitude and direction for better quality than plain LoRA at similar cost.
LoRA remains the default because it is simple, flexible, mergeable and widely supported.
How much GPU memory does full fine-tuning need?
Memory has four parts: weights, gradients, optimizer states, and activations.
With mixed precision and Adam, a rule of thumb is about 16 bytes per parameter: 2 for BF16 weights, 2 for gradients, 4 for FP32 master weights, and 8 for the two Adam moments. A 7B model needs roughly 112 GB before counting activations.
Activations grow with batch size and sequence length; gradient checkpointing, smaller micro-batches and shorter sequences reduce them.
LoRA cuts gradient and optimizer memory to the adapter parameters only; QLoRA also shrinks the stored base weights to about 0.5 bytes per parameter.
You have 2,000 labeled examples and one 40 GB GPU. How do you fine-tune a 13B model?
Full fine-tuning is out: it would need well over 200 GB. Use QLoRA: the 4-bit base takes roughly 7 to 8 GB, leaving room for adapters, optimizer states and activations.
Typical setup: rank 16 with alpha 32, adapters on all linear layers, learning rate around 2e-4, 2 to 3 epochs, a small micro-batch with gradient accumulation, gradient checkpointing, BF16 compute, and a max sequence length that covers your data.
Hold out a validation split, evaluate against a strong prompting baseline, and check general-capability regressions.
With only 2,000 examples, quality and overfitting control matter most; and say honestly if prompting plus retrieval would already be enough.
How do gradient checkpointing, gradient accumulation and mixed precision help?
Gradient checkpointing stores only some activations and recomputes the rest during the backward pass, trading about 20% to 30% more compute for large memory savings.
Gradient accumulation sums gradients over several micro-batches before one optimizer step, giving a large effective batch size without the memory cost.
Mixed precision (BF16 or FP16) halves memory for activations and speeds up math; BF16 usually needs no loss scaling, while FP16 does.
Other savings: 8-bit or paged optimizers, FlashAttention, and shorter sequences.
How do you train across multiple GPUs: data parallelism, FSDP, DeepSpeed ZeRO and model parallelism?
Data parallelism replicates the model on each GPU and splits the batch; simple, but every GPU holds a full copy of weights, gradients and optimizer states.
FSDP and DeepSpeed ZeRO shard optimizer states (stage 1), gradients (stage 2) and parameters (stage 3) across GPUs, so models far larger than one GPU's memory can be trained with data-parallel simplicity.
Tensor parallelism splits individual matrix operations across GPUs; pipeline parallelism splits layers into stages. These are used for very large models and combine with sharding.
Communication cost matters: fast interconnects (NVLink, InfiniBand) and overlapping communication with compute are key.
Which libraries and tools do you use for fine-tuning?
Hugging Face Transformers with PEFT (LoRA and QLoRA adapters) and TRL (SFT, DPO and other trainers) are the standard stack, with bitsandbytes for quantization.
Axolotl, LLaMA-Factory and Unsloth wrap these with configuration-driven training, and Unsloth adds optimized kernels for speed and memory.
Use Accelerate, FSDP or DeepSpeed for multi-GPU runs, and an experiment tracker (Weights and Biases, MLflow) to log hyperparameters, data versions and metrics.
How would you serve many LoRA adapters on one base model?
Load the base model once and keep each adapter as a small set of low-rank weights, applying the right one per request; servers such as vLLM support multi-LoRA batching.
Benefits: many tenants or tasks share the expensive base weights, and adapters are small to store and hot-swap.
Considerations: adapter loading and caching (GPU memory for hot adapters), a small per-request overhead compared with a merged model, and routing requests to the correct adapter.
If one adapter serves most traffic, merging it into the base gives the lowest latency.
Can LoRA teach a model new knowledge?
LoRA and SFT mainly shape behavior, style, format and skills; they can add some facts, but memorization is unreliable, hard to update, and can encourage confident hallucination about facts the model only half learned.
For knowledge that is large or changing, use retrieval; use continued pretraining only when you need broad domain vocabulary and have a lot of data.
Measure factual recall on held-out questions rather than assuming the model learned the content.
Why can't we just use supervised fine-tuning for alignment?
SFT teaches the model to imitate demonstrations, but it only sees good examples; it never learns what to avoid or which of two plausible answers is better.
Writing ideal answers for every situation is expensive, while ranking two answers is easier and more reliable for humans.
Preference methods use comparisons to push the model toward better responses and away from worse ones, shaping helpfulness, honesty, safety and tone beyond what imitation captures.
Walk me through how RLHF works end to end.
Step 1, SFT: fine-tune the base model on high-quality instruction and response data to get a reasonable starting policy.
Step 2, reward model: sample several responses per prompt, have humans rank them, and train a model to score responses so that preferred ones get higher rewards.
Step 3, reinforcement learning (typically PPO): generate responses, score them with the reward model, and update the policy to increase reward while a KL penalty keeps it close to the SFT reference model.
It works but is expensive and unstable: it needs several models in memory, careful tuning, and good preference data.
How is a reward model trained, and what can go wrong?
It is usually the language model with a scalar output head, trained on preference pairs with a Bradley-Terry style loss: the probability that the chosen response beats the rejected one is the sigmoid of the difference of their scores.
Problems: noisy or inconsistent labels, biases such as preferring longer or more confident answers, poor generalization outside the training distribution, and exploitation by the policy (reward hacking).
Evaluate it on held-out preference pairs, check agreement across annotators, and refresh it as the policy changes.
What is PPO doing in RLHF, and what does the KL penalty do?
Proximal Policy Optimization updates the policy using the reward signal and an advantage estimate (usually from a learned value model, the critic), with ratio clipping to stop any single update from changing the policy too much.
The KL penalty subtracts a term proportional to the divergence between the current policy and the frozen reference model from the reward, preventing the policy from drifting into gibberish or exploiting the reward model.
Too small a KL weight invites reward hacking; too large prevents learning. Some pipelines adapt the coefficient to hit a target KL.
What is DPO, and how does it work?
Direct Preference Optimization skips the explicit reward model and the RL loop. It trains directly on preference pairs: (prompt, chosen response, rejected response).
The loss is the negative log-sigmoid of beta times the difference between the log-probability ratios (policy over frozen reference) of the chosen and rejected responses, so the model increases the relative likelihood of the chosen one.
Beta controls how strongly the policy may deviate from the reference model; lower beta allows larger changes.
Data requirements: two responses per prompt with a reliable label; often start from an SFT model that matches the data distribution.
Why did DPO largely replace PPO-based RLHF in many pipelines, and when is PPO still better?
DPO is simpler (a supervised-style loss), cheaper (no reward model, critic, or sampling loop), and more stable to tune.
It is limited by the offline preference data you have: it cannot explore new outputs, can be sensitive to noisy pairs and length bias, and may degrade the likelihood of both chosen and rejected responses.
Online RL methods (PPO, GRPO) can still win when you have a strong reward signal and want the model to improve on its own samples, for example in math and code with verifiable rewards.
What are KTO, ORPO and IPO, and how do they differ from DPO?
KTO learns from unpaired good-or-bad labels (thumbs up and down) instead of pairs, which is easier to collect from products.
ORPO folds a preference penalty into the SFT objective, so alignment and SFT happen in one stage without a reference model.
IPO modifies the DPO objective with a regularized form to reduce overfitting to the preference data.
The choice depends on data format (pairs versus binary labels), whether you want a single stage, and how noisy your labels are; compare them empirically on your evaluation set.
What is GRPO, and how does it differ from PPO?
Group Relative Policy Optimization samples a group of responses for each prompt, scores them, and uses each response's reward relative to the group mean (often divided by the group standard deviation) as the advantage; no separate critic (value) model is needed.
Example: rewards of 1, 0, 0, 1 have mean 0.5 and standard deviation 0.5, giving advantages of +1, -1, -1, +1.
Savings: lower memory than PPO because there is no value network. Costs: several samples per prompt, and when all responses in a group score the same (all correct or all wrong) the learning signal vanishes.
It keeps PPO-style clipping and usually a KL penalty to a reference model.
What is RLVR (reinforcement learning with verifiable rewards), and when should you use it instead of a learned reward model?
RLVR uses a programmatic checker as the reward: unit tests for code, exact-answer matching for math, format and constraint validators.
Use it whenever correctness can be checked automatically: it is cheap, objective, and removes the reward-model failure mode where the policy exploits the scorer.
Use learned reward models or preference tuning for subjective qualities (helpfulness, tone, safety), and combine both when a task has verifiable and subjective parts.
What is reward hacking, and how do you detect and mitigate it?
Reward hacking is when the policy finds ways to score highly without doing the intended task: very long answers, flattery, formatting tricks, or exploiting bugs in a checker.
Detect: watch for a growing gap between training reward and a held-out evaluation or human ratings, rising response length, KL spikes, odd repeated patterns, and falling business metrics such as escalations to humans.
Mitigate: stronger or ensemble reward models, KL regularization, length normalization or penalties, deterministic checkers for hard constraints, adversarial examples added to reward-model training, and periodic human audits.
How do you collect and improve preference data?
Sources: human annotators ranking pairs, product feedback (thumbs up or down, edits, regenerations), and AI feedback (RLAIF), where a strong model judges responses against written principles.
Quality controls: clear guidelines, annotator calibration and agreement checks, removing ties and noisy pairs, balancing response length so length is not the deciding signal, and varying prompts and difficulty.
Prefer on-policy pairs (sampled from the model being trained) because they reflect its real mistakes, and verify any AI-judge labels on a human-checked sample.
What are rejection sampling and best-of-n fine-tuning?
Sample several responses per prompt from the model, score them with a reward model or checker, keep the best (or those above a threshold), and fine-tune on them with plain SFT.
It is simple, stable and often a strong baseline; it can be repeated in rounds as the model improves.
Best-of-n can also be used at inference time, trading extra compute for quality.
How would you design a preference-tuning pipeline for a product LLM?
Start from an SFT model that already does the task well, and define the behaviors you want to improve plus metrics for them.
Collect prompts from real usage, generate multiple candidates, label preferences (human, product signals, or calibrated AI judge), and clean the pairs.
Train with DPO (or a related method) on the pairs, or use RLVR for verifiable parts; keep a KL or reference constraint to avoid drift.
Evaluate on held-out preferences, task metrics, safety and a general regression suite; roll out gradually with monitoring and a rollback path, and feed new failures back into the data.
Which hyperparameters matter most in fine-tuning, and what are typical values?
Learning rate is the most sensitive: roughly 1e-5 to 5e-5 for full fine-tuning, and 1e-4 to 2e-4 for LoRA or QLoRA (adapters tolerate higher rates).
Epochs: usually 1 to 3; more epochs on small data quickly lead to memorization.
Effective batch size (micro-batch times accumulation times GPUs), warmup (a few percent of steps), a cosine or linear decay schedule, weight decay, and for LoRA the rank, alpha and target modules.
Tune on validation metrics, change one thing at a time, and fix random seeds so runs are comparable.
What is the role of warmup and learning-rate scheduling?
Warmup ramps the learning rate up from near zero over the first steps, avoiding large destabilizing updates while optimizer statistics (such as Adam's moments) are still noisy.
Decay (cosine or linear) lowers the rate later in training so the model settles into a good minimum instead of bouncing around.
Too high a peak rate causes loss spikes and forgetting; too low wastes compute and underfits.
How do you tell if a fine-tune is overfitting, and what do you do?
Training loss keeps falling while validation loss flattens or rises; outputs become repetitive, copy training examples, or fail on rephrased prompts.
Stop earlier (pick the checkpoint with the best validation score), reduce epochs, lower the learning rate or LoRA rank, add dropout or weight decay, and increase data diversity.
Evaluate with prompts phrased differently from the training set, not only the training distribution.
How do you detect and mitigate catastrophic forgetting?
Detect: evaluate on a broad general-capability suite (for example MMLU-style questions, instruction following, safety and multilingual checks) before and after tuning; a significant drop signals forgetting.
Mitigate: use LoRA or other parameter-efficient methods that freeze base weights, a low learning rate and fewer epochs, replay (mix general instruction data into the fine-tuning set), regularization toward the base model, and smaller rank.
Less forgetting is not none: LoRA at high rank or many epochs on a narrow dataset can still erode general abilities, so keep the regression suite in every release gate.
Your training loss is not decreasing, spikes, or becomes NaN. How do you debug it?
Check data and labels first: correct chat template, loss masking, no empty or garbled targets, sensible token counts, and shuffled data.
Check the learning rate (the usual culprit): lower it, add warmup, and use gradient clipping; for FP16 check loss scaling, or switch to BF16.
Overfit a tiny subset of examples; if the model cannot memorize 20 examples, there is a bug in the code or setup.
Inspect gradient norms, confirm the right parameters are trainable (adapters actually receiving gradients), and compare against a known-good configuration.
Training loss looks fine but the model's outputs are bad, or it never stops generating. What do you check?
Template and tokens: training and inference must use the same chat template, and the end-of-turn or end-of-sequence token must appear in the training targets and be unmasked, or the model learns not to stop.
Decoding settings at inference (temperature, max tokens, stop sequences) and a mismatch between training-time and serving-time prompts.
Data issues such as truncation cutting off answers, duplicated or contradictory examples, or labels that do not reflect the desired behavior.
Evaluate generations directly on real examples, since loss alone does not measure usefulness.
How do you evaluate a fine-tuned model?
Task metrics on a held-out test set that was never used for training or tuning: exact match, F1, pass rate, schema validity, or a rubric scored by an LLM judge and human reviewers.
Compare against baselines: the base model with a good prompt, with and without retrieval, and the previous production version.
Run a regression suite for general capabilities, instruction following, safety and refusals, and your critical edge cases.
Pair offline evaluation with online checks (shadow traffic, canary, user feedback) before full rollout.
What should a promotion gate for a new fine-tuned model include?
Minimum scores on the task test set and no significant drop on the regression and safety suites, with confidence intervals so noise is not mistaken for progress.
Latency, cost and format checks (schema validity, stop behavior), plus a review of sampled outputs by humans.
Full traceability: dataset version, code commit, hyperparameters and base model recorded in a registry, with a tested rollback path to the previous model.
Merge the adapter or keep it separate? How do you deploy a fine-tuned model?
Merging LoRA into the base weights gives the lowest inference latency and a single deployable artifact; keeping adapters separate allows many tasks or tenants to share one base model and swap quickly.
For serving, you can quantize the merged model (for example AWQ or GPTQ for GPU servers, GGUF for local runtimes) after confirming quality on your evaluation set.
Roll out behind a feature flag or canary with monitoring of quality, latency and cost, and keep the previous version warm for rollback.
How do you estimate the cost and time of a fine-tuning run?
Rule of thumb for full fine-tuning: compute is about 6 times parameters times tokens processed (FLOPs). For a 7B model on 10 million tokens for 3 epochs: 6 x 7e9 x 3e7, about 1.3e18 FLOPs, which is roughly 3.5 GPU-hours at an effective 100 TFLOPs per GPU.
Adjust for LoRA (fewer weight-gradient computations, but the forward and backward passes through the base model still dominate), for QLoRA dequantization overhead, gradient checkpointing recompute (about 30% more), and realistic GPU utilization (often 30% to 50%).
Add costs for evaluation runs, hyperparameter sweeps, data preparation and storage; cut spend with QLoRA, spot instances with frequent checkpoints, and smaller models where quality allows.
When would you use a managed fine-tuning service instead of training yourself?
Managed services (provider fine-tuning APIs) are quick to start, need little infrastructure, and are enough for format, tone or classification improvements on hosted models.
Self-hosting gives control over the base model, data residency, advanced methods (custom losses, preference tuning), cost at scale, and the freedom to move providers.
Consider data privacy terms, model portability (you often cannot export a provider-tuned model), lock-in, and the volume of inference you expect.
When would you distill a model instead of fine-tuning it directly?
Fine-tuning adapts a model's behavior; distillation transfers a large teacher model's behavior into a smaller student, usually to cut latency and cost.
A common recipe: use a strong model to generate (and filter) high-quality outputs for your task, then fine-tune a small model on them, evaluating the student on human-verified data.
Choose distillation when inference cost or latency is the bottleneck and the teacher is already good enough at the task; check the teacher's license terms for training use.
How do you version and govern fine-tuned models?
Version everything that determines the model: base model checkpoint, dataset snapshot and its provenance, code commit, hyperparameters and random seeds, and evaluation results.
Use a model registry with stages (candidate, staging, production), access controls and approval steps for promotion.
Keep audit logs so you can answer which model produced a given output and which data it was trained on, and have a policy for retraining, deprecation and handling data deletion requests.
What are the privacy risks of fine-tuning on private data, and how do you reduce them?
Models can memorize and later reproduce rare strings from training data (personal data, secrets, proprietary text), especially with many epochs or small datasets.
Reduce by removing or masking sensitive data before training, deduplicating, limiting epochs, testing for memorization with extraction probes, and restricting who can query the model.
Where required, use differential privacy training (with some quality cost), isolate tenants' data and adapters, and keep a clear retention and deletion process.