Use a held-out validation set and stop early when validation loss rises, even if training loss keeps falling.
Use fewer epochs (often 1 to 3), a small learning rate, LoRA with moderate rank, weight decay, and dropout.
Increase data diversity, remove near-duplicates, and test on prompts phrased differently from the training set.
Loading…