Post-Training Models
Harsh Raj · IIT Mandi · Oct 25, 2026
Language models forget older tasks when they are fine-tuned on new ones, a problem called catastrophic forgetting. This talk reviews how researchers deal with it when old training data cannot be replayed and the model is not told which task a question belongs to. Harsh Raj walks through the main families of methods (rehearsal, regularization and architectural), explains LoRA as background, and then covers O-LoRA, which trains a small adapter for each task and keeps the adapters orthogonal so new learning does not overwrite old learning. The talk ends with the reported results and the limits of the approach.
Talks like this are given by GroupBrain students and researchers. They pick a topic, read the papers, run experiments and present what they learn to the group. You can do the same.