GroupBrain Labs
← All talks

Post-Training Models

Language Model Continual Learning: Literature Review

Harsh Raj · IIT Mandi · Oct 25, 2026

About this talk

Language models forget older tasks when they are fine-tuned on new ones, a problem called catastrophic forgetting. This talk reviews how researchers deal with it when old training data cannot be replayed and the model is not told which task a question belongs to. Harsh Raj walks through the main families of methods (rehearsal, regularization and architectural), explains LoRA as background, and then covers O-LoRA, which trains a small adapter for each task and keeps the adapters orthogonal so new learning does not overwrite old learning. The talk ends with the reported results and the limits of the approach.

Watch the talk

Want to give a talk like this?

Talks like this are given by GroupBrain students and researchers. They pick a topic, read the papers, run experiments and present what they learn to the group. You can do the same.

About the fellowship Email us

Slides

Title slide of the talk Language Model Continual Learning: Literature ReviewOpen the slides (PDF)