Vinay C.
GenAI Engineering3 min read

LoRA and QLoRA

Lesson 4.3 · 3 min read · by Vinay C

We'll cover the following

Full fine-tuning updates every weight of a model with billions of them. That needs lots of GPU memory and leaves you a full-size copy per task. LoRA (Low-Rank Adaptation) freezes the original weights and trains two tiny matrices next to each big one. Their product is the "change".

Example: a 4096 x 4096 matrix has about 16.7M numbers. LoRA with rank 8 adds a 4096 x 8 and an 8 x 4096 matrix, about 65K numbers, roughly 0.4% as many. At inference the small update is added to the frozen weights.

  • Far less memory, cheaper and faster training, quality often close to full fine-tuning.
  • The adapter is a small file (MBs). Keep one adapter per task or customer and swap them on one shared base model.
  • QLoRA: keep the frozen base model in 4-bit and train LoRA adapters on top, so a large model fits on a single GPU.
Check yourselfA SaaS has 200 customers who each want their own brand voice. Why is LoRA a good fit?

One shared base model plus 200 small adapters, swapped per request, instead of 200 full model copies. Training and storage are cheap.

Summary: say it in the interview

  • "I would use LoRA: it freezes the base model and trains tiny low-rank adapters, so it is cheap and I can swap adapters per task or customer."