Model customization is having a moment. If I use an evolution analogy, I see three phases so far:

1. Pre-foundational (pre-2022)

Transfer learning matured in vision (think the ImageNet era), then jumped to NLP with ULMFiT and ELMo in 2018.

2. Foundation-model Breakout (2018 → 2022+)

GPT-1, BERT, and especially GPT-3 raised the ceiling, then late-2022’s ChatGPT moment pulled customization into the mainstream. Parameter-efficient tuning like LoRA (and later QLoRA) made “downstream” fine-tuning feel practical.

3. The Agentic-AI Wave (2025)

Customization is back with a purpose: fine-tuning for reliable tool use and schema-following, plus distillation for the “inner loop.” Meanwhile, adaptor-style methods keep getting sharper — see the recent LoRA Without Regret work showing when LoRA can match full fine-tuning.

Economics Matter

As models scale, blanket fine-tuning everywhere makes less sense. Targeted customization — adapters, schema adherence, tool-use competence, and distillation where it counts — does.