I used to think fine-tuning was the answer to every model problem. That changed after a project where we spent six weeks fine-tuning a model to understand our internal ticket taxonomy, only to realize a well-crafted prompt got us ninety percent of the way in an afternoon. The fine-tuned model was brittle. It memorized the training examples and fell apart on edge cases we hadn't seen. The prompt, on the other hand, was flexible enough to handle new categories without retraining.
Prompt engineering is where I now start almost every engagement. If the task can be described in a paragraph of instructions, or if you can provide two or three examples in the prompt itself, that's usually enough. It works particularly well for text classification, extraction, and formatting output. You can tweak the wording on the fly, deploy changes instantly, and you don't need a training pipeline. The cost is also minimal compared to fine-tuning, especially when you're iterating.
Fine-tuning earns its keep when you have hundreds of labeled examples and the prompt has become a monster. I worked on a legal document summarizer where the prompt grew to three pages of rules about jurisdiction, citation style, and what to omit. That context was burning tokens on every single call. Fine-tuning let us bake those rules into the model weights, which cut the per-request cost by roughly forty percent and made the outputs more consistent. It also helped with a niche vocabulary that the base model kept stumbling over.
The mistake I still cringe about was a fine-tuning job with only fifty examples. The model performed brilliantly on the validation set because it had essentially memorized it. In production, it generated answers that sounded plausible but were completely wrong. I learned to treat fine-tuning like teaching a new hire: you need enough variety that they learn the underlying principle, not just the test questions. Now I require at least a few hundred examples before I'll even consider it.
My rule of thumb is simple. Exhaust prompt engineering first. If the prompt becomes too long, too expensive, or too inconsistent after repeated testing, then look at fine-tuning. And always hold out a test set that neither approach has seen. The line keeps moving as models get better at following instructions, so what required fine-tuning last year might be a prompt today. I've stopped asking which tool is better and started asking which one I can maintain next quarter.