I wasted two weeks fine-tuning a model for a legal contract summarizer when a 400-word prompt would have done the job. My team assumed training was the pro move. We scraped five thousand contract summaries, paid for compute, and watched the eval metrics climb. Then we deployed it and the output was barely different from what GPT-4 gave us out of the box with a decent system prompt and a few examples. That mistake cost us a sprint and a chunk of budget.
Prompt engineering is your default. It should always be your default. You iterate in seconds, not hours. You tweak a phrase, add a constraint, paste in a couple of shot examples, and you know immediately if it worked. When the stakes are formatting a response, following a specific output schema, or guiding the model through a reasoning chain, prompts handle it. I've shipped customer-facing features where the entire secret sauce was a six-paragraph system prompt telling the model exactly what tone to use, what fields to extract, and what to ignore.
Fine-tuning enters the picture when prompting hits a wall. That wall is usually context length or style transfer. If you need to cram three hundred pages of proprietary medical jargon into the model's behavior so it responds like your senior triage nurse, you cannot fit that into a prompt. You train it. I worked on an internal support bot for a legacy ERP system. The product had its own language, weird error codes, and niche workflows. We tried stuffing the documentation into the prompt. We hit token limits, costs exploded, and latency made the bot unusable. Fine-tuning on historical ticket resolutions compressed all that institutional knowledge into the weights. The model just knew what code 8774 meant without us spelling it out every single time.
There is also the consistency factor. A prompt can drift. You ask for JSON, and nine times out of ten you get JSON, but that tenth time the model rambles and breaks your parser. Fine-tuning bakes the format into the model's brain. It becomes native behavior, not a requested behavior it can choose to ignore.
The trap is reaching for training because it feels more sophisticated. Vendors push it because compute costs money. But fine-tuning freezes your model at a moment in time. You train on data that ages. When the underlying base model updates, you might need to retrain. Your prompt from last week still works today.
My rule now is simple. I start with a prompt. I spend a few days pushing it hard, throwing edge cases at it, stretching the examples. If it works, I ship it. I only call the training pipeline when I am hitting token limits on every request, when the latency of a massive prompt is killing the user experience, or when the domain is so specialized that the base model fundamentally lacks the vocabulary. Most problems are prompt problems. Recognizing that saves months of work.