Every conversation about LLMs in enterprise starts with chatbots. And yeah, they're useful. But after a few years of building these things, I've found the real wins come from the boring stuff nobody talks about at conferences. The stuff that doesn't make a flashy demo but saves hours every day.
Let me start with a failure. We were building a customer-facing assistant for a logistics company. Smart thing, could answer questions about shipping status, reroute packages, all that. Spent months tuning it. Launch day comes and... nobody used it. Turns out the users already had a perfectly good tracking interface. They didn't need a chatbot to tell them what they could see in three clicks.
That's when I stopped thinking about conversations and started thinking about automation.
The first practical win came from document extraction. Every enterprise has a pile of unstructured documents — invoices, contracts, compliance forms. Traditional OCR can get you the text, but it can't make sense of it. We built a pipeline that feeds PDFs into an LLM with a structured output schema. It spits back a JSON object with all the fields we need. No training data, no custom models. We went from 40 hours of manual data entry per week to about 2 hours of validation. The key was keeping a human in the loop for edge cases. The LLM was right about 85% of the time on the first pass. That last 15% required someone to sanity-check it, but that still cut the workload by more than half.
Another one that surprised me was test case generation. I worked with a team that had a massive legacy system with no test coverage. They had thousands of API endpoints documented in a sprawling wiki. We fed that wiki into an LLM and asked it to generate edge case tests for each endpoint. It found things nobody had thought about — race conditions, boundary values, weird authentication flows. The output wasn't perfect, but it gave the QA team a starting point that saved them weeks of brainstorming. They took the generated tests, tweaked them, and had a coverage baseline in days instead of months.
Then there's policy enforcement. This one is harder to sell because it involves trust. We had a client who needed to review every outbound email for compliance with data protection rules. They'd been using regex and keyword lists, which missed everything that wasn't spelled exactly right. We hooked an LLM into the email pipeline to flag messages that might contain sensitive information. It wasn't a hard block — just a flag for review. The false positive rate was annoying at first, but we tuned it with examples of what they actually considered a violation. After a few weeks, it was catching things the old system would have missed entirely, like a manager referencing a project codename in a message to an external partner.
What I've learned from all of this is that LLMs are terrible at being autonomous agents but great at being assistants to existing workflows. The moment you try to let them run unsupervised, they'll do something weird. But when you put them in a narrow lane with clear boundaries and a human check at the end, they're remarkably effective.
The pattern is always the same: find a task that's repetitive, involves unstructured data, and has a clear success criteria. That's where LLMs shine. Not in the flashy demos, but in the grunt work nobody wants to do. And honestly, that's probably where they belong.