A lot of AI in products today is a demo that made it to production by accident. It impressed someone in a meeting, so it shipped, and now it sits there: a chat box nobody uses, a "summarize" button that produces worse text than the original, a feature that exists to say the product has AI.
We build AI into our own products, so we have shipped some of these and killed others. The useful question is never "can we add AI here." It is "is there a real job here that intelligence does better than the alternative." Those are very different questions, and most teams only ask the first.
The test, before we build
Before any AI feature, we make it pass three plain checks:
- Name the task. Not "AI assistant", the actual unit of work: classify this, extract that, draft a first version, triage these. If you cannot name it in one sentence, there is no feature, there is a vibe.
- Name the failure cost. The model will be wrong sometimes. Is that cheap and caught, or expensive and silent? A mis-classified support ticket is recoverable. An auto-sent message or an irreversible write is not.
- Name the alternative. Sometimes a rule, a filter, or a well-placed form beats a model, and is faster, cheaper, and predictable. AI should win on merit, not because it is the exciting option.
If a feature cannot answer those, it is a gimmick, and we do not ship it. That single filter has saved us more time than any model has.
An LLM is equally confident whether it is right or wrong. Everything you build around it exists to handle that.
Where it earns its place
AI pays off when the job is repetitive, judgement-light, and verifiable. In our own products that has meant: reading a messy document and pulling the structured fields out of it, drafting a first version a human then edits, classifying an incoming item so it routes correctly, and answering a known-answer question so a person does not have to. Boring, useful, measurable. The win is real time saved, and you can watch the number move.
The common thread: the model does the first 80 percent of a tedious task, and a human owns the last 20 that needs judgement. That split is where intelligence belongs.
Where it does not
AI is the wrong tool when being wrong is expensive and hard to catch, when the task is genuinely novel each time, or when a deterministic rule would do the job better. We have pulled AI out of features where a plain validation rule was more reliable and a tenth of the cost. That is not a retreat; it is picking the right tool. Using a model where a rule belongs is as much an engineering mistake as the reverse.
And anything irreversible stays a recommendation, not an action. The model can propose the refund, the message, the change; a human confirms it. The moment you let a confident guess commit something you cannot undo, you have built a liability, not a feature.
The part that is actually the work
The prompt is the easy 10 percent. The real engineering is everything around the model: validating its output before you trust it, constraining what it can touch, building the fallback for when it fails, and measuring whether it is actually better than what it replaced. Teams that treat the prompt as the product ship the demo. Teams that treat the model as one unreliable component in a system they control ship the feature.
That is the whole difference between shipping with AI and shipping around it. Around it, you bolt a model on and hope. With it, you decide where intelligence genuinely helps, wrap it in the discipline that makes it safe, and let the product quietly get better.
AI is one of our disciplines, not our pitch. The products that win with it will not be the ones that added the most AI. They will be the ones that added it exactly where it earned its seat, and had the discipline to leave it out everywhere else.
Good to know
When does AI actually belong in a product?
When it removes a real, repeated unit of human effort with a verifiable result, and when being occasionally wrong is cheap or caught. Classification, extraction, drafting, triage. If you cannot name the task and the failure cost, it is a demo, not a feature.
What is the most common AI mistake in production?
Trusting the output. An LLM is confident whether or not it is right. The engineering is not the prompt; it is the validation, the guardrails, and the fallback for when the model is wrong, because it will be.
Should the user always know AI is involved?
The best AI features are usually invisible, the product is just better. But any action that is hard to reverse should surface a human-readable result the user can confirm, not silently commit the model's guess.
We engineer ideas into working software.
Tarka · logic · Shila · stone