There is a simple operating rule that captures most of the value of these tools while avoiding most of the ways they cause damage: use them to produce a draft, and have a person responsible for what actually goes out.

It sounds obvious. It is routinely broken, usually by connecting a tool directly to a channel where its output reaches a customer with no one in between.

Why the draft is where the value is

Most business writing has a blank-page problem, not a quality problem. The quotation follow-up, the response to a complaint, the description of a service, the summary of a long thread -- the hard part is starting, and the second draft is always faster than the first.

A tool that produces a serviceable first version turns a forty minute task into a ten minute one. The ten minutes are spent on judgement, which is the part that needed you anyway.

That is a real gain, available today, with no integration work and very little risk.

Why sending is where the damage is

Three things go wrong when output reaches a customer unreviewed.

Confident wrongness. These systems produce fluent, plausible text regardless of whether the underlying claim is true. An invented policy, a misremembered price, a procedure that does not exist -- all delivered in the same assured tone as everything else. A human reading it catches this. A pipeline does not.

Tone that misses the situation. A customer writing about a serious failure gets a breezy, structured reply with bullet points. Technically responsive, and it reads as contempt.

Commitments you did not make. The most expensive category. Text that promises a delivery date, a refund, a discount or a capability. Whether or not it was authorised, the customer now has it in writing from your business.

The liability does not move

This is the part worth being clear about. If an automated reply from your business gives a customer wrong information and they act on it, that is your problem. Not the vendor's, not the model provider's.

"The system generated it" is not a position that helps you with a customer, a regulator, or a court. You chose to put it in front of people without review. The accountability follows that choice.

Which means the question is not whether the tool is usually right. It is whether you are willing to own the times it is not.

Where the line sits in practice

Draft, always reviewed: quotes and proposals, replies to complaints, anything about price, anything with a date or a commitment, anything going to a regulator or a lawyer, anything published under your name.

Automatable without review: stable factual answers you would put on a sign -- hours, location, whether you deliver somewhere. Acknowledgements that promise only that a person is coming. Status read out of a real system.

The dividing line is whether the answer is the same for everyone who asks. Same for everyone, and verifiable, can be automated. Situational cannot.

Making review real

The rule fails quietly when review becomes rubber-stamping. If somebody is approving forty drafts an hour, they are not reading them.

Keep the volume low enough that reading is possible. Make the reviewer the person who would have to handle the consequences -- people check more carefully when the fallout lands on them. And make sure the reviewer is allowed to rewrite it entirely, not just approve or reject, because sometimes the draft is simply wrong in premise.

The one exception worth naming

Internal drafts that stay internal need less ceremony. Summarising a long thread for your own use, restructuring your own notes, drafting an agenda -- the consequences of an error are that you notice and fix it.

Keep the discipline for anything that leaves the building. That is where the cost of being wrong stops being yours alone.