Honest Claims for AI Products: Accuracy, Limits, Data

Claims are where trust is won or lost. An AI product can be genuinely useful and still damage its reputation by promising too much. A buyer who discovers a gap between the page and the product does not just stop using it. They tell colleagues, leave a review and become sceptical of the next tool too.

This article is about making claims you can stand behind. It covers accuracy, limits, safety, bias and data, with practical wording suggestions. It is not legal advice, and some claims, especially in regulated areas, may have specific rules. If in doubt, take professional advice.

A simple rule: evidence before adjective

Before you write an adjective such as "accurate", "fast", "secure" or "fair", ask whether you have evidence for it. If you do, state the evidence. If you do not, either gather it or leave the adjective out.

This rule is demanding but easy to remember, and it helps in every area below. It also aligns with the principle in Google's guidance on helpful content that trust is built through clear sourcing, evidence and transparency about how something was made.

Claims about accuracy

Accuracy is the claim most often abused. The trouble is that accuracy is not a single number. It depends on the task, the data, the measure and the conditions.

What to say.

• The task: "Extracts invoice totals from PDF invoices."
• The test: "Tested on a set of three hundred invoices from twelve suppliers."
• The result: "The correct total was extracted in most cases. Most errors involved scanned invoices with poor image quality."
• The date: "Tested in March."
• The caveat: "Results will vary with your documents."

What to avoid.

• Round, unsupported figures with no method.
• Comparisons with "humans" with no description of who or how.
• Quoting the best run as typical.
• Mixing results from different tests.

If your system is probabilistic, as most AI systems are, say so. The ICO's guidance on AI and data protection includes statistical accuracy as a consideration for fairness, which signals that regulators expect the topic to be handled thoughtfully.

Claims about speed and cost

Speed and cost claims are easy to make false by omission. "Results in seconds" should say under what load. "Saves 80% of your time" needs a source. If you quote savings from a customer, say who, in what context and how they measured, and obtain their permission.

For cost, say what is included and what is extra. If fees depend on usage, explain the units. Buyers resent surprises more than they resent prices.

Claims about safety and security

Be specific about the measures you use: encryption in transit and at rest, access controls, logging, independent audits and certifications you actually hold. Name certifications only if you have them, and say the scope. A general claim such as "bank-grade security" means nothing and invites scepticism.

If you have not done an audit, do not imply you have. It is acceptable to say what you do and what you plan.

Claims about bias and fairness

"Unbiased AI" is a claim that few can substantiate. Bias can arise in data, in design and in use. A more honest and useful approach is to describe the steps you take.

• What groups or conditions you tested for differences in performance
• What you found
• What you changed
• What limitations remain
• How users can report concerns

The NIST AI Risk Management Framework is voluntary guidance that aims to help organisations manage risks to individuals, organisations and society and to build trustworthiness into AI systems. You do not need to claim compliance with it, but you can use its approach of identifying risks, measuring them and communicating about them as a template for your own statements.

Claims about data

Data handling statements should be easy to find and plain.

• What you collect, and why
• How long you keep it
• Whether and how it is used to train or improve models
• Whether customers can opt out
• Where it is stored and processed
• Who you share it with
• How to request deletion

Do not claim "we never use your data" if some of it is used for diagnostics or improvement. Do not claim "fully private" when staff can read content for support. If the position differs between plans, say so.

Claims about human involvement

If humans review outputs, or if parts of the process are manual, state it. Some products labelled "AI" depend on people behind the scenes. That is not shameful, but hiding it is misleading, and it matters to people who care about who sees their data.

Claims about being "the best" or "the first"

Superlatives are difficult to prove. "The first AI tool to do X" can be falsified by a single counter-example. "The best" depends on criteria. If you want to use such claims, qualify them: "Ranked first in category by community votes during the week of 3 March". Make sure the basis is clear and verifiable, and keep paid and voted signals separate. On this site, votes cannot be bought and paid placements are labelled; mention each accurately.

Warnings and appropriate use

Good products tell customers when not to use them. If your tool should not be the sole basis for medical, legal, financial or safety decisions, say so prominently. If outputs should be checked by a person, say so. A warning that is easy to find and plain to read helps users and protects you.

Having a route for problems

Claims are tested when something goes wrong. Make sure customers can report errors easily, that someone reads reports and replies, and that you track patterns. If you correct a significant error, tell affected users. A track record of responding well is itself a powerful form of reassurance.

Review your claims regularly

Products change. Models are updated. Data policies evolve. Put a recurring reminder in the calendar, perhaps quarterly, to check every factual claim on your site, in your listings and in your sales material. Remove or update anything that has become untrue. Date your evidence.

A claims checklist

For each claim on your page, answer:

1. What exactly am I asserting?
2. What is my evidence?
3. When was it gathered?
4. Under what conditions does it hold?
5. Would a sceptical customer accept it?
6. Is there a limit I should mention alongside?

If you cannot answer the first three, rewrite or delete the claim.

Writing the page

On your listing, a short "What we can and cannot promise" box can be very effective. List three things you are confident about and three limits. Link to a fuller page. On this site, the submit page starts your listing, you can put these points in its description, and the categories page shows where it will appear.

An example of a claims audit

To see how this works in practice, imagine an AI writing assistant for estate agents, and audit three sentences from its website.

"Writes perfect property descriptions in seconds." The word "perfect" has no evidence and invites a screenshot of a mistake. "In seconds" can be tested and is probably true on a normal connection. Rewrite: "Drafts a property description from your listing details in under a minute. Always read it through before publishing, as it can occasionally state features that are not in your details."

"Trusted by thousands of agents." Is it true, and what does trusted mean? If thousands of accounts exist but most are inactive, the claim is stretched. Rewrite with a figure you can verify, such as active accounts in the past thirty days, with the date.

"Your data is completely safe." Nobody can promise that. Rewrite: "Listing details are encrypted in transit and at rest and are not used to train our models. Staff access is limited to support requests that you initiate." Each part can be checked and, if it ever changes, corrected.

After the audit, the page is more modest and more credible. The sales team may complain at first, but support will see fewer disputes and fewer disappointed customers.

When you discover you were wrong

Sooner or later you may find that a claim was inaccurate. Correct it promptly. Update the page, note the date of the change, tell customers who relied on it if the error mattered, and review how it got there. Quiet correction is better than none, and open correction is better still when the error was material. A company that handles mistakes well tends to keep customers who have seen it do so.

Why this pays off commercially

Honest claims can feel like a handicap in a market full of bold promises. In practice, buyers who evaluate carefully, especially businesses, are increasingly weary of inflated language and look for suppliers who describe their products plainly. Precise statements shorten sales conversations because they answer the questions buyers would otherwise ask. They reduce churn because expectations match reality. And they give you a store of reusable, defensible text for proposals, security questionnaires and listings. The short-term cost is a less exciting headline. The long-term benefit is a reputation that does not need repairing.

Frequently asked questions

Is it fine to use customer testimonials about accuracy? Yes, with permission, and as opinion rather than proof. Do not present a testimonial as a measurement.

What if a competitor makes bigger claims? Resist the temptation to match them. Buyers who compare carefully will notice who offers evidence.

Should I publish my test results? If you can, yes. Even a short summary with a method note builds trust.