How the scoring works

What a score is

You give the key points: what every reader must take away. Each AI reader reads your letter the way a person with their traits would, then says in their own words what it told them. Each answer is marked against each key point: understood, partly, missed or got it wrong. Partly counts as half. A score is the share of the key points the AI readers took in.

The four outcomes

• Understanding: all the key points.
• Price and value: the key points about what people pay.
• Products and services: the key points about what the product does and what is changing.
• Support: whether readers know what to do next and where to get help.

The verdict

From the understanding score: Most readers understood it at 75% or more, Some readers struggled in between, Many readers struggled below 55%. A key point missing from the letter, or AI readers likely to do something that harms them, can also make it amber or red.

Everyone is held to the same bar, including AI readers in vulnerable circumstances. Scores are not weighted. The report shows the gap between those readers and everyone else.

Who the AI readers are

About 200 read each test. Half are built from your customers, using FCA Financial Lives 2024 figures for the products you offer. Half are going through something harder, such as a bereavement, money worries or poor health, so you can see who struggles. Each has an age, a confidence with money and online, and sometimes a circumstance. Meet them on Your panel.

What it is good for

Finding the sentences and points people are likely to miss before a letter goes out, and seeing who struggles most.

What it is not good for

It is not a judgement on whether a letter meets any rule. The AI readers are modelled, not real people.

How sure you can be

With 100 AI readers, a score is within about 10 points either way. Groups under 20 AI readers are not shown. The AI readers have not yet been compared with real customers. The model is checked against 20 made-up letters whose results are known.

Versions

Every report records the model and scoring versions and the seed, so any result can be run again and checked. Every rule, in detail