WDYRI scam SMS checker

obile-money scams arrive as text messages. A relative needs help, an agent sent money by mistake, a landlord has a new number. In Tanzania a common hook is "ni tumie kwa namba hii": send it to me on this number.

Published Swahili scam detectors report 98.7% to 99.86% accuracy. That sounds solved. We rebuilt them and asked a simpler question: what are they actually reading?

Mostly, the numbers. In the BongoScam data 86% of scams, and no genuine texts, contain a phone number or link. Add a phone number to a harmless message and a fine-tuned transformer called it a scam 96% of the time.

A detector that only reads numbers is easy to fool and quick to cry wolf.

So WDYRI masks every number, trains its models on messages where numbers appear on both sides, and pairs a transformer with a character model that fail on different messages. The phone-number false alarms fell from 96% to 0%.

It is still a research demo. When in doubt, call your provider on its official number.

FAQ

How it works

What happens to your message, how the check works and how far to trust it.

How does it decide if a message is a scam?

Two models read your message and each gives the chance that it is a scam. One is a transformer fine-tuned on Swahili SMS (AfroXLMR), which reads the words in context. The other is a character n-gram model, which looks at short runs of letters.

They tend to be wrong on different messages, so the message is flagged if either one flags it. That is why the result shows both of them, and why they sometimes disagree.

Is my message sent anywhere?

No. Everything runs in your browser: your message is never sent anywhere or stored.

The transformer (about 360 MB) downloads once and your browser caches it. The n-gram model answers immediately while it loads.

What does "Undo disguise tricks" do?

Scammers disguise words so filters miss them. This removes invisible characters, turns lookalike Cyrillic and Greek letters back into Latin ones, undoes digit swaps such as 4 for a, and rejoins s p a c e d words before the models read the text.

Why are numbers replaced with <PHONE> and <AMOUNT>?

Before reading, the models replace phone numbers, amounts and links with <PHONE>, <AMOUNT> and <URL>, so they judge the words rather than whether a number is there.

Most scams in the training data carried a number, and models trained on them learned that shortcut. Both models here were trained with numbers in both kinds of message, so a number alone no longer decides.

Does it work in other languages?

It is built for Swahili SMS from Tanzania.

It saw no other language in training, so it does worse elsewhere: on real scam SMS in Chichewa, from Malawi, it is much less accurate.

Can I trust the verdict?

It is a research demo, not a guarantee. On Swahili test messages it never saw in training it scores 0.994 F1 (1 would mean every scam caught with no false alarms), and stays at 0.95–0.99 when letters are disguised or words split. But heavily disguised text is treated as suspicious even when it is genuine, and scam scripts unlike anything in training can slip through.

When in doubt, call your provider on its official number. The code, data and full results are on GitHub.

Why did you
redeem it?

Transformer

N-gram model

What drove it

How it works

"Why did you redeem it?!"