Insights / Threat defense
AI-written phishing and deepfakes: why verification beats detection
Spelling mistakes and awkward phrasing no longer identify a phishing email. Organizations need verification procedures that hold up even when a message looks genuine.
Lookalike email quarantined before delivery
Security awareness training has long taught employees to look for errors: poor grammar, generic greetings, a logo that looks slightly wrong. Language models remove those signals. They produce fluent, specific messages in any language, and they can draw on a company’s website, press releases and staff profiles to make each message credible.
Voice and video impersonation has advanced in the same way. A few minutes of recorded speech is enough to clone a voice, and most executives have far more than that online. In January 2024, a finance employee at the engineering firm Arup joined a video call with what appeared to be the company’s CFO and several colleagues. Every other participant was a deepfake. The employee made 15 transfers totaling about US$25 million before the fraud was discovered.
FromDana Ruiz, CFO <dana.ruiz@yourcornpany.com>rn, not m
SubjectConfidential: acquisition wire, today please
I need this transfer processed before 3pm. I’m in back-to-back meetings, so please don’t call. Keep it between us until the announcement.
Verify requests instead of judging messages
Detection depends on people recognizing a fake, and fakes are improving faster than people can learn to spot them. A more reliable approach is to make high-risk decisions independent of how genuine a message appears.
- Confirm on a second channel. Requests to move money, change bank details or reset access are confirmed by calling a number already on file, never one provided in the message.
- Issue code phrases to staff who can authorize payments. Agree on them in person and change them regularly.
- Require two approvers for irreversible actions. One employee under time pressure can be deceived. Two independent approvals are much harder to defeat.
- Support staff who take time to verify. Leadership should state clearly that verifying a request will never count against an employee.
The durable fix is to make the decision not depend on whether the message looks real.
Practice the procedures
Procedures that have never been practiced tend to fail under pressure. Run consented exercises, such as a cloned-voice call to the finance team or a spoofed vendor requesting new bank details, and review the results with staff afterward. The objective is to make verification routine.
Our AI Threat Defense service includes the exposure review, the verification procedures and the simulation exercises.