Can AI Stop Phishing? The Truth Behind the Cybersecurity Arms Race
Artificial intelligence is everywhere in cybersecurity right now. Every vendor claims their product uses AI. Every conference talk promises that machine learning will transform threat detection. Every board-level presentation includes a slide about AI-powered defence.
But can AI actually stop phishing which is the most prevalent, most damaging, and most stubbornly persistent cyber threat on the internet?
The honest answer is nuanced, and it matters. Understanding what AI can and cannot do against phishing is not an academic exercise, it directly determines whether your organisation is genuinely protected or operating under a false sense of security.
What AI Is Actually Very Good At ?
Let’s start with where AI genuinely delivers.
Volume-Scale Detection
The sheer scale of phishing means human analysis alone is impossible. Over 3.4 billion phishing emails are sent every day. No team of analysts can review that volume. AI can.
Machine learning models trained on billions of historical phishing samples have become extremely effective at catching commodity phishing; the mass-produced, generic campaigns that make up the majority of phishing volume. Google reports that Gmail’s AI-powered filters block over 99.9% of spam, phishing, and malware before it reaches users’ inboxes. That is an extraordinary achievement at extraordinary scale, and it’s only possible with AI.
Microsoft Defender for Office 365 uses AI to detonate suspicious links in sandboxed environments in real time, following every URL in an email, rendering the resulting webpage, and analysing it for phishing indicators before the email is delivered. This catches thousands of novel phishing pages daily that no threat intelligence feed has seen before.
Lookalike Domain Detection
AI is effective at detecting domain impersonation. Natural language processing models can identify that natwest-secure-login.com is almost certainly impersonating NatWest. Computer vision models can detect that a webpage’s visual design matches a known brand even when the URL is completely different. Graph analysis can map relationships between newly registered domains and known phishing infrastructure, flagging suspicious registrations within minutes.
Behavioural Anomaly Detection
AI-powered identity protection systems monitor user behaviour continuously and build a baseline of normal activity. When that baseline is violated with events like a login from a new country, an unusual time of day, an impossible travel scenario where a user authenticates from London and then from Romania twenty minutes later…AI flags it immediately.
This is particularly valuable for catching the post-compromise phase of phishing attacks. Even when a credential theft succeeds, the subsequent abnormal behaviour of the stolen account can trigger automated responses: blocking the session, requiring step-up authentication, or alerting the security team.
Natural Language Understanding
Modern large language models can read email body text and identify the linguistic patterns that characterise social engineering i.e. urgency language, authority impersonation, fear triggers, artificial scarcity, requests for unusual actions. These patterns persist even when attackers vary the specific wording, and AI can detect them at scale across millions of emails simultaneously.
Where AI Falls Short; The Honest Assessment
Here is where the conversation gets more uncomfortable for vendors, and more important for defenders.
The Quishing Blind Spot
AI-powered email security gateways scan text and URLs. They cannot decode a QR code embedded in an image.
This is not a temporary limitation awaiting a software update. It is a fundamental architectural challenge. When a phishing email contains a QR code image rather than a clickable link, the AI sees pixels and not a URL to analyse. The malicious link is completely invisible to every traditional detection mechanism.
Attackers discovered this blind spot and have exploited it aggressively throughout 2025 and 2026. Some vendors have begun implementing QR code image scanning, but coverage is inconsistent and attackers have already adapted — embedding QR codes inside PDF attachments, using high-contrast variants that are harder for scanners to decode, and routing QR codes through legitimate URL shorteners before the malicious page.
Adversary-in-the-Middle Attacks
AiTM phishing kits like Evilginx3 and Modlishka work by proxying a legitimate website in real time. The victim is accessing a real Microsoft login page — relayed through the attacker’s server. The login succeeds. MFA is completed. The session token is stolen.
From the AI’s perspective, a legitimate authentication just occurred. The user entered their credentials correctly. MFA was approved. The session was issued by Microsoft’s genuine servers. There is nothing anomalous to detect at the email gateway layer — because the attack completes at the browser layer, not the email layer.
Behavioural AI can potentially catch the subsequent token replay — a session token used from a different IP address than where it was issued. But by that point, the attacker is already authenticated.
AI-Generated Phishing Content
This is perhaps the most significant emerging threat to AI-based email defences.
Traditional AI phishing detectors were trained predominantly on phishing emails written by humans — often non-native English speakers, frequently grammatically imperfect, structurally formulaic. The models learned to associate certain linguistic features with phishing risk.
Attackers now use the same large language models powering legitimate AI tools to generate phishing content. A GPT-4 or Claude-generated phishing email is grammatically flawless, stylistically appropriate for the impersonated sender, perfectly personalised with the target’s name and context, and free of the linguistic tells that trained historical models to detect phishing.
PhishScout research has analysed samples of suspected AI-generated phishing emails and found that they score significantly lower on traditional NLP-based phishing detectors than human-written equivalents — precisely because they lack the features those detectors were trained to identify.
The irony is stark: the same AI tools being deployed to defend against phishing are being used to generate phishing that evades AI defence.
Zero-Day Lures and Novel Techniques
AI models are trained on historical data. They are inherently backward-looking. A genuinely novel phishing technique — a new delivery method, a brand being impersonated for the first time, a lure exploiting a news event that broke hours ago — may evade detection until the model is retrained on examples of that technique.
This window of vulnerability — between a novel technique being deployed and AI defences being updated to detect it — is exactly when attackers strike. The most sophisticated threat actors deliberately test their campaigns against known AI detection systems before launching, specifically to exploit the gap.
The Human Layer AI Cannot Reach
No amount of AI can stop a person who has been sufficiently manipulated into choosing to take an action.
A finance director who has been groomed over weeks through a business email compromise campaign, who receives a payment instruction that appears to come from a trusted colleague on a legitimate email chain — the AI may flag nothing because the email is genuinely from that colleague’s compromised account. The human element of the attack operates at a layer AI cannot access.
Vishing (voice phishing) and smishing (SMS phishing) operate largely outside the email security perimeter. AI can’t intercept a phone call. It can’t scan a text message in real time on a personal device. And it certainly can’t prevent someone from scanning a QR code on a physical piece of mail that arrives at their home.
The Arms Race: AI vs AI
The most consequential development in phishing in 2025-2026 is not any individual technique — it is the emergence of AI as a tool on both sides of the attack.
Attackers Are Using AI To:
Scale personalisation: spearphishing previously required significant manual research per target. AI can now generate hundreds of personalised phishing emails per hour, each tailored with the target’s name, employer, role, recent LinkedIn activity, and known colleagues. What was once the preserve of nation-state actors is now accessible to mid-tier criminal groups.
Improve evasion: AI models can be used to test phishing content against known detection systems before deployment, iterating until the content scores below detection thresholds. This is adversarial machine learning applied to attack optimisation.
Generate convincing deepfake content: voice cloning AI can replicate a CEO’s voice from a few minutes of publicly available audio. Video deepfakes can impersonate executives in video calls. These capabilities are not theoretical — they have been used in successful fraud cases in 2025.
Automate reconnaissance: AI tools can scrape LinkedIn, company websites, news articles, and professional databases to build detailed target profiles automatically, feeding personalised attack content generation pipelines.
Defenders Are Responding With:
Adversarial training — training detection models on AI-generated phishing content specifically, to improve detection of LLM-produced emails.
Behavioural signals over content signals — shifting detection emphasis from what an email says to how it behaves — sender reputation history, sending infrastructure, authentication patterns — which AI-generated content cannot easily forge.
Real-time threat intelligence sharing — platforms like PhishScout aggregating community-submitted phishing reports to identify novel campaigns before they reach scale.
Phishing-resistant authentication — moving beyond password and TOTP-based MFA to FIDO2 hardware keys and passkeys, which are cryptographically immune to credential phishing regardless of how convincing the attack is.
What This Means For Your Organisation
The practical implications of the AI-vs-AI dynamic are clear for security teams:
AI email filtering is necessary but not sufficient. Deploy it and maintain it — it catches the vast majority of commodity phishing. But do not mistake 99.9% email filtering for 100% protection. The 0.1% that gets through is increasingly sophisticated.
Layer your defences beyond the email gateway. DMARC enforcement, phishing-resistant MFA, Conditional Access policies, endpoint detection, and user awareness training each address attack vectors that email AI cannot.
Invest in behavioural detection. AI that monitors what users do after authentication — where sessions originate, what files are accessed, what emails are sent — catches compromises that email gateway AI misses entirely.
Update your awareness training for AI-generated attacks. If your phishing simulation programme uses poorly written, obviously suspicious test emails, you are training people to spot last decade’s phishing. Introduce AI-generated simulation content that reflects current attack quality.
Build community intelligence capability. Individual AI systems see only the attacks targeting their own environment. Community intelligence platforms aggregate signals from thousands of organisations, identifying novel campaigns earlier. Subscribe to threat feeds. Report phishing you encounter. The network effect of shared intelligence is something no single AI system can replicate alone.
The Verdict
AI has transformed phishing defence. The volume of phishing that reaches human inboxes today is a fraction of what it was before AI-powered filtering — and that reduction represents an enormous reduction in successful attacks, financial losses, and compromised accounts.
But AI has not solved phishing. It has shifted the battleground.
The commodity end of phishing — generic, poorly crafted, mass-distributed — has been largely defeated by AI. What remains is increasingly sophisticated: AI-generated content, novel delivery vectors, targeted attacks that exploit human psychology rather than technical vulnerabilities, and attack techniques deliberately engineered to evade AI detection.
The organisations that understand this distinction — that AI handles the volume while humans and layered technical controls handle the sophistication — will be far better positioned than those who deploy AI email filtering and consider the problem solved.
Phishing is not an AI problem waiting for an AI solution. It is a human problem with a human attacker at one end and a human target at the other. AI is an extraordinarily powerful tool in the defence stack. But it is a tool — not a resolution.
PhishScout tracks live phishing campaigns and publishes community-verified threat intelligence at phishscout.net. Submit phishing reports to help protect the community.