How artificial intelligence is turning familiar attack vectors into something businesses are not equipped to defend against — and what that means for how you share information externally
The phishing email has been a known threat for thirty years. The fraudulent payment instruction has existed as long as banking. The impersonation of a senior executive to authorise an urgent transfer is a well-documented attack pattern that every finance team has been warned about.
None of these are new. What is new is that artificial intelligence has removed almost every constraint that previously limited how effectively they could be executed.
Volume, personalisation, linguistic quality, visual authenticity, voice replication, video synthesis — every dimension along which a human attacker was previously limited by time, skill, and resources has been transformed by AI into something that scales effortlessly and costs almost nothing. The attacks themselves have not fundamentally changed. The economics of executing them at scale and at quality have changed completely.
For businesses that share sensitive information, communicate financial instructions, and manage external relationships through email and link-based file sharing, this is not a future risk. It is a present one. And the defences that were already insufficient against the pre-AI version of these threats are not going to hold against what is coming.
What AI Has Changed About Phishing
The tell-tale signs that trained users to spot phishing emails — poor grammar, generic salutations, implausible scenarios, mismatched domains that looked wrong on close inspection — were always imperfect signals. They worked because most phishing was mass-produced at low quality. Volume was the attacker’s strategy, not precision.
AI inverts this. A large language model can produce a phishing email that is grammatically flawless, tonally appropriate to the relationship, contextually accurate about the recipient’s role and organisation, and calibrated to the specific platform being spoofed — in seconds, at no marginal cost, at whatever volume is required. The low-quality mass phishing campaign does not disappear. It is joined by high-quality targeted attacks that are indistinguishable from legitimate communication by any standard that does not involve verifying the underlying infrastructure.
This matters enormously for link-based file sharing. The standard notification email — a document is ready for your review, click here to access it — is the most commonly spoofed format in existence precisely because it is so widely used and so familiar. An AI-generated version of that notification, tailored to the specific platform your organisation uses, referencing a plausible document name, sent from a domain that differs from the legitimate one by a single character, is not something that user training is going to reliably catch.
The human in the loop — the last line of defence in almost every security awareness programme — is being systematically outpaced. Not because people are becoming less careful, but because the quality of what they are being asked to evaluate is becoming indistinguishable from the legitimate.
The Deepfake Problem Is Already Here
Deepfake technology — the synthesis of realistic audio and video of real individuals — has moved faster than almost any other AI capability and is already being used in financial fraud at scale.
In 2024, a finance worker at a multinational firm was deceived into transferring approximately $25 million after attending what appeared to be a video call with the company’s CFO and several colleagues. Every person on the call was a deepfake. The employee had no reason to doubt what they were seeing. The verification mechanism they were implicitly relying on — visual and audio confirmation that they were speaking to who they thought they were speaking to — had been completely defeated.
This was not an isolated incident. It was a high-profile example of an attack category that is growing rapidly as the cost of generating convincing deepfake audio and video continues to fall. What required significant technical capability and resources two years ago can now be accomplished with consumer tools and a few minutes of source material scraped from public video or audio.
The implications extend well beyond video calls. AI-generated voice cloning is being used to impersonate executives in phone calls authorising transfers or changes to banking details. It is being used to answer verification calls — the “call to confirm” workaround that organisations were advised to use as a defence against email-based fraud. The verification mechanism that was supposed to catch the fraud is now itself a target.
Why These Threats Converge on the Same Vulnerability
AI-enhanced phishing and deepfake impersonation are different techniques, but they exploit the same underlying vulnerability: the reliance on open, unverified communication channels for interactions where identity and authenticity matter.
Email is an open channel. It cannot verify the identity of the sender at the level of certainty that AI-generated content now requires. A shared link is an open access point. It cannot distinguish between the intended recipient and an attacker who obtained it. A phone call or video conference cannot, by itself, verify that the person on the other end is who they appear to be — a fact that was always technically true but is now practically exploitable at scale.
The common thread is that these channels were designed for a world where impersonation was hard and expensive, where producing a convincing fake required skill and resources that most attackers did not have, and where the effort required to deceive a specific individual was proportionate to the return. AI has eliminated all three of those constraints simultaneously.
The attack surface has not changed. The cost of attacking it has collapsed.
The Training Problem
The security industry’s primary response to social engineering attacks has been user education. Teach people to spot phishing. Run simulated attacks. Build a culture of scepticism. These programmes have genuine value and are not without effect.
But they are built on an assumption that is becoming increasingly untenable: that human judgement, properly trained, can reliably distinguish between legitimate and fraudulent communications. That assumption was already strained before AI. It is now being broken systematically.
A user awareness programme that trains people to look for grammatical errors cannot help them when the fraudulent email is grammatically perfect. A programme that trains people to verify unusual requests by phone cannot help them when the voice on the phone is a convincing clone of their CFO. A programme that trains people to be suspicious of unexpected file share notifications cannot help them when the notification is pixel-perfect and contextually plausible.
The problem is not the training. The problem is that training is a defence built around human perception, and human perception is no longer a reliable instrument for detecting AI-generated deception. The gap between what a well-trained person can catch and what a well-resourced AI can produce is widening, not narrowing.
This does not mean user education is worthless. It means it cannot be the primary line of defence for the interactions that matter most.
What a Structural Defence Looks Like
If the vulnerability is open, unverified communication channels, the structural defence is channels that are closed and verified by design — where identity is established cryptographically rather than assumed perceptually, and where the attack vectors that AI has made so effective simply do not exist.
Eliminating the link eliminates the phishing vector
An AI-generated phishing email that tells a recipient to click a link to access a document is only effective if the recipient’s normal behaviour involves clicking links in emails to access documents. Remove the link from the legitimate workflow and the attack loses its camouflage. If your external sharing process does not involve sending links by email — if documents appear in an authenticated dashboard that recipients access directly, without any email-based trigger — then an email purporting to send a document link is immediately anomalous. Not because the recipient spotted a grammatical error, but because the format of the attack no longer matches the format of the legitimate process.
This is a structural defence, not a perceptual one. It does not rely on the recipient being more observant than the AI generating the phishing email. It relies on the attack having nowhere to hide because the legitimate process it is trying to mimic does not exist.
Verified identity replaces assumed identity
The deepfake problem is fundamentally an identity problem. The finance worker who transferred $25 million believed they were talking to their CFO because everything they could perceive told them they were. What they lacked was a verification mechanism that did not rely on perception.
Cryptographic identity verification — where access to a channel, approval of a transaction, or confirmation of an instruction requires authentication through a verified credential rather than through visual or audio recognition — cannot be defeated by a convincing deepfake. A deepfake can replicate what someone looks like and sounds like. It cannot replicate their cryptographic identity or their device-bound passkey. The verification happens at a layer the AI cannot reach.
Applied to financial workflows, this means that payment instructions, banking detail changes, and high-value approvals should require verification through an authenticated channel rather than confirmation through a call or video meeting. Not because calls and video meetings are useless, but because they are no longer sufficient on their own for interactions where the cost of being deceived is measured in six or seven figures.
The audit trail as a detection mechanism
AI-enhanced attacks are becoming harder to prevent at the point of attempt. They are not, yet, becoming harder to detect after the fact — provided the right records exist. An immutable audit trail that captures every access event, every document interaction, every communication in an authenticated channel creates a forensic baseline against which anomalous activity becomes visible.
When an instruction arrives through an unverified channel, or when an access pattern deviates from the established baseline, the audit trail provides both the detection signal and the evidence needed for response. This is not a complete defence against AI-driven attacks. It is a meaningful backstop for the cases where the attack succeeds at the attempt stage but can be contained, investigated, and remediated before the full damage is done.
The Compounding Effect
What makes the AI threat landscape particularly challenging for organisations that have not yet changed their external communication architecture is that the individual capabilities — phishing generation, voice cloning, deepfake video, automated reconnaissance — do not operate in isolation. They compound.
An attacker can use AI to research a target organisation’s key relationships and communication patterns from publicly available information. They can use that intelligence to generate a targeted phishing email that references a real ongoing matter. They can use voice cloning to follow up with a phone call that sounds like the expected contact. They can use deepfake video to confirm the instruction in a meeting. Each step makes the next more convincing. The full chain of an attack can be orchestrated with a level of coordination and personalisation that previously required a team of skilled human attackers working over an extended period.
Against this kind of compounded, AI-orchestrated attack, the defence cannot be primarily perceptual. No amount of training prepares a person to maintain scepticism across multiple, mutually reinforcing deceptions from different channels. The defence has to be structural — removing the open channels through which the attack operates and replacing them with verified ones that the attack cannot successfully mimic.
The Architecture AI Can’t Reach
Every AI-driven attack described in this article shares a common dependency: it needs something to target. A link to spoof. A notification email to replicate. A login page to fake. A public-facing access point to probe. A communication channel open enough to insert itself into. Remove those dependencies and the attack has nowhere to go — not because the defence is stronger, but because the attack surface it requires does not exist.
This is the principle behind what might be called an invisible architecture. A collaboration environment with no public-facing URLs, no link-based entry points, no email-triggered access notifications, and no anonymous access layer presents AI-driven reconnaissance with nothing to work with. There is no notification email format to replicate because legitimate notifications do not arrive by email. There is no link to spoof because legitimate access does not involve links. There is no login page to fake because legitimate authentication happens through device-bound credentials that a fake page cannot capture. The attack surface that AI-enhanced phishing and impersonation attacks depend on simply is not there.
This is meaningfully different from having a strong defence. A strong defence assumes the attack reaches the perimeter and tries to stop it there. An invisible architecture means the attack cannot find the perimeter in the first place. You cannot phish a door that isn’t there. You cannot spoof a notification that is never sent.
Peer-to-peer channels and the closed loop
When two businesses connect directly through a verified, peer-to-peer channel — where documents and conversations flow between two authenticated dashboards without any public-facing intermediary — the entire class of link-based and notification-based AI attacks becomes irrelevant. The channel is closed. Access is by verified identity only. There is no public URL for an attacker to discover, no link format to replicate, and no email-based trigger that can be mimicked. An AI that has scraped every piece of publicly available information about both organisations cannot identify the channel’s existence, let alone insert itself into it.
For external parties who are not yet connected through a peer-to-peer channel, authenticated access through existing verified credentials — with the authentication happening at the device level rather than through a password entered on a web page — removes the credential theft vector that AI-generated phishing is specifically designed to exploit.
Passkeys: the layer AI cannot fake
The specific weakness that AI-generated phishing exploits at the authentication stage is the password. A convincing fake login page captures a password because the password is something a person knows and types — and therefore something that can be entered into any page that asks for it, legitimate or otherwise.
Passkeys eliminate this vulnerability entirely. A passkey is a cryptographic credential stored on the user’s own device, tied to their biometric or device PIN, and bound to the specific domain it was created for. It cannot be entered into a fake login page because it is never typed. It cannot be phished because it is never transmitted. It cannot be stolen from a database because it is never stored on a server. And it cannot be replicated by an AI that has synthesised a perfect copy of the legitimate login page — because the authentication happens between the user’s device and the legitimate domain at a cryptographic level that the fake page cannot participate in.
Even in the scenario where an AI-generated phishing email is convincing enough to make a user navigate toward a fake login page, passkey authentication stops the attack cold at the point of sign-in. There is nothing for the fake page to capture. The credential that would grant access does not exist in a form that can be intercepted.
Layered defence: each layer defeats a different stage
The value of a properly constructed security architecture against AI-driven attacks is not that any single layer is impenetrable. It is that each layer defeats a different stage of the attack, so that defeating one layer gains the attacker nothing because the next is already in place.
No public links or notification emails — the AI-generated phishing campaign has nothing to mimic. Closed peer-to-peer channels — the automated reconnaissance finds no access points to probe. Verified identity access — possession of a spoofed link or a stolen credential provides no entry. Passkey authentication — a convincing fake login page captures nothing useful. HSM-backed, application-level encryption — a breach of the data layer produces only ciphertext with no keys present. Immutable audit trails — any anomalous access or instruction is immediately visible against the established baseline.
An AI-orchestrated attack that successfully navigates the first layer encounters the second. One that navigates the second encounters the third. The compounding of AI capabilities that makes these attacks so formidable is met by a compounding of defensive layers, each independently effective, each reinforcing the others. This is not defence in depth as a marketing phrase. It is defence in depth as a structural reality — where the attacker’s task is not to find one weakness but to simultaneously defeat multiple independent systems, each of which would stop the attack on its own.
Who Needs to Act and When
The organisations most exposed to AI-enhanced external communication threats are those that regularly share sensitive documents, communicate financial instructions, or manage high-value external relationships through email and link-based file sharing. In practice, this means law firms, accountancy practices, financial services organisations, real estate teams, investment managers, and any business operating in a sector where individual transactions are large enough to justify a targeted, sophisticated attack.
The timeline for action is not “when AI attacks become more common.” They already are. The $25 million deepfake fraud was 2024. The volume of AI-generated phishing campaigns is already orders of magnitude higher than pre-AI baselines. The organisations that are acting now are not being paranoid. They are responding to a threat environment that has already changed, with defences appropriate to what it has changed into.
Those that are waiting for a more compelling reason to act should consider that the most compelling reason — a successful attack — is the one they are trying to avoid.
Moving Forward
AI has not invented fraud. It has not invented phishing, impersonation, or social engineering. What it has done is remove the skill, effort, and cost barriers that previously limited how effectively these attacks could be executed — and done so faster than most organisations have been able to respond.
The appropriate response is not more training, better spam filters, or additional verification steps bolted onto communication processes that remain fundamentally open and unverified. It is a change in the underlying architecture — closed, authenticated channels for external communication; identity-based access rather than link-based access; cryptographic verification for high-stakes interactions; and audit trails that create accountability regardless of how convincing the attack was at the point of attempt.
The threat has changed structurally. The defence needs to match it.
AI can generate a perfect phishing email. It can clone a voice and synthesise a face. What it cannot do is forge a cryptographic identity or insert itself into a closed, verified channel. That is where the defence has to live.
