He Approved $25 Million. Every Person on the Video Call Was a Deepfake.
An engineering firm in Hong Kong lost $25 million to a video call where every face except the victim's was a deepfake. The same AI tools that made it possible are also reshaping creative work. Here's how to tell which is which.
The Promise
- Diffusion image and video tools now do real commercial work — concept art, product visualization, storyboards, localized video — at a fraction of the time and cost a studio used to need.
- A plain-English model of how diffusion actually generates an image and a video, so the capability stops being magic and becomes something you can evaluate and govern.
- C2PA Content Credentials and provenance standards give organizations a real, adoptable way to sign and trace authentic media before the fakes arrive.
- The four-D framework applied to image and video AI — a concrete way to decide where these tools belong and where they don't.
The Risk
- The Arup case: a finance employee in Hong Kong approved $25 million on a video call where every other face and voice was a deepfake. Video is no longer proof of presence.
- Training-data lawsuits — Getty v. Stability AI, Silverman v. OpenAI, NYT v. OpenAI — are still redrawing what's commercially safe to use, and the liability lands on whoever ships the output.
- Style theft and displaced creative livelihoods: a model trained on an artist's catalog can reproduce the look without the artist, and the Hollywood AI provisions cover only part of the gap.
- Detection is losing the arms race to generation — every detector becomes training data for the next generator, so 'we'll just catch the fakes' is not a durable control.
The call that broke the last safe channel
A finance employee at the engineering firm Arup joined a video call with his CFO and five colleagues. He approved $25 million in transfers. Every face on that call except his own was a deepfake.
He was not reckless. He was suspicious enough to demand a video call instead of trusting an email. A year ago, that was the right answer. Email was the channel you didn’t trust. A live video call with people you recognized was the channel you did. That assumption is now gone, and most authorization processes still run on it.
How the image actually gets made
Strip away the marketing and a diffusion model does one thing: it starts with pure noise and removes it, step by step, until an image that matches your prompt emerges. It learned to do that by watching hundreds of millions of captioned images and practicing the reverse — adding noise, then predicting how to take it back out. Video adds one more constraint, temporal consistency, so the same face holds shape from frame to frame.
That is the whole trick. Once you see it as denoising rather than magic, the promise and the risk both get easier to reason about.
Where it earns its place
The upside is real and already in production. Concept art, product visualization, storyboards, localized training video — work that used to need a studio and a week now takes an afternoon. The same convergence of cheap image and voice generation that powers the Arup fraud also powers a marketing team shipping twelve campaign variants before lunch.
This is not a question of whether the tools are useful. They are. It is a question of which output you can trust, and who carries the bill when you are wrong.
Where it breaks
Three places. First, the training data. Getty v. Stability AI, Silverman v. OpenAI, and NYT v. OpenAI are still deciding what was legal to learn from, and the liability for an infringing output follows whoever publishes it, not the model vendor. Second, livelihoods: a model trained on one illustrator’s catalog can reproduce the style without the illustrator, and the Hollywood AI provisions only fenced off part of that field. Third, detection. Every detector you deploy becomes training data for the next generator. The fakes get better specifically because the detectors exist.
After 25 years in cybersecurity, I have watched this exact shape before. A new attack class outruns the verification protocol. Phishing did it in the 2000s. Ransomware did it in the 2010s. The organizations that rebuild the protocol early stay out of the case studies. The ones that wait become them.
What actually works
Detection is losing, so the defenses that hold are the ones that don’t depend on spotting the fake. Out-of-band verification on any instruction to move money. A second authorization channel the caller can’t fake on the spot. C2PA Content Credentials to sign the media you produce. And a verification policy that treats a video call as a claim to be checked, not a fact already proven.
The tools are genuinely useful and the fraud is genuinely here. The deciding factor is not the technology. It is whether your authorization process was rewritten for a world where seeing a face is no longer the same as verifying a person.