The Promise of AI Accountability. The Risk of Penalizing the Wrong Actor.

In 2019 I argued that organizations should be required to make their data breach victims whole — for at least seven years, the time it typically takes a victim of identity theft to fully recover. That position aged well, and most of the regulatory architecture that has emerged since then has been moving in that direction.

Now the same question is in front of us for AI. How should organizations be penalized when they deploy AI that produces harm they could have prevented? The answer that regulators arrive at over the next 24 months will define what AI looks like in the enterprise for the next decade.

The Promise

For the first time since the original 2019 piece, the regulatory architecture for serious AI accountability is coalescing.

The EU AI Act creates explicit risk-tier categories with corresponding penalty structures. Social scoring is prohibited entirely. Biometric categorization on sensitive attributes is prohibited entirely. High-risk uses (employment, education, critical infrastructure, law enforcement, migration) carry obligation structures with real teeth. The maximum penalty is 7% of global revenue, higher than GDPR.

Beyond the EU, the U.S. state-level patchwork is filling in faster than expected. Colorado’s 2024 AI Act. NYC’s automated employment decision tools rule. California’s algorithmic discrimination provisions. Plus an increasingly active plaintiff’s bar developing AI-specific class action theories.

And NIST AI RMF gives organizations a framework that, when documented and followed, creates a “we did the reasonable work” defense — the AI analog of “we followed CIS Top 20” for cyber. It doesn’t eliminate liability. It establishes a defensible baseline.

This is the promise: the architecture is finally there to make AI prevention economically rational. Organizations that adopt the framework, document the inventory, and build the oversight cadence will be on the defensible side of any penalty regime that emerges.

The Risk

Two failure modes need to be priced in.

First, under-penalization. If AI failures continue to be priced as a regulatory cost rather than as actual harm to actual people, the same cycle that produced the data-breach economy will produce an AI-incident economy. Hallucinated medical advice. Biased hiring outcomes. Discriminatory pricing in insurance. Defamatory AI-generated content. The 2017 pattern was that breached organizations rationally calculated the breach cost less than the prevention cost. If AI failures get priced the same way, we’ll see the same behavior.

Second, over-penalization aimed at the wrong actor. Regulation tends to penalize the most visible actor in the chain — usually the deployer, not the developer. The EU AI Act distinguishes reasonably between providers, deployers, importers, distributors, and authorized representatives. The U.S. patchwork does not, consistently. The risk is a regulatory regime where a small business using a third-party AI tool ends up bearing penalties intended for the foundation model provider, while the foundation model provider operates behind a model-card disclaimer.

Both failure modes share a structural root cause: the difficulty of attributing an AI harm to a specific decision by a specific actor. This is the technical problem that needs to be solved for any penalty regime to function. Until it is, the regime will be either too weak to prevent harm or too blunt to be fair.

The Verdict

The Promise & Risk needle leans toward Promise — for the first time on this question — but the verdict has a deadline. The next 24 months will define whether AI accountability becomes meaningful or merely expensive.

The 2019 question still holds. How should organizations be penalized for AI failures they could have prevented? Enough that prevention becomes the rational economic choice. The architecture to get there is forming. Whether it lands in time depends on whether the technical attribution problem gets solved, whether the penalty structures land on the right actors, and whether organizations participate in shaping the regime rather than waiting for it.