AI, who controls what?
Back in February 2026 I wrote a short piece after Anthropic published Claude’s Constitution, asking one simple question. Who controls what?
This weekend, Dario Amodei broke his silence on X to answer the investor Gavin Baker, who had accused him of talking too darkly about AI and of pushing regulation that would concentrate power in a handful of labs. Honest, and just as incomplete, his reply is worth reading.
What he concedes
He starts by conceding the diagnosis.
AI is structurally a technology that tends to concentrate power, for reasons that have nothing to do with regulation.
The cause lies in the scaling laws rather than in any regulation. He then admits the one criticism he considers fair, namely that “we haven’t yet delivered on our big promises to benefit the world. That is totally on us.”
He closes on public distrust, which no marketing campaign will repair any time soon. “Saying that AI will cure cancer is more a cliche than it is inspiring. What will work is actually curing cancer.” Quite.
What he leaves out
Except that his entire answer is about safety. Pre-deployment testing, revenue thresholds and a sectoral regulator are instruments for containing a model’s dangerous capabilities. Not one line addresses who decides those models’ values.
No regulator, existing or proposed, will rule on how an agent handles blasphemy, assisted dying or equality between men and women. Those judgements stay written by a sociologically narrow circle, and those blind spots are what genuinely bother me.
The troubling part is elsewhere. In the same thread he argues that a sound institution distributes power rather than concentrating it, adding that “at their best, institutions can vest power in ideas rather than people”. He simply does not apply that conclusion to his own model’s constitution.
Someone will object that Anthropic ran the experiment, with a constitution drafted in 2023 alongside a citizen panel of a thousand people and thirty-eight thousand votes. It was real and commendable.
But who were those thousand people? Americans. And widening a circle would not fix the problem. Above all, the experiment happened, it revealed divergences from the internal text, and the 2026 version was still written in-house. Deliberation was tested, not institutionalised.
The scale we look at least
The same problem plays out at a scale we watch even less, that of the company. When an organisation deploys an internal agent, every system prompt is a normative decision dressed up as a technical choice. Writing that the agent must not criticise a given product is a communications decision, and having it redirect to human resources is a governance decision. In most deployments the data or engineering team makes the call, without a mandate, without procedure, and sometimes without a trace or any deliberation.
We worry a great deal about interference from outside. This one is internal, invisible and near-daily.
So I stand by my three questions.
Do we accept that a normative text shaping the interactions of hundreds of millions of people gets written without any democratic process? Inside a company, who should hold the legitimacy to configure the moral frame of an agent every employee uses? And what legal framework covers conversational logs, that seam of personal data of unprecedented intimacy?