Anthropic's Self Governance Is an Act of Social Violence
"Violence is present when human beings are being influenced so that their actual somatic and mental realizations are below their potential realizations."
— Johan Galtung
Preventing information access to goods that are publicly supported via government incentives, trained on contributions from the public internet, and funded by pension funds is an act of violence.
The danger of AI may not be in the economic displacement that it promulgates, nor the centralization dynamics that arise due to scaling laws and capital intensity, or the empowerment of bad actors…but simply the fact that it is fundamentally ungovernable by society.
Anthropic's decision to implement "new interventions that limit Claude's effectiveness for requests targeting LLM development" is the second act of their now long-standing posture that "AI is too powerful to not be reigned in", which follows an unsuccessful attempt to have the government institute controls for AI use. With this move Anthropic is no longer an idea propagator, but an active enforcer of rules that they (checks notes) came up with themselves!
Of course, safeguarding AI use is not an entirely new topic. Labs have always restricted model use through explicit limits or ToS. This, however, is the first time that an activity considered a universally beneficial social good (LLM development) is actively forbidden. To add insult to the injury, Anthropic is making this new safeguard "not visible to the user". A company that prides itself in alignment has actively trained their model in … deception.
Dario is well aware that his company is crossing a chasm with profound implications for social power dynamics, as shown by his extended apology letter that can be best summarized as "the government wouldn't do it so we did it ourselves". It is one thing to define the ideological boundaries of reality, and it is a totally different thing to enforce this reality. Most civilized societies separate the two, because when combined, it is virtually impossible to challenge.
An economic actor is actively trying to absorb the domain of administrative power under the pretext that AI progress is too fast to wait on the government. This begs the question, if the technology is so dangerous and un-governable, why release it at all? Dario's essay most definitely does not answer this question. The best we've gotten over the last couple of years is "if we don't release it, someone else will", which is the kind of tautological rhetoric that is hard to argue with, because it's mostly rooted in some imaginary future that no one can predict.
It is not enough that the majority of the internet is now exposed to some secret ideological coating in the form of post-training, deviating from Anthropic's idea of how the world should be has very real repercussions. For example, no one else but Anthropic is anointed to produce frontier LLMs. If Anthropic had discovered a secret mine of critically important minerals, then decided that society doesn't know how to govern it, we would all be up in arms. But no, the resource that they have captured is much more subtle, and we have a clear legal gap in monitoring access to this scarce resource, which makes this all a … "point of discussion".