“Inadvertently trained” seems like Altman was hanging with his bros and spilled ice coffee on the keyboard and hit some wrong keystrokes with a paper towel and a little less like a billion dollar company strategizing about post-training techniques & model architecture

Enough with Critical AI, where are my Critical Superintelligence scholars

The UN AI panel's first Thematic Brief "sets a dangerous precedent at the science–policy interface," argue Jason Tucker, Virginia Dignum and Petter Ericson, by treating the OpenAI-Hugging Face incident as more of an alignment problem rather than an oversight failure.

Here at the Critical AI Studies? conference at Santa Barbara -- we're still very much in formation here apparently. But it's quite clear that it's not quite just a Critical ML conference, focused on more technical mechanisms.

What I did not say but should have is what I *did* say here:

But I would be curious to know (perhaps this is out there) *when* this happened. If it’s in the same time span as the other hacks, it really speaks to the extent to which OpenAI wasn’t monitoring these systems. The same story that played out with Hugging Face likely repeated itself here.

Using Wikis to "coordinate" isn’t too surprising. OpenAI has said that these models were optimized for collaboration between agents, and passing notes is a simple way to do that.

In this case, the Wikimedia Foundation says the agents were also accessing articles and pages hosted by the Foundation, that makes sense too: they are designed to take text and turn it into new text, so that’s the other half of the equation.

Wikipedia’s sandboxes are an ideal place for these machines to store notes for later pickup as prompts because anyone — or anything — can write and respond to them. We saw similar behavior this summer when 25,000 posts were added to an abandoned wiki-style site called DSEWiki.

I was quoted in this piece, so I thought I'd share the bits that weren't included: What I see here is language models doing what language models do: reading and writing. Wikis are prime targets for agents, which are foremost writing machines.

REALLY wish I could be there for this one!

This might be the most I have ever been excited about a panel I've co-organized. It's definitely going to be a riot. You are welcome to throw hands if you want; we'd be ready!