While I agree to some extent, it’s not an emergency brake: LLMs are opaque, and prone to derail in their generation of text… they should never be able to act without human supervision.

Ma tu guarda… chi avrebbe mai potuto immaginarlo tranne ogni singolo esperto di tecnologia che ne ha parlato all’epoca? Milioni di euro buttati per un singolo referendum di nessun effetto.

They didn’t “evade instructions”. LLMs are not built to “follow instructions”, they generate text and if you let them “cogitate” long enough they will end up doing unpredictable things. Giving models unfettered internet access without any supervision is so irresponsible it beggars belief.

TIL

THIS:

It is not an engineering problem, and more significantly it is not a “problem”. It is how these systems work. It’s like saying that a combustion engine has the “problem” of burning fuel. It is how it works. You “solve” the problem by changing technology to something different.

Your entire job right now is to pet them. I don’t make the rules.

Artificial intelligence systems do not think, feel, want or understand. Avoid language that gives them human characteristics.

This. This, not “superintelligence”, is the existential risk posed by AI. This and also all other lesser risks: reliance without cross checking on a useful but INTRINSICALLY unreliable tool.

“AI” did not do anything. Humans who improperly used a tool or properly used the wrong tool and now want to pass the blame did it.

FWIW I don’t think that AI is an existential threat, nor on track to become one. I do however think that the leaders of the biggest AI companies, AI’s strongest boosters, and some of the folks who are most adept at using it to exploit others are actively hastening along OTHER existential threats.