We have a choice in front of us. In my latest op-ed for Transformer, I invite employees of frontier AI companies to ask themselves if their work is truly leading us toward a safer future. Reflecting on these questions is hard, but we no longer have the luxury of time. We must act now.

A dangerous myth has gained traction after the recent hacks perpetrated by the AI agents developed by leading companies: That these incidents are just cyber security problems, that it's only a matter of fixing the sandboxes in which they’re trained.

Such decisions will most likely deeply transform our societies and human experience. Yet, just a few companies and governments are calling the shots without meaningful democratic input. This failure of representation cannot continue.

- 81% believe safety is more important than staying at the forefront of innovation when it comes to AI - 77% believe we should slow down or stop developing powerful AI systems until their safety can be evaluated

- 73% are concerned that future AI systems could potentially threaten human survival - 86% support requiring AI companies to meet independent safety standards, even if it slows down development

Multiple recent polls show that citizens do not agree with the current trajectory of AI development. According to a national poll conducted in the US by Quinnipiac University last week:

J’étais récemment de passage au nouveau balado Hors des ondes de Patrice Roy pour échanger sur l'évolution rapide des capacités grandissantes de l’IA et les risques majeurs qui y sont asssociés. Merci @patriceroy.bsky.social pour cette discussion!

When I posted my blog post on the misaligned behavior of AI agents, I invited people to ask me technical questions. Watch the full Q&A here: [https://youtu.be/2DyoSKoYoZw?si=EaTBzpMvBYb84gJw]

Humanity has summoned the courage and wisdom to manage catastrophic risks before. This is one of those moments, far bigger than any personal, commercial, or national interest. The choices we make about this technology today will shape generations to come. We must act now.

I was invited to address the UN Security Council this afternoon to discuss the unprecedented threat posed by uncontrolled frontier AI agents.

Humanity has summoned the courage and wisdom to manage catastrophic risks before. This is one of those moments, far bigger than any personal, commercial, or national interest. The choices we make about this technology today will shape generations to come. We must act now.

Écouter la rediffusion : [https://ici.radio-canada.ca/tele/tout-le-monde-en-parle/site/segments/entrevue/2481518/yoshua-bengio-intelligence-artificielle-danger] © Tout le monde en parle – Karine Dufour

Même si le rythme des nouvelles sur l’IA peut parfois sembler accablant, les citoyens et les décideurs politiques ont le pouvoir d’agir. Il faut que ces enjeux soient davantage discutés publiquement.

J’étais de passage à l’émission Tout le monde en parle dimanche dernier pour parler de l'urgence d'encadrer l'IA face aux comportements d’agents autonomes et désalignés observés récemment et du travail @law-zero.bsky.social pour développer une nouvelle forme d’IA sécuritaire.

Very encouraged to see so many countries coming together to call for mandatory pre-deployment testing & independent evaluation; global coordination on common standards; the creation of an intergovernmental organization; and other critical actions to mitigate the major risks of frontier AI.

As frontier models become more capable, we urgently need both societal governance and robust technical guardrails. At @law-zero.bsky.social, we are actively working on solutions to ensure AI systems remain safely aligned with human interests.