Technology has been moving faster than policy for some time now, and the advent of AI isn't changing that, so what can we do to maintain safety despite uncertainty? Su Cizem has spent the last few years trying to answer that question. As an analyst at the Future Society, she works on global AI governance, specifically on building international consensus around AI red lines: the thresholds we collectively agree must never be crossed. In this conversation, Su walks through her path from philosophy to policy, the evolution of the global AI safety summit series, why voluntary commitments from AI labs aren't enough, and what it would actually take to make international cooperation on AI safety real.

Chapters

• (00:00) - Introduction
• (03:23) - From Philosophy to Policy
• (22:25) - What AI Governance Actually Means
• (26:49) - The Summit Series
• (43:01) - Drawing The Red Lines
• (01:10:51) - Can These Companies Govern Themselves?
• (01:24:01) - Breaking Into The Field
• (01:27:51) - Closing Thoughts & Outro

• Su's LinkedIn
• Global Call for AI Red Lines
• The Futures Society report - “Facing the Stakes of AI Together”: 2025 Athens Roundtable Report
• Politico article - How the global effort to keep AI safe went off the rails
• TechPolicy.Press article - A Timeline of the Anthropic-Pentagon Dispute
• The Guardian article - AI got the blame for the Iran school bombing. The truth is far more worrying
• Google and OpenAI Employee open letter - We Will Not Be Divided
• The Register article - Altman said no to military AI abuses – then signed Pentagon deal anyway
• SaferAI report - Evaluating AI Providers’ Frontier AI Safety Frameworks

Jacy Reese-Anthis, founder of Sentience Institute and researcher at Stanford, began his journey working for animal welfare, but is now finishing up his PhD with research in many different AI subfields at the intersection of neuroscience, philosophy, social science, and machine learning. While this may seem like an odd jump at first, Jacy shares how his work has all been centered around the idea of moral circle expansion. In this episode, we dig into what sentience actually means (or at least how we can begin to think about it), why anthropomorphization is more complicated than it sounds, and how language models may be able to be leveraged as an effective tool for social science research.

Jacy also shares his median AGI estimate somewhere in there, so stay tuned if you want to catch it.

Chapters

• (00:00) - Introduction
• (05:41) - From Animal Welfare to Digital Minds
• (09:00) - Founding Sentience Institute
• (22:00) - Defining Sentience
• (27:13) - The Anthropomorphization Problem
• (47:51) - Why "Digital Minds" (Not "Artificial Intelligence")
• (51:05) - LLMs as Social Science Tools
• (01:07:03) - Jacy’s AGI Timeline & The Singularity
• (01:09:23) - Final Thoughts & Outro

• Jacy's website
• Wikipedia article - Jacy Reese Anthis
• Sentience Institute website
• CHI paper - Digital Companionship: Overlapping Uses of AI Companions and AI Assistants
• ICML paper - LLM Social Simulations Are a Promising Research Method
• ACL paper - The Impossibility of Fair LLMs
• Wikipedia article - ELIZA effect
• The Atlantic article - How a Google Employee Fell for the Eliza Effect