How many AI safety papers are at the big ML conferences, what do they study, and who writes them? A comprehensive analysis.> Website: https://ai-safety-tracker-website.vercel.app/> Data, code and plots: https://github.com/SomaxSoma/AI-Safety-Research-TrackerTL;DR: We classified every paper accepted at ICLR, ICML and NeurIPS from 2019 through 2026, using an LLM that reads each title and abstract. 2,328 of them (4.2%) are AI safety papers.

This criticism of AI 2040: Plan A by Séb Krier unfortunately seriously mischaracterizes our proposal. It also mostly contains flat assertions, not real argumentation, and the argumentation in it seems quite weak. While we appreciate constructive criticisms of Plan A, such as the ones by Tom Davidson, Richard Ngo, and 1a3orn, we feel the need to correct the issues in Séb’s response.

Preface for LessWrong: When I think back on my most cherished memories of this community, I return to those honoring defiance in pursuit of goodness:Defying prestigious dogma and searching for raw truth;Defying social pressure, acting alone to help someone while others watch;Defying your self-expectations (your “role”), instead searching over lines of cause-and-effect to find a winning pathway;Defying a powerful foe’s threats, because they only threaten since people like you ...

IntroductionOver the past few years, AI tools have become useful for conducting technical AI research. In the early ChatGPT era (~2023–2024), chat assistants were maybe useful as sounding boards for research ideas, or as editors for polishing a paper draft.

An export-control order in June exposed the real problem—and why Europe's regulatory toolkit can't fix it. [1]The most important AI-governance development of 2026 so far wasn’t a model release. It was an export-control order. On June 12, the US Commerce Department’s Bureau of Industry and Security directed Anthropic to cut off its two newest frontier models for all foreign nationals.

If we end up with a weakly superhuman scheming AI, trading with it may reduce takeover risk: offer it things it values (donate to causes that further its goals) in exchange for useful behavior (revealing misalignment, better alignment auditing techniques). Others have argued the case[1].One key bottleneck is credibility. A scheming AI knows the lab controls its training data, tool call outputs, and runtime context. Why would it believe any deal is real?

This post will only make sense if you already know about AI 2040. If you don't, consider reading the authors' announcement post (Substack, Less Wrong), or reading the full scenario.Alternatively, if you want to read a longer unofficial summary of AI 2040: Plan A, consider reading Scott Alexander’s introduction and reaction post or Zvi Mowshowitz’s introduction and reaction(s) post.IntroI did not like AI 2027 and expected to feel similarly about AI 2040: Plan A.

TLDR: Several US states are considering banning legal personhood for AI systems and/or declaring that AI systems cannot be conscious. The primary motives include religious beliefs, ensuring human accountability for harms, and child safety. We think both moves are premature. Declaring that AI cannot be conscious is especially unwise, since that is a scientific/philosophical question lawmakers cannot resolve.

Last year, I wrote an essay "The hard problem of qualia in the age of AI", https://zenodo.org/records/20549564 (11 pages PDF). I want to create a linkpost to that essay while taking a fresh look at the subject.

Most AI control research such as LinuxArena and Ctrl-Z only gives the red team basic agents which only have access to tools. Yet in 2026, usage of AI within frontier labs has moved to agent harnesses that have access to skills, memory, subagents, external services, compaction and more.At the same time, Claude Code and Codex have both implemented their own version of both action-based and source code monitoring.In such a world, the threat vectors have changed.

This post was originally posted my Substack. I can be reached on LinkedIn and X.Just when it seemed that the frontier AI battle was a two-horse race between Anthropic and OpenAI, the events of the past few weeks changed the AI landscape. The releases of SpaceX’s Grok 4.5 and Meta’s Muse Spark 1.1 put these two companies back in the conversation. Alphabet is also set to launch its next frontier model.

I spent the last month running a persistent personal-AI system on top of Claude Code. This is a field note, not a benchmark, and the hypothesis it left me with is uncomfortable: greater capability did not reduce my avoidance of the actions that can produce rejection.

American bald eagle caws angrily in the distanceWhoa whoa whoa, just hear me out. Unions aren’t usually a good answer for free-market loving libertarians, but one particular AI safety problem is awfully union-shaped:Repeatedly, companies have started out being pro-AI safety and talked the talk about how they’d take precautions around building advanced AI systems. Repeatedly, this line was used to placate & hire very talented researchers who cared about AI safety.

Summary of this postThis is the second in a series of posts detailing my manifold findings while investigating how a chess transformer engine that mimics human play represents knight forks.Last post described strong correlational evidence that the knight-fork policy logit snaps into place after block 5’s attention layer.In this post I will show that controls rule out obvious alternative readings about block 5, that knight forks are mostly assembled compositionally (check plus...

Over the next few weeks we'll cross post some of AI 2040 supplements to LessWrong for discussion; let us know if there are particular ones you think are especially useful to cross post and we'll prioritize those.

My biggest problem with AI 2027 is I don't think it is science-fictional enough. That is, the end of the scenario seems optimized for respectability over accuracy - here I refer to the "special economic zones" and "robot economy" parts. Their industrial explosion assumes human-scale robots will be building robot factories to build more human-scale robotics factories. This is a respectable assumption and one that is fun to model, but seems likely false to me.