This post covers our recent paper: Out-of-Distribution Generalization of Risk Aversion in Language Models. It gives the intro, main results table, and example prompts from the training and evaluation sets.

Following up on my previous experiment - studying Gemma's behavior on agentic tasks when given the number of steps left across the run - I endeavored to see whether giving Gemma a stop_run tool meaningfully changes it's behavior.My initial assumption was that it would not change the completion rate of these tasks, and that has been correct.What I wanted to test:Whether Gemma would "give up" when nearing the end of the run after hitting a dead end.

It’s a quiet week so let’s do the monthly right on schedule.

Aleksandr Bowkis* and David Africa*TL;DRChain of thought (CoT) monitorability may be fragile, and natural language autoencoders (NLAs) may provide a helpful, decorrelated monitoring surface.We tried to read NLAs from the monitor itself, where the NLA readout surfaces what the monitor internally represents while judging an agent's trajectory.NLAs can be useful for monitoring in two ways:Monitor-side: Eliciting latent capabilities from weak monitors by surfacing unverbalised kn...

The dialecticianI crave contact and direct haptic feedback from the world, seeking it far beyond my nominal area of expertise. I engage passionately in discussions with others, even (or perhaps especially) with those I disagree with. I observe the world curiously but critically, chomping at the bit to find flaws in established frameworks. When observation is insufficient, I will data into existence through trials and experiments that answer my questions.

OpenAI’s GPT-5.6-Sol is finally here, along with the cheaper Terra and Luna.

We propose synthetic scalable oversight, a technique for studying scalable oversight by creating graphical abstractions of real-world problems and training tiny models inside these synthetic environments as a proxy for training LLMs at scale.We thank Oliver Richardson, Christian Szegedy, Michael Douglas, and countless others for the many conversations that inspired this work.

I was poking around at a chessformer which mimics human play and made this fun companion app to visualize a lot of the internals of the engine.

ForewordWe will continue where we left off. In the last post, I introduced a few important observables along with the setup. We are interested in their asymptotics, and the behaviors of these random variables. But before we dive into them, we first see some motivation for their definitions, and how they are all connected to each other (the WAIC definition was pretty non-intuitive to me the first time I saw it).