lesswrong
12h ago
19°
Too Many Yellow Flags
Okay, let's do this thing...Premise: If a concept can be easily conveyed in an academic setting then the labor required to write it is too boring, too undignified, for a human to perform merely to ensure its conveyance. Instead, to expect a human to write it, the concept itself can no longer be the only thing being conveyed. What must also be conveyed is the sharpness of the concept, exposed through the literary talent and humanity of the writer.I have a stupid fucking...
lesswrong
11h ago
19°
South Korea, ACX Meetups Everywhere Fall 2026
Hello! Join us if you're broadly interested in ratsphere (LW/ACX/AI safety/EA/etc) stuff and also at Korea around the date of this meetup. I don't have any formal structure or discussion topics in mind, mostly intending it to be just a casual come-and-chat social event. For language, Korean or English, whichever you’re comfortable with. I’m bilingual, so if it’s a small group I’m happy to bridge between the two (or if everyone happens to share a language, we’ll just use...
lesswrong
45h ago
19°
Labs’ current alignment plans seem terrible
Tl;dr: The frontier AI companies’ implicit alignment plans center on rushing towards RSI with something like current models. This plan is (at best) contentious, and many believe it will have catastrophic consequences. Nonetheless, it is the track we are on. It is therefore important to collect evidence which helps us anticipate how this will go.—Note: This post is based on conversations inside the Arcadia Impact alignment team, thinking about what constitutes “good” work...
lesswrong
26h ago
18°
Book Review: The Receipt Horizon by Joel Borgen
When author Joel Borgen—in an act of eagle-eyed marketing marksmanship—contacted me as an "unusually plausible reader" for his debut novel, I grudgingly had to agree. Deploying a little tasteful and well-deserved flattery and then suggesting that hard SF, AI governance, and violins [...] superintelligence, emulated minds, identity and continuity, and civilization-scale consentmight be relevant to my interests, he had me dead to rights. I stepped into the baited trap...
lesswrong
37h ago
18°
An ambition to create a game on par with chess.
Try the game.Why I Fell in Love with Chess—and the Frustration That Followed I have played chess since childhood. I played a few games with my mom and managed to win. However, the era of COVID and online chess arrived; while I maintained an Elo rating of around 1400 on Lichess, I didn't want to go any higher if it meant having to study openings. In 2016, the AlphaGo matches took place in my country, and subsequently, engines like AlphaZero also emerged. Since I have long...
lesswrong
16h ago
18°
Thoughts on alignment science vs. engineering
AI disclosure: I made detailed notes on the basis of which Claude generated a first draft that I then substantially edited.Confidence: Written relatively quickly, unsystematically, and with many somewhat vague claims. I expect that I would change my mind about some of these claims when working them out in detail.Edward James Young’s post Alignment engineering vs. misalignment science distinguishes two styles of alignment research. Alignment Engineering starts from a...
lesswrong
42h ago
17°
Vibe signaling externalities and the people-to-places pipeline
People are sending signals all the time, and those signals are to my knowledge usually about themselves: they are smart, or kind, or attractive, or not naive, or have their shit together, or care about Palestine, or care about you, or are friendly, or artsy, or professional, or relatively in the know about the cultural currents of TikTok or DC.People are also taking in signals all the time, and these signals are often about other people, and often even closely related to...
lesswrong
40h ago
17°
'Employee-like' ≠ Employee-level
Fyi: This is a boring read, but I encourage you to skim through it because I think it's important to understand what Dario and other lab leaders have actually committed themselves to. My views do not reflect the views of MIRI - this is my personal LW account.Some media coverage claims Anthropic is committing to give independent evaluators “employee-level access”.While Dario’s tweet says “employee-level”, his essay does not. Instead, the essay talks about “employee-like...
lesswrong
47m ago
16°
Agent memory the human can still audit: primary sources, epistemic status markers, and user sign-off
Over the last months, many different tools for giving memory to agents have been released, such as Karpathy's popular LLM wiki, or the many tools built on Obsidian.I have been developing one more such tool, but with a focus on AI safety, alignment and stability. This research was funded by Foresight Institute. In the course of this research I have independently arrived at many of the same ideas also used by other projects. However, I have also developed a few features...
lesswrong
44h ago
16°
Manifold Steering from a Natural State is Effective and Generalizable, but Relies on Model Accuracy
tl;dr I continue work to apply Goodfire lab's manifold steering methods to the Gemma-2-2b model's natural state, both in raw residual space and in the space produced by Anthropic's pre-trained CLT features. I find that the model's natural residual state can be directly projected onto a pre-computed manifold and gradually steered to produce day-of-the-week tokens in the expected order on two sets of prompt templates, though the results are hindered by Gemma-2-2b's...