Philosophy

At the local AI safety co-working space, there are ~two kinds of regulars.There's the kind of regular who's been thinking seriously about AI safety and alignment since pre-2022, who have passing to intimate familiarity with the funding ecosystem, the Sequences, and various conferences that happen at Lighthaven. Let's call them rationalists.Then there's the kind of regular who comes in with many years of impressive industry or government experience, who realized in the last few years that it is i

For the last twenty years I haven't paid much attention to my State Senate race: Pat Jehlen hasn't had a close race since she won the seat in 2005. I think she's done a lot of good work: while allocating credit is hard, more people want to live here than did two decades ago, with more high-paying jobs, better schools, and being a better place to walk and bike. The biggest missing piece has been building housing to keep up, because a larger number of people with the same pool of houses means so

Large language models often take actions running on one computer (via an agentic harness such as Claude Code or Codex), however the LLMs’ responses to prompts are computed on a different computer with GPU access. Could a malicious LLM gain control of the host machine where its weights are loaded? Such a machine is a high-value target: it has sufficient compute to run a frontier LLM, offers easy access to the LLM’s weights, and has privileged access to other computers in the datacentre compared w

This post is about the major role alignment researchers played in advancing the frontier of AI capabilities over the last decade, and how the distinction between “alignment research” and “capabilities research" thereby lost most of its meaning.[1] In particular, I’ll chronicle the development of what I’ll call the “pragmatic alignment” paradigm, and how it helped the three leading AGI companies push hard on the path to AGI under the banner of safety.[2] This was not a subtle effect—it’s apparent

I'm sharing preliminary results of a suite of experiments I ran with claudecode on a small LLM (gpt2-small, no Layer Norm version, courtesy of Apollo research. most of these are on the layer-6 MLP). The github repo for the experiments is here. The success of these experiments given the method's simplicity surprised me, and I would appreciate criticism and bug-finders.This is the headline result. This is not an abstract cartoon, but an exact experimental graph. Yes, I will explain.The key idea in

This post is somewhat niche, and I will sometimes not give context or link relevant background.There’s a big debate that has played out in slow motion on LessWrong over the past two decades, between two broad ways of putting a measure over all possible realities (Tegmark IV):Some “objective” prior (a “reality fluid”), usually a simplicity prior[1]: This is the position taken by Max Tegmark, Jürgen Schmidhuber and UDASSA.A “caring measure”, where we say that our preferences determine our probabil

TLDR: In recent work, Roy Fox proposes to understand an agent's capabilities in terms of the set of environment dynamics it can bring about.[1] This leads to an intriguing duality between probabilities and utilities via the Legendre-Fenchel transform.IntroductionSome agents are more powerful than others. Indeed, some can yield a wider range of outcomes, maybe because they are capable long-term planners or because they have built rich world models. Being able to clearly delineate the capabilities

Industrial explosion is what will make the next-model building loops (and thus learning) with LLMs 1000x faster by about 2050, if indeed the slow-learning prosaic RSI becomes AGI before the big compute buildout slowdown of 2032+ that is already starting. This puts an upper bound on how long it takes to invent ASI that sets off software-only singularity, implementing efficient online learning and fixing all the other hobblings of the likely near-future AGI technology (LLMs/pretraining/RL). The in

TLDR: After briefly outlining the problems with the current review ecosystem, I detail in-depth a research & review platform which addresses these points and seems plausible to me. I further illustrate 4 imaginary researchers to better understand what this would mean in practice and try answering some additional questions the reader might have, like how this would actually be implemented.I don't know a single person who would argue that the current AI research ecosystem is working out. Conferenc

This is a timed post. Every 5 minutes while writing this post, I had to stop to do 15 push-ups, and when I could no longer complete my required number, I had to upload it. My friends thought this would be a fun challenge, but it means that it will likely be less polished than some of my other output.Summary: There has been a lot of talk in the EA/AI safety community recently about what it means to be a “generalist,” but many disagree on what this means. In this post, I give my take on the role:

This essay is adapted from Traversal. “In the great chain of cause and effect,” Alexander von Humboldt wrote as he was teaching science to read the poetry of nature, “no single fact can be considered in isolation.” When the first European colonists made landfall on New Zealand’s shores in Humboldt’s lifetime, the cats and rats that descended from their ships began decimating the native population of black robins — sparrow-sized birds with yellow-soled feet that had

“If equal affection cannot be, let the more loving one be me.” This essay and poem are now part of the Universe in Verse book. In 1865 — a year before the German marine biologist Ernst Haeckel coined the word ecology, the year Emily Dickinson composed her stunning pre-ecological poem about how life-forms come into being — the German physicist Rudolf Clausius coined the word entropy to describe the undoing of being. The thermodynamic collapse of physical systems into incr

What is a question? It seems obvious, but perhaps it isn't. Lani Watson has just published a book on the topic. She thinks that true questions can be identified by their function, not by their form. She explains what she means by this and why questions matter so much in philosophy and elsehwere.

[Revised entry by Willi Goetschel on August 21, 2026. Changes to: Bibliography] Margarete Susman stands out as a distinct voice among the first generation of German women in philosophy in the 20th century. In her capacity as a philosopher, cultural critic, and essayist, she was a regular member of the circle of Georg Simmel, which included Martin Buber, Ernst Bloch, and Georg Lukacs, among others. Jewish thought and the question of the status and role of women in modern society are the two cen

[Revised entry by Philip Atkins, Fabrice Correia, and Teresa Robertson Ishii on August 21, 2026. Changes to: Main text, Bibliography, notes.html] The terms 'essential property' and 'accidental property' have many distinct senses in philosophy. Two have figured prominently in analytic metaphysics of the last 70 years or so. There is the modal sense, according to which (at least as a first pass) an essential property of an object is a property that it must have while an accidental property of an

The great cruelty of life, and also its great mercy, is that only hindsight ever knows each last — the last time your lover put their hand on the small of your back, the last sip of the last cup at your favorite coffee shop before it vanished overnight, the last living note of your mother’s voice. Bittersweet or downright devastating, lasts are the currency of loss with which we pay the price for loving. They are the necessary losses that make us who we are, our training ground for m

“Life is a process of becoming, a combination of states we have to go through. Where people fail is that they wish to elect a state and remain in it. This is a kind of death.” “When you surrender, the problem ceases to exist,” Henry Miller wrote in his stunning letter to Anaïs Nin (February 21, 1903–January 14, 1977). “Try to solve it, or conquer it, and you only set up more resistance.” But we, the controlling species, the conquering species, have a hard time wit

“While we weaken friendships by expecting too little of them, we undermine romantic relationships by expecting too much of them.” We move through the world largely unaware that our emotions are made of concepts — the brain’s coping mechanism for the blooming buzzing confusion of what we are. We label, we classify, we contain — that is how we parse the maelstrom of experience into meaning. It is a useful impulse — without it, there would be no science or story

Tom Rockmore, professor emeritus of philosophy at Peking University and distinguished professor emeritus at Duquesne University, has died. (The following memorial notice was written by Gabriel Gottlieb.) Tom Rockmore, professor emeritus at Peking University, passed away on August 6th. He was known for his work on German Idealism, where he emphasized its constructivist methods in epistemology. Tom’s dissertation, Man as Activity in Fichte and Marx, defended in 1973 at Vanderbilt University, was

“You may have to break your heart, but it isn’t nothing to know even one moment alive.” The morning after a relationship of depth and significance long bending under the weight of its own complexity had finally broken with an exhausted thud, I opened the kiln to discover a month’s worth of pottery shattered — two pieces had exploded, the shrapnel ruining the rest. All that centering, all that glazing, all the hours of pressing letterforms into the wet clay — all of