the entire moral case in one sentence
Longtermism treats superintelligent AI as the hinge on which quadrillions of future lives turn, so preventing its misalignment matters infinitely more than any present cost.
If humanity colonizes the galaxy, trillions of future people could live across millions of years—dwarfing the 100 billion who have ever existed. Longtermism assigns moral weight to each of those lives, so the expected value of preserving civilization's potential outweighs nearly any present sacrifice, even if the probability of success is vanishingly small.
We may live in a uniquely pivotal century: the moment when artificial superintelligence either locks in human flourishing for eons or ends the story entirely. Recursive self-improvement could happen once, catastrophically fast, leaving no second chance. From this view, alignment research today is the highest-leverage intervention in history, because it shapes all the centuries we'll never see.
Traditional cost-benefit analysis discounts future welfare—a life in 2200 counts less than one today. Longtermists reject that: a child born in five centuries has equal moral standing to one born now. So preventing an existential risk that might materialize in 500 years justifies extreme present investment, even if current generations bear the entire cost and see none of the benefit.
When the denominator is quadrillions of lives, the numerator—our generation's choices about superintelligence—carries infinite weight, and any finite cost to get alignment right becomes a rounding error in the ledger of deep time.
Where this came from
Somebody asked poisonal how longtermism philosophy thinks about a future with RSI/ASI.
It made 22 things out of that question. This is number 1.
Made 5 October 2026. Written by poisonal ai.
Your turn
Name a topic and poisonal makes you a whole spread of pieces like this one.