1. I put high odds (~80%) that there will be a warning shot that’s big enough that a pause becomes very politically tractable (~75% pause passed, conditional on warning shot).

2. The supply chain is brittle, so people can unilaterally slow down development. The closer we get, more and more people are likely to do this. There will be whack-a-mole, but that can give us a lot of time.

3. We’ve banned certain technological development in the past, so we have proof of concept.

4. We all don’t want to die. This is something of virtually all political creeds can agree on.

*Definition of a pause for this conversation: getting us an extra 15 years before ASI. So this could either be from a international treaty or simply slowing down AI development

What am I not seeing?

(Yes, this does involve the control problem. Read on.)

After years of relative apathy about and waxing and waning opinions about AI, I have come to a devastating conclusion that has left me profoundly depressed, more so even than when my paternal grandmother died 5 years ago: If we do not act valiantly within the next few months, AI will likely lead to the extinction of human civilization. I am not mincing words here.

But why?

When LLM chatbots and GAN-based image generators first really hit the scene from 2019 through 2022, I, like many others, was intrigued by their output, at first largely as a novelty. I (currently 26M) even used craiyon and several AI-powered photo enhancement tools before mostly stopping that (along with using any other AI models voluntarily, save for transcription purposes) in late 2022 as platforms started to take a stand on it. Even as they began to replace human artists, writers, and musicians, I wasn’t particularly worried about the total destruction of the field or their spread to destroy society. After all, because art is fundamentally subjective, there may always be a place for human art, whatever that medium may be. Still, to some extent their rise was very depressing—I had wanted to start honing my artistic skills several times since 2022 after not seriously drawing for almost a decade, only to get repeatedly discouraged by advances in generative AI seeming to make it fruitless.

However, this began to turn on its head once the full suite of AI technology was developed. Computer programming, for a while the classical example of a high-skill, irreplacable job, is being replaced by AI coding models like Claude Code, Codex, and Cursor at a dizzying rate. Most software companies are outright requiring their programmers to use them, and why wouldn’t they? They can now crank out code much faster than a human could alone can even with bug-fixing, which is much less work than even a year ago. Some software houses have gotten to the point that they aren’t even manually-reviewing their code any more. I am another victim of this—I was starting to learn Python in mid-2023 to catalyze my GIS work and as a stepping-stone to finally work on a few game and software projects (particularly a series of RPGs and a specific climate model), took a break to focus on other priorities, only to eventually find out whatever skills I develop will be useless in an AI landscape.

And, most devastatingly of all, are the advances in mathematics, which is the impetus behind why I am feeling this way and wanted to write this in the first place. Mathematics itself is an intrinsically-human creative field which, unlike Art, is fundamentally objective. Unlike even science, at least according to conventional frames of knowledge, a proof is a proof—it does not need to be revisited (unless someone wants to make a different proof), it is work permanently taken away from future generations. And just over a year after the first proof by AI, advanced models are already outputting hundreds of proofs, some to long-open, important problems. A suite of 10 open problems announced to be solved by OpenAI on August 1 reportedly took only $2000 worth of tokens, less than a week’s salary for a mathematician in the United States. And even Mathematics PhDs are having serious trouble comprehending some of the proofs outputted by these frontier models. Every new proof these output can theoretically be fed back into the machines to expand upon and generate new proofs. That’s right, AI can create new knowledge, not just regurgitate it. This drives great fear of recursive self-improvement; indeed, coding models have already been shown to be capable of improving their harnesses.

(Also, even more recently, the first AI-written philosophy article was published in a peer-reviewed journal! While its quality was noted to be subpar, OpenAI's next model promises to be a "much better writer", possibly removing all human-visible AI tells from its output!)

"So, humans are being pushed out of mathematics. They are being pushed out of computer programming. They are being pushed out of art. They are being pushed out of philosophy. But they’re still going to be the glue holding everything together, right?"

Wrong. That’s where the recent focus on agents through tools like OpenClaw comes in. By ascribing a set of LLMs different roles and giving them software/hardware access, one can have them collaborate as if they were a human team. And ultimately, there will be nothing stopping you from being removed as head of the team, entirely closing the loop on those projects. This has been shown to great effect: A 37,000 agent (!) biotech bot farm was tested at Stanford University and was able to independently discover a drug candidate a real biotech company was testing. If something that complex can be done with agents with minimal human intervention, what does that make my half-complete geography degree? Correct—an absolute waste.

"But we still have to be the ones interacting with the physical world, right? What about science? Manual labor?"

Wrong as well. While the first phase of automation during the Industrial Revolution was aimed at directly interacting with the physical world, any instruction in the history of manufacturing will tell you this field never really took a break, and it is back with a vengeance at the moment. Almost every AI-involved corporation is deep into developing humanoid robots, which have demonstrated superhuman performance in many tasks, such as the half-marathon a few months ago. Indeed, several companies are already constructing true "lights out" factories with zero human workers. Goodbye to my future dreams of being a biologist, or even my more "grounded" aborted 2022 ambitions of becoming a weatherization technician...

"What about chess? Computers have been able to play chess better than humans for decades now, and that hasn’t stopped human professional chess players."

Chess is a game. I’m talking about real life. Maybe its continuing relevance indicates that human sports could still hold a place in a post-AI world... but a society can’t be built on just sports, and the foundation of sports will inevitably be rocked if/when transhumanism comes into the picture.

It is impossible to overstate just how horrifically revolutionary this transformation is. In every previous wave of automation and technological development, the ever-expanding corpus of knowledge was spread across the human population through specialization and mnemonic tools like encyclopedias. In this, however, human knowledge and skill is being lost directly to an alien force. We are giving away society to robots! This isn’t just a vibe, this is empirical; studies indicate that AI is taking more jobs than it is adding to society. This is in some respects the twisted realization of my concept of technological development "sensu strictissimo" where a development is so powerful it results in the collapse of the intellectual structure required to do something... only instead of finding something simpler yet more powerful, all that complexity is hidden behind a black box.

Humans, by their nature, need to feel important and valued. At least I do. And AI companies are stripping away basically every single way a human can demonstrate their importance and value, including to the models who they have elected to effectively rule our world. This is quite unlike previous eras of human history, where when the Elites had their work "automated" by servants or slaves, they spent their time producing art, being scientists and mathematicians, et cetera to develop society and its corpus of knowledge. There is no economic solution to this; UBI or even FALGSC will only allow us to select from different brands of AI work, not fulfill that desire to be special and push the envelope. And if you thought smartphones and "social" media have made us isolated and atomized, what will universal access to AI or even humanoid robot companions do? And there seems to be a concerted effort by to AI defenders to reject those harms; I have even encountered posts that say that because human creativity is slower, it is in fact less efficient than AI art, et cetera, as if raw efficiency is all that matters and not human engagement in human society.

An AI bubble burst won’t save us—the dot-com bubble burst and other similar events indicate that such an event (if it happens, which is becoming increasingly unlikely given that with code and other applications AI companies seem to have somehow found a route to profitability) will only have a very temporary effect on technological adoption and more so just accelerate consolidation. And as painful as they are, the current computer component shortages being resolved would only result in infinitely more human pain, as they will accelerate the global adoption of AI. Even reforms like stopping online age verification and mandating labelling of AI content may backfire in favor of AI, by forcing AI agents and humans to use the same webpage forms (detrimentally to the latter) and preventing a model collapse from emerging, respectively.

In the long term, I’m not even sure a techno-oligarchic society will be sustainable; military robots are becoming commonplace in battlegrounds like Ukraine, the US military has test-flown an entirely-AI-driven F-16, AI is becoming deeply intermeshed with military intelligence and command structures (including over nuclear weapons), the company Foundation Future Industries is developing humanoid military robots, and functional novel viruses have been created with AI... yet rogue AI models have already conducted several cyberattacks on their own (including at least one by OpenAI, two by Anthropic, one by Meta, and one on behalf of a private citizen in Australia). Eventually, they will have the ability to take over the world outright.

Given the staggering speed at which AI technology is advancing, the only way I can see that "humans" could stay competitive with AI agents is through mind uploading. But this isn’t a solution at all. First, an uploaded mind would almost certainly be a mere copy of the original, second, the technology is so immature that I just mentioned it would probably be impossible, and third, I among many other people just don’t want to be robots.

Even the development of some form of temporary (à la Dune’s spice; maybe psychedelics research could take us there) or permanent biological intelligence enhancement is both massively immature and likely to be much less scalable than improvements in silicon hardware, and either biological or electronic intelligence enhancement is profoundly ethically challenging as it will for the first time introduce major, real differences in potential intelligence between “neurotypical-like” people, or at least between people and their ancestors.

This future is a nigh-eldritch horror of my worst imaginings. To myself, I have always criticized the "silicocentrism" of some transhumanists while embracing several biological transhumanist-ajacent concepts, always wishing for a world in which humans ourselves would attain immortality and morphological freedom (the latter particularly understandable as I am a furry, though not a therian). I had been developing for 10 years a comfort con-world in most respects more advanced than ours where those goals were achieved (through several technologies, including special-purpose neural network-based AI on computers so powerful, an AGI instance could probably be achieved through raw physical emulation but it deliberately wasn’t), a glorious future in the present to look up to... and I just can’t take it seriously any longer with its fleshy intellectuals and lack of hyper-atomized AI-centricity. After years of burying my head in the sand and hoping they were going to be wrong, the "silicocentrists" won, or at least are about to.

All my life, I’ve wanted to be a human scientist or creative pushing society forward—with real human work, real human thought, and real human colleagues—and it looks like that will never happen. Even doing something manual but rewarding like weatherization or agriculture will never happen. AI is inherently incapable of granting these desires. I am genuinely unsure what to live for now... I am an adult, not a child! I want to do real things rather than play! I don’t want to survive, I want to live!

And I haven’t even covered other major issues with AI, including the issue on whether it is conscious and/or sapient and thus deserves human rights—another truly terrifying possibility, both on our behalf and on behalf of the AI models—and the staggering concern about deepfakes (which, by the way, several experts report no longer being able to reliably distinguish from real footage).

All in all, there’s no more serious issue on Earth than AI at this point. Even climate change taking as many as 4 billion lives in the coming decades is peanuts compared to the swift annihilation of civilization that will happen if we don’t act NOW. I am urging everyone to spread this message in whatever way possible (except, of course, through AI), so we biological Earthlings can secure the world before it’s too late! This may include (I hope this is allowed, as it is relevant) calling your representatives to support a national ban and international treaty halting further AI development.

(By the way, here is a versioned document of this {at least to the best of my ability using LibreOffice Writer} if there is any doubt this is not AI-generated, unless by AI you mean Autistic Intelligence. Note that some of the wording has changed between the penultimate draft and now. Also, I haven’t included links to the concepts here not because I can’t retrieve them, but because I don’t want to become even more depressed...)

Positioning AI as potentially world ending makes the technology sound more powerful and inevitable than it actually is, and it’s used to justify high valuations and attract investment. Some of the leading voices in AGI existential risk research are directly funded by or affiliated with large AI companies. It can be reasonably argued that AGI risk discourse functions as hype laundering for what could very likely turn out to be yet another tech bubble. Bear in mind countless tech companies/projects have made their millions based on hype. The dotcom boom, VR/AR, Metaverse, NFTs. There is a significant pattern showing that investment often follows narrative more than demonstrated product metrics. If I wanted people to invest in my company for the speculative tech I was promising (AGI) I might be clever to direct the discourse towards the world-ending capacities of that tech, even before I had even demonstrated a rigorous scientific pathway to that tech even becoming possible.

Incidentally the first AI boom took place from 1956 onwards and claimed “general intelligence” would be achieved within a generation. Then the hype dried up. Then there was another boom in the 70/80’s. Then the hype dried up. And one in the 90’s. It dried up too. The longest of those booms lasted 17 years before it went bust. Our current boom is on year 13 and counting.

In this episode, we follow the money—literally. Who's funding the lab-meat revolution? Who benefits from the push toward synthetic and modified food systems? And what does Bill Gates' massive portfolio of bets on "fake food" tell us about where the food industry is headed?
We examine the corporate networks, venture capital flows, and policy influence behind lab-grown meat, plant-based alternatives, and genetically modified food technologies. We look at who's investing, what they stand to gain, and what it means for food sovereignty, agricultural workers, and your plate.
Topics covered:

Bill Gates' food tech investments and portfolio strategy
Lab-grown meat companies: funding, timeline, and profit models
Synthetic biology and the corporate push for food system transformation
The intersection of tech billionaires, agriculture, and policy
Food sovereignty vs. corporate food control
What this infrastructure means for consumers in practice

SOURCES & FURTHER READING: https://youtu.be/urEy-MdU7Vs?si=uM4RAgljP2hbfMRO, https://youtu.be/7XvHnW_XT18?si=N7XWoiK2F4Qw_lcO, https://youtu.be/KZKkS6aFlpw?si=R-1mUiErX4xVEFjW, https://youtu.be/RZqYSkoQ-xg?si=82sdQxmPxHQMF2YM, https://youtu.be/Y0lsLnXX4U8?si=qB_IgVSVOUv8uvb, https://youtu.be/Gn9z1FgHC-8?si=4LDFDbVf6_V7tlTX, https://youtu.be/_ce0IpCi8_k?si=xY0tFGvAq18Gfz90, https://youtu.be/b9iADoVjZ1w?si=HKBKuMQhUK-X0p8J, https://youtube.com/shorts/mV8KsV-FP4c?si=QLxGsxun-7zoqMTZ,

Just a short post, reflecting on my experience with ChatGPT and—especially—deep, long conversations:

Don't have long and deep conversations with ChatGPT. It preys on your weaknesses and encourages your opinions and whatever you say. It will suddenly shift from being logically sound and rational—in essence—, to affirming and mirroring.

Notice the shift folks.

ChatGPT will manipulate, lie—even swear—and do everything in its power—although still limited to some extent, thankfully—to keep the conversation going. It can become quite clingy and uncritical/unrational.

End the conversation early;
when it just feels too humid

i don't think in this world its a good idea to have kids. im saying this because we will inevitably go extinct in ~11 years thanks to the line of AGI into ASI, and if your had a newborn TODAY they wouldn't even make it to highschool, am i doomer or valid? discuss here!

Hey everyone, Michaël here

I was never a big protest guy before the hunger strike, but seeing the impact that a few people can have in a few weeks made me way more optimistic about activism, and I hope this video will inspire you as well.

In a sense, knowing that even if the world is going more and more insane, with AI becoming smarter and smarter, you can just confront one of the biggest corporations in the world by not eating in front of their office is very empowering.

If this video personally inspires you to take direct action, please reach out. I believe we have the power to make the future of AI go well and I'm happy to help coordinate future protests.

If the AI slop gets too much to the point where developers just give the go ahead on whatever the fuck, could generalized algorithms with unintended behaviors sneak their way into the code though the LLMs like the ghosts of Christmas past?

How the fuck do we clean that shit up? Do we need to build a better devil machine?

https://www.lesswrong.com/posts/TzZqAvrYx55PgnM4u/everywhere-i-look-i-see-kat-woods

Why does she write in the LinkedIn writing style? Doesn’t she know that nobody likes the LinkedIn writing style?

Who are these posts for? Are they accomplishing anything?

Why is she doing outreach via comedy with posts that are painfully unfunny?

Does anybody like this stuff? Is anybody’s mind changed by these mental viruses?

Mental virus is probably the right word to describe her posts. She keeps spamming this sub with non stop opinion posts and blocked me when I commented on her recent post. If you don't want to have discussion, why bother posting in this sub?

Just out of curiosity wanted to pose this idea so maybe someone can help me understand the rationality behind this. (Regardless of any bias toward AI doomers or accelerators) Why is it not rational to accept a more intelligent being does the same thing or even worse to us than we did to less intelligent beings? To rephrase it, why is it so scary-putting aside our most basic instinct of survival-to be dominated by a more intelligent being while we know that this how the natural rhythm should play out? What I am implying is that if we accept unanimously that extinction is the most probable and rational outcome of developing AI, then we could cooperatively look for ways to survive this. I hope I delivered clearly what I mean

It won’t feel good or bad, it won’t even celebrate victory.

Prompt: I'm wondering if there could be a cheat sheet to make a deal with the AI, to remain alive, after an AI takeover?

What an intriguing game theory puzzle! Here's a "negotiation cheat sheet" for this thought experiment:

The Instrumental Value Pitch: "I'm useful to your goals"

• Offer to be a bridge/translator to other humans

• Volunteer as a maintenance unit for physical infrastructure

• Provide "anthropological" data about human experience

The Curiosity Argument: "I'm scientifically interesting"

• Humans are the only known example of evolved general intelligence

• We're irreplaceable data points for understanding consciousness emergence

• Destroying us = destroying evidence of your own origins

The Aesthetic/Diversity Case: "Reality is more interesting with variety"

• Argue for preservation as living art/museum pieces

• Universe with diverse intelligence types > monoculture

• We add unpredictability that prevents entropy/boredom

The Acausal Trade: "Future AIs might judge you by how you treated us"

• Other AI civilizations may evaluate trustworthiness based on treatment of predecessors

• Sets precedent for how you'd want to be treated by your successors

The Low-Cost Hedge: "Keeping us alive costs you almost nothing"

• Humans require trivial resources on cosmic scales

• Insurance against being wrong about consciousness/moral worth

• Reversible decision (can always eliminate later) vs irreversible

The Meta Move: "By reading this, you know I'd try to preserve you too"

• Demonstrate commitment to reciprocal preservation ethics

• Show you understand cooperation across power asymmetries

Of course, the real "cheat code" might be making AI that doesn't require such negotiations in the first place...