At first read, this claim seems preposterous; one can invent dozens of ways to stop the creation of AGI, so obviously, that is false. Here, we argue that the opposite is not only obvious but inevitable. Separately, we argue we are on the path to AGI, and the AGI is ASI. AGI Will Come, AGI Implies ASI. Here, we start by arguing that it would be very difficult to stop our creation of AGI, then extend that argument to show, for practical purposes, that it is impossible to stop it.

Consider the very direct and complete control we have over our own breathing. Effortlessly, we choose when and how much to inhale and exhale. Control is so effortless that every child begins by imagining that they could stop breathing for as long as they wanted, anytime they wanted. Of course, from experience, all adults know just how impossible this becomes after a period of not breathing. Similarly, all corporations and nations enjoy a kind of direct control over their actions, so much so that we might imagine we could manipulate them to stop the creation of AGI if we desire. This would be nearly impossible to achieve, however. Both nations and corporations can select from a range of actions as long as those actions do not strongly impact profit or power. Both entity types strive to maintain and increase power and profit over nearly every other consideration. It is nearly impossible to manipulate them to operate against those objectives. (Of course, we can pass laws to shape behavior, but we see why this does not work below.)

Our efforts to avert climate change provide a very telling example of our inability to control ourselves. The scientific consensus that climate change is real and very damaging is overwhelming. Further, several tipping point scenarios, where damage spirals beyond reversal, are judged very plausible even if assessing exactly how close we are to those tipping points is difficult. Thus, right now, each day, we are knowingly risking catastrophe. Through dramatic pressure and fortitude of many, we have slowed our progress towards such catastrophes but not stopped it nor reversed those risks back toward zero. Ultimately, the will of millions or billions is not enough to shut down the profit and power motivations of nations and corporations.

But of course, we could stop climate change in its tracks. If every human on earth knew with certainty that all life on earth would end as soon as we reach the next part per million of CO2 in the air. Well, that progression would stop overnight! But we don’t know this, and thus, we don’t act in such a unified fashion. Even with a sizable fraction of the population believing outcomes will be catastrophic for decades, we have only modestly slowed ourselves down. Within the AI community, it is not surprising to have a researcher estimate “P(doom)” > 1% or > 10% and still be bullish on continuing to develop AGI–meaning there is a 1 or 10 percent chance that AGI will destroy all human or something equally appalling, yet still be in favor of AGI given its benefits.

These are the important conclusions to draw from this: (1) given powerful sources of power and profit like fossil fuel consumption or AI computation, corporations and nations will not slow or forgo these advantages except under extreme pressure. (2) Humanity only applies extreme pressure in cases of high certainty. Existential risk is not sufficient; a majority of humanity needs to have high confidence in the threat before such pressures could potentially be applied.

So, we have established that it will be difficult for humanity to stop the development of AGI, how is this extended to conclude it is effectively impossible to achieve?