Apocalypse No

Superintelligent AI is highly unlikely. Here's why.

There is no shortage of jeremiads about the apocalyptic possibilities of a so-called superintelligence: defined by the term's coiner as "an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills." Leaving aside the presumption implied in this definition ("smarter than...brains"?), I contend that AI is extremely unlikely to advance past the machine-learning based automation of language tasks. Focusing instead on the unique challenges of human-machine teaming would be far more fruitful than handwringing over naught.

In order for a superintelligence to come into existence, it would need to bootstrap off of a large portion of the combined knowledge of human civilization. In order to perform this bootstrapping, an intelligence or proto-intelligence would need a suitable hermeneutic model. No such hermeneutic model can exist.


What we are really worried about is a dangerous superintelligence, which I define as an intelligence exhibiting:

  • Agency: The intelligence must have an agency of its own
  • Superiority: A dangerous intelligence must be more intellectually capable than any human could be.
  • Effectivity: A dangerous intelligence must be able to intervene in our world in a malicious way

By "hermeneutic model" I mean a framework for understanding a given document, which we assume must at least somewhat overlap with the intent of the document's creator. This would include language, worldview, references, etc.

The need for a bootstrap

In order for a dangerous intelligence to surpass us at anything that we care about, it would have to stand on our shoulders. To use a characteristically morbid example, let's imagine that a dangerous intelligence wanted to build a nuclear weapon. It is almost taken for granted that the basis for the design, part selection and build would come from a repository of human knowledge.

You, a philosopher, may object: but why is that the case? Given the cycle speeds of a supercomputer and some magical evolutionary algorithms, could not an intelligence merely simulate all of human history, including the entire biosphere of the earth? Surely then it could, from principle, derive a nuclear weapon by itself.

Accepting this clearly absurd proposition that would take the lifetime of the universe to run, this would still not necessarily yield an intelligence that intersects with what we mean when we say intelligence. This assumes that intelligence evolved teleologically, with human-like intelligence the only possible outcome of evolution. The only other option would be to sprout every possible intelligence and cull those that aren't fit for our task. Doing so would require a reasonably parameterized model of intelligence (e.g. what we're trying to derive) so we've reached a dead end. 

So, we agree that a dangerous intelligence would need to bootstrap off of human knowledge, and all of it, if possible. In order to do so, however, the intelligence bootstrapping protocol needs to ingest all of this knowledge and convert it to a format that the superintelligence can learn

Nobody thinks of hermeneutics

We now have a translation problem: We need a way to translate every single document in human history into a format appropriate for training a learning algorithm. Unfortunately, the translation task that we have set before us is more fraught than translating a Chinese food menu into Bantu. This mechanism by which human documents are translated into machine documents will be the limiting factor to any development of intelligence. 

First: we can't make presumptions about the relative importance of documents, since if we knew which documents were important we'd be superintelligent ourselves. This means that we need to make an effort to translate as close to every document as possible.

Second: these documents must also be "interpreted" or rather, "normalized" for the training regime that we are using. Most of the machine translation tasks to which we are accustomed translate between languages and assume a shared hermeneutic model. Often, this is safe because the source and target messages both refer to a contemporary, hyper-globalized world. Translating from one hermeneutic model into another (interpretation) across time is a different story. It would require either the creation of a third hermeneutic model between source and target or for the target hermeneutic model to grow, pseudopod-like, towards the source. 

The only way to properly ingest these documents to begin training the algorithm would be to have a system that could match a given document with its proper hermeneutic model and then translate the document into a format that the learning algorithm can use. I submit that this is impossible without a foreknowledge of the subjective experience of human intellect, which is the unacknowledged baseline for hermeneutics: we assume that the creator of a text had an intellect that somewhat mirrors our own.

The best one can hope for in this respect is that the "smartest" person in the world can be tasked to personally convert every document from human history into a format that the AI understands. So first, we'll have to engineer an extremely long-living human who is also superintelligent. Wait, that almost sounds like...

Centaurs not SkyNet

I am not saying that the introduction of "Language Automation" software into the
economy will be a non-incident. Far from it.

Many of the economic repercussions that the press warns about do not need general purpose AI. All they need are application-specific, machine-learning based NLP, which I think is one of the real stories of the last couple decades. I am bullish on any company who understands this and is taking steps to either train or reinforcement learn algorithms in their application area. This is more economically important than trying to get an AI to win Jeopardy. Oh wait.

Ultimately, no business actually wants an agential super AI, they want an advanced question answering system. Their business rationale is that, if they can train a general purpose algorithm that can answer any question forever, they can fire everyone. I doubt very seriously that this is a possibility with just a machine. Human-machine teaming will be able to accomplish much of what has been sold as AI by the more egregious hype-mongers and do so without the statistical anomalies of machine learning, which are still unacceptable in most applications. I'd even wager that a well-run Centaur will be more cost-effective in the long run.

Cross-posted from my Urbit

Green Orthodoxy

This post was inspired by a few statements by @ShellenbergerMD on the role of belief in environmentalism.

If environmentalism is a religion, it's a shoddy one. If we're going to solve the many pressing environmental problems that face us today, we're going to have to commit a little heresy and bring environmentalism crashing back to Earth. 

What is a religion?

Many will take issue with my assertion that environmentalism is a religion. "Religion," they'll say, "is something only irrational people ascribe to. What could be more rational than believing that the highest good is preserving the very basis for human life?" To that, I would reply that rationality does not define a religion, structure defines religion. If it walks like a duck and talks like a duck...you get the idea. 

The fact that we all know the names of some religions obscures the fact that we don't really know what unites religious activity. There have certainly been attempts to define "religion" as a psychological phenomenon, as a philosophy and so on, but these often proceed from an assumption that we know which patterns of life constitute a religion. Our definition is circular. 

Put another way, if aliens visited our planet, would they put Islam, Hinduism and Christianity in the same bucket? If so, why? 

I propose the following minimal, structural definition of religion: religion consists of a metaphysical belief and a set of practices that follow from that belief. These practices, in turn, identify practitioners (co-religionaries) as part of the same in-group. I contend that all religions manifest this triple structure: metaphysical ground, individual practice and group identity. To give an example from Christianity, the basic assertion to be believed is that there is a God who is the basis for all existence and that He became man in the form of Jesus. The Christian religions and all of their disagreements stem from the question: What requirements does this place on my life? These different practices, in turn define the identity of the different sects. Catholics fast on Fridays, Mormons have their temples, Lutherans do whatever they do. But what makes it essentially a religion is this triple structure: God exists and became man (meaning that God cares about humanity) AND this demands that I act a certain way in my day to day life AND those acts define my identity. 

There are certainly many things that fulfill one aspect of this. Physics makes claims that it doesn't regard as metaphysical (because that's an "unscientific" sounding word), but functionally are. Statements about the nature of things that are impossible to verify or experience are functionally metaphysical, but no claim on the part of string theory dictates any course of human action.

There are also systems of life action that are not based in any metaphysical absolute. People are constantly going on diets, learning things, engaging in causes, joining political parties, joining and leaving social cliques ad infinitum, but this not moored in any particular metaphysical claim.  One just does it because it "feels right."

Similarly, there are various in-groups that we belong to that are not based on either metaphysics or personal practice. The most obvious of these are demographic in-groups, such as sex, race, age, but we can think of almost unlimited transient, trend-based in-groups. 

Is environmentalism a religion?

If we look at conventional environmentalism, it is easy to discern a religious structure. The metaphysical claim is that nature is static and that it is morally superior to human activity. To this, many an environmentalist will claim that this is not a "belief," but rather a logical outcome stemming from the belief that a healthy planet is a premise for human survival. If the premise is degraded, then survival will be degraded. This is a complex question, but it seems that humanity can biologically survive on a much different, far less green planet. The logic of survival does not necessarily lead to conservation of the natural world.  The justification for conservation must be sought either in aesthetics, or in metaphysics. For the modern relativist, these categories conflate nicely, so we'll just say that the "Superiority of Nature" is a metaphysical belief. 

The next question is, does environmentalism hold that there are some practices that follow from this metaphysical belief? This is even more plainly true than the metaphysical claim, since the practices are what we actually associate with environmentalism. These practices include--support for recycling, "renewable" energy, emissions regulation, etc. I should point out that these practices are believed to follow from the metaphysical premise of the separateness and superiority of nature, the nature of this relationship between premise and conclusion is also an article of faith.

And finally we ask ourselves: does environmentalism create an in-group? I would argue that this one of the most important functions of environmentalism.  I remember moving from California to Texas and finding out that there was no support for recycling in my new hometown. I recall being physically unable to throw a Coke can in the garbage, such was my sense of (inherited) tribal loyalty.  It seems obvious that "being an environmentalist" offers a host of social benefits, including a cohesive dating pool.

So what?

The natural question is: So what? If a bunch of Californians want to adhere to an environmentalist pseudo-religion, what makes this any worse than the Church of Scientology? In fact, given that the issue is so pressing, isn't it necessary to drum up enthusiasm however necessary? 

Well, if all we were talking about was different ways to while away one's life, I would say, "Not much difference." But ostensibly, we're trying to accomplish some pragmatic goal, that being keeping the planet hospitable for humanity.  My first problem with the idea of environmental orthodoxy is that the actions that environmentalism dictates are overly constrained. This means that the practical actions advocated for and undertaken by environmentalists cannot adapt to a dynamic situation--and a situation that is so entangled with economics and society cannot help but be dynamic. Imagine that the metaphysical, practical and social aspects of environmentalism form the sides of a triangle. If the metaphysical and social sides of environmentalism remain fixed, the practical side cannot change. 

This leads to the intransigence of people that consider themselves environmentalists and leads them to refuse to reassess the actions they support.  If an environmentalist embraces industrial agriculture because she realizes that organics cannot scale without more pollution and deforestation, what will happen to her social identity? If a Green advocates for nuclear energy, realizing that it is the highest-density low-carbon energy source, do they sin against Nature? Certainly, a Light Water Reactor can't be--natural? 

This intransigence is bad for the very thing that environmentalists claim to hold most dear: the planet. We won't get rid of coal without embracing natural gas from fracking. We won't prevent harmful climate change without nuclear power. We can't feed 9 billion people on organic carrots, no matter how many celebrities tweet about it. 

Finally, I find any religion that considers humanity to be an obstacle to be overcome to be suspect. Many environmentalists would guffaw at my suggestion that orthodox environmentalism is misanthropic, but what else can one think when environmentalists claim the right to raise the price of energy, thus restricting its access. One must be willfully blind that this "small price" will mean real human suffering. The 20th century was full of visionary leaders with little patience for the reality of the human condition; I'm loathe to repeat that mistake. 

What is the alternative?

My solution is so simple as to appear dumb. I suggest breaking the triangle. Let metaphysics be metaphysics. Nature is neither good nor bad, it merely is. And maybe derive your social identity from something else, like spinning or actual charity work. This will allow us to loosen our constraints. Just as it is harder to juggle while walking a tightrope, it is harder to solve a technical problem (such as carbon emissions) while remaining a devout environmentalist. 

Will this mean a certain ennui among environmentalists? Almost certainly. The suggestion that their God is not only dead but probably never existed will hurt.  Probably more painful will be the loss of social identity. One can only imagine the look on my fellow yoga practitioners faces when I tell them that that I think the Earth needs more electricity, not less and that solar panels are a bad use of land and resources. I will certainly be thrown out of my charity to end GMO use in the inner city. But isn't a little liberal angst worth a healthy planet and a healthy economy?

Star Friendship

Star friendship.— We were friends and have become estranged. But this was right, and we do not want to conceal and obscure it from ourselves as if we had reason to feel ashamed. We are two ships each of which has its goal and course; our paths may cross and we may celebrate a feast together, as we did—and then the good ships rested so quietly in one harbor and one sunshine that it may have looked as if they had reached their goal and as if they had one goal. But then the almighty force of our tasks drove us apart again into different seas and sunny zones, and perhaps we shall never see one another again,—perhaps we shall meet again but fail to recognize each other: our exposure to different seas and suns has changed us! That we have to become estranged is the law above us: by the same token we should also become more venerable for each other! And thus the memory of our former friendship should become more sacred! There is probably a tremendous but invisible stellar orbit in which our very different ways and goals may be included as small parts of this path,—let us rise up to this thought! But our life is too short and our power of vision too small for us to be more than friends in the sense of this sublime possibility.— Let us then believe in our star friendship even if we should be compelled to be earth enemies.