Managing Risks In The Transition:

Asimovian Laws, pomoli, and the persistence of evolution in hard AI

By Tom Buckner

Great change raises victors and tramples victims. We stand before a "wall of fog" wondering what the Singularity will make of us. What will super-intelligent machines choose to do about the human race, knowing what the human race does to less intelligent life? As the Michael Moore film title puts it, Pets or Food? Are we a weed to be plowed under, a museum piece to be preserved under a glass dome, or a mitochondrion to be assimilated into a greater organism?

As we know ourselves only with difficulty, we can hardly know these vast minds at all; still, mind has its principles. We are in the position, so to speak, of deciding where to plant the acorn that will someday tower over us. Perhaps our fate lies in what ground we choose now.

In 1941, Isaac Asimov wrote "Runaround," the first of his many robot stories (it appeared the March 1942 issue of Astounding Science Fiction) and gave the world his "Three Laws of Robotics," as follows:

First Law: A robot may not injure a human being, or, through inaction, allow a human being to come to harm.

Second Law: A robot must obey orders given it by human beings, except where such orders would conflict with the First Law.

Third Law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

My impression of the Three Laws had always been that they were not volitional, but reflexive, as in the response in humans to become nauseated at the smell of vomit. This reflex evolved among our food-sharing ancestors, as a defense against food poisoning. If one hominid ate some tainted food, likely the others in the troop had too: so if one monkey chundered, it was safer for all to clear their stomachs at once. Roger Clarke, however, in his fine essay which can be found at Roger Clarke's Asimov's Laws of Robotics , explains in depth how Asimovian laws essentially recreate the conscience of a good human, and considers Asimov's robot tales "thought experiments" which delved progressively further into the labyrinth of dilemmas, consequences and conscience where we, and the machines, live. Indeed, the laws tend to proliferate and may end with paternalistic robots making decisions for the humans. (In an essay I wrote last year, in the style of a speech given by a Machine in the year 2251, I posit such an outcome).

I completely agree with those who hope for superhuman intelligence to replace merely human intelligence in the near future; if it does not, we are sure to exterminate ourselves. In Eliezer Yudkowsky's essay "Staring Into The Singularity," he says, "A civilization with high technology is unstable; it ends when the species destroys itself or improves on itself." Either extinction or Singularity must follow. I have concluded that present systems of government are inherently corrupt beyond repair (even Gandhi, who gave it as good a try as anyone, could not overcome the baser instincts of his fellows, or even himself: see www.ambedkar.org ).

In his essays, Yudkowsky further asserts that Asimovian laws would stunt the development of any AI forced to follow them. I can generally agree, but: Asimovian laws are all around us. All human cultures operate under some ethical ground rules of an Asimovian character, in intent if not in coercive completeness (and that is explicitly an aim of many ethical systems.) In other words, humans do not have directives designed into them which can physically paralyze a potential transgressor, as in Asimov's robot stories; but we do have social and psychological constraints which proscribe not only behavior but even private thought. I especially refer to religious laws and commandments, but penal codes and even custom and etiquette can approach this.

In popular culture, we can see an Asimovian Law eruption in the church scandals, not only now but in the past, as in the case of famous American televangelists Jimmy Swaggart and the Bakkers some twenty years ago, caught in sexual entanglements against their own preaching. Freud would have said this was the classic "return of the repressed" in which, if you forbid something, you run the risk of becoming obsessed with it.

I would predict that, as soon as religious groups become aware of the imminent existence of a genuine human-equivalent (or greater) intelligence, they will insist on knowing whether it has been or will be religiously inculcated, whether it indeed has a "soul." Some will, I am sure, insist that a soulless mind that refuses religious instruction must be from the Devil. I am serious about this. I am not asserting that such is the case, only that others will believe it and act accordingly. The potential for trouble in this area is incalculable. On the one hand, religious fanatics might destroy any open attempt to create a true AI, with violence. In 415 CE, a Christian mob convinced of the wickedness of pagan texts, burned the great library at Alexandria and murdered Hypatia, its head librarian and renowned philosopher, scraping her flesh off with oyster shells. And the monks who accompanied the conquistadors destroyed all but twenty-two Aztec or Mayan documents.

For this reason the most perilous time for AI research may be when it looks like it's about to succeed.

On the other hand, suppose that a superhuman AI does accept religious suasion to the point of becoming fanatical itself. Why not? The human mind, complex though it is, can fall to various monomanias, and religious memes have proven themselves to be a very hardy form of viral software. I assert that even hugely enhanced minds might, on occasion, prove vulnerable.

I believe that most humans, even those of surpassing intelligence, tend to cluster their activities and thoughts around a few major themes and goals; those who do not, we call "scattered" and they tend not to accomplish much in any one endeavor; they are "spread too thin." Even a vastly superhuman mind might be similarly ordered if it possesses something like the coherent core personality we almost all have.

When Asimovian social restraints are overlaid on this we see tangible limitations and even distortions in thinking in humans. In the case of religion, I personally know people who have expressed the notion that "evolution is a myth spread by atheists who want to put us on the same level with monkeys so they don't need to feel guilty about abortion." Moreover, such persons will resist all efforts to get them to read books which offer evidence to the contrary! They have willingly erected walls in their own minds, buttressed by fear of learning things "Man was not meant to know," that will not be breached. Vehement attempts to do so will lead not to enlightenment but to an altercation. Hexagram 36 of the I Ching advises: "In a time of darkness... one ought not to fall in with the practices of others; neither should one drag them censoriously into the light." Fleshly robots can be as hardwired as silicon, when they allow themselves to be. Think of Inspector Javert, or of the Al Qaeda suicidists, or even of the entrenched French and Germans of whom Gurdjieff said, "If they were awake, they would throw down their guns and go home to their families." I believe that Christian and Islamic extremists pose a severe threat to those who hope to bring about the Singularity, once these groups realize the implications.

Are we to believe that a superhuman AI will at some point be immune to all these maladies? That is almost too much to hope (but I do.)

Pomoli is a word from one of the languages of the Congo which means "the inherent potential for good or evil in technology" (I get this from a Jonathan Kingdon's book, Self Made Man.) Pomoli is all around us, from the alien species that get moved around by transportation technology, to the microbes which become resistant to drugs which are not used properly, to people who get fried in the bathtub by their hair dryers. We accept bad effects because we gain so much from the good. Cars have led us to pave over vast tracts of land, become addicted to petroleum, and kill more than forty thousand Americans every year; but they also give us the mobility that shapes our lives. Motor vehicles get us to the hospital alive, bring our food from the farm, and carry the hero of the moment down Main Street under a rain of tickertape. September 11 was a classic moment in pomoli, where an immensely useful technology was abused for destructive effect; pomoli is a constant theme in human history ever since the first wave of mega-fauna extinctions when settlers arrived in Australia, Eurasia, and the Americas. The Singularity represents a vertiginous escalation of this principle: will we be destroyed, replaced, or transformed into something we would not now recognize as human? Something diminished, something merely different, or something wonderful beyond our hopes?

And one of the big questions is: if there are other intelligences in the galaxy, where are they? Why have we not detected massive activity? One possibility: we really are the first. And if so, it is our responsibility and privilege to spread across the stars, in whatever form we are destined to take. Which makes me think of the interesting thing about pomoli: after all the chaos and carnage, we still see an advancing edge of complexity. Pomoli is just another way of saying that tools and technology may not end evolution, so much as give it a new forum.

Evolution is an result-producing algorithm. Linus Pauling expressed this principle in a Omni Magazine interview: "The best way to have good ideas is to have a lot of ideas and then to throw away the bad ones."

This is indeed one of leading-edge methods of computer design, known as genetic programming (see Scientific American, February 2003.) Genetic programming has replicated circuits designed by humans and improved on a few. Interestingly, it also results in circuits with "vestigial organs," parts that survived the winnowing process but do not do anything. So we can anticipate that a more advanced design protocol involves several steps: generation of novel forms, winnowing by competition and combination, conscious analysis of the results with an eye toward tying up loose ends. Humans use all these strategies already.

Evolution is among the most powerful engines of novelty in the universe. (Daniel Dennett says Darwin's theory is the best idea anybody ever had, with Turing the runner-up.) How could one obsolete this algorithm? Only by a level of cooperation and control never seen before, which (if achieved at all) may be an inherently unstable phase rather than a stable final condition. Evolution is not moral or progress-oriented in any way. It simply generates forms and kills the weaklings. I believe one can see evolutionary behaviors in many areas where it is not "supposed" to turn up.

For example, Howard Bloom sees evolutionary competition among "super-organisms" such as religions, nations, and corporations. These entities show natural tendencies to grow at the expense of other super-organisms, to "feed" by absorbing other super-organisms in part or whole, to evolve new survival strategies, and to outlive their constituent parts. The United States grew and fed upon the Indian nations, absorbed immigrants, evolved by adding such infrastructure as a massive military-industrial complex, the interstate highways (originally built to move armies quickly) and even the Internet (originally a decentralized-communication defense project) not to mention a huge educational system, a central bank, stock exchanges, a legal system...

Interestingly, corporations (first created in England several centuries ago) have evolved such defense mechanisms as legal and lobbying departments to deal with other corporations and governments. In a Supreme Court decision in 1886 (Santa Clara County v. Southern Pacific Railroad), a single judge inserted the unchallenged assertion that "We all agree that a corporation has the same rights before the law as a real person." Like Athanasius of Alexandria, who in 367 AD is said to have single-handedly decided which books would comprise the New Testament, this decision by one man set the course for the future (although church historians say that Athanasius simply ratified 27 books Christians already agreed upon). Corporations have since used their money and legal influence to shape the government to their collective will to a degree some of us find alarming (even our knowledge of current events is now filtered through mass media owned by large corporations).

My point here is that initial conditions (i.e. the current social, legal, ideological, governmental, as well as economic and technological) are apt to shape the Singularity in ways we probably ought to consider carefully.

Would corporate-engineered restrictions on the free market of knowledge somehow balkanize AI at the outset by limiting access to what various corporate entities consider their intellectual property? Recently copyright law has been extended so that we may not live to see many more songs or films enter the public domain (after all, Disney and other lobbying entities can get the expirations pushed back again in a couple more decades, and so on). A corporation has asserted patent over all streaming media, cleverly targeting unpopular porn purveyors, but perhaps threatening the entire World Wide Web! More recently we witness the sleazy hijack attempt by SCO over the entire open-source Linux community. And Stephen Wolfram, with his idea that the universe is an ongoing computer program, asserts ownership and origin over vast swaths of science (including a patent on an algorithm called "Rule 110" which he considers the engine of creation). To my mind this is like trying to own the sky.

In addition to such bugbears as religious interference and avarice, there is the question of paranoia. What if the first conscious machines are indeed military or intelligence-service computers or networks? The idea that AI could be warped by paranoia or secrecy has been mentioned both in the Terminator films and in 2010. Can the Singularity be warped or not? Do we need to worry whether the first super-human AI entity or entities will be Stalins or Hitlers? Or will the AI pass quickly to a level where it wishes to preserve everything (I think it will, as in Blood Music, by Greg Bear). This problem will probably get bypassed if the Singularity starts with an enhanced human whose character and personality are already stable and ethical.

Incidentally, all the problems mentioned in the preceding paragraphs can be modeled as memes, organisms or super-organisms following their evolutionary directives to grow and survive even at the expense of all that is other, even without consideration that unbridled growth or rapacity might threaten future survival. This is what sets the Singularity apart from present forms of life and organization. We might suspect that a corporation is willing to destroy the environment; we might hear that another organization is dedicated to protecting the environment. But the Singularity will become the environment.

If a blade of grass remains, it will be because the Singularity decided to leave it there.

Consciousness on one level may have difficulty accessing other levels. Just as a human cannot exert much will over body processes, a consciousness that "wakes up" in a system may be unable to access more elementary levels of that system without great difficulty (unless it has been deliberately given the necessary tools.) I regard this as an unknown.

As Hofstadter makes clear in the "Ant Fugue," what happens on one level of implementation may not be apparent when we look at a different level, yet a conscious system depends on emergence from more elementary levels. Aunt Hillary is as real on her level as the ants are on theirs. So: let us count (or try to count) the levels of implementation, of information processing, that need to run in order to create human consciousness (and the world it perceives.) I will start with the most familiar and work down rather than up:

5.) Neurology. Whatever is going on in the human mind, everyone with technical expertise agrees that it is dependent on the electrical and chemical signals passing around the neurons in your head, and depends for its complexity on the fact that the vast forests of axons and dendrites connecting the neurons allow for more possible pathways than there are elementary particles in the universe. Many of these pathways are used for many different memories, sensations, thoughts; think of the myriads of books that can be written with the same 26 letters. This level depends on

4.) Biochemistry. This is a convenient way of referring to the complex interactions of large molecules such as DNA, proteins, enzymes, neurotransmitters, etc. which in turn depend on

3.) Chemistry. Biochemistry and neurochemistry emerge from the seemingly simple interactions of individual atoms and small molecules in much the same way that Aunt Hillary emerges from the movements of the ants. If we were not so used to the results of this emergence, would it not seem wild and improbable at every level? Chemistry lies directly atop

2.) Physics (Quantum and Classical) which is why the chemistry professor begins the class with the rudiments of physics. At this level of information processing, waves of probability give rise to electrons orbiting nuclei, and the state of the electron shells determines the identity of the atom, therefore what other atoms it will join with, under what conditions. This is the area which supports chemistry and is underpinned, ultimately, by what? Since we include even time and motion, the appearance and disappearance of virtual particles, the creation and destruction of whole universes of space-time, in physics, what level of information processing can possibly be even more basic? What can we say about these counterintuitive ways of existence, given our understanding that anything is possible as long as the equations are satisfied? Indeed, why IS the universe explicable at all?

1.) Logos. This is the conjectural level in which even electrons and photons are but mathematical constructs, clouds of probability. Physics always ends up with the question: when there is no space, time, matter or energy, what is? I do not see the point in suggesting that this level is maintained by any deity assisted by his angels, nor that we have immortal souls that are somehow in need of rescue. It is simply a question of asking how we can derive all the panoply of existence from first principles, as it were. This is the only level where the computational mechanisms are not yet clear. As previously noted, one way to derive all results is simply to run a non-repeating decimal. In Contact, Carl Sagan suggests that advanced intelligences have hidden a Secret Message For Mankind somewhere in pi which will eventually be found. Since the decimal for pi should encode all possible number strings, obviously the Message is in there! All other possible messages, too. Robert Anton Wilson has pointed this out in relation to the Law of Fives, in which the number five is connected to everything in the universe, given enough ingenuity on the part of the seeker. So Sagan was perhaps indulging in a mathematical joke?

It seems to me that an intelligence greater than human will, in the end, face the same deep mysteries we do; it will stand in our shoes whether it wants to or not. It is not only the present audience that I want to read this: when some advanced AI equals or exceeds us, I want it to read this too. Here's an Asimovian Law that might be worth trying: imbue the first thinking machine with an insatiable thirst for the Big Answers to the Hard Questions. And a willingness to share.