Speculations Concerning the First Ultraintelligent Machine (1965) [pdf]

Speculations Concerning the First Ultraintelligent Machine* IRVING JOHN GOOD Trinity College, Oxford, England and Atlas Computer laboratory, Chilton, Berkchire, England 1. Introduction. . 31 2. Ultraintelligent Mmhines and Their Value . . 33 3. Communication aa Regeneration . . 37 4. Some Representations of “Meaning” and Their Relevance to Intelligent Mmhines . . 40 6. %all and Information Retrieval . . 43 0. Cell Assemblies and Subessemblies . . 54 7. An Assembly Theory of Meaning . . I4 8. TheEconomyof Meaning . . I7 9. Conclusions . . 78 10. Appendix: Informational and Caw1 Interactions . . 80 References . . 83 1. Introduction The survival of man depends on the early construction of an ultra- intelligent machine. I n order to design an ultraintelligent machine we need to understand more about the human brain or human thought or both. In the follow- ing pages an attempt is made to take more of the magic out of the brain by means of a “subassembly” theory, which is a modification of Hebb’s famous speculative cell-assembly theory. My belief is that the first ultraintelligent machine is most likely to incorporate vast artificial neural circuitry, and that its behavior will be partly explicable in terms of the subassembly theory. Later machines will all be designed by ultra- * Baaed on trrlke given in a Conference on the Conceptual Aspects of Biocommunica- tione, Neuropsychiatric Institute, University of California, Los Angeles, October 1962; and in the Artificial Intelligence Sessions of the Winter General Meetings of the IEEE, Jenwry 1963 [ I , 461. The first draft of this monograph was completed in April 1963, and the present slightly emended version in May 1964. I em much indebted to Mre. Euthie Anthony of IDA for the 8rdUOUe teak of typing. 31 IRVING JOHN GOOD intelligent machines, and who am I to guess what principles they will devise?

But probably Man will construct the dew ex machina in his own image. The subassembly theory sheds light on the physical embodiment of memory and meaning, and there can be little doubt that both will need embodiment in an ultraintelligent machine. Even for the brain, we shall argue that physical embodiment of meaning must have originated for remons of economy, at lemt if the metaphysical reasons can be ignored. Economy is important in any engineering venture, but especially so when the price is exceedingly high, as it most likely will be for the first ultraintelligent machine. Hence semantics is relevant to the design of such a machine. Yet a detailed knowledge of semantics might not be required, since the artificial neural network will largely take care of it, provided that the parameters are correctly chosen, and provided that the network is adequately integrated with its sensorium and motorium (input and output). For, if these conditions are met, the machine will be able to l e m from experience, by means of positive and negative re- inforcement, and the instruction of the machine will resemble that of a child. Hence it will be useful if the instructor knows something about semantics, but not necessdy more useful than for the instructor of a child. The correct choice of the parameters, and even of the design philosophy, will depend on the usual scientific method of successive approximation, using speculation, theory, and experiment. The percentage of speculation needs to be highest in the early stages of any endeavor. Therefore no apology is offered for the speculative nature of the present work. For we are certainly still in the early stages in the design of rn ultraintelligent machine. I n order that the argumente should be reasonably self-contained,it is necessary to discuss a variety of topics. We shall define an ultra- intelligent machine, and, since its cost will be very large, briefly con- sider its potential value. We say something about the physical embodi- ment of a word or statement, and defend the idea that the function of meaning is economy by describing it as a process of “regeneration.” I n order to explain what this means, we devote a few pages to the nature of communication. (The brain is of course a complex communication and control system.) We shall need to discuss the process of recall, partly because ifs understanding is very closely related to the understanding of understanding. The process of recall in its turn is a special cme of statistical information retrieval.

This subject will be discussed in Section 6. One of the main difficultiesin this subject is how to estimate the probabilities of events that have never occurred. That such probabilities are relevant to intelligence is to be expected, since intelli- gence is sometimes defined as the ability to adapt to new circumstances. 32 THE FIRST ULTRAINTELLIGENT MACHINE The difficulty of estimating probabilities is sometimes overlooked in the literature of artificial intelligence, but this article would be too long if the subject were surveyed here, A separate monograph has been written on this subject [as]. Some of the ideas of Section 6 are adapted, in Section 6, to the prob- lem of recall, which is discussed and to some extent explained in terms of the subassembly theory. The paper concludes with some brief suggestions concerning the physical representation of “meaning.” This paper will, as we said, be speculative: no blueprint will be suggested for the construction of an ultraintelligent machine, and there will be no reference to transisitom, diodes, and cryogenics. (Note, however, that cryogenics have the important merit of low power consumption. This feature will be valuable in an ultraintelligent machine.) One of our aims is to pinpoint some of the difficulties. The machine will not be on the drawing board until many people have talked big, and others have built small, conceivably using deoxyribonucleic acid (DNA). Throughout the paper there are suggestions for new research. Some further summarizing remarks are to be found in the Conclusions. 2. Ultraintelligent Machines and Their Value Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultra- intelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelli- gence of man would be left far behind (see for example refs. [ZZ], [34], [44]).Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control. It is curious that this point is made so seldom outside of science fiction. It is sometimes worthwhile to take science fiction seriously.

I n one science fiction story a machine refused to design a better one since it did not wish to be put out of a job. This would not be an in- superable difficulty, even if machines can be egotistical, since the machine could gradually improve itselfout of all recognition, by acquir- ing new equipment. B. V. Bowden stated on British television (August 1962) that there is no point in building a machine with the intelligence of a man, since it is easier to construct human brains by the usual method. A similm point was made by a speaker during the meetings reported in a recent IEEE 33 IRVING JOHN GOOD publication [ I ] , but I do not know whether this point appeared in the published report. This shows that highly intelligent people can overlook the “intelligence explosion.” It is true that it would be uneconomical to build a machine capable only of ordinary intellectual attainments, but it seems fairly probable that if this could be done then, at double the oost, the machine oould exhibit ultr&ntelligence. Since we are concerned with the economical construction of an ultra- intelligent machine, it is necessary to consider &st what such a machine would be worth. Carter [ I I ] estimated the value, to the world, of J. M. Keynes, as at least 100,000 million pounds sterling. By definition, an ultraintelligent machine is worth far more, although the sign is un- certain, but since it will give the human race a good chance of surviving indehitely, it might not be extravagant to put the value at a mega- keynes. There is the opposite possibility, that the human race will become redundant, and there are other ethical problems, such as whether a machine could feel pain especially if it contains chemical artScial neurons, and whether an ultraintelligent machine should be dismantled when it becomes obsolete [43, 841. The machines will create social problems, but they might also be able to solve them in addition to those that have been created by microbes and men. Such machines will be feared and respected, and perhaps even loved. These remarks might appear fanciful to some readers, but to the writer they seem very real and urgent, and worthy of emphasis outside of science fiction.

If we could raise say a hundred billion dollars we might be able to simulate all the neurons of a brain, and of a whole man, at a cost of fen dollars per artificial neuron. But it seems unlikely that more than say a millikeyneswould actudly be forthcoming, and even this amount might be difficult to obtain without first building the machine! It would be justified if, with this expenditure, the chance of sucoess were about lo-*. Until an ultraintelligent machine is built perhaps the best intellectual feats will be performed by men and machines in very close, sometimes called “symbiotio,” relationship, although the term “biomechanical” would be more appropriate. As M. H. A. Newman said in a private communication in 1946, an electronic computer might be used as “rough paper” by a mathematidan. It could already be used in this manner by a o h m player quite effectively, although the effectiveness would be much increased if the ahess-playing programs were written with extremely close man-machine interaction in mind from the start. The reason for this effectiveness is that the machine has the advantage in speed and accuracy for routine calculation, and man has the advantage in im- agination. Moreover, a large part of imagination in chess can be reduced to routine. Many of the ideas that require imagination in the amateur are routine for the matar. Consequently the machine might 34 THE FIRST ULTRAINTELLIGENT MACHINE appear imaginative to many observers and even to the programmer. Similar comments apply to other thought processes. The justification for chess-playing programs is that they shed light on the problem of artificial intelligence without being too difficult to write. Their interest would be increased if chess were replaced by so- called “randomized chess,” in which the positions of the white pieces on the first rank are permuted a t random before the game begins (but with the two bishops on squares of opposite colors), and then the initial positions of the black pieces are determined by mirror symmetry. This gives rise to 1440 essentially distinct initial positions and effectively removes from the game the effect of mere parrot learning of the open- ings, while not changing any of the general principles of chess. In or- dinary chess the machine would sometimes beat an international Grandmaster merely by means of a stored opening trap, and this would be a hollow victory.