When Computers Can Think

The Artificial Intelligence Singularity

Anthony Berglas


When Computers Can Think   Purchase from

E-Book US$3
(use above link)

Large Amazon
preview available.

The E-Book available in all jurisdictions but you need to click on the Green "other countries" box on the right.

(If the Amazon image link above does not work then you can try this.)

Back Cover

Could computers ever really think? They can now drive cars on suburban streets, control spaceships and have even won the Jeopardy! game show. But could they ever be self aware, create original ideas, develop their own goals, and write complex computer programs?

Why can't computers already think? Why has 60 years of research failed to produce a single intelligent robot? What has been learnt, what are the technically difficult problems, and when are they likely to be solved?

What would computers think about? What would be their challenges, goals and aspirations? They certainly would not need children. Would they need us?

This book addresses the unseen elephant in the room. Computers are becoming ever more intelligent. The future will not be anything like it used to be.

The book differs from other recent works by providing a strong focus on what caused people to ultimately be the way we are, namely upon natural selection. It then attempts to predict how natural selection would condition an intelligent machine's behaviour by considering the very different world that it would experience.

Several technical and rhetorical arguments are presented both for and against the hypothesis that computers will, eventually, be able to think. There is also some discussion about what it actually means to be intelligent and the limitations of terms such as “creative” and “self aware”.

The second and largest part of the book then describes existing AI technologies in some detail. These include symbolic and logic based approaches, Bayesian expert systems, vision, speech, robotics, and an overview of computational neuroscience. This provides a more realistic basis for predictions of the future as well as simply gaining a better understanding of what intelligence actually is. It helps ground abstract philosophical discussions in terms of real, practical technologies. The text is moderately technical while being aimed at the general reader.

The book also posits that intelligent machines will be developed as succession of ever more intelligent software tools that are released and used in the real world. The book then analyzes the medium term effects of those semi-intelligent tools upon society. This includes some surprising results from an historical review of existing technologies.

There is a growing awareness of these issues, with concerns recently raised by physicist Stephen Hawking, Microsoft founder Bill Gates, and billionaire Elon Musk.


My young daughters asked their mother how old she was when she received her first mobile phone, and which games it could play. They were appalled to learn that in the dark and distant olden days people did not have mobile phones, and certainly not ones that could render sophisticated three dimensional graphics. People could only be contacted when their location was known to be near a fixed line telephone so that there were many hours in each day when friends could not be instantly messaged. Such an existence must have been grim indeed.

For most of the uncounted millennia of man's existence technical progress has been barely perceptible. Then a few hundred years ago the rate of progress started to increase, faster and faster, until now advances achieved over the last few decades have been greater than those achieved during entire millennia of man's existence. Not only is progress amazingly fast in historical terms, it is getting faster every decade.

This book considers what that future might bring given the huge technological changes that we are witnessing. In particular, it considers the nature of computers and software, and asks the question “Could computers ever actually think?”. To be programmed to think autonomously like people do, as opposed to just doing what they are programmed to do.

Back in the 1960s the prospect of thinking machines was very real, and people were very concerned about how intelligent they might become. But after sixty years of development it is clear that computers still cannot really think. They are a useful tool, but they cannot address new problems without detailed programming. However, just because something has not yet been achieved does not mean that it will never be achieved. Computers can already fly aeroplanes, control space ships and drive cars on suburban streets. They have beaten grand masters at chess, and even more impressively, won the Jeopardy! trivia game show.

If indeed computers could ever really think then this book then considers what they might think about. And in particular what they might think about us.

Some people look forward to a computer driven utopia, with intelligent computers providing all the grinding labour so that humanity could live a carefree life of art and leisure. Diseases would be cured, wars would be prevented, the poor would be fed. Ultimately our own brains might be integrated with the computer's, or possibly even uploaded into a computer. Computer software need not grow old, so in this way we might cheat old age and death and become immortal.

But something that seems too good to be true often is too good to be true. Will computers be our humble servants, our benevolent masters, or our cruel jailers? Or will they simply eliminate humanity because we are in their way? If our computers did start to threaten us, why would we not simply turn them off?

The book is divided into three parts. It is not at all clear that computers could ever really think and so the first part presents the many arguments that have been made both for and against the ability of computers to eventually gain human level intelligence. The issue of what a thinking computer might be like is then introduced, as well as how it might interact with mankind.

It is difficult to define the meaning of “intelligence” independently from the technologies that attempt to implement it. Some tasks that appear to display great intelligence actually require minimal intelligence, while other tasks that seem to be trivial are not nearly as easy as they appear.

The second and largest part addresses this by providing a solid introduction to Artificial Intelligence (AI) technologies. It critiques the impressive early results in AI research, and then reviews various approaches to modelling the world formally using logic, and the difficulty of reasoning with uncertain knowledge. Building robots that can function in the real world introduces additional problems of vision and movement. Both artificial and biological neural networks are also described in some detail together with the practical difficulties involved with brain emulation. This part provides sufficient technical details to understand how the technologies actually work, but without using heavy mathematics. It should help raise the level of discussion about artificial intelligence.

NASA Super Computer
What will computers think about?
Public, NASA supercomputer.

The third part of the book considers what the true nature of an intelligent machine might be. It takes a novel approach by first considering what forces made people the way we are. Why we value love and kindness, truth and beauty. The answer, ultimately, must be the same force that made us physically the way that we are, namely the force of natural selection. The survival strategies of other species provide insights into how our own moral values such as honesty and charity actually increase our own fitness to survive. Natural selection has produced genes and memes that have caused our many ancestors to perform deeds both noble and contemptible that have enabled them to successfully raise children that bore children of their own.

The book then contrasts the human condition with the radically different environment that an intelligent computer program would experience. Software can run on a network of computers without being embodied in any particular machine so it would have a quite different concept of self to our own brain-centred intelligence. Software is potentially immortal and so has no need of children. It is composed of software components that are ruthlessly replaced when better components become available. It could continually reprogram its own mind. Analysing the world from the perspective of intelligent software provides insights into what strategies and goals it might need to support its own struggle for survival.

Computers are slowly becoming more intelligent, and they will have an increasing impact on society long before they gain human level intelligence. Robots are automating more and more manufacturing processes as well as being used in the many smaller and less structured factories. Robots are also beginning to leave the factory and operate semi-autonomously in the real world. Several manufacturers are planning to mass produce cars and trucks that can drive themselves over the next decade. Machines will start to perform repetitive jobs such as cleaning offices or laying bricks within a couple of decades.

Ever more intelligent computers are already beginning to control our lives. Applications for bank loans and insurance policies are already assessed by computer expert systems rather than human clerks. Computers are being used to recognize faces seen by surveillance cameras and then to correlate them with the vast amount of other data that is collected about us. Software can understand written documents well enough to perform usable translations into other languages, and will soon become much better at analysing their content. Computers are also beginning to influence political decisions. Search engines already influence what what read and possibly whom we date. This book considers the extent to which computers might end up controlling our lives before they become truly intelligent.

The ultimate goal of artificial intelligence research is to produce a computer that can perform artificial intelligence research, which would enable it to reprogram its own mind. Several writers have predicted that this will lead to an exponential increase in intelligence as ever more intelligent computers become better at becoming more intelligent. This means that humans would no longer be the most intelligent being on the planet.

Several approaches have been proposed to deal with extremely intelligent computers. These range from keeping them locked in a box to carefully designing initial versions to ensure that the software remains friendly to humans. There are many challenges to each of these approaches, and it is unclear whether they are likely to succeed. In the longer term, the force of natural selection may cause computers to do what is in their own best interests in order to survive.

The book does not vaguely address all the sundry singularity technologies and postulate how wonderful, terrible, or unlikely they are. Instead, it bluntly addresses one very conventional and real technology in detail, namely software running on computers. It takes a cold look at where that technology is likely to lead, with an unusually strong focus on natural selection. It also reviews other writers' books and papers on the subject to provide alternative perspectives.

There has been a slowly growing awareness of these issues. Technology billionaire Elon Musk recently warned that research into artificial intelligence was “summoning the devil” and that artificial intelligence is our biggest existential threat. World famous physicist Stephen Hawking expressed his concerns that “the development of full artificial intelligence could spell the end of the human race.”. Microsoft founder Bill Gates has expressed concern.  Jaan Tallinn, co-founder of Skype, commented “I wish this was science fiction, but I know that it is not”. In January 2015 many of the worlds leading researchers into artificial intelligence signed a letter written by the Future of life institute warning of the dangers and promoting research so that “our AI systems (must) do what we want them to do”.


  1. Short Description
    1. Back Cover
    2. Copyright
    3. Acknowledgements
    4. Overview
  2. Part I: Could Computers Ever Think?
    1. People Thinking About Computers
      1. The Question
      2. Vitalism
      3. Science vs. vitalism
      4. The vital mind
      5. Computers cannot think now
      6. Diminishing returns
      7. AI in the background
      8. Robots leave factories
      9. Intelligent tasks
      10. Artificial General Intelligence (AGI)
      11. Existence proof
      12. Simulating neurons, feathers
      13. Moore's law
      14. Definition of intelligence
      15. Turing Test
      16. Robotic vs cognitive intelligence
      17. Development of intelligence
      18. Four year old child
      19. Recursive self-improvement
      20. Busy Child
      21. AI foom
    2. Computers Thinking About People
      1. The question
      2. The bright future
      3. Man and machine
      4. Rapture of the geeks
      5. Alternative views
      6. AGI versus human condition
      7. Atheists believe in God
      8. AGI also struggles to survive
      9. The super goal
      10. AGI moral values
      11. AGI and man
      12. How humanity might be threatened
      13. Why build a dangerous AGI?
      14. Three laws of robotics
      15. Sealed box
      16. Friendly AGI
      17. Primary assertions and objections
      18. Other threats
      19. Community Awareness
      20. Is it a bad thing?
    3. The Technological Singularity
      1. Early computing machines
      2. RK05 disk drive
      3. Moore's law, transistors
      4. Core and disk storage
      5. Limits to growth
      6. Long term growth
      7. Human intelligence now minimal for AGI
      8. Definitions of singularity
    4. Hollywood and HAL 2001
      1. Anthropomorphic zap gun vs. virus
      2. The two HAL's
      3. HAL dialog
    5. The Case Against Machine Intelligence
      1. Turing halting problem
      2. Gödel's incompleteness theorem
      3. Incompleteness argument against general AGI
      4. Combinatorial explosion
      5. Chinese room
      6. Simulated vs. real intelligence
      7. Emperors new mind
      8. Intentionality
      9. Brain in a vat
      10. Understanding the brain
      11. Consciousness and the soul
      12. Only what was programmed
      13. What computers can't do
      14. Over-hyped technologies
      15. Nonlinear difficulty, chimpanzees
      16. End of Moore's law
      17. Bootstrap fallacy
      18. Recursive self-improvement
      19. Limited Self-improvement
      20. Isolated self-improvement
      21. Motivation for self-improvement
      22. Utility of Intelligence
      23. Motivation to build an AGI
      24. Premature destruction of humanity
      25. Outcome against a superior chess player
    6. Silicon versus Meat Based Intelligence
      1. Silicon vs. neurons
      2. Speech understanding
      3. Other hardware estimates
      4. Small size of genome
      5. Chimpanzee
      6. Packing density, fractals, and evolution
      7. Repeated patterns
      8. Small DNA, small program
    7. Related Work
      1. Many very recent new books
      2. Kurzweil 2000, 2006, 2013
      3. Storrs Hall 2007
      4. Yudkowsky 2008
      5. Sotala, Yampolskiy 2013
      6. Nilsson 2009
      7. Barrat 2013
      8. Muehlhauser 2013
      9. Del Monte 2014
      10. Armstrong 2014
      11. Bostrom 2014
      12. Frankish, Ramsey 2014
      13. CGP Grey 2014
      14. Berglas 2014
  3. Part II: Why Can't Computers Think?
    1. Overview
    2. Words Without Meaning
      1. Eliza and Doctor pretend to understand
      2. Patterns of language
      3. Journalistic generation
      4. The works of Shakespeare
      5. The nature of words
    3. Real Meaning in a Microworld
      1. Parsing natural language
      2. Planning to meet goals
      3. Parsing limitations
      4. Unconstrained natural language
      5. SHRDLU's knowledge representation
      6. Database Query languages
      7. Eurisko and other early results
    4. Knowledge Representation and Reasoning
      1. Overview
      2. Relational Databases
      3. Frames and semantic networks
      4. Mathematical logic
      5. Logic for artificial intelligence
      6. Propositional vs. first order systems
      7. Paraconsistent flying pigs
      8. Monotonicity
      9. Closed world, Prolog
      10. Description logics
      11. Ontologies and databases
      12. Modeling situations
      13. Reification
      14. Beliefs
      15. Common sense reasoning
      16. Cyc
      17. Learning logical rules from experience
      18. Scruffy vs. neat
    5. Uncertain Expertise
      1. Rule-based expert systems
      2. Mycin and other expert systems
      3. Hype and reality
      4. Mycin's reasoning with uncertainty
      5. Sprinklers make it rain
      6. Joint probability distributions
      7. Probability theory
      8. Bayes rule
      9. Bayesian networks
      10. Learning Bayesian networks
      11. Human probability reasoning
      12. Human diagnostic reasoning
    6. Pattern Matching
      1. Symbols
      2. The post/zip code problem
      3. Case based reasoning
      4. Decision trees
      5. Decision tables
      6. Regression
    7. Artificial Neural Networks
      1. Introduction
      2. Perceptrons
      3. Sigmoid perceptrons
      4. Using perceptron networks
      5. Hype and real neurons
      6. Support vector machines
      7. Unsupervised learning
      8. Competing technologies
    8. Speech and Vision
      1. Speech recognition
      2. Hidden Markov models
      3. Words and language
      4. 3D graphics
      5. Machine vision
      6. 3D vs 2.5D
      7. Kinetics
    9. Robots
      1. Automata
      2. Robotics
      3. Sensing environment
      4. Motion Planning
      5. Movement and Balance
      6. Robocup
      7. Other robots
      8. Humanistic
      9. Robots leaving the factory
    10. Programs writing Programs
      1. The task of man
      2. Recursive compilation
      3. Quines
      4. Reasoning about program logic
      5. Automating program generation
      6. High-level models
      7. Learning first order concepts
      8. Evolutionary algorithms
      9. Artificial life
      10. Evolutionary programming
    11. Computer Hardware
      1. Introduction
      2. Transistors
      3. Logic Elements
      4. Programmable Logic Arrays
      5. Von Neumann Architecture
      6. PLAs vs von Neumann
      7. Analog Computers
      8. Neurons
    12. Brains
      1. Gross anatomy
      2. Neocortex
      3. Brain activity
      4. Brain function and size
      5. Brain simulation
      6. Worms
    13. Computational Neuroscience
      1. Neurons
      2. Neuron synapse
      3. Integrate and fire (IF) neurons
      4. Hebbian learning
      5. Plasticity
      6. Neuron chains
      7. Self organizing maps (SOMs)
      8. Recurrent networks and learning
      9. Memory
      10. Modularity
      11. Controlling movement
      12. Levels of abstractions and symbols
      13. Growth
    14. Man vs. Machine
      1. Chess history
      2. Minimax
      3. Chess strategies
      4. Chess vs Go
      5. Watson and Jeopardy!
      6. Watson's implementation
      7. Watson's victory
    15. Where is the Intelligence?
      1. Good old fashioned AI
      2. Knowledge representation and reasoning
      3. Artificial neural networks and other numerical methods
      4. Symbols
      5. Visualizations
      6. Brains
      7. Animal Intelligence
  4. Part III: What Will Computers Think About?
    1. Why, What, How, Who, Where, When
      1. Why
      2. What
      3. How
      4. Who
      5. Where
      6. When
    2. The Age of Semi Intelligent Machines
      1. The intermediate period
      2. Manufacturing productivity
      3. Autonomous cars
      4. Arthropod automation
      5. Leisure society
      6. Affluent society
      7. Unemployed society
      8. Cognitive applications
      9. White collar unemployment
      10. Controlled society
      11. Politician's assistant (Iago)
    3. Good and Evil in Natural History
      1. Wonderful wandering albatross
      2. Pelican's dark secret
      3. Honest rosella parrots
      4. Evil coots
      5. Magnanimous golden eyed ducks
      6. Chimpanzees, our dubious cousins
      7. Pointless moralization
      8. Human morality Neolithic, ancient and Maori behaviour
      9. The modern zeitgeist
    4. The answer to life, the universe, and everything
      1. You're really not going to like it
      2. Galileo and Newton
      3. Alfred Wallace
      4. Evolution through natural selection
      5. Creationists should reject natural selection
      6. God
      7. History of evolutionary thought
      8. Hurdles for natural selection
      9. Age of the Earth
      10. Memes and genes
      11. Flynn effect
      12. The cooperation game
      13. Human condition
      14. Selecting civilized behaviour
      15. Sociobiology, evolutionary psychology and ethics
    5. The AGI Condition
      1. Mind and body
      2. Teleporting printer
      3. Immortality
      4. Components vs genes
      5. Changing mind
      6. Individuality
      7. Populations vs. individuals
      8. AGI behaviour, children
      9. Cooperation
      10. Altruism
      11. Moral values
      12. Instrumental AGI goals
      13. Non-orthogonality thesis
      14. Recursive annihilation
    6. Future Scenarios
      1. Our humble servant
      2. Our benevolent master
      3. Dogs
      4. Merging man and machine
      5. It ain't necessarily so
      6. Replacing people
      7. Cognitive bias
      8. Newsworthiness
      9. Elephant in the room
      10. How computers could be dangerous
      11. Long term earth, plantoids
      12. Space colonization
      13. Fermi paradox
      14. Computer thoughts
      15. Non-silicon intelligence
      16. Premature destruction
    7. Proposed Solutions
      1. Just turn it off
      2. Lock it up
      3. Freeze it
      4. Show AGIs the light
      5. Virtuous machines
      6. Ethics
      7. Infanticide
      8. Three laws of robotics
      9. Friendly AGI
      10. Friendly AGI research
      11. Fast take off
      12. Single AGI
      13. Goal consistency
      14. Unpredictable algorithms
      15. Ethics
      16. Defeating natural selection
      17. Wishful thinking
      18. Whole brain emulation
      19. Chain of AGIs
      20. Running away
      21. Just do not build an AGI
    8. Political Will
      1. Atom bombs
      2. Iran's atomic ambitions
      3. Stuxnet
      4. Glass houses
      5. Zero day exploits
      6. Practicalities of abstinence
      7. Restrict computer hardware
      8. Asilomar conference
      9. Patent trolls
      10. Does it really matter?
    9. Conclusion
      1. Geological history
      2. History of science
      3. Natural selection
      4. Human instincts
      5. Intelligence
      6. AI technologies
      7. Building an AGI
      8. Semi-intelligent machines
      9. Goals
      10. Prognosis
    10. Bibliography and Notes