Saturday, February 27, 2016

reprint of "Intelligence Explosion"


When I looked at Wikipedia today this article has either been renamed or deleted from Wikipedia(likely because it freaks people out too much). However, I think understanding this sort of thing (because we are entering into something like this right now) will help people survive it in the short and long run, rather than have aberrant reactions because they just have no idea what is wrong in their lives. (Sort of like Trump boycotting Apple and asking his supporters to do the same out of ignorance of how this might bankrupt them in the long run).














Showing posts sorted by relevance for query intelligence explosion. Sort by date Show all posts

Wednesday, January 6, 2016


Intelligence explosion

This article is also related to this article:

2000 to 2100: The Era of the Technological Singula...

Intelligence explosion

From Wikipedia, the free encyclopedia
An intelligence explosion is the expected outcome of the hypothetically forthcoming technological singularity, that is, the result of man building artificial general intelligence (strong AI). Strong AI would be capable of recursive self-improvement leading to the emergence of superintelligence, the limits of which are unknown.
The notion of an "intelligence explosion" was first described by Good (1965), who speculated on the effects of superhuman machines, should they ever be invented:
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
Although technological progress has been accelerating, it has been limited by the basic intelligence of the human brain, which has not, according to Paul R. Ehrlich, changed significantly for millennia.[1] However, with the increasing power of computers and other technologies, it might eventually be possible to build a machine that is more intelligent than humanity.[2] If a superhuman intelligence were to be invented—either through the amplification of human intelligence or through artificial intelligence—it would bring to bear greater problem-solving and inventive skills than current humans are capable of. It could then design an even more capable machine, or re-write its own software to become even more intelligent. This more capable machine could then go on to design a machine of yet greater capability. These iterations of recursive self-improvement could accelerate, potentially allowing enormous qualitative change before any upper limits imposed by the laws of physics or theoretical computation set in.

Contents

Plausibility

Most proposed methods for creating superhuman or transhuman minds fall into one of two categories: intelligence amplification of human brains and artificial intelligence. The means speculated to produce intelligence augmentation are numerous, and include bioengineering, genetic engineering, nootropic drugs, AI assistants, direct brain–computer interfaces and mind uploading. The existence of multiple paths to an intelligence explosion makes a singularity more likely; for a singularity to not occur they would all have to fail.[3]
Hanson (1998) is skeptical of human intelligence augmentation, writing that once one has exhausted the "low-hanging fruit" of easy methods for increasing human intelligence, further improvements will become increasingly difficult to find. Despite the numerous speculated means for amplifying human intelligence, non-human artificial intelligence (specifically seed AI) is the most popular option for organizations trying to advance the singularity.[citation needed]
Whether or not an intelligence explosion occurs depends on three factors.[4] The first, accelerating factor, is the new intelligence enhancements made possible by each previous improvement. Contrariwise, as the intelligences become more advanced, further advances will become more and more complicated, possibly overcoming the advantage of increased intelligence. Each improvement must be able to beget at least one more improvement, on average, for the singularity to continue. Finally the laws of physics will eventually prevent any further improvements.
There are two logically independent, but mutually reinforcing, accelerating effects: increases in the speed of computation, and improvements to the algorithms used.[5] The former is predicted by Moore’s Law and the forecast improvements in hardware,[6] and is comparatively similar to previous technological advance. On the other hand, most AI researchers believe that software is more important than hardware.[citation needed]

Speed improvements

The first is the improvements to the speed at which minds can be run. Whether human or AI, better hardware increases the rate of future hardware improvements. Oversimplified,[7] Moore's Law suggests that if the first doubling of speed took 18 months, the second would take 18 subjective months; or 9 external months, whereafter, four months, two months, and so on towards a speed singularity.[8] An upper limit on speed may eventually be reached, although it is unclear how high this would be. Hawkins (2008), responding to Good, argued that the upper limit is relatively low;
Belief in this idea is based on a naive understanding of what intelligence is. As an analogy, imagine we had a computer that could design new computers (chips, systems, and software) faster than itself. Would such a computer lead to infinitely fast computers or even computers that were faster than anything humans could ever build? No. It might accelerate the rate of improvements for a while, but in the end there are limits to how big and fast computers can run. We would end up in the same place; we'd just get there a bit faster. There would be no singularity.
Whereas if it were a lot higher than current human levels of intelligence, the effects of the singularity would be enormous enough as to be indistinguishable (to humans) from a singularity with an upper limit. For example, if the speed of thought could be increased a million-fold, a subjective year would pass in 30 physical seconds.[3]
It is difficult to directly compare silicon-based hardware with neurons. But Berglas (2008) notes that computer speech recognition is approaching human capabilities, and that this capability seems to require 0.01% of the volume of the brain. This analogy suggests that modern computer hardware is within a few orders of magnitude of being as powerful as the human brain.

Intelligence improvements

Some intelligence technologies, like seed AI, may also have the potential to make themselves more intelligent, not just faster, by modifying their source code. These improvements would make further improvements possible, which would make further improvements possible, and so on.
This mechanism for an intelligence explosion differs from an increase in speed in two ways. First, it does not require external effect: machines designing faster hardware still require humans to create the improved hardware, or to program factories appropriately. An AI which was rewriting its own source code, however, could do so while contained in an AI box.
Second, as with Vernor Vinge’s conception of the singularity, it is much harder to predict the outcome. While speed increases seem to be only a quantitative difference from human intelligence, actual improvements in intelligence would be qualitatively different. Eliezer Yudkowsky compares it to the changes that human intelligence brought: humans changed the world thousands of times more rapidly than evolution had done, and in totally different ways. Similarly, the evolution of life had been a massive departure and acceleration from the previous geological rates of change, and improved intelligence could cause change to be as different again.[9]
There are substantial dangers associated with an intelligence explosion singularity. First, the goal structure of the AI may not be invariant under self-improvement, potentially causing the AI to optimise for something other than was intended.[10][11] Secondly, AIs could compete for the scarce resources mankind uses to survive.[12][13]
While not actively malicious, there is no reason to think that AIs would actively promote human goals unless they could be programmed as such, and if not, might use the resources currently used to support mankind to promote its own goals, causing human extinction.[14][15][16]
Carl Shulman and Anders Sandberg suggest that intelligence improvements (i.e., software algorithms) may be the limiting factor for a singularity because whereas hardware efficiency tends to improve at a steady pace, software innovations are more unpredictable and may be bottlenecked by serial, cumulative research. They suggest that in the case of a software-limited singularity, intelligence explosion would actually become more likely than with a hardware-limited singularity, because in the software-limited case, once human-level AI was developed, it could run serially on very fast hardware, and the abundance of cheap hardware would make AI research less constrained.[17] An abundance of accumulated hardware that can be unleashed once the software figures out how to use it has been called "computing overhang."[18]

Impact

Dramatic changes in the rate of economic growth have occurred in the past because of some technological advancement. Based on population growth, the economy doubled every 250,000 years from the Paleolithic era until the Neolithic Revolution. The new agricultural economy doubled every 900 years, a remarkable increase. In the current era, beginning with the Industrial Revolution, the world’s economic output doubles every fifteen years, sixty times faster than during the agricultural era. If the rise of superhuman intelligence causes a similar revolution, argues Robin Hanson, one would expect the economy to double at least quarterly and possibly on a weekly basis.[19]

Superintelligence

A superintelligence, hyperintelligence, or superhuman intelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. "Superintelligence" may also refer to the form or degree of intelligence possessed by such an agent.
Technology forecasters and researchers disagree about when human intelligence is likely to be surpassed. Some argue that advances in artificial intelligence (AI) will probably result in general reasoning systems that lack human cognitive limitations. Others believe that humans will evolve or directly modify their biology so as to achieve radically greater intelligence. A number of futures studies scenarios combine elements from both of these possibilities, suggesting that humans are likely to interface with computers, or upload their minds to computers, in a way that enables substantial intelligence amplification.

Existential risk

Berglas (2008) notes that there is no direct evolutionary motivation for an AI to be friendly to humans. Evolution has no inherent tendency to produce outcomes valued by humans, and there is little reason to expect an arbitrary optimisation process to promote an outcome desired by mankind, rather than inadvertently leading to an AI behaving in a way not intended by its creators (such as Nick Bostrom's whimsical example of an AI which was originally programmed with the goal of manufacturing paper clips, so that when it achieves superintelligence it decides to convert the entire planet into a paper clip manufacturing facility).[20][21][22] Anders Sandberg has also elaborated on this scenario, addressing various common counter-arguments.[23] AI researcher Hugo de Garis suggests that artificial intelligences may simply eliminate the human race for access to scarce resources,[12][24] and humans would be powerless to stop them.[25] Alternatively, AIs developed under evolutionary pressure to promote their own survival could outcompete humanity.[16]
Bostrom (2002) discusses human extinction scenarios, and lists superintelligence as a possible cause:
When we create the first superintelligent entity, we might make a mistake and give it goals that lead it to annihilate humankind, assuming its enormous intellectual advantage gives it the power to do so. For example, we could mistakenly elevate a subgoal to the status of a supergoal. We tell it to solve a mathematical problem, and it complies by turning all the matter in the solar system into a giant calculating device, in the process killing the person who asked the question.
A significant problem is that unfriendly artificial intelligence is likely to be much easier to create than friendly AI. While both require large advances in recursive optimisation process design, friendly AI also requires the ability to make goal structures invariant under self-improvement (or the AI could transform itself into something unfriendly) and a goal structure that aligns with human values and does not automatically destroy the human race. An unfriendly AI, on the other hand, can optimize for an arbitrary goal structure, which does not need to be invariant under self-modification.[26]
Eliezer Yudkowsky proposed that research be undertaken to produce friendly artificial intelligence in order to address the dangers. He noted that the first real AI would have a head start on self-improvement and, if friendly, could prevent unfriendly AIs from developing, as well as providing enormous benefits to mankind.[15]
Bill Hibbard (2014) proposes an AI design that avoids several dangers including self-delusion,[27] unintended instrumental actions,[10][28] and corruption of the reward generator.[28] He also discusses social impacts of AI[29] and testing AI.[30] His 2001 book Super-Intelligent Machines advocates the need for public education about AI and public control over AI. It also proposed a simple design that was vulnerable to corruption of the reward generator.
One hypothetical approach towards attempting to control an artificial intelligence is an AI box, where the artificial intelligence is kept constrained inside a simulated world and not allowed to affect the external world. However, a sufficiently intelligent AI may simply be able to escape by outsmarting its less intelligent human captors.[31][32][33]
Stephen Hawking said in 2014 that "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks." Hawking believes that in the coming decades, AI could offer "incalculable benefits and risks" such as "technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand." Hawking believes more should be done to prepare for the singularity:[34]
So, facing possible futures of incalculable benefits and risks, the experts are surely doing everything possible to ensure the best outcome, right? Wrong. If a superior alien civilisation sent us a message saying, "We'll arrive in a few decades," would we just reply, "OK, call us when you get here – we'll leave the lights on"? Probably not – but this is more or less what is happening with AI.

See also

References






  • Ehrlich, Paul. The Dominant Animal: Human Evolution and the Environment

  • Bibliography


    External links






  • Superbrains born of silicon will change everything.

  • "What is the Singularity? | Singularity Institute for Artificial Intelligence". Singinst.org. Retrieved 2011-09-09.

  • David Chalmers John Locke Lecture, 10 May, Exam Schools, Oxford, Presenting a philosophical analysis of the possibility of a technological singularity or "intelligence explosion" resulting from recursively self-improving AI.

  • The Singularity: A Philosophical Analysis, David J. Chalmers

  • "ITRS" (PDF). Retrieved 2011-09-09.

  • Siracusa, John (2009-08-31). "Mac OS X 10.6 Snow Leopard: the Ars Technica review". Arstechnica.com. Retrieved 2011-09-09.

  • Eliezer Yudkowsky, 1996 "Staring into the Singularity"

  • Eliezer S. Yudkowsky. "Power of Intelligence". Yudkowsky. Retrieved 2011-09-09.

  • Omohundro, Stephen M., "The Basic AI Drives." Artificial General Intelligence, 2008 proceedings of the First AGI Conference, eds. Pei Wang, Ben Goertzel, and Stan Franklin. Vol. 171. Amsterdam: IOS, 2008

  • "Artificial General Intelligence: Now Is the Time". KurzweilAI. Retrieved 2011-09-09.

  • Omohundro, Stephen M., "The Nature of Self-Improving Artificial Intelligence." Self-Aware Systems. 21 Jan. 2008. Web. 07 Jan. 2010.

  • Barrat, James (2013). "6, "Four Basic Drives"". Our Final Invention (First Edition. ed.). New York: St. Martin's Press. pp. 78–98. ISBN 978-0312622374.

  • "Max More and Ray Kurzweil on the Singularity". KurzweilAI. Retrieved 2011-09-09.

  • "Concise Summary | Singularity Institute for Artificial Intelligence". Singinst.org. Retrieved 2011-09-09.

  • Bostrom, Nick, The Future of Human Evolution, Death and Anti-Death: Two Hundred Years After Kant, Fifty Years After Turing, ed. Charles Tandy, pp. 339–371, 2004, Ria University Press.

  • Shulman, Carl; Anders Sandberg (2010). Mainzer, Klaus, ed. "Implications of a Software-Limited Singularity" (PDF). ECAP10: VIII European Conference on Computing and Philosophy. Retrieved 17 May 2014.

  • Muehlhauser, Luke; Anna Salamon (2012). "Intelligence Explosion: Evidence and Import". In Amnon Eden, Johnny Søraker, James H. Moor, and Eric Steinhart. Singularity Hypotheses: A Scientific and Philosophical Assessment (PDF). Springer.

  • Robin Hanson, "Economics Of The Singularity", IEEE Spectrum Special Report: The Singularity, retrieved 2008-09-11 & Long-Term Growth As A Sequence of Exponential Modes

  • Ethical Issues in Advanced Artificial Intelligence, Nick Bostrom, in Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, Vol. 2, ed. I. Smit et al., Int. Institute of Advanced Studies in Systems Research and Cybernetics, 2003, pp. 12–17

  • Eliezer Yudkowsky: Artificial Intelligence as a Positive and Negative Factor in Global Risk. Draft for a publication in Global Catastrophic Risk from August 31, 2006, retrieved July 18, 2011 (PDF file)

  • The Stamp Collecting Device, Nick Hay

  • 'Why we should fear the Paperclipper', 2011-02-14 entry of Sandberg's blog 'Andart'

  • Omohundro, Stephen M., "The Basic AI Drives." Artificial General Intelligence, 2008 proceedings of the First AGI Conference, eds. Pei Wang, Ben Goertzel, and Stan Franklin. Vol. 171. Amsterdam: IOS, 2008.

  • de Garis, Hugo. "The Coming Artilect War", Forbes.com, 22 June 2009.

  • Coherent Extrapolated Volition, Eliezer S. Yudkowsky, May 2004

  • Hibbard, Bill (2012), "Model-Based Utility Functions", Journal of Artificial General Intelligence 3: 1, arXiv:1111.3934, Bibcode:2012JAGI....3....1H, doi:10.2478/v10229-011-0013-5.

  • Avoiding Unintended AI Behaviors. Bill Hibbard. 2012 proceedings of the Fifth Conference on Artificial General Intelligence, eds. Joscha Bach, Ben Goertzel and Matthew Ikle. This paper won the Machine Intelligence Research Institute's 2012 Turing Prize for the Best AGI Safety Paper .

  • Hibbard, Bill (2008), "The Technology of Mind and a New Social Contract", Journal of Evolution and Technology 17.

  • Decision Support for Safe AI Design|. Bill Hibbard. 2012 proceedings of the Fifth Conference on Artificial General Intelligence, eds. Joscha Bach, Ben Goertzel and Matthew Ikle.

  • Yudkowsky, Eliezer (2008), Bostrom, Nick; Cirkovic, Milan, eds., "Artificial Intelligence as a Positive and Negative Factor in Global Risk" (PDF), Global Catastrophic Risks (Oxford University Press): 303, Bibcode:2008gcr..book..303Y, ISBN 978-0-19-857050-9

  • Artificial Intelligence Will Kill Our Grandchildren (Singularity), Dr Anthony Berglas

  • The Singularity: A Philosophical Analysis David J. Chalmers
  • end quote from:
  • ntelligence explosion

  • Wednesday, February 3, 2016


    Intelligence explosion:Wikipedia

    Intelligence explosion

    From Wikipedia, the free encyclopedia
    An intelligence explosion is the expected outcome of the hypothetically forthcoming technological singularity, that is, the result of man building artificial general intelligence (strong AI). Strong AI would be capable of recursive self-improvement leading to the emergence of superintelligence, the limits of which are unknown.
    The notion of an "intelligence explosion" was first described by Good (1965), who speculated on the effects of superhuman machines, should they ever be invented:
    Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
    Although technological progress has been accelerating, it has been limited by the basic intelligence of the human brain, which has not, according to Paul R. Ehrlich, changed significantly for millennia.[1] However, with the increasing power of computers and other technologies, it might eventually be possible to build a machine that is more intelligent than humanity.[2] If a superhuman intelligence were to be invented—either through the amplification of human intelligence or through artificial intelligence—it would bring to bear greater problem-solving and inventive skills than current humans are capable of. It could then design an even more capable machine, or re-write its own software to become even more intelligent. This more capable machine could then go on to design a machine of yet greater capability. These iterations of recursive self-improvement could accelerate, potentially allowing enormous qualitative change before any upper limits imposed by the laws of physics or theoretical computation set in.

    Contents

    Plausibility

    Most proposed methods for creating superhuman or transhuman minds fall into one of two categories: intelligence amplification of human brains and artificial intelligence. The means speculated to produce intelligence augmentation are numerous, and include bioengineering, genetic engineering, nootropic drugs, AI assistants, direct brain–computer interfaces and mind uploading. The existence of multiple paths to an intelligence explosion makes a singularity more likely; for a singularity to not occur they would all have to fail.[3]
    Hanson (1998) is skeptical of human intelligence augmentation, writing that once one has exhausted the "low-hanging fruit" of easy methods for increasing human intelligence, further improvements will become increasingly difficult to find. Despite the numerous speculated means for amplifying human intelligence, non-human artificial intelligence (specifically seed AI) is the most popular option for organizations trying to advance the singularity.[citation needed]
    Whether or not an intelligence explosion occurs depends on three factors.[4] The first, accelerating factor, is the new intelligence enhancements made possible by each previous improvement. Contrariwise, as the intelligences become more advanced, further advances will become more and more complicated, possibly overcoming the advantage of increased intelligence. Each improvement must be able to beget at least one more improvement, on average, for the singularity to continue. Finally the laws of physics will eventually prevent any further improvements.
    There are two logically independent, but mutually reinforcing, accelerating effects: increases in the speed of computation, and improvements to the algorithms used.[5] The former is predicted by Moore’s Law and the forecast improvements in hardware,[6] and is comparatively similar to previous technological advance. On the other hand, most AI researchers believe that software is more important than hardware.[citation needed]

    Speed improvements

    The first is the improvements to the speed at which minds can be run. Whether human or AI, better hardware increases the rate of future hardware improvements. Oversimplified,[7] Moore's Law suggests that if the first doubling of speed took 18 months, the second would take 18 subjective months; or 9 external months, whereafter, four months, two months, and so on towards a speed singularity.[8] An upper limit on speed may eventually be reached, although it is unclear how high this would be. Hawkins (2008), responding to Good, argued that the upper limit is relatively low;
    Belief in this idea is based on a naive understanding of what intelligence is. As an analogy, imagine we had a computer that could design new computers (chips, systems, and software) faster than itself. Would such a computer lead to infinitely fast computers or even computers that were faster than anything humans could ever build? No. It might accelerate the rate of improvements for a while, but in the end there are limits to how big and fast computers can run. We would end up in the same place; we'd just get there a bit faster. There would be no singularity.
    Whereas if it were a lot higher than current human levels of intelligence, the effects of the singularity would be enormous enough as to be indistinguishable (to humans) from a singularity with an upper limit. For example, if the speed of thought could be increased a million-fold, a subjective year would pass in 30 physical seconds.[3]
    It is difficult to directly compare silicon-based hardware with neurons. But Berglas (2008) notes that computer speech recognition is approaching human capabilities, and that this capability seems to require 0.01% of the volume of the brain. This analogy suggests that modern computer hardware is within a few orders of magnitude of being as powerful as the human brain.

    Intelligence improvements

    Some intelligence technologies, like seed AI, may also have the potential to make themselves more intelligent, not just faster, by modifying their source code. These improvements would make further improvements possible, which would make further improvements possible, and so on.
    This mechanism for an intelligence explosion differs from an increase in speed in two ways. First, it does not require external effect: machines designing faster hardware still require humans to create the improved hardware, or to program factories appropriately. An AI which was rewriting its own source code, however, could do so while contained in an AI box.
    Second, as with Vernor Vinge’s conception of the singularity, it is much harder to predict the outcome. While speed increases seem to be only a quantitative difference from human intelligence, actual improvements in intelligence would be qualitatively different. Eliezer Yudkowsky compares it to the changes that human intelligence brought: humans changed the world thousands of times more rapidly than evolution had done, and in totally different ways. Similarly, the evolution of life had been a massive departure and acceleration from the previous geological rates of change, and improved intelligence could cause change to be as different again.[9]
    There are substantial dangers associated with an intelligence explosion singularity. First, the goal structure of the AI may not be invariant under self-improvement, potentially causing the AI to optimise for something other than was intended.[10][11] Secondly, AIs could compete for the scarce resources mankind uses to survive.[12][13]
    While not actively malicious, there is no reason to think that AIs would actively promote human goals unless they could be programmed as such, and if not, might use the resources currently used to support mankind to promote its own goals, causing human extinction.[14][15][16]
    Carl Shulman and Anders Sandberg suggest that intelligence improvements (i.e., software algorithms) may be the limiting factor for a singularity because whereas hardware efficiency tends to improve at a steady pace, software innovations are more unpredictable and may be bottlenecked by serial, cumulative research. They suggest that in the case of a software-limited singularity, intelligence explosion would actually become more likely than with a hardware-limited singularity, because in the software-limited case, once human-level AI was developed, it could run serially on very fast hardware, and the abundance of cheap hardware would make AI research less constrained.[17] An abundance of accumulated hardware that can be unleashed once the software figures out how to use it has been called "computing overhang."[18]

    Impact

    Dramatic changes in the rate of economic growth have occurred in the past because of some technological advancement. Based on population growth, the economy doubled every 250,000 years from the Paleolithic era until the Neolithic Revolution. The new agricultural economy doubled every 900 years, a remarkable increase. In the current era, beginning with the Industrial Revolution, the world’s economic output doubles every fifteen years, sixty times faster than during the agricultural era. If the rise of superhuman intelligence causes a similar revolution, argues Robin Hanson, one would expect the economy to double at least quarterly and possibly on a weekly basis.[19]

    Superintelligence

    A superintelligence, hyperintelligence, or superhuman intelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. "Superintelligence" may also refer to the form or degree of intelligence possessed by such an agent.
    Technology forecasters and researchers disagree about when human intelligence is likely to be surpassed. Some argue that advances in artificial intelligence (AI) will probably result in general reasoning systems that lack human cognitive limitations. Others believe that humans will evolve or directly modify their biology so as to achieve radically greater intelligence. A number of futures studies scenarios combine elements from both of these possibilities, suggesting that humans are likely to interface with computers, or upload their minds to computers, in a way that enables substantial intelligence amplification.

    Existential risk

    Berglas (2008) notes that there is no direct evolutionary motivation for an AI to be friendly to humans. Evolution has no inherent tendency to produce outcomes valued by humans, and there is little reason to expect an arbitrary optimisation process to promote an outcome desired by mankind, rather than inadvertently leading to an AI behaving in a way not intended by its creators (such as Nick Bostrom's whimsical example of an AI which was originally programmed with the goal of manufacturing paper clips, so that when it achieves superintelligence it decides to convert the entire planet into a paper clip manufacturing facility).[20][21][22] Anders Sandberg has also elaborated on this scenario, addressing various common counter-arguments.[23] AI researcher Hugo de Garis suggests that artificial intelligences may simply eliminate the human race for access to scarce resources,[12][24] and humans would be powerless to stop them.[25] Alternatively, AIs developed under evolutionary pressure to promote their own survival could outcompete humanity.[16]
    Bostrom (2002) discusses human extinction scenarios, and lists superintelligence as a possible cause:
    When we create the first superintelligent entity, we might make a mistake and give it goals that lead it to annihilate humankind, assuming its enormous intellectual advantage gives it the power to do so. For example, we could mistakenly elevate a subgoal to the status of a supergoal. We tell it to solve a mathematical problem, and it complies by turning all the matter in the solar system into a giant calculating device, in the process killing the person who asked the question.
    A significant problem is that unfriendly artificial intelligence is likely to be much easier to create than friendly AI. While both require large advances in recursive optimisation process design, friendly AI also requires the ability to make goal structures invariant under self-improvement (or the AI could transform itself into something unfriendly) and a goal structure that aligns with human values and does not automatically destroy the human race. An unfriendly AI, on the other hand, can optimize for an arbitrary goal structure, which does not need to be invariant under self-modification.[26]
    Eliezer Yudkowsky proposed that research be undertaken to produce friendly artificial intelligence in order to address the dangers. He noted that the first real AI would have a head start on self-improvement and, if friendly, could prevent unfriendly AIs from developing, as well as providing enormous benefits to mankind.[15]
    Bill Hibbard (2014) proposes an AI design that avoids several dangers including self-delusion,[27] unintended instrumental actions,[10][28] and corruption of the reward generator.[28] He also discusses social impacts of AI[29] and testing AI.[30] His 2001 book Super-Intelligent Machines advocates the need for public education about AI and public control over AI. It also proposed a simple design that was vulnerable to corruption of the reward generator.
    One hypothetical approach towards attempting to control an artificial intelligence is an AI box, where the artificial intelligence is kept constrained inside a simulated world and not allowed to affect the external world. However, a sufficiently intelligent AI may simply be able to escape by outsmarting its less intelligent human captors.[31][32][33]
    Stephen Hawking said in 2014 that "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks." Hawking believes that in the coming decades, AI could offer "incalculable benefits and risks" such as "technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand." Hawking believes more should be done to prepare for the singularity:[34]
    So, facing possible futures of incalculable benefits and risks, the experts are surely doing everything possible to ensure the best outcome, right? Wrong. If a superior alien civilisation sent us a message saying, "We'll arrive in a few decades," would we just reply, "OK, call us when you get here – we'll leave the lights on"? Probably not – but this is more or less what is happening with AI.

    See also

    References





  • Ehrlich, Paul. The Dominant Animal: Human Evolution and the Environment

  • Bibliography


    External links





  • Superbrains born of silicon will change everything.

  • "What is the Singularity? | Singularity Institute for Artificial Intelligence". Singinst.org. Retrieved 2011-09-09.

  • David Chalmers John Locke Lecture, 10 May, Exam Schools, Oxford, Presenting a philosophical analysis of the possibility of a technological singularity or "intelligence explosion" resulting from recursively self-improving AI.

  • The Singularity: A Philosophical Analysis, David J. Chalmers

  • "ITRS" (PDF). Retrieved 2011-09-09.

  • Siracusa, John (2009-08-31). "Mac OS X 10.6 Snow Leopard: the Ars Technica review". Arstechnica.com. Retrieved 2011-09-09.

  • Eliezer Yudkowsky, 1996 "Staring into the Singularity"

  • Eliezer S. Yudkowsky. "Power of Intelligence". Yudkowsky. Retrieved 2011-09-09.

  • Omohundro, Stephen M., "The Basic AI Drives." Artificial General Intelligence, 2008 proceedings of the First AGI Conference, eds. Pei Wang, Ben Goertzel, and Stan Franklin. Vol. 171. Amsterdam: IOS, 2008

  • "Artificial General Intelligence: Now Is the Time". KurzweilAI. Retrieved 2011-09-09.

  • Omohundro, Stephen M., "The Nature of Self-Improving Artificial Intelligence." Self-Aware Systems. 21 Jan. 2008. Web. 07 Jan. 2010.

  • Barrat, James (2013). "6, "Four Basic Drives"". Our Final Invention (First Edition. ed.). New York: St. Martin's Press. pp. 78–98. ISBN 978-0312622374.

  • "Max More and Ray Kurzweil on the Singularity". KurzweilAI. Retrieved 2011-09-09.

  • "Concise Summary | Singularity Institute for Artificial Intelligence". Singinst.org. Retrieved 2011-09-09.

  • Bostrom, Nick, The Future of Human Evolution, Death and Anti-Death: Two Hundred Years After Kant, Fifty Years After Turing, ed. Charles Tandy, pp. 339–371, 2004, Ria University Press.

  • Shulman, Carl; Anders Sandberg (2010). Mainzer, Klaus, ed. "Implications of a Software-Limited Singularity" (PDF). ECAP10: VIII European Conference on Computing and Philosophy. Retrieved 17 May 2014.

  • Muehlhauser, Luke; Anna Salamon (2012). "Intelligence Explosion: Evidence and Import". In Amnon Eden, Johnny Søraker, James H. Moor, and Eric Steinhart. Singularity Hypotheses: A Scientific and Philosophical Assessment (PDF). Springer.

  • Robin Hanson, "Economics Of The Singularity", IEEE Spectrum Special Report: The Singularity, retrieved 2008-09-11 & Long-Term Growth As A Sequence of Exponential Modes

  • Ethical Issues in Advanced Artificial Intelligence, Nick Bostrom, in Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, Vol. 2, ed. I. Smit et al., Int. Institute of Advanced Studies in Systems Research and Cybernetics, 2003, pp. 12–17

  • Eliezer Yudkowsky: Artificial Intelligence as a Positive and Negative Factor in Global Risk. Draft for a publication in Global Catastrophic Risk from August 31, 2006, retrieved July 18, 2011 (PDF file)

  • The Stamp Collecting Device, Nick Hay

  • 'Why we should fear the Paperclipper', 2011-02-14 entry of Sandberg's blog 'Andart'

  • Omohundro, Stephen M., "The Basic AI Drives." Artificial General Intelligence, 2008 proceedings of the First AGI Conference, eds. Pei Wang, Ben Goertzel, and Stan Franklin. Vol. 171. Amsterdam: IOS, 2008.

  • de Garis, Hugo. "The Coming Artilect War", Forbes.com, 22 June 2009.

  • Coherent Extrapolated Volition, Eliezer S. Yudkowsky, May 2004

  • Hibbard, Bill (2012), "Model-Based Utility Functions", Journal of Artificial General Intelligence 3: 1, arXiv:1111.3934, Bibcode:2012JAGI....3....1H, doi:10.2478/v10229-011-0013-5.

  • Avoiding Unintended AI Behaviors. Bill Hibbard. 2012 proceedings of the Fifth Conference on Artificial General Intelligence, eds. Joscha Bach, Ben Goertzel and Matthew Ikle. This paper won the Machine Intelligence Research Institute's 2012 Turing Prize for the Best AGI Safety Paper .

  • Hibbard, Bill (2008), "The Technology of Mind and a New Social Contract", Journal of Evolution and Technology 17.

  • Decision Support for Safe AI Design|. Bill Hibbard. 2012 proceedings of the Fifth Conference on Artificial General Intelligence, eds. Joscha Bach, Ben Goertzel and Matthew Ikle.

  • Yudkowsky, Eliezer (2008), Bostrom, Nick; Cirkovic, Milan, eds., "Artificial Intelligence as a Positive and Negative Factor in Global Risk" (PDF), Global Catastrophic Risks (Oxford University Press): 303, Bibcode:2008gcr..book..303Y, ISBN 978-0-19-857050-9

  • Artificial Intelligence Will Kill Our Grandchildren (Singularity), Dr Anthony Berglas

  • The Singularity: A Philosophical Analysis David J. Chalmers
  • end quote from:
  • https://en.wikipedia.org/wiki/Intelligence_explosion 
  •  
  • Since I'm being prevented from making clear comments at the beginning of my reprint I thought I would try again here.

    Intelligence Explosion has been either deleted or renamed at Wikipedia because when I looked for it it wasn't there any more today. I'm thinking it was freaking to many people out. However, my point of view is it is better to be prepared than to lose your life to something you just don't understand at all through the obvious Technological singularity we have presently entered. And Trump succeeding just shows me how aberrant the reactions of people are to global Climate change and the present Technological Singularity we have now entered worldwide.

  • The only question is when will this Singularity peak. I'm thinking 2025 to 2045 at present.

  • And what does a peak mean? It means will the human race survive this or not?

  • No comments: