What if automating AI R&D triggers an intelligence explosion?

This could be an interesting new research paper by Geoffrey Hinton, Yoshua Bengio, Thore Graepel, Philip H. S. Torr, Jeff Clune their team.

I would say the fear that the progress of AI could get out of control is quite overblown. Humans can handle and manage it and they will grow with it. We have done it before with nuclear power and genetics.

Keep government intervention to a minimum!

Maybe in a few years we will see super intelligent humans.

"In contrast to even a year ago, AI systems now write most of the code inside the companies that build them.

As more of the AI research and development (R&D) pipeline is automated, could AI progress radically accelerate in an "intelligence explosion," where years of advances are compressed into months or less?

Preliminary evidence suggests that it could. In this work, we assess this evidence, analyze an intelligence explosion's potential impacts, and propose policy responses. AI systems are on track to automate most AI R&D work within a few years, and possibly all of it. If this triggers an intelligence explosion, it could dramatically bring forward AI's benefits, but also pose extreme risks: capabilities growth could accelerate far beyond what society can keep up with, humanity could lose control over superhuman AI systems, and checks on power within and between states, companies, and branches of government could be severely eroded.

Although there remains much uncertainty about these possibilities, the high stakes warrant serious further attention. Policymakers should urgently obtain more visibility into the automation of AI R&D, develop ways to steer and constrain an intelligence explosion, and prepare society to adapt to an intelligence explosion's impacts."

[2609.36054] What if automating AI R&D triggers an intelligence explosion?

Global human population is pushing Earth past its breaking point. Really!

Such or similar false apocalyptic claims have been made frequently for the past 50 years or so!

Human ingenuity has already defeated many of those and other apocalyptic claims in the past!

Look at the chart below, this is primitive trend extrapolation. Yes, the growth of human population since about 1800 AD was extreme and a miracle, but it will not continue like this!

This is superstition, not science! Alarmism and hysteria! Perhaps very lucrative for these pseudo scientists behind this study!

When scientists call for authoritarian central government watch out!

I am not sure why any journal today would publish such rubbish again!

Actually, world population may even decrease over the next several decades. The population of e.g. China and Japan is already shrinking. E.g. Germany's German population may also be shrinking. I bet also India's population growth is already slowing and so on.

"Earth has already exceeded its ability to support the global population sustainably, with new research warning of increasing pressure on food security, climate stability, and human well-being. However, slowing population growth and raising global awareness could still offer humanity some hope. ..."

From the abstract:

"The ecological concept of human carrying capacity is necessarily complicated because human beings are the ‘ultimate ecosystem engineers’ who moderate the environment for their benefit.

For at least the last few hundred years, human ingenuity, access to massive stocks of fossil fuels, and technological development have driven facilitation whereby increasing human abundance has promoted higher population growth rates.

However, this positive relationship broke down during the 1950s, and by 1962, the global human population entered a phase where the growth rate consistently declined as population increased.

The onset of this negative phase occurred 8 years before a global biocapacity deficit began in 1970. The onset of the negative phase also varies regionally, with the lowest-income and highest fertility regions entering this phase later than higher-income regions.

A Ricker logistic model fitted to the negative phase predicts that the global population could reach 11.7–12.4 billion people between 2067 and 2076. The same model fitted to the facilitation phase predicts a maximum population of 2.5 billion people that Earth might be able to maintain [???]. 

The negative phase also correlates strongly with the trend in global temperature anomaly, ecological footprint, and total emissions, with more of their variation explained by increasing population size rather than increasing per-capita consumption.

The Earth cannot sustain the future human population, or even today’s, without a major overhaul of socio-cultural practices [???] for using land, water, energy, biodiversity, and other resources."

Global human population is pushing Earth past its breaking point

Global human population has surpassed Earth's sustainable carrying capacity

Figure 1. Global human population size from 1000 to present (main figure) and from 1800 to present

US DNI Director Tulsi Gabbard On Her Visit To Hiroshima and warning of the imminent threat of a nuclear holocaust

What made her release such a strange video at this time?

She is the United States Director of National Intelligence. Does she know something we don't?

Is she referencing Putin the Terrible of the mullahs in Iran or the dictator in North Korea?

City rats are on the rise, and climate change and urban population growth are to blame. Really!

More alarmism and hysteria from the AAAS! Another case of junk science!

Like in the Medieval Ages for the American Association for the Advancement of Science climate change explains everything! 

Apparently, the AAAS is not aware of the urban heat island effect!

What about the lack of the usual, natural predators in urban areas?

"The warmer a city gets and the more people who move into it, the worse the rat infestations will likely become. That’s the conclusion of a first-ever global analysis of how and why rat numbers have changed over time. Smart, cooperative, and resilient, rats have co-evolved with humans for millennia and have fine-tuned their ability to take advantage of garbage, debris piles, sewers, and small postage-stamp sized plots of soil along sidewalks for food and nesting. The animals can transmit disease, spoil food and animal feed supplies—costing the United States $27 billion per year—and cause mental anguish in city dwellers.

To get a better handle on the problem, Jonathan Richardson and his team compiled inspection, trapping, and rat sighting records kept by 16 cities over an average of 12 years and compared trends in rat populations to how much temperatures had risen, how much more crowded the municipalities became and what was happening to parks, open lots and other green spaces.

Cities that experienced greater rates of temperature increase and more people moving in were more likely to have bigger rat problems; climate change emerged as a driving factor behind urban rat swarms, the researchers report. Disappearing green space also seemed to benefit rat populations."

From the abstract:

"Urban rats are commensal pests that thrive in cities by exploiting the resources accompanying large human populations. Identifying long-term trends in rat numbers and how they are shaped by environmental changes is critical for understanding their ecology, and projecting future vulnerabilities and mitigation needs.

Here, we use public complaint and inspection data [???] from 16 cities around the world to estimate trends in rat populations. Eleven of 16 cities (69%) had significant increasing trends in rat numbers, including Washington D.C., New York, and Amsterdam. Just three cities experienced declines. Cities experiencing greater temperature increases over time saw larger increases in rats. Cities with more dense human populations and more urbanization also saw larger increases in rats. Warming temperatures and more people living in cities may be expanding the seasonal activity periods and food availability for urban rats. Cities will have to integrate the biological impacts of these variables into future management strategies."

ScienceAdviser

Warmer, more crowded cities bring out the rats "A global study of urban rat populations explores why some urban environments are more infested than others"

Increasing rat numbers in cities are linked to climate warming, urbanization, and human population

When you combine two myth like the Global Warming hoax and the Population Bomb (published 1968)!

'Godfather of AI' Geoffrey Hinton (age 76) speaks on threat of tech surpassing humanity. Really!

Sometimes old men worry too much! Pardon my sarcasm!

Or did Nikkei misrepresent his opinion? The article is behind paywall.

So what! With this kind of thinking we would be still living in the Stone Ages! I have full confidence that younger generations will learn how to deal with AI! The potential of benefits far outweigh the risks!

'Godfather of AI' speaks on threat of tech surpassing humanity - Nikkei Asia Geoffrey Hinton believes AI is already having experiences akin to humans'

Paul R. Ehrlich, der absurdeste Weltuntergangsprophet

Empfehlenswert!

Mormon state battles ‘biblical’ grasshopper invasion

Headline of the day! A day of judgement for the polygamous sect (just kidding, no insult intended, mea culpa)? 😊

"Earlier this spring, Neil Sorensen’s farm looked downright apocalyptic.

Juvenile grasshoppers covered almost every square inch. They leapt in swarms as he walked through his grain and alfalfa fields. They even blanketed the bare ground. It’s the worst infestation the Sanpete County-based grower said he’s ever seen. ..."

Here’s what Utahns are doing to battle ‘biblical’ grasshopper invasion

A Collection of Writings on the Religious Character of the Environmental and Climate Movement

This might be interesting reading! The hoarse horsemen and horsewomen of the apocalypse!

".. through and around the UN’s signature annual [United Nations Climate Change Conference], describing the jockeying among the climate elite for prominence inside the conference and relaying the apocalyptic religiosity of the activists – like “Brother Spirit” – who make up the true believers in the climate movement. He introduces readers to Les U. Knight, the founder of the Voluntary Human Extinction Movement, who is “mainly concerned with speciecide, annihilating Homo sapiens and leaving behind a planet liberated even from the memory that human beings had once existed.” And he explains why nuclear power offers a zero-carbon alternative to fossil fuels available right now, but the Biden Administration is only supporting projects in countries like Romania ..."

CEI Publishes Collection of Kevin D. Williamson Writing on the Religious Character of the Environmental Movement - Competitive Enterprise Institute

The Four Horsemen, from "The Apocalypse" by Albrecht Dürer

The hot mess theory of AI misalignment: More intelligent agents behave less coherently. Really!

This hot and messy blog post was written by a moderately cited worry wart of a machine learning researcher working for Google, who has written/co-authored about 130 papers on machine learning! He is also one of the researchers I am following in my studies of AI/machine learning.

How does it feel to be turned into a "paper clip" or what is a "toilet tank mechanism" you have to "jiggle just right"? The Walt Disney fantasy of an  AI researchers!

See also my very recent post touching on similar issues regarding AI.

It is not up to Jascha or his compatriots "to get to wonderful outcomes" whatever this might be!

Elitist PhD experts like Jascha will also not get to define what is misaligned with human values!

As far as his exaggerated fear of "a subtle misalignment might pose an existential risk to humanity" is concerned, he might be too young to realize humanity managed so far very well to live with ABC weapons! The suspected lab leak of SARS-Cov-2/Covid-19 also did not kill all humans! However, still existing totalitarian dictatorships like in Russia, China, or Iran pose a serious risk to the potential of AI being used for malicious purposes.

The young man is also afraid of superintelligent machines! Scary or is it more like alarmism and hysteria? Is it not more likely that superintelligence will benefit humans too by becoming themselves superintelligent like in symbiosis?

"Many machine learning researchers worry about risks from building artificial intelligence (AI). This includes me ...and we will need to work hard to get to the wonderful outcomes ...
One popular AI risk centers on AGI misalignment. It posits that we will build a superintelligent, super-capable, AI, but that the AI's objectives will be misspecified and misaligned with human values. ... a subtle misalignment might pose an existential risk to humanity ...
I am extremely glad people are worrying about and trying to prevent negative consequences from AI. ...
I am extremely glad people are worrying about and trying to prevent negative consequences from AI. ...
An accidentally misaligned superintelligence which poses an existential risk to humanity seems about as likely as any other specific hypothesis for the future which relies on a dependency chain of untested assumptions. ..."

The hot mess theory of AI misalignment: More intelligent agents behave less coherently | Jascha’s blog

Credits: Synced Global AI Weekly

Following is a very nice cartoon! 😊 Imagined corner cases, oh well! Nice exercise!

Figure 1: The space of intelligence and coherence. Each corner corner represents an extreme of intelligence and coherence, and is labeled with an example of a machine demonstrating those attributes.

Closest Known Asteroid to Fly By Earth on 8/16/2020

That was a near miss! Would the asteroid have completely burnt up in earth's atmosphere? How many close by flights do we miss?

This asteroid was discovered by human researchers in India and Taiwan scanning that day's images. High time to employ computer vision and machine learning to do this automatically.

"On August 16, the ... Palomar Observatory ... spotted an asteroid that had, just hours earlier, traveled only 1,830 miles (2,950 kilometers) above Earth's surface. The previous known record-holder is asteroid 2011 CQ1, discovered by the Catalina Sky Survey in 2011, which passed above Earth about 1,550 miles (2,500 kilometers) higher than 2020 QG. Asteroid 2020 QG is about 10 to 20 feet (3 to 6 meters) across, or roughly the size of an SUV, so it was not big enough to do any damage even if it had been pointed at Earth; instead, it would have burned up in our planet's atmosphere. ... Asteroid 2020 QG was identified by Kunal Deshmukh, a student at the Indian Institute of Technology Bombay. Deshmukh had been scanning that day's images along with Kritti Sharma, also at the Indian Institute of Technology Bombay, and Chen-Yen Hsu at National Central University in Taiwan."


ZTF Finds Closest Known Asteroid to Fly By Earth | www.caltech.edu

Commonalities Between Conspiracy And Apocalyptic Theories

Posted: 10/18/2019

The commonalities are:

1. Both are typical human follies and obsessions
2. Both are produced again and again in abundance by humans
3. Both are usually unsubstantiated conjectures that are never verified once in circulation in the public opinion. Even successfully and credibly refuted theories live on, because plenty of humans prefer ignorance
4. Both have extremely long lifespans (possibly several decades) before they fade away or become obscure
5. Both reflect primitive, ancient and medieval thinking
6. Both were supposed to become close to extinct with the spread of Enlightenment and Rationalism. Here we are in the 19th year of the 21st century and both types of theories are still alive and kicking. Both are now spread much more easily through the Internet ...

Definition of insanity: To do the same thing over and over again and expecting a different outcome/result.

This definition also fully applies to conspiracy and apocalypse theories! 

Why are humans so obsessed with obsessions?

When Will The Genes Of Human Existential Threat Be Discovered?

Posted: 9/25/2015

Trigger

Yesterday, I listened as usual to National Public Radio while on my commute to work. Well, NPR interviewed an environmentalism expert on climate change in conjunction with the pope visit to the U.S. This man kept repeating at least three times or so during this short interview what an existential threat climate change was to humans.

Positive Gene Editing

If we only could identify and modify the genes responsible for why so many humans are afflicted by perceived and excessive existential threats, we could finally put all these doomsayers out of business.

The corollary would be to identify and strengthen the genes responsible for human optimism and ingenuity.

Doomsayers Finally Run Out Of Minutes

Posted: 1/25/2015

Trigger

Just read “Bulletin of the Atomic Scientists moves Doomsday Clock 2 minutes closer to midnight”

To quote from above article (emphasis added):

“The board that runs the Bulletin of the Atomic Scientists (BAS) has decided to move the minute hand on its symbolic Doomsday Clock 2 minutes closer to disaster. The clock now shows 3 minutes before midnight because the “probability of global catastrophe is very high” as a result of continuing climate change and efforts to modernize nuclear weapons stockpiles.”

Humans Have Been In Love With Apocalypse For Ages

There are too many Cassandras in this world and since ancient times. Which one is right? Human ingenuity trumps!