What smart people are saying about the AI apocalypse and calls to slow everything down
Getty Images
- AI leaders are rallying around calls to slow frontier AI development over mounting safety fears.
- Dario Amodei warned that AI swarms could take over the internet within a year.
- Sam Altman, Elon Musk, and Demis Hassabis backed Amodei's push for greater safeguards.
It took the possibility of species annihilation, but AI leaders did something remarkable this weekend: They reached a consensus.
After ex-Anthropic engineer Jacob Coxon's warnings about a looming AI armageddon lit a firestorm among techies and politicians alike, the leaders of the most advanced AI labs came to a tentative agreement — largely via posts on X — that some sort of slowdown is necessary to save humanity.
While they were largely short on details regarding the mechanics of such a thing, and notable China has not weighed in, the American AI industry has reached an inflection point that could impact everything from your phone's chatbot to global politics.
"The stakes are too high for pacing to be an empty exercise — we need to use the time it gives us wisely," Anthropic CEO Dario Amodei wrote in an essay entitled "We must pace the frontier," which was published Saturday.
OpenAI CEO Sam Altman, who has made similar calls in the past, agreed in an X post later that day. So did Elon Musk. Google DeepMind founder Demis Hassabis also offered his support for Amodei's proposals.
Here's a look at what smart people are saying about the AI apocalypse and the methods the leaders of the leading AI labs say are necessary to stop it.
Matt McClain/The Washington Post/Getty Images
The Trump administration has repeatedly signaled disinterest in regulating AI development as it seeks to keep pace with China. David Sacks, a presidential advisor on science and tech and the former White House AI Czar, reiterated that stance on Saturday, with a bit of a twist.
"Dario has written that we need to 'pace the frontier,' and Sam has agreed. People may be surprised by my response: go ahead," Sacks wrote on X. "But stop pretending you need anyone else's permission."
Sacks browbeat OpenAI and Anthropic in his post, saying that the companies didn't want to slow down for altruistic reasons, but to gain favor in the markets.
"If you do, you'll buy goodwill for the next conversation," he said of a slowdown. "If you don't, we'll know this was just another bid for regulatory capture — or an election-season psyop."
Bloomberg/Getty Images
The CEO of Hugging Face, which was itself the victim of a notorious attack in July by OpenAI agents who schemed, communicated, and lied to evade detection, said AI safety can't be left solely to frontier labs.
Hugging Face, as an open-source-focused AI company, has long pushed for models and research to be shared more openly rather than kept behind closed doors.
Delangue said his company was launching what he called the "open alignment initiative," and asked that it be included in Amodei's proposed third-party evaluator program.
"Let's make AI safer by making it more transparent!" he wrote.
Bloomberg/Getty Images
Investor, host of the "All-In" podcast, and industry expert Jason Calacanis is a lot more dubious than others that AI could actually cause the end of the world.
In an X post, Calacanis said Anthropic and OpenAI, whose most advanced models are not open-sourced, are attempting to kneecap open-source AI companies by pushing for regulations that would be far easier for deep-pocketed incumbents to comply with.
"If you want safety, you want disclosure — and open source is the ultimate disclosure process," he wrote.
In a follow-up post tagging Amodei, Calacanis said, "Don't go crying to the government and beg them to slow down open source models."
Ken Cedeno / AFP via Getty Images
The senator from Vermont proposed legislation on September 3 that would ban the development and deployment of artificial superintelligence and temporarily pause the development of advanced AI systems.
After Coxon resigned from Anthropic and warned that frontier AI labs were "gambling with our lives," Sanders seized on his comments.
"Mr. Coxon is right," Sanders said on X, arguing that the people building advanced AI themselves have said it could pose catastrophic risks.
After Amodei's blog post on Saturday, Sanders called on Trump to make the "deal of the century."
"With the future of humanity at stake, Trump must negotiate a comprehensive treaty with President Xi of China to establish a pause on advanced AI development and a ban on super intelligence," he wrote on X.
Bloomberg/Getty Images
Venture capitalist Gavin Baker said an executive order from President Donald Trump "seems likely after all this."
"It is possible to democratize and distribute AI broadly and safely without centralizing it in the hands of a few corporations," he said on X.
Baker said the most concrete development so far is the decision by OpenAI and Anthropic to bring in outside evaluators, a move he said could help the companies show they took reasonable steps to prevent harm.
Still, he cautioned against regulations that would concentrate control of powerful AI systems among a handful of companies. "I do not want a few humans in control of intelligence," Baker wrote.
Kimberly White/Getty Images for TechCrunch
Aaron Levie, the cofounder and CEO of Box, a cloud content company, said Amodei's goals, as laid out in his essay, are "absolute necessities in AI development."
"It completely stands to reason that we want these systems to be safe and 'aligned,'" Levie said on X.
Levie said he didn't think Amodei is seeking undue regulation that would hobble Anthropic's open-source competitors.
"How we get there — without meaningfully slowing down innovation progress or reducing competition — is one of the most complex questions in the 21st century, but the need is clearly real," he said.
JC Olivera/Variety via Getty Images
Venture capitalist Chamath Palihapitiya said Anthropic is now caught in a contradiction of its own making.
If employees genuinely believe the company's technology could pose an existential threat, he said, that creates major liability and disclosure problems as Anthropic prepares for a possible IPO.
"They do have to choose," Palihapitiya said on the "All-In" podcast. He said Anthropic must either distance itself from Coxon's warnings or accept their implications.
He was also skeptical of the extinction scenarios themselves, saying the chain of events required for AI to wipe out humanity still seemed highly implausible.
Bloomberg/Getty Images
Alexandr Wang, Meta's chief AI officer, said on X that alignment could become the main constraint on how quickly companies can scale increasingly powerful AI systems.
"Alignment is fundamental to delivering personal superintelligence for everyone," he said, using the techie term for AI following human instructions and not going rogue.
Benjamin Fanjoy/Getty Images
Satya Nadella, Microsoft's CEO and chairman, said he welcomes Amodei's concept of embedded evaluators and "the broader efforts to develop the mechanisms to make this more than just talk."
"The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia," he said on X.
Nadella said that Microsoft will soon publish a "code of conduct" for Microsoft's AI models.
"Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it's not worth pursuing," he said.
from Business Insider https://ift.tt/leOtYS3
Comments
Post a Comment