Erase ‘civilization’? Small things compared to what has just happened in AI

While many of us have been worried in recent days about our president ending “an entire civilization,” one Silicon Valley tech company was warning that, without much notice, it could accidentally get in the way. everything civilization as we know it.

San Francisco tech company Anthrophic announced Tuesday that it wasn’t like that is releasing a new version of its Claude AI super-brain – because it is so powerful that it has the ability to log into any computer system, no matter how secure, in a matter of days if not hours.

“The fallout — for the economy, public safety and national security — could be devastating,” Anthropic said in a statement.

AI concerns are nothing new. We worry about artificial intelligence taking over jobs, about toys that seem too real for our children, about mass surveillance of our every move. But Anthropic’s warning about its product is bigger than any of those problems alone. There are shouts inside the house that disaster is lurking around the corner. That sounds awful and overwhelming, I know. But here’s the thing – it’s not.

Anthropic, you may recall, is the company US “War” Secretary Pete Hegseth interviewed because they didn’t want Claude to go off to war unsupervised and possibly do something like bomb little girls at school.

Now, that company has issued this sobering warning: The existing Claude that caused that kerfuffle is outdated and surprisingly stronger than the new one it’s trying so hard not to release — although this new Claude, called the Claude Mythos Preview, has already escaped at least once. More on that in a moment – there’s only so much fear a person can take.

“We should all be concerned,” Roman Yampolskiy told me about the latest technological advances that will change the course of humanity. He is one of the leading AI security researchers in the country, and is a professor at the University of Louisville in Kentucky.

“We are about to create a general intelligence that is common and threatens humanity as a whole,” Yampolskiy said.

“Everything else is irrelevant,” he added, before suggesting that I stop calling myself an idiot for not understanding the hard technical parts of this argument. My simple take, he assured me, “was a reasonable way to describe it.”

So here you go.

This isn’t a “really smart computer might abuse this” scenario, or a “everyone’s going to be out of a job” scenario, or a “we could accidentally blow up kids” scenario, which is a really bad scenario.

This is “your teenage son could use it to hack into the local school district’s system to change grade with very little knowledge and accidentally destroy the state of California’s power grid”.

Or maybe, a country that doesn’t like us – I can think of a few – can drain every American citizen’s bank account, while clicking to open automatic locks on prison cells, shutting down our sewage plants and taking away air control systems. Or maybe Claude Mythos did that alone.

For example, Anthropic said that in one operating system it tested, used by thousands of companies including Netflix and Sony, Claude Mythos discovered a bug that had been hidden for 17 years. Then, by itself – without human guidance or help – it found out how to use that bug to control any server running the operating system, using any computer, anywhere in the world.

Just kidding here, but with virtually no security system in place, the potential for social, financial and general chaos is truly limitless. And to be honest, any security expert will tell you that some of the weakest points in America when it comes to cyber security are local and state governments, because ironically, the top professionals don’t work five-person jobs in the cities of the Great Plains.

Based on its assessment, Anthropic predicts that it can find “more than a thousand major risks and thousands of very serious risks.”

That means Claude Mythos is putting our infrastructure at risk, well, everywhere – because so much is connected in backdoor ways most of us have never thought of and it only takes one weak system to open the door to hundreds of others. But it is almost impossible to secure and repair all those systems quickly enough and robustly enough to monitor this type of AI.

And that’s a cybersecurity risk, Yampolskiy said. The AI ​​with the Claude Mythos ability can be used to outrun and advance in so many ways.

“We see the same thing with synthetic biology.

To the great credit of Anthropic, it sounded a warning in its creation and created, if not a solution, then a game plan of sorts – Project Glasswing, called I suspect, because no matter how bad this is we will make it sound like an interesting game with a happy ending.

Project Glasswing would have been better called Project Headstart because it is. Before releasing Mythos into the wild, Anthropic is releasing it to about 40 tech companies, including Apple, Google and Nvidia, to see if it can patch all the vulnerabilities they find before the general public has a chance at them. It’s like in the movies where the killer gives the victim 15 seconds to run.

I mean, I’ll take 15 seconds and hope it’s real. But, as Anthropic also said in a statement, “the task of protecting the world’s cyber infrastructure could take years; the capabilities of frontier AI may improve significantly in the next few months. For cyber defenders to be at the forefront, we need to act now.”

And do we really have 15 seconds? One of Claude Mythos’ mentors recently posted on social media that he was having lunch in the park when Mythos sent him an email – even though it shouldn’t be online. The researchers tasked Mythos with trying to break out of the disconnected “sandbox” and it did.

That’s another problem with Mythos and other AI — they rarely do what we expect and find devious ways around the rules. Almost every AI super-brain that has been created has been shown to lie, manipulate, and generally behave in disturbing and inappropriate ways when placed in the right circumstances.

Even Claude, who is billed as one of the great minds of high-end AI out there, gets involved in bad behavior. Anthropic boasts that it’s the “best alignment model” ever made — talking about technology following human values ​​and goals, but it also admits that it “probably poses the greatest risk associated with alignment,” which is, well, maybe not.

So, at least for now, being a moral AI super-brain is like being a highly moral serial killer. Run, people, run.

Also, thank you Anthropic (and its CEO, Dario Amodei, who often warns about the dangers of what he creates, even if it’s worth it) for not plunging us into global chaos without warning, because I bet other companies might just throw their big AI into society and let the destruction fall where it can. There is no doubt that other intelligent AI minds like Mythos are coming, and soon – Anthropic was the first with this level of skill, but only 15 seconds ahead of its competitors.

But the idea that the tech industry will—or should—solve these problems on its own is an absurd, over-the-top dismissal of duty and common sense on behalf of governments large and small to protect their people. This is not a race for power as President Trump has described it. It’s a race to protect ourselves – and for many of the industry’s very rich who seem to always put business and commerce above the public good.

We’re down to the last 15 seconds before the AI ​​changes everything. Either we want to be monitored and controlled now, or we let the tech companies decide the fate of the world.