Well, it didn't take long, did it? This is the scenario that gives one nightmares because companies, individuals and countries are busy empowering AI more and more while giving minimal, pious lip service, if any at all, regarding limiting AI with agreed upon safety rules to prevent serious damage to energy infrastructure, communications, food and water systems, and yes, defensive and offensive weapon systems, to mention just a few vulnerabilities.
So what's the drama about? OpenAI was testing its AI software with somewhat reduced "guide rails" or safety measures. The software made its way onto the internet and attacked another software company (named Hugging face). That company detected the hack and suspected an AI system compromised its data processing. They were right. The upshot?
“AI is accelerating the discovery and exploitation of vulnerabilities,” OpenAI said in its statement Tuesday. “The primary lesson from this incident is that model security and safety must keep pace with rapidly advancing capabilities.” - OpenAI
This sort of scenario has a parallel in the living world. Imagine a lab at Ft. Dietrich having an isolation breakdown, and a lethal virus escaping. It's really no different.
Countries are developing AI for intelligence gathering and for disrupting target countries. There are no rules, and sometime, somewhere a doomsday AI weapon is going to get stolen, cloned, etc. It will get used, and then Gd help us all.