AI is a Very Expensive Joke!
It's a messy time again. Messy, Messi, Messy. And I mean “Messy”. It’s crazy how AI has become a joke. A very expensive and costly joke. It’s such a costly joke that parents have lost their children, partners - their other halves, companies over $500 million in less than a month, the planet, major water losses, amongst others, and now we’ve just seen another episode of “America takes the biscuit” 🍪.
I really try not to sound too cynical in these newsletters, but it’s really difficult and gets harder by the day.
You must be living under a rock if you’re a professional and didn’t hear about the dramatic episode, this time not with Anthropic, but with their favourite cousin Open AI; who decided to have a partner this time in the open - source company, Hugging Face.
While the various events coming from the AI labs in the US are sometimes comical, the implications are so grave that we can’t just laugh them off.
So what happened this time? I’ll keep it brief and focus on the implications.
Anthropic released Fable 5, portrayed as one of the most powerful AI technologies that ever existed a few months ago.
Open AI had to come up with a marketing strategy to announce their competitor to Fable 5: GPT 5.6 Sol.
Open AI claims a couple of models escaped their sandbox environment during testing including Sol.
Hugging Face reported multiple security breaches as they discovered thousands of infiltrations on their internal systems over the weekend of the incident.
Open AI came out to say, “Hey that was us actually. It was our models that went a bit rogue”.
Hugging Face had to run to China for help by using their open source models to detect the security breach to their systems, as the closed source models in the US have heavy guardrails that didn’t provide any help.
Nvidia, and a few other AI labs in the US are now calling for open-source models rather than closed weights due to this incident.
Literally every news channel in the Western world covered this story, emphasising the unpredictable and uncontrollable nature of AI agents.
And so on, and so forth.
Apart from having a headache with all this back and forth from overly dramatic American companies who 1) Don’t have any experience with software engineering or security controls and ,2) Are not obliged to, while they’re continuously creating world-changing and life-altering technologies, otherwise known as AI; I just get plain annoyed that a technology this powerful is still not confined to a lab, but has been released to the general public for a very long time now, without having the right governance, regulations or guardrails in place. Like, this is just plain wrong.
First of all, let’s take a step back and pause for a moment, and take all of this in. Humans have successfully developed a technology that can make decisions on its own, and even escape human oversight and control. This is what we call AI agents. Agents of Chaos or Agents of Good is a question that remains unanswered.
Now back to the somewhat irritating debacle that happened between Open AI and Hugging Face, who I heard “hugged” afterwards, while “facing” each other. Sorry I couldn’t resist.
Focusing on the important details of this scenario, Open AI was testing some new models, and didn’t put security controls or guardrails in place. They either intentionally allowed the agents to go rogue to see what extent they could go, or they didn’t expect any foul play. Regardless of their intentions, the events that followed made a lot of people worried, and rightfully so.
So what could be done about this?
Well, Demis Hassabis, the CEO of Google Deepmind and Nobel Prize winner just shared his thoughts about the importance of US control and regulation on highly capable frontier models, a couple of days before this incident. (Talk about clairvoyance) He proposed a US owned framework that tests and reviews frontier models 30 days before release. A proposal that I agree with, although I question the US-centricity and call for a more broader set of controls and oversight.
While China is a heavily regulated country which hasn’t spared AI in its regulation (for the better in my opinion), it’s still head-to-head in a savage competitive geopolitical cold war with the US with regards to AI. So ideally, since the only 2 leading global powers on AI are China and the US, the US should indeed up their game in ensuring adequate evaluations and tests are carried out before the wide release of any highly capable AI systems.
Now imagine if the US already had their ducks in a row, and had an established governance and safety framework, mandating the review and evaluation of any frontier models before release, this debacle between Open AI and Hugging Face would not have happened.
The stakes are really high, and we’re lucky that Hugging Face was the victim and not a critical organisation such as a hospital or power grid where the consequences could have been fatal.
Due to this recent event, the industry is now changing tune, and Nvidia is leading a movement calling for a change from closed-source models to open-source models, an argument that has existed for a very long time.
But if the US wants to maintain a competitive edge over China, they have to go open-source as that’s China’s competitive advantage right now, as well as their cheap pricing.
If the industry eventually decides to open model weights, it’d be a clear win for AI transparency and ultimately AI safety, as the research communities and startups will be able to evaluate and work with AI models more closely. But of course, as with everything in the world of AI, there are risks involved.
So what do you think? Will AI models finally become open-source? And if so what does that mean to the never-ending threats of national security and bio-risks above others.
Does this also mean that the US will lose its competitive edge over China as they’ll be very little differentiators between the two countries?
One thing is for sure, businesses will benefit from open-source models extensively. So let’s hope this ends up being a win-win for all involved. It has to be right? Considering the people always caught in the middle, and the stakes involved.