So Mythos and Sol come around, and they are weapons of mass destruction in the hands of civilians. At least, that’s what Anthropic tried to say, when they delayed its release to civilians — a brilliant marketing move on the heels of OpenAI back-talking to Pentagon to force them out of the US government market.
Despite this being a very extremist viewpoint, I actually believe it. Three years ago, if you wanted to launch a cyber-attack, you would need to be able to hire one of the black-hats available on the market. There aren’t that many of them, and they are not necessarily cheap (I hope, at least). If that knowledge is now available for everyone in LLM models, and with enough money you can run agents in the cloud, launching cyber-attacks with just two ingredients: money for compute and tokens, and a simple prompt.
LLMs are having the unfortunate effect of making the richer (Nvidia, OpenAI, Microsoft, …) even richer. Whoever has the money to buy infrastructure will reap its benefits the most. This seems to me the same as the Industrial Revolution that made farm owners even richer, and workers even poorer (despite the increased quality of life).
If I was the CTO of a Bank, State or any critical infrastructure, I would be panicking. How much more budget is needed to defend your systems from attacks? How much does it cost to repair the damage of people losing all their money, or their homes? The economy of attacks vs defense has changed a lot. Yes, you can use LLMs to fix your leaks, but you need to cover all of them and having software closed source does not help anymore. When attacking, you just need to attack one.
The recent Hugging Face unfortunate attack shows that these vulnerabilities exist and can be attacked with enough budget. I am not talking about a theoretical attack, this can be happening this exact moment. And the European Central Bank agrees with me.
As a CTO, I would ask to have offline (paper, even) copies of all critical data, and I would consider how much we can move into an air-gapped system. This could end all online banking (at least for large amounts), and we could go back to having special-purpose terminals in banks as an entry to the air-gapped network. This would be in parallel to improving the defenses, investing in open-source software and having teams to maintain them, and making sure they are up to date. I would create teams to manage supply-chain attacks (I have noticed a surge in these types of attacks).
And I’m usually a very positive person.