During my current visit to UCL’s SOLAR group, I attended this week’s SSE Seminar on the challenges posed by the AI Act, presented by Paolo Falcarin.
The European Union Artificial Intelligence act is EU’s first attempt of regulating AI products and services. It defines different requirements based on the risk level of the application (ranges between high, medium and low).
In particular, high-risk usage (healthcare, toys, security, …) have stricter requirements. To begin with, they need to be registered in an European database, and frequently updated as the implementation or requirements change. Furthermore, the service or product should be documented, should be traceable, transparent, secure and overridable by humans. Despite these requirements, there is no clear definition, or path forward on how these properties can be ensured, especially when applications are closed-source and frequently trained and updated. One of the challenges we addressed in CAMELOT was how to build interprable Machine Learning models. We explored the use of Genetic Programming and Domain-Specific Languages to create inherently interpretable models. My research team is continuing exploring that possibility.
General-Purpose models have a special categorization within the AI act: Providers of general-purpose models (think OpenAI or Google) must provide a good understanding of the capabilities and limitations, comply with European copyright law, and provide a sufficiently detailed summary about the content used for training of the general-purpose model, following a given template.
Despite all the effort in understanding the potential of deep neural networks, and generative models in particular, it is not clear year what exactly are the capabilities or limitations of what they can produce. Without a clear standard of what is expected, organizations might be able to completely ignore this requirement.
As an example, take the Gandalf AI game, in which you can (easily) trick the LLM into telling you the password of the next level, even though it was instructed not to. Even with the aditional blocks introduced by each level, it is still easy to win the game. This is the state of the art in protecting LLMs from producing a known output. On a larger scale, an earlier version of Microsoft’s AI-powered Bing also generated output that went against the policies it was purposely trained against.
As for the copyright compliance, this goes in direct conflict with what OpenAI is defending. In fact, using copyrighted materials for free in the context of learning is allowed by European law. As such, it is not clear what this article entails in practice. My guess is that this is going to require a reform on the copyright law, possible to distinguish human versus automated learning. Otherwise, models this efficient may never (legally) exist again. This is something the law and computer science communities should debate before politicians take the initiative.
Finally, general-purpose AI with systemic risks (probably all of Large Language Models) have stricter restrictions: they need to evaluate models based on standardised protocols and tools, documenting adversarial testing of the model. While there are good practices for evaluating models, I do not believe the community will agree on an universal metric for general-purpose AI, and different metrics will arise.
Overall, I think it is positive that the EU is trying to regulate the use of AI. Unfortunately, I think it is a lost battle, as the technology is very new and evolves at a very high pace (something that beaurocracy might slow down). I defend that the EU should invest more on the evaluation and monitoring of AI, maybe more than on its development. After all, we cannot compete with the likes of NVIDIA, OpenAI/Microsoft, Google, Apple or Amazon, since they aquire all the relevant hardware before Europe does. Not even Intel or TSMC investing in Europe will allow us to beat US companies (or adjacent universities) in AI research. However, we can beat them in studying the societal impact of AI.
[1]