As AI technology gets adopted at a never-seen pace, we also need more people to work on AI Safety to make sure that AI applications are actually safe and aligned.

Because AI generates content and decisions faster than humans, this is a very interesting research problem: humans cannot verify AI alone, so we need smarter technology to do it.

In the interview below Ryan Kidd, the CEO of MATS, talks about the efforts they (and others) are doing to train people in AI Safety. Throughout the conversation he also addresses how many of these AI Safety companies/non-profits have been founded, and even goes into neural networks vs evolutionary algorithms.

But AI Safety is a very large umbrella that encompasses soft skills like Policy Making and writing Model Cards to more technical challenges like interpreting neural networks or building guardrails.

Ankur Pandey argues that AI Safety training is not focusing on building tools and that's probably where the gains are for society. I actually believe existing researchers from technical domains (like formal verification) are equipped to build the tools, they just need funding and support. As an example, Harmonic has been doing interesting stuff recently and I know they recruited the right people.

Unfortunately for those of us who do not want to relocate, many of these AI Safety companies are in-person in Berkeley, DC, Cambridge, NYC and London (with India and Singapore also investing in these areas). If I was funding these efforts, I would hire remotely and I would try to get funding from NATO/EU/UNESCO and other multinational organizations.

As for training, I think we should make sure that all universities offer an AI Safety course at least at the Masters level, but here I am, daydreaming again.