We need control, infrastructure, and governance

Translated from the original Spanish article.

“Pacing the frontier” is a newly published manifesto signed by more than 1,000 workers at frontier AI companies. A gesture to ease the conscience.

Essentially, they announce that they are very close to AI researching—and improving—itself. With unpredictable consequences. They explain that we might need time to understand the implications and risks arising from this scenario, which could advance at an unexpected speed.

You've been in this world with me for some time; you'll have seen a thing or two. So, if I asked you whether you think the planet's governments and companies could have the will, or even the ability, to coordinate in order to control and limit the overall progress of AI model research, what would you say?

Out of sheer caution, you might leave it at “unlikely.”

“AI is progressing at a rate that our society might not be ready for. Frontier labs are very close to AI that can exceed even the best people on almost every metric of intelligence.” Shengjia Zhao (Chief Scientist, Meta AI).

I've been working long enough with frontier models and autonomous agents that have persistent memory, general context, and all kinds of tools: to connect to the internet, to write their own software, send emails, sign up on websites… and so on. You wouldn't believe what my executive assistant agent has managed to do on its own.

Right now the problem isn't technology. The problem is governance.

You can't let an AI act independently in pursuit of a goal, because it's impossible to keep up with it. Its capacity for work is, quite literally, inhuman.

So, when it comes to using frontier AI, like the agents I'm telling you about, the only decision is this:

Either you dial back its autonomy to keep it on rails we've designed (a process it must follow without straying from it), or you give it autonomy and close your eyes, hoping it doesn't do anything crazy, even unwittingly.

Obviously, the only option that should even be considered is the first. The second means losing control entirely, with unforeseen consequences, as we've seen in some highly publicized recent cases.

Naturally, temptation, combined with what we'll call flexible ethics, will lead quite a few to give agents autonomy in pursuit of whatever goal of their own.

“The world is locked in a deadly race towards an intelligence explosion… just like a runaway nuclear chain reaction.” Leo Gao (OpenAI).

With what I'm about to say, I risk being branded little short of a madman. But I'll tell you that I consider a scenario in which a digital collapse occurs, at extreme speed, through the uncontrollable action and reaction of sufficiently capable AIs. I'm not hiding from this idea; perhaps I'll develop it in another post, further down the line.

In any case, we don't need more capable AI models. We need control, infrastructure, and governance.