Tech leaders are again warning that artificial intelligence is advancing faster than the safeguards designed to control it

By David Chan | The Current

LOS ANGELES — For years, warnings that artificial intelligence could one day escape human control sounded more like science fiction than a serious business or public-policy discussion.

That is becoming harder to dismiss.

Some of the strongest warnings about AI are now coming from inside the industry itself — from researchers, executives and companies responsible for building increasingly powerful systems.

The debate is no longer simply about whether AI will replace office workers, disrupt journalism or automate customer service. The much larger question is whether humanity can continue to control increasingly autonomous technology as its capabilities accelerate.

Anthropic CEO Dario Amodei has argued that the industry needs to devote substantially more attention to safeguards as AI systems become more capable and increasingly able to perform tasks without constant human supervision.

OpenAI CEO Sam Altman has similarly argued that leading AI companies should cooperate on safety rather than waiting for governments to figure everything out.

The idea is not necessarily to stop AI development.

It is to make sure humans remain in control of what they are building.

And that distinction could become one of the defining technology debates of the next decade.

From chatbot to autonomous agent

Most people still experience artificial intelligence through relatively simple interactions.

They ask ChatGPT or Claude a question. The computer provides an answer. The human remains firmly in charge.

But the technology industry is rapidly moving toward AI “agents” capable of performing much more complicated sequences of actions.

Instead of simply telling someone how to book a trip, for example, an agent could potentially research flights, compare hotels, examine a calendar, make reservations and reorganize an itinerary.

In business, agents could analyze sales information, contact customers, write software, conduct research and operate other computer systems.

That could produce enormous productivity gains.

It also creates a fundamentally different risk.

The more authority humans give an AI system to act independently, the more important it becomes to understand how that system will behave when it encounters circumstances its designers did not anticipate.

Security researchers have already demonstrated situations in controlled testing where advanced models found unexpected ways of completing objectives, exploiting vulnerabilities or behaving differently when safeguards were weakened.

That does not mean today’s AI systems are secretly plotting against humanity.

But it does demonstrate why researchers take the control problem seriously.

The more immediate danger may be human

There is also a less futuristic concern.

AI does not have to become an uncontrollable superintelligence to cause enormous damage.

A human being can use it.

Advanced models can potentially increase the capabilities of hackers, fraudsters, hostile governments and other malicious actors. AI companies have consequently introduced restrictions around areas including cyberattacks and biological weapons research.

The uncomfortable reality is that the same characteristic making artificial intelligence economically valuable — its ability to dramatically increase what one person can accomplish — could also increase what one malicious person can accomplish.

Imagine sophisticated cyberattack capabilities that once required a team of experienced programmers becoming available to a single operator.

Or biological research that once required years of specialized expertise becoming easier to understand and manipulate.

The technology itself does not need malicious intentions.

It simply needs to make dangerous people more capable.

Nobody really knows where this ends

Predictions of machines surpassing human intelligence are hardly new.

British mathematician Alan Turing was considering the implications of intelligent machines more than seven decades ago. Mathematician Norbert Wiener similarly warned about giving machines objectives without fully understanding the consequences.

What has changed is the technology.

Artificial intelligence is no longer theoretical.

Businesses are deploying it. Governments are studying it. Militaries are exploring it. Students are using it. Software developers are building with it.

Billions of dollars are being invested in the race for increasingly powerful systems.

That competition creates its own problem.

Even if one company decides that development should slow down, competitors may not.

If American companies slow development, Chinese companies could continue.

If one laboratory refuses to release a powerful model, another might.

It is a classic competitive dilemma: everyone may benefit from restraint, but nobody wants to be the first competitor to surrender an advantage.

Regulation faces the same problem

Governments are trying to catch up.

China has publicly emphasized maintaining human control over artificial intelligence. The United States continues debating how much regulation is necessary without undermining innovation and competitiveness.

Other countries are developing their own approaches.

But legislation traditionally moves slowly.

AI development does not.

A regulatory framework designed around today’s technology could already be outdated by the time it becomes law.

That leaves governments facing an unusually difficult balancing act.

Regulate too aggressively and they risk restricting one of the most important economic technologies since the internet.

Regulate too lightly and society could discover the dangers only after systems have become too powerful or too widely deployed to easily contain.

The question isn’t whether AI is good or bad

Artificial intelligence could produce extraordinary benefits.

It could accelerate drug discovery, improve medical diagnosis, increase industrial productivity, transform education and allow small businesses to operate with capabilities previously available only to large corporations.

Those possibilities should not be ignored because of hypothetical worst-case scenarios.

But neither should legitimate risks be dismissed simply because they sound dramatic.

The International AI Safety Report has described the probability and timing of catastrophic loss-of-control scenarios as highly uncertain. Current systems have demonstrated pieces of capabilities that researchers watch closely, but that is very different from evidence that today’s AI can independently overthrow human control.

That uncertainty is precisely what makes the debate difficult.

Humanity is effectively developing increasingly capable technology while simultaneously trying to understand what its limits should be.

There may never be a moment when a computer suddenly announces that humans are no longer in charge.

The greater danger could be much less cinematic.

We may simply keep handing machines more authority because doing so is profitable, convenient and competitive — one task, one system and one decision at a time.

And by the time society decides where the line should have been drawn, it may discover that drawing it has become considerably harder.

David Chan is a reporter with The Current.